Hi all!
I'm thinking about (re-)implementing a web-based soccer sweepstake[1]
that I'm doing regularly with some of my friends in Common Lisp. We
already had different versions (World Cup, European Championship,
Bundesliga, ...) with different rules and I now have to write a new
version.
I could overhaul one of the old versions which were written in Perl or
PHP but I'd rather write it using Common Lisp (actually LispWorks or
CMUCL with Apache/mod_lisp) because I think it'd be more fun for me
and a good way to learn more Lisp.
The old versions were all based on an RDMBS (PostGreSQL) because I
needed persistent data[2] and didn't want to care about things like
concurrent access. However, with Lisp I have the option to have
everything in one process and can thus keep the data structures in
memory (which would make the whole thing much more elegant and faster
BTW).
I see no problems with transactions as I think this can be done with
locking but I wonder how to implement backups.[3] With an RDBMS that's
easy but I'd rather avoid the complexity of UncommonSQL if
possible. Would dumping the whole image from time to time be a good
solution? Or should I just use PRINT or FORMAT to write the data to a
plain text file and READ it back in[4] if necessary? Any reason to use
something like Berkeley DB instead of plain files? (I'm not familiar
with Berkeley DB.) Or something completely different that I missed?
(I'm aware of Plob but I think this is a bit too much for this little
game.)
What do experienced Lispers do in such a situation?
Thanks in advance for your hints,
Edi.
[1] Is that the right word? I want to say that we're betting on the
matches.
[2] The different Apache processes of course have to share data like
the match result and the players' rankings.
[3] I like the idea of having the data still available in a consistent
state even if the program crashes (I make mistakes...) or the
server has to be restarted or whatever.
[4] I understand that this approach is not always possible.
Edi Weitz <···@agharta.de> writes:
> I see no problems with transactions as I think this can be done with
> locking but I wonder how to implement backups.[3] With an RDBMS that's
> easy but I'd rather avoid the complexity of UncommonSQL if possible.
here's the deal. Persistence (in CL) doesn't come cheap. There are a
few options that I think minimize the burden. One of them is Common SQL
(via a LispWorks enterprise ed.) Another (cheaper) is UncommonSQL,
which you might have seen other posts about. Another is PLOB!, which
you know about, and which I think is the best option -- especially if
you just want to keep it simple and in CL, without major interface
headaches. PLOB! has, IMO, a better interface than AllegroStore (from
Franz). It's nice that it's free for non-commercial use.
dave
Dave Bakhash <·····@alum.mit.edu> writes:
> Edi Weitz <···@agharta.de> writes:
>
> > I see no problems with transactions as I think this can be done with
> > locking but I wonder how to implement backups.[3] With an RDBMS that's
> > easy but I'd rather avoid the complexity of UncommonSQL if possible.
>
> here's the deal. Persistence (in CL) doesn't come cheap. There are a
> few options that I think minimize the burden. One of them is Common SQL
> (via a LispWorks enterprise ed.) Another (cheaper) is UncommonSQL,
> which you might have seen other posts about. Another is PLOB!, which
> you know about, and which I think is the best option -- especially if
> you just want to keep it simple and in CL, without major interface
> headaches. PLOB! has, IMO, a better interface than AllegroStore (from
> Franz). It's nice that it's free for non-commercial use.
Well, not to be blunt, but bullshit! Persistence is way cheap in CL,
wether you hand roll it or use USQL. I can't think of another
language where persistence is cheaper actually!
On the way cheap, with the CVS version of ODCL you can use the
filesystem-db in filesystem.lisp, with the lightweight transaction
monitor in transaction.lisp, which USQL editing contexts are built
upon, to get transaction managment for throwing whatever
representation of the data you want, lists, binary blobs etc... into
the filesystem-db. We use alists for most of the "metadata" about
objects in our system, but we also have a whole MARC catalog store
built on top of it that stores the MARC records ina lightly parsed
sexp representation of MARC.
We use this, as well as some indexing support that allows us to
construct indexes of the blobs stored in the filesystem-db and ensure
they get update when the data changes. The indexes are presently
mapped into RDBMS tables. One top of this we have a query engine
which uses the indexes from the filestore and the core view-class
entities from our application framework.
None of this is theoretically pure, well-formed, or even politicall
correct, and I'm sure we will get many critiques about how it's not
REALLY persistence, and not REALLY OO or whatever, but I think it has
succesfully tied together a few different data storage paradigms.
Craig Brozefsky <·····@red-bean.com> writes:
> Dave Bakhash <·····@alum.mit.edu> writes:
> here's the deal.
> Persistence (in CL) doesn't come cheap. There are a > few options
> that I think minimize the burden. One of them is Common SQL...
>
> Well, not to be blunt, but bullshit! Persistence is way cheap in CL,
> wether you hand roll it or use USQL. I can't think of another
> language where persistence is cheaper actually!
First off, when I say "not cheap", I mean in the same way that Linux is
not cheap. You might have heard that "Linux is only free if your time
is worth nothing." To some extent, that has been true for me. The same
goes for the O/R mappings. It's not an easy problem, and I've come to
realize that the solutions are not quite there yet (for me). I don't
think that USQL is at a point where it is usable in my programs, and so
I'm just projecting when I steer people away from it. As it is, I think
that LW's Common SQL is also deficient. I don't have the answer just
yet, but I'm working toward it at this time. I've directed a lot of
time (and money) into this precise direction, and have had the benefit
of working with database experts in this area. If you ask me, I'd say
that USQL should be scrapped (and I mean that bluntly), though parts of
it might prove useful in its successor.
At one company, we were able (painfully and expensively) to make LW's
SQL package behave in a way that made sense. But I am aware of the
limitations of that effort as well, and it's quite depressing to see
that people are satisfied with these products.
At this time, the truth is that persistence is probably now most
developed in Java. Java is not nearly the best language for this (CL
is), but it is sufficient for a decent solution, and there are a whole
host of these for Java, many of which are superior to their Lisp
counterparts. It's mostly a function of the amount of resources that
have gone into them.
In a few months, god willing, I will have a simple O/R mapping. If so,
it will be intuitive, quick (relatively speaking), transactionally safe,
and will work at the very least with LispWorks and with
InterBase/FireBird. It's a realistic effort. I am not starting from
the Common SQL framework, which I believe to be glitchy.
This is not to say that any of these existing packages can't work for
your application. I think people should explore them all, if possible,
and use them if possible. But we'll be seeing much better alternatives
in the not-too-distant future, for sure.
dave
On 02 Aug 2002 14:00:19 -0400, Dave Bakhash <·····@alum.mit.edu> wrote:
>At one company, we were able (painfully and expensively) to make LW's
>SQL package behave in a way that made sense. But I am aware of the
>limitations of that effort as well, and it's quite depressing to see
>that people are satisfied with these products.
Can you please elaborate on this? What do you consider broken and how would
you fix it?
-----------------------
Fernando Rodriguez
Dave Bakhash <·····@alum.mit.edu> writes:
> At this time, the truth is that persistence is probably now most
> developed in Java. Java is not nearly the best language for this (CL
> is), but it is sufficient for a decent solution, and there are a whole
> host of these for Java, many of which are superior to their Lisp
> counterparts. It's mostly a function of the amount of resources that
> have gone into them.
Could you elaborate on these advantages the Java components have? One
of the issues I have with USQL presently is that we haven't really
defined the DBMS interface sufficiently to allow the easy creation of
new drivers for new DBs, and that presently it does not support the
translation of DB values into lisp values at a sufficiently low level
to have them handled by the DB drivers. Java JDBC is more concrete
and documented if not anything else.
> In a few months, god willing, I will have a simple O/R mapping. If so,
> it will be intuitive, quick (relatively speaking), transactionally safe,
> and will work at the very least with LispWorks and with
> InterBase/FireBird. It's a realistic effort. I am not starting from
> the Common SQL framework, which I believe to be glitchy.
Could you be more specific with regards to "glitchy" as it manifests
in USQL? What particular aspects of the O/R model do you dislike? As
an author of USQL I could list quite a few things that I feel are
wrong with it, but fresh eyes often give insightful critique.
> This is not to say that any of these existing packages can't work for
> your application. I think people should explore them all, if possible,
> and use them if possible. But we'll be seeing much better alternatives
> in the not-too-distant future, for sure.
I hope so too, and good luck.
--
Sincerely,
Craig Brozefsky <·····@red-bean.com>
Free Scheme/Lisp Software http://www.red-bean.com/~craig
Craig Brozefsky <·····@red-bean.com> writes:
> ...and that presently it does not support the translation of DB values
> into lisp values at a sufficiently low level to have them handled by
> the DB drivers. Java JDBC is more concrete and documented if not
> anything else.
This is definately one of the problems that I'm trying to address now.
My system, which is currently under development, integrates Lisp types
and SQL types more naturally, and manages how they are translated from
one world to another. You can also define new types, and they may be
parameterized. This is integrated into the O/R mapping. In the
definition of a new type, you define:
o what a valid CL typespec is for the type
o what a valid SQL typespec is for the type
o bilateral translation of values to/from CL/SQL
> Could you be more specific with regards to "glitchy" as it manifests
> in USQL? What particular aspects of the O/R model do you dislike? As
> an author of USQL I could list quite a few things that I feel are
> wrong with it, but fresh eyes often give insightful critique.
I'm so happy to hear that the USQL people are open. I recall once
trying to suggest that CLOCC socket stuff was not my speed, and the
result was equivalent to popping a quaalude in his newsreader.
My main beef with USQL has to do with the added requirements on slot
definitions. I feel that there is enough symmetry to prevent a lot of
additional mess that ends up obfuscating O/R mappings.
Moreover, I think that defining a schema that results in the definition
of one or more persistent classes should engender not just the DML
mapping, but the DDL mapping as well.
I also believe that this mapping should not just reflect the inherent
O/R mapping between CLOS and SQL, but also the storage mechanisms in CL
(namely, gc in SQL, which is possible, and could easily be defined
through the O/R mapping interface).
Another aspect of USQL and Common SQL that I feel is deficient is in
transaction management. I have a lot to say about this, and I've tried
to make improvements there. I got help and feedback from several
experts, including someone who is, without question, an expert in
database issues, and who knows how it's done in Java (EJB), and other
systems. Just as gc will be brought to the SQL side, so too will
transactional security (including nested transactions) be brought to the
Lisp side.
> > This is not to say that any of these existing packages can't work
> > for your application. I think people should explore them all, if
> > possible, and use them if possible. But we'll be seeing much better
> > alternatives in the not-too-distant future, for sure.
>
> I hope so too, and good luck.
Yeah...there's so much to a good O/R mapping...so many subtleties...that
you don't need to get everything right to have something that is useful,
and something that lessens the overall burden of programming.
But to stop at any of these places means that you're satisfied...and I
know that even with what I'm trying to build now, I won't be satisfied.
dave
Dave Bakhash <·····@alum.mit.edu> writes:
> This is definately one of the problems that I'm trying to address now.
> My system, which is currently under development, integrates Lisp types
> and SQL types more naturally, and manages how they are translated from
> one world to another. You can also define new types, and they may be
> parameterized. This is integrated into the O/R mapping. In the
> definition of a new type, you define:
>
> o what a valid CL typespec is for the type
> o what a valid SQL typespec is for the type
> o bilateral translation of values to/from CL/SQL
I think I can be more specific in what I find to be USQL's fault here.
First, the method for defining the translation of a value into SQL for
a given type definition can really only specialize on symbols for type
specifiers. However, the most pressing is an extra translation from
the lisp value into its SQL string representation at too high a level.
This is an artifact of wanting to get a readable SQL representation
bootstrapped ASAP. A refactoring of the output-sql methods for the
various SQL expression objects would allow the conversion to be put
off until the RDBMS layer.
Perhaps it's because I just glued presentation types from McCLIM into
IMHO, but I am thinking that a similiar model, if not the same
presentation type system, would make this protocol simple, coherent
with other mechanisms in CL for describing representations of lisp
values in an external form. Postponing that conversion and allowing
the database drivers to optimize the representations of various lisp
types would be a big win. It would also do good to create a operator
in the functional SQL interface and syntax to handle such
presentation, as there presently is no way other than a roundabout
hack thru the [slot-value 'class 'slotname value] operator to control
the conversion of a lisp value via a type specifier.
> Moreover, I think that defining a schema that results in the definition
> of one or more persistent classes should engender not just the DML
> mapping, but the DDL mapping as well.
Hmm, the workable schema support in USQL handles the creation and
destruction of the various tables, indexes and sequences generated
from a def-view-class, and I see that as part of the DDL. Perhaps you
mean that a constructor or something for instances of those persistent
classes should be defined from the information it has about the
persistence of the class, but I am failing to see a real win over the
existing make-instance protocol sufficient to warrant such automation.
> I have a lot to say about this, and I've tried to make improvements
> there. I got help and feedback from several experts, including
> someone who is, without question, an expert in database issues, and
> who knows how it's done in Java (EJB), and other systems. Just as
> gc will be brought to the SQL side, so too will transactional
> security (including nested transactions) be brought to the Lisp
> side.
You should see Editing Contexts as they appear in Apple/NeXT EOF if
you have not already. USQL does have ECs which provide transaction
mgmt for in-memory objects and ensures they are updated in the store
appropriately. However it does not support nested contexts and nested
rollbacks. Given the lowest common denominators of the existing set
of DBMS drivers available for USQL, I don't see that changing soon.
> Yeah...there's so much to a good O/R mapping...so many
> subtleties...that you don't need to get everything right to have
> something that is useful, and something that lessens the overall
> burden of programming.
>
> But to stop at any of these places means that you're satisfied...and I
> know that even with what I'm trying to build now, I won't be satisfied.
Well, I can appreciate this level of abstraction while recognizing
that it's not the current direction or goal of USQL. I can't speak
for anyone else who writes code for USQL, but I see it not as aspiring
towards the most advanced O/R mapping mechanism available for CL. The
functional SQL interface and syntax do no just exist as implementation
sublayers for the O/R mapping, just as its editing contexts do not
existing only to provide cacheing and transaction control for the O/R
mapping.
There are many different ways to store data and a super clean O/R
system just doesn't seem to payoff whereas a more ecletic approach
tying together files, abstract tables, view-classes, metadata, and
persistent alists into a single transaction controlled system supports
a wider variety of my data access, indexing, searching and storage
needs.
--
Sincerely,
Craig Brozefsky <·····@red-bean.com>
Free Scheme/Lisp Software http://www.red-bean.com/~craig
Craig Brozefsky <·····@red-bean.com> writes:
> > Moreover, I think that defining a schema that results in the
> > definition of one or more persistent classes should engender not
> > just the DML mapping, but the DDL mapping as well.
>
> Hmm, the workable schema support in USQL handles the creation and
> destruction of the various tables, indexes and sequences generated
> from a def-view-class, and I see that as part of the DDL.
I have seen (years ago) how def-view-class generates tables. I don't
recall all of the reasons I wasn't satisfied with it, but it's something
that I should look into. But I think that a problem I had with the
whole thing also had to do with how inheritance was implemented (at the
database level). I'm not too sure I actually flushed that all out,
though.
> Perhaps you mean that a constructor or something for instances of
> those persistent classes should be defined from the information it has
> about the persistence of the class
no...that's not what I was referring to wrt DDL.
> You should see Editing Contexts as they appear in Apple/NeXT EOF if
> you have not already. USQL does have ECs which provide transaction
> mgmt for in-memory objects and ensures they are updated in the store
> appropriately. However it does not support nested contexts and nested
> rollbacks. Given the lowest common denominators of the existing set
> of DBMS drivers available for USQL, I don't see that changing soon.
I see. Well, this is a great note. I'll definately look into ECs,
since it seems like something I've wanted for in-memory objects.
> There are many different ways to store data and a super clean O/R
> system just doesn't seem to payoff whereas a more ecletic approach
> tying together files, abstract tables, view-classes, metadata, and
> persistent alists into a single transaction controlled system supports
> a wider variety of my data access, indexing, searching and storage
> needs.
While I mostly agree, the bottom line that I'm going for is a maximally
transparent, safe, and efficient interface for persistent data. The
different designs you get seem to be based on different weights on these
objectives.
Thanks for explaining more about USQL.
dave
From: Tim Bradshaw
Subject: Re: Data persistence for a small app
Date:
Message-ID: <ey3adoenwpi.fsf@cley.com>
* Edi Weitz wrote:
> I see no problems with transactions as I think this can be done with
> locking but I wonder how to implement backups.[3] With an RDBMS that's
> easy but I'd rather avoid the complexity of UncommonSQL if
> possible. Would dumping the whole image from time to time be a good
> solution? Or should I just use PRINT or FORMAT to write the data to a
> plain text file and READ it back in[4] if necessary? Any reason to use
> something like Berkeley DB instead of plain files? (I'm not familiar
> with Berkeley DB.) Or something completely different that I missed?
> (I'm aware of Plob but I think this is a bit too much for this little
> game.)
dribbling a log of transactions to a file (or several) is quite a good
approach. You can then recreate a state of the system by loading the
initial state and then rolling the system forward from the real state.
At some point you need to save the current state and then restart the
log, and this is fiddly to get right, especially if you can not afford
downtime. If you care a lot about the data you also need to be *very*
careful that it is on disk when you think it should be. I've worked
on a system that did just this trick.
--tim
On 25 Jul 2002 18:55:52 +0200, Edi Weitz <···@agharta.de> wrote:
> solution? Or should I just use PRINT or FORMAT to write the data to a
> plain text file and READ it back in[4] if necessary? Any reason to use
> something like Berkeley DB instead of plain files? (I'm not familiar
> with Berkeley DB.) Or something completely different that I missed?
You may have a look at the CLOS I/O support code which comes as part of
CLOCC's cllib.
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://www.paoloamoroso.it/ency/README
Edi Weitz <···@agharta.de> wrote in message news:<··············@dyn164.dbdmedia.de>...
> ......
> I see no problems with transactions as I think this can be done with
> locking but I wonder how to implement backups.[3] With an RDBMS that's
> easy but I'd rather avoid the complexity of UncommonSQL if
> possible.
>
UncommonSQL is not the only choice. Eric Marsden has published a socket
level interface to PostgreSQL (http://www.chez.com/emarsden/downloads/pg.lisp)
which works both with CMUCL and CLisp. If this does not fulfil your needs,
I would be curious to know why this is not so.
regards,
-Klaus.
Edi Weitz <···@agharta.de> writes:
> locking but I wonder how to implement backups.[3] With an RDBMS that's
> easy but I'd rather avoid the complexity of UncommonSQL if
> possible. Would dumping the whole image from time to time be a good
> solution? Or should I just use PRINT or FORMAT to write the data to a
> plain text file and READ it back in[4] if necessary? Any reason to use
> something like Berkeley DB instead of plain files? (I'm not familiar
> with Berkeley DB.) Or something completely different that I missed?
> (I'm aware of Plob but I think this is a bit too much for this little
> game.)
>
> What do experienced Lispers do in such a situation?
Maybe there's a reason why experienced lispers didn't mention
make-load-form (and if so, I'd certainly like to know it), but that is
what I use for persistence.
In more detail: I have structures called cat, rootcat, sparse-vector,
and lots of instances of them which are all reachable from *root*
(an instance of rootcat). The way I save *root* (with all its
children) is this:
(defmethod make-load-form ((s cat) &optional environment)
(make-load-form-saving-slots s :environment environment))
(defmethod make-load-form ((s rootcat) &optional environment)
(make-load-form-saving-slots s :environment environment))
(defmethod make-load-form ((s sparse-vector) &optional environment)
(make-load-form-saving-slots s :environment environment))
(defun save-results (path)
(compile-file "save-aux.cl" :output-file path))
In save-aux.cl I have
(setf *rootcat* #.*rootcat*)
(setf *the-greatest-prime-we-found-so-far*
#.*the-greatest-prime-we-found-so-far*) ;;-) there is no need for
;; make-load-form for simple things
To restore the data, I (load "results.x86f") or whatever name I gave
it in save-results.
Andras
> Maybe there's a reason why experienced lispers didn't mention
> make-load-form (and if so, I'd certainly like to know it), but that is
> what I use for persistence.
It's sometimes not very complete. Ever seen :JUST-DUMP-IT-NORMALLY?
joelh
Joel Ray Holveck <·····@juniper.net> writes:
> > Maybe there's a reason why experienced lispers didn't mention
> > make-load-form (and if so, I'd certainly like to know it), but that is
> > what I use for persistence.
>
> It's sometimes not very complete. Ever seen :JUST-DUMP-IT-NORMALLY?
No, I haven't. What is it? And when is it not complete? What are the
issues one should be aware of?
Andras
······@math.bme.hu (Simon Andr�s) writes:
> Joel Ray Holveck <·····@juniper.net> writes:
>
>> > Maybe there's a reason why experienced lispers didn't mention
>> > make-load-form (and if so, I'd certainly like to know it), but that is
>> > what I use for persistence.
>>
>> It's sometimes not very complete. Ever seen :JUST-DUMP-IT-NORMALLY?
>
> No, I haven't. What is it? And when is it not complete? What are the
> issues one should be aware of?
:JUST-DUMP-IT-NORMALLY is the return value of various make-load-form
methods on CMUCL (and likely SBCL), which is used to indicate to
CMUCL's FASL dumper to "just dump it normally" ;).
However, as far as I'm aware, CMUCL's behaviour is actually
non-conforming, it should really return a pair of useful forms, as per
the spec on make-load-form.
That said, as long as you rely on the FASL dumper of an implementation
(as the OP did in his code), you shouldn't have to care about this.
This only gets problematic if you want to call make-load-form
yourself, and do something with the return values, however that is
always going to be tricky, even on conforming systems (you have to do
various magic with the forms returned, to be able to use them to
reconstruct arbitrary instances).
Regs, Pierre.
······@math.bme.hu (Simon Andr�s) writes:
> Joel Ray Holveck <·····@juniper.net> writes:
>
>> > Maybe there's a reason why experienced lispers didn't mention
>> > make-load-form (and if so, I'd certainly like to know it), but that is
>> > what I use for persistence.
>>
>> It's sometimes not very complete. Ever seen :JUST-DUMP-IT-NORMALLY?
>
> No, I haven't. What is it? And when is it not complete? What are the
> issues one should be aware of?
:JUST-DUMP-IT-NORMALLY is the return value of various make-load-form
methods on CMUCL (and likely SBCL), which is used to indicate to
CMUCL's FASL dumper to "just dump it normally" ;).
However, as far as I'm aware, CMUCL's behaviour is actually
non-conforming, it should really return a pair of useful forms, as per
the spec on make-load-form.
That said, as long as you rely on the FASL dumper of an implementation
(as the OP did in his code), you shouldn't have to care about this.
This only gets problematic if you want to call make-load-form
yourself, and do something with the return values, however that is
always going to be tricky, even on conforming systems (you have to do
various magic with the forms returned, to be able to use them to
reconstruct arbitrary instances).
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
Pierre R. Mai <····@acm.org> writes:
> ······@math.bme.hu (Simon Andr�s) writes:
>
> > Joel Ray Holveck <·····@juniper.net> writes:
> >
> >> > Maybe there's a reason why experienced lispers didn't mention
> >> > make-load-form (and if so, I'd certainly like to know it), but that is
> >> > what I use for persistence.
> >>
> >> It's sometimes not very complete. Ever seen :JUST-DUMP-IT-NORMALLY?
> >
> > No, I haven't. What is it? And when is it not complete? What are the
> > issues one should be aware of?
>
> :JUST-DUMP-IT-NORMALLY is the return value of various make-load-form
> methods on CMUCL (and likely SBCL), which is used to indicate to
> CMUCL's FASL dumper to "just dump it normally" ;).
Ha. No, SBCL currently returns :SB-JUST-DUMP-IT-NORMALLY. The
motivation for changing it (it wasn't change for change's sake, I
promise!) was that there was a nasty bootstrap bug that manifested
when compiling SBCL from OpenMCL; SBCL's fasl reader was depending on
the host having this special symbol for MAKE-LOAD-FORM.
> However, as far as I'm aware, CMUCL's behaviour is actually
> non-conforming, it should really return a pair of useful forms, as per
> the spec on make-load-form.
I think this is true; in particular, currently in cmucl/sbcl the user
can't do:
(defmethod make-load-form-saving-slots ...
(multiple-value-bind (create init)
(call-next-method)
(values `(let ... (prog1 ,create (frob-some-associated-resource))) init)))
I think, because the symbol :JUST-DUMP-IT-NORMALLY is only magical at
top-level.
I'm not certain that this cramps anyone's style, but I think it is an
actual bug.
Cheers,
Christophe
--
Jesus College, Cambridge, CB5 8BL +44 1223 510 299
http://www-jcsu.jesus.cam.ac.uk/~csr21/ (defun pling-dollar
(str schar arg) (first (last +))) (make-dispatch-macro-character #\! t)
(set-dispatch-macro-character #\! #\$ #'pling-dollar)
From: Andreas Hinze
Subject: Re: Data persistence for a small app
Date:
Message-ID: <3D47C29B.27679B36@smi.de>
"Simon Andr�s" wrote:
>
> Maybe there's a reason why experienced lispers didn't mention
> make-load-form (and if so, I'd certainly like to know it), but that is
> what I use for persistence.
>
> In more detail: I have structures called cat, rootcat, sparse-vector,
> and lots of instances of them which are all reachable from *root*
> (an instance of rootcat). The way I save *root* (with all its
> children) is this:
>
> (defmethod make-load-form ((s cat) &optional environment)
> (make-load-form-saving-slots s :environment environment))
>
> (defmethod make-load-form ((s rootcat) &optional environment)
> (make-load-form-saving-slots s :environment environment))
>
> (defmethod make-load-form ((s sparse-vector) &optional environment)
> (make-load-form-saving-slots s :environment environment))
>
> (defun save-results (path)
> (compile-file "save-aux.cl" :output-file path))
>
> In save-aux.cl I have
>
> (setf *rootcat* #.*rootcat*)
> (setf *the-greatest-prime-we-found-so-far*
> #.*the-greatest-prime-we-found-so-far*) ;;-) there is no need for
> ;; make-load-form for simple things
>
> To restore the data, I (load "results.x86f") or whatever name I gave
> it in save-results.
>
I tried to figure out what it will do. But i can't understand it. Even
looking up make-load-form-saving-slots in CLHS only results in a big
question mark :-(
Can anyone please be so kind to explain how that code works ? I didn't find
a explanation anywhere. If anyone has a working example i would like to study
it to find out for myself what is going on.
I found in CLHS that make-load-form-saving-slots returns an "copy" of the
given argument. But why make save-results the structures permanent ?
Does a call to make-load-form-saving-slots mean "a copy with all that is
included into the object" ?
Any help is welcome.
Thanks in advance
AHz
> I see no problems with transactions as I think this can be done with
> locking but I wonder how to implement backups.[3]
One possibility is to periodically make core images. Most Lisp
implementations support the ability to create a core image that you
can reload later. The typical use for this is to shorten load time
for product delivery.
I used to do this in the program I'm working on now, until I realized
the data could be easily reconstructed anyway. That may be another
avenue: keep a log of what you need to do to reconstruct your data.
Write to this log every time your database changes.
Neither one of these are very well thought-out, but they are
alternatives.
joelh