I've got some classes, with millions of instances. So much that it is
easy to get running out of memory.
My simple question is:
Is there any way to manage all of this clases through a database? I'm
not saying just persist them, but also getting them out of memory so
I can work from the database when needed?
I don't want to reimplement all of the program, I'm looking for a
solution like MOP or something like that.
Any help?
Javier <·······@gmail.com> writes:
> I've got some classes, with millions of instances. So much that it is
> easy to get running out of memory.
> My simple question is:
>
> Is there any way to manage all of this clases through a database? I'm
> not saying just persist them, but also getting them out of memory so
> I can work from the database when needed?
> I don't want to reimplement all of the program, I'm looking for a
> solution like MOP or something like that.
>
> Any help?
Take a look at the CL-SQL system and see if that meets your needs.
--
Thomas A. Russ, USC/Information Sciences Institute
We use a combination of Berkeley DB (formerly Sleepycat, now owned by
Oracle) and CL-STORE. Very fast, very easy to use.
Berkeley DB: http://www.oracle.com/technology/products/berkeley-db/index.html
CL-STORE: http://common-lisp.net/project/cl-store/
-- david
On 13 nov, 17:35, David Young <··········@gmail.com> wrote:
> We use a combination of Berkeley DB (formerly Sleepycat, now owned by
> Oracle) and CL-STORE. Very fast, very easy to use.
>
> Berkeley DB:http://www.oracle.com/technology/products/berkeley-db/index.html
> CL-STORE:http://common-lisp.net/project/cl-store/
>
> -- david
It seems to be very easy, thanks.
But, unfortunately, it seems that CL-STORE can't free memory (or I'm
wrong). My main problem is that memory is exausted. I'd like to work
with database like if I were working with memory. Is that possible?
I don't understand. As far as I know, CL-STORE only consumes memory
when (re)storing objects. Once this operation is done that memory is
available for garbage collection.
Perhaps I've misunderstood your needs. Are you looking for something
like AllegroCache? Are you trying to work with "millions of instances"
at one time? What Lisp platform are you using? If your Lisp has a
generational garbage collector, is it promoting some of your instances
to a generation that isn't usually swept at gc time?
--david
On 13 nov, 18:48, David Young <··········@gmail.com> wrote:
> I don't understand. As far as I know, CL-STORE only consumes memory
> when (re)storing objects. Once this operation is done that memory is
> available for garbage collection.
>
> Perhaps I've misunderstood your needs. Are you looking for something
> like AllegroCache? Are you trying to work with "millions of instances"
> at one time? What Lisp platform are you using? If your Lisp has a
> generational garbage collector, is it promoting some of your instances
> to a generation that isn't usually swept at gc time?
Yes, I'm looking for something like AllegroCache, but I'm not a
professional, so I won't pay for it.
Basically what I'm doing is some AI simulations, in my spare time,
which consumes lot of memory. So yes, I'm trying to work with millions
of instances "at the same time".
CL-STORE would be nice If I find a way to keep some instances in
memory while accessing others from database, but I think this would be
very difficult to implement, as I've to manually manage how the GC is
collecting things.
I hope you have now a better understanding. Thanks.
J> Basically what I'm doing is some AI simulations, in my spare time,
J> which consumes lot of memory. So yes, I'm trying to work with millions
J> of instances "at the same time".
if you don't actually need persistence and if you have access to 64-bit
system, you can just allocate freaking large swap file.
OS will move unused objects into swap file automatically, and move them back
when they are used.
the caveat is that full GC can be terribly slow in this situation, but
probably it can be tweaked so it will be usable.
Alex Mizrahi wrote:
> if you don't actually need persistence and if you have access to 64-bit
> system, you can just allocate freaking large swap file.
> OS will move unused objects into swap file automatically, and move them back
> when they are used.
No, it won't. What an OS will do is move least-recently-used PAGES to
disk, and page them back when any address on that page is accessed again
again. If your objects are much smaller than a memory page, performance
can be horrendous -- a lot depends on locality of reference: If objects
that are referenced in close temporal proximity are kept in memory
proximity, you might not lose too badly. But generally relying on the
OS to manage your memory for you doesn't scale very well because the OS
(and the GC) cannot know what is important about your memory usage. A
good GC will try to keep lists together, and a good OS will try to
predict when you are accessing file or memory pages in sequence, but
they can't do much to predict random traversal of linked objects
scattered all over memory. Sooner or later you will reach a knee point
and performance will die.
Page size on a 64-bit Linux is typically 4K bytes. Remember that on a
64-bit system conses and pointers are typically twice as large as on
32-bit systems.
> the caveat is that full GC can be terribly slow in this situation, but
This is indeed true.
> probably it can be tweaked so it will be usable.
Maybe. Maybe not.
P� Wed, 14 Nov 2007 02:40:43 +0100, skrev Steven M. Haflich
<···@alum.mit.edu>:
> Alex Mizrahi wrote:
>
>> if you don't actually need persistence and if you have access to 64-bit
>> system, you can just allocate freaking large swap file.
>> OS will move unused objects into swap file automatically, and move them
>> back when they are used.
>
> No, it won't. What an OS will do is move least-recently-used PAGES to
> disk, and page them back when any address on that page is accessed again
> again. If your objects are much smaller than a memory page, performance
> can be horrendous -- a lot depends on locality of reference: If objects
> that are referenced in close temporal proximity are kept in memory
> proximity, you might not lose too badly. But generally relying on the
> OS to manage your memory for you doesn't scale very well because the OS
> (and the GC) cannot know what is important about your memory usage. A
> good GC will try to keep lists together, and a good OS will try to
> predict when you are accessing file or memory pages in sequence, but
> they can't do much to predict random traversal of linked objects
> scattered all over memory. Sooner or later you will reach a knee point
> and performance will die.
Actually less horrendous under Windows which has a file based cache rather
that unix's volume based cashe.
--------------
John Thingstad
Den Wed, 14 Nov 2007 12:53:50 +0100 skrev John Thingstad:
>> generally relying on the OS to manage your memory for you doesn't scale
>> very well because the OS (and the GC) cannot know what is important
>> about your memory usage. A good GC will try to keep lists together,
>> and a good OS will try to predict when you are accessing file or memory
>> pages in sequence, but they can't do much to predict random traversal
>> of linked objects scattered all over memory. Sooner or later you will
>> reach a knee point and performance will die.
>
> Actually less horrendous under Windows which has a file based cache
> rather that unix's volume based cashe.
I fail to see even the slightest trace of relevance here.
Cheers,
Maciej
On Nov 13, 1:10 pm, Javier <·······@gmail.com> wrote:
> Yes, I'm looking for something like AllegroCache, but I'm not a
> professional, so I won't pay for it.
> Basically what I'm doing is some AI simulations, in my spare time,
> which consumes lot of memory. So yes, I'm trying to work with millions
> of instances "at the same time".
Take a look at Elephant (http://common-lisp.net/project/elephant/).
I'm not sure when or for how long it instantiates objects in memory
vs. reading them from disk, but it might be worth a try, and you might
be able to make it do what you want using its extensions to the MOP.
--Eli
Den Tue, 13 Nov 2007 08:15:42 -0800 skrev Javier:
> I've got some classes, with millions of instances. So much that it is
> easy to get running out of memory.
> My simple question is:
>
> Is there any way to manage all of this clases through a database? I'm
> not saying just persist them, but also getting them out of memory so I
> can work from the database when needed? I don't want to reimplement all
> of the program, I'm looking for a solution like MOP or something like
> that.
>
> Any help?
There is CL-perec (http://common-lisp.net/project/cl-perec/), which does
a lot of MOP magic to provide transparently persistent classes. It's very
much darcs software though, and getting it to run is not exactly trivial
(if you do try, *remember* to see if there isn't a branch under cl-dwim
project darcs for each dependency, if you take an upstream branch, it
might break in creative ways if the branch hasn't been merged yet).
Disclaimer: I'm not a dev, nor have I actually used cl-perec for anything
than playing around yet, so I dunno how well it will support your needs.
I guess Attila can tell more about suitability for millions of records
scenario, though he wrote if for a big gov't app, so I assume it was
intended for things like that.
Cheers,
Maciej
On 13 nov, 18:49, Maciej Katafiasz <········@gmail.com> wrote:
> There is CL-perec (http://common-lisp.net/project/cl-perec/), which does
> a lot of MOP magic to provide transparently persistent classes. It's very
> much darcs software though, and getting it to run is not exactly trivial
> (if you do try, *remember* to see if there isn't a branch under cl-dwim
> project darcs for each dependency, if you take an upstream branch, it
> might break in creative ways if the branch hasn't been merged yet).
>
> Disclaimer: I'm not a dev, nor have I actually used cl-perec for anything
> than playing around yet, so I dunno how well it will support your needs.
> I guess Attila can tell more about suitability for millions of records
> scenario, though he wrote if for a big gov't app, so I assume it was
> intended for things like that.
Thanks.
I've downloaded it and didn't found any documentation. I supose I've
got to read source code in order to understand it. This might take
some days, I think.
On Nov 13, 8:15 am, Javier <·······@gmail.com> wrote:
> I've got some classes, with millions of instances. So much that it is
> easy to get running out of memory.
> My simple question is:
>
> Is there any way to manage all of this clases through a database? I'm
> not saying just persist them, but also getting them out of memory so
> I can work from the database when needed?
> I don't want to reimplement all of the program, I'm looking for a
> solution like MOP or something like that.
>
> Any help?
AllegroCache is awesome, http://www.franz.com/products/allegrocache/index.lhtml
but it's only for Allegro lisp, if that's Ok for your purpose you
should contact Franz if you may find suitable license.
>From the opensource staff you might look at http://common-lisp.net/project/elephant/
supports Berkeley, Postgre, and many other RDBMS with cl-sql. I found
it's Postgre backend, through postmodern very responsive, there's is a
good community around it.
Submarine is also interesthing http://common-lisp.net/project/submarine/
but only supports Postgre. This could be as well feature.
If you give us more data about your lisp, os and what kind of rdbms
you're planning to use you might get better answers. Before you
settle on a lib, check what kind of lisp & os combination it's
community , especially main developers are using. Authors are doing
their best to support at least the big five ( ACL, LW, SBCL, OpenMCL &
CMUCL) but it would be wise to choose their native lisp & OS unless
you're ready to help with development.
cheers
Slobodan
On 13 nov, 19:26, Slobodan Blazeski <·················@gmail.com>
wrote:
> AllegroCache is awesome, http://www.franz.com/products/allegrocache/index.lhtml
> but it's only for Allegro lisp, if that's Ok for your purpose you
> should contact Franz if you may find suitable license.
Yes, something like AllegroCache is what I'm looking for. But I'm
playing around in my spare-time (I'm not currently a full-time
developer, although I was), so I'm not going to buy it.
>From the opensource staff you might look athttp://common-lisp.net/project/elephant/
Unfortunately, there are some irritating limits, these two:
"Garbage collection is only supported via an offline migrate interface
which will compact the database by only copying reachable instances.
Explicit deletion of data is also possible and some data stores will
exploit reuse freed space to reduce the total disk utilization."
Which, if I understand well, means that I still need to explicitly
delete data, which is something that I do not desire (it will increase
the complexity of my program too much).
and
"Instances are lightweight, storing only a unique ID and a parent
store reference"
Probably it is enough right now to store an ID (I supose is internaly
an integer) and a reference. But is is suposed that my system is going
to grow up so much that even this would mean that the program is going
to run out of memory.
> supports Berkeley, Postgre, and many other RDBMS with cl-sql. I found
> it's Postgre backend, through postmodern very responsive, there's is a
> good community around it.
> Submarine is also interesthinghttp://common-lisp.net/project/submarine/
> but only supports Postgre. This could be as well feature.
> If you give us more data about your lisp, os and what kind of rdbms
> you're planning to use you might get better answers. Before you
> settle on a lib, check what kind of lisp & os combination it's
> community , especially main developers are using. Authors are doing
> their best to support at least the big five ( ACL, LW, SBCL, OpenMCL &
> CMUCL) but it would be wise to choose their native lisp & OS unless
> you're ready to help with development.
My Lisp is SBCL, Debian, 32 bits (probably 64bits in the future), and
the type of database is probably BerkeleyDB, as I have read that it is
fast and don't need the IP stack (slow), but just internal library
calls.
Thanks.
On Nov 13, 12:29 pm, Javier <·······@gmail.com> wrote:
> On 13 nov, 19:26, Slobodan Blazeski <·················@gmail.com>
> wrote:
>
> > AllegroCache is awesome, http://www.franz.com/products/allegrocache/index.lhtml
> > but it's only for Allegro lisp, if that's Ok for your purpose you
> > should contact Franz if you may find suitable license.
>
> Yes, something like AllegroCache is what I'm looking for. But I'm
> playing around in my spare-time (I'm not currently a full-time
> developer, although I was), so I'm not going to buy it.
>
> >From the opensource staff you might look athttp://common-lisp.net/project/elephant/
>
> Unfortunately, there are some irritating limits, these two:
>
> "Garbage collection is only supported via an offline migrate interface
> which will compact the database by only copying reachable instances.
> Explicit deletion of data is also possible and some data stores will
> exploit reuse freed space to reduce the total disk utilization."
>
> Which, if I understand well, means that I still need to explicitly
> delete data, which is something that I do not desire (it will increase
> the complexity of my program too much).
>
> and
>
> "Instances are lightweight, storing only a unique ID and a parent
> store reference"
>
> Probably it is enough right now to store an ID (I supose is internaly
> an integer) and a reference. But is is suposed that my system is going
> to grow up so much that even this would mean that the program is going
> to run out of memory.
>
> > supports Berkeley, Postgre, and many other RDBMS with cl-sql. I found
> > it's Postgre backend, through postmodern very responsive, there's is a
> > good community around it.
> > Submarine is also interesthinghttp://common-lisp.net/project/submarine/
> > but only supports Postgre. This could be as well feature.
> > If you give us more data about your lisp, os and what kind of rdbms
> > you're planning to use you might get better answers. Before you
> > settle on a lib, check what kind of lisp & os combination it's
> > community , especially main developers are using. Authors are doing
> > their best to support at least the big five ( ACL, LW, SBCL, OpenMCL &
> > CMUCL) but it would be wise to choose their native lisp & OS unless
> > you're ready to help with development.
>
> My Lisp is SBCL, Debian, 32 bits (probably 64bits in the future), and
> the type of database is probably BerkeleyDB, as I have read that it is
> fast and don't need the IP stack (slow), but just internal library
> calls
>
> Thanks.
Berkeley is not free either I'm interested about the price but unable
to find it at their site , according to
http://searchoracle.techtarget.com/originalContent/0,289142,sid41_gci1218680,00.html
be prepared for ~ $750 per processor. So maybe PostgreSQL could do
the trick as it seems that is the only option available for free.
Slobodan
SB> Berkeley is not free either I'm interested about the price but unable
SB> to find it at their site , according to
SB>
http://searchoracle.techtarget.com/originalContent/0,289142,sid41_gci1218680,00.html
SB> be prepared for ~ $750 per processor.
unless you want to sell proprietary stuff it's free.
also iirc they've said once that using it from dynamic languages is free
(kinda only bindings themselves subject to GPL).
SB> So maybe PostgreSQL could do the trick as it seems that is the only
SB> option available for free.
postgresql ain't only free database!
SQLite seems to be a viable option -- it runs in same process, just like
BDB, but offers SQL-ish front-end, although accessors are nicely low-level.
On Nov 14, 1:20 pm, "Alex Mizrahi" <········@users.sourceforge.net>
wrote:
> SB> Berkeley is not free either I'm interested about the price but unable
> SB> to find it at their site , according to
> SB>http://searchoracle.techtarget.com/originalContent/0,289142,sid41_gci...
> SB> be prepared for ~ $750 per processor.
>
> unless you want to sell proprietary stuff it's free.
> also iirc they've said once that using it from dynamic languages is free
> (kinda only bindings themselves subject to GPL).
I'm not an lawyer but according to their site http://www.oracle.com/database/berkeley-db.html
you could use their dbms for free if you don't redistribute it, else
you have to buy a commercial license or release your code under gpl or
bsd license .It doesn't matter would you charge or not as long as you
don't redistribute it, putting behind a public website is considered
redistribution.
There is one tricky part about php/python scripts that I don't
understand. But I would rather avoid
Berkeley now , also the guys from elephant switched from Berkeley to
PostgreSQL because of the licensing. So consult your lawyer.
As for noncommercial use, you can use whatever you want. Nobody gonna
sue you if you don't have any money.
>
> SB> So maybe PostgreSQL could do the trick as it seems that is the only
> SB> option available for free.
>
> postgresql ain't only free database!
Agreed, there's other options like firebird etc but Postgre is
definately most popular after mysql. Even closed source companies
are giving their entry level dbms for free like Oracle Express, SQL
Server 2005 and especially DB2 express
http://www-306.ibm.com/software/data/db2/express/support.html .
But unless you plan to upgrade to payed option I would stick with
Postgre or some other open source solution. If you don't have a payed
support you need a good community to help you when problems arise .
And they will arise.
>
> SQLite seems to be a viable option -- it runs in same process, just like
> BDB, but offers SQL-ish front-end, although accessors are nicely low-level.
SQLite is very limited solution but depends on your needs.
Slobodan
SB> I'm not an lawyer but according to their site
http://www.oracle.com/database/berkeley-db.html
SB> you could use their dbms for free if you don't redistribute it, else
SB> you have to buy a commercial license or release your code under gpl or
SB> bsd license .It doesn't matter would you charge or not as long as you
SB> don't redistribute it, putting behind a public website is considered
SB> redistribution.
http://www.oracle.com/technology/software/products/berkeley-db/htdocs/licensing.html
where do you see "putting behind a public website is considered
redistribution"?
SB> There is one tricky part about php/python scripts that I don't
SB> understand.
----
Do I have to pay for a Berkeley DB license to use it in my Perl or Python
scripts?
No, you may use the Berkeley DB open source license at no cost. The Berkeley
DB open source license requires that software that uses Berkeley DB be
freely redistributable. In the case of Perl or Python, that software is Perl
or Python, and not your scripts. Any scripts you write are your property,
including scripts that make use of Berkeley DB. None of the Perl, Python or
Berkeley DB licenses place any restrictions on what you may do with them.
----
is there something you don't understand here? BDB is free for all dynamic
languages :)
SB> But I would rather avoid Berkeley now , also the guys from elephant
SB> switched from Berkeley to PostgreSQL because of the licensing.
elephant supports multiple backends. postgresql-only backend offers
additional features comparing to BDB one.
??>> SQLite seems to be a viable option -- it runs in same process, just
??>> like BDB, but offers SQL-ish front-end, although accessors are nicely
??>> low-level.
SB> SQLite is very limited solution but depends on your needs.
what limited do you find in SQLite? that it doesn't support TCP/IP
connections?
but OP explicitly said that he doesn't want to use TCP/IP, and i'd say you
he is right -- TCP/IP will simply kill performance.
Alex I'm not a lawyer and I doubt you are so If you plan to use it
why don't you just consult one or even better contact Oracle and ask
them is Berkeley free to be used as a backend for a public web site
powered by lisp? That would bring more knowledge than you and I
trying to decipher and navigating their confusing designed site.
cheers
Slobodan
On 2007-11-15 07:27:05 -0500, "Alex Mizrahi"
<········@users.sourceforge.net> said:
> do you need to be a lawyer to understand plain english?
when that "plain english" is in a legal document such as a license,
sadly the answer is "yes, you do."
On Nov 15, 4:47 pm, Raffael Cavallaro <················@pas-d'espam-
s'il-vous-plait-mac.com> wrote:
> On 2007-11-15 07:27:05 -0500, "Alex Mizrahi"
> <········@users.sourceforge.net> said:
>
> > do you need to be a lawyer to understand plain english?
>
> when that "plain english" is in a legal document such as a license,
> sadly the answer is "yes, you do."
Excellent answer.
Slobodan
On Thu, 15 Nov 2007 14:27:05 +0200, "Alex Mizrahi"
<········@users.sourceforge.net> wrote:
> SB> Alex I'm not a lawyer and I doubt you are
>
>do you need to be a lawyer to understand plain english?
The text may contain English words, but the language is Legalese
(actually it's a confusing variant of Legalese called PlainSpeak,
which is quite easy to mistake for English).
However much it may look like English, you have to understand that it
is _not_ and that you will likely get yourself into trouble if you try
to read it as if it were.
George
--
for email reply remove "/" from address
J> Is there any way to manage all of this clases through a database? I'm
J> not saying just persist them, but also getting them out of memory so
J> I can work from the database when needed?
J> I don't want to reimplement all of the program, I'm looking for a
J> solution like MOP or something like that.
some persistence layers (like ELEPHANT, for example) should be able to do
this,
however quality differs: most were not tested with "millions of instances",
so it's tricky.
i believe there some 4-5 viable solutions out there, so you just can try to
evaluate all of them, which fits better..
for example, CommonSQL-like ORM should be very stable and scalable, but it
requires you to write your classes is quite special way.
other persistence libs are more liberal, but more likely to fail same time.
there's also commercially supported AllegroCache, but that costs $$$.
if you were using Java, there's nice almost magical solution called
Terracotta <grin>
On Tue, 13 Nov 2007 08:15:42 -0800, Javier wrote:
> I've got some classes, with millions of instances. So much that it is
> easy to get running out of memory.
> My simple question is:
>
> Is there any way to manage all of this clases through a database? I'm
> not saying just persist them, but also getting them out of memory so I
> can work from the database when needed? I don't want to reimplement all
> of the program, I'm looking for a solution like MOP or something like
> that.
>
> Any help?
I've been looking around for a persistence library lately, and as far as
I can tell, clsql is /probably/ good enough to do this. It has a fairly
basic (but apparently sane and straight-forward) OO-relational mapping
and the documentation seems to be quite good.
See http://clsql.b9.com/
I've also had a look at cl-perec, and it looks a lot fancier, but the
documentation is pretty much non-existent (or it may be just hard to find)
Anyway, I can't tell you anything about the memory efficiency of either -
maybe someone else can tell you more, or just try it out yourself.
Joost.
Joost Diepenmaat wrote:
> On Tue, 13 Nov 2007 08:15:42 -0800, Javier wrote:
>
>> I've got some classes, with millions of instances. So much that it is
>> easy to get running out of memory.
>> My simple question is:
>>
>> Is there any way to manage all of this clases through a database? I'm
>> not saying just persist them, but also getting them out of memory so I
>> can work from the database when needed? I don't want to reimplement all
>> of the program, I'm looking for a solution like MOP or something like
>> that.
>>
>> Any help?
>
> I've been looking around for a persistence library lately, and as far as
> I can tell, clsql is /probably/ good enough to do this. It has a fairly
> basic (but apparently sane and straight-forward) OO-relational mapping
> and the documentation seems to be quite good.
>
> See http://clsql.b9.com/
>
> I've also had a look at cl-perec, and it looks a lot fancier, but the
> documentation is pretty much non-existent (or it may be just hard to find)
>
> Anyway, I can't tell you anything about the memory efficiency of either -
> maybe someone else can tell you more, or just try it out yourself.
>
> Joost.
Is Rucksack dead? kt
On Tue, 13 Nov 2007 18:07:39 -0500, Ken wrote:
> Is Rucksack dead? kt
I don't know, is it? I've seen it on my brief tour of CL persistence
frameworks, but I haven't tried it (yet).
Joost.
On Nov 14, 12:23 am, Joost Diepenmaat <·····@zeekat.nl> wrote:
> On Tue, 13 Nov 2007 18:07:39 -0500, Ken wrote:
> > Is Rucksack dead? kt
>
> I don't know, is it? I've seen it on my brief tour of CL persistence
> frameworks, but I haven't tried it (yet).
Some months ago, the lead developer (Arthur Lemmens) opined that he
likely wouldn't have any time for it this year (2007).
Project mailing lists are generally the easiest way to get the pulse
of a project.
Incidentally, people might enjoy his outline of Lisp persistence,
though of course it's missing some newer libs like cl-perec and
Postmodern:
http://www.pentaside.org/paper/persistence-lemmens.txt
Tayssir
On Nov 13, 3:07 pm, Ken <·················@optonline.net> wrote:
> Joost Diepenmaat wrote:
> > On Tue, 13 Nov 2007 08:15:42 -0800, Javier wrote:
>
> >> I've got some classes, with millions of instances. So much that it is
> >> easy to get running out of memory.
> >> My simple question is:
>
> >> Is there any way to manage all of this clases through a database? I'm
> >> not saying just persist them, but also getting them out of memory so I
> >> can work from the database when needed? I don't want to reimplement all
> >> of the program, I'm looking for a solution like MOP or something like
> >> that.
>
> >> Any help?
>
> > I've been looking around for a persistence library lately, and as far as
> > I can tell, clsql is /probably/ good enough to do this. It has a fairly
> > basic (but apparently sane and straight-forward) OO-relational mapping
> > and the documentation seems to be quite good.
>
> > Seehttp://clsql.b9.com/
>
> > I've also had a look at cl-perec, and it looks a lot fancier, but the
> > documentation is pretty much non-existent (or it may be just hard to find)
>
> > Anyway, I can't tell you anything about the memory efficiency of either -
> > maybe someone else can tell you more, or just try it out yourself.
>
> > Joost.
>
> Is Rucksack dead? kt
Author doesn't have time to develop it, though concept is very
interesthing.
Slobodan
Slobodan Blazeski wrote:
> On Nov 13, 3:07 pm, Ken <·················@optonline.net> wrote:
>> Joost Diepenmaat wrote:
>>> On Tue, 13 Nov 2007 08:15:42 -0800, Javier wrote:
>>>> I've got some classes, with millions of instances. So much that it is
>>>> easy to get running out of memory.
>>>> My simple question is:
>>>> Is there any way to manage all of this clases through a database? I'm
>>>> not saying just persist them, but also getting them out of memory so I
>>>> can work from the database when needed? I don't want to reimplement all
>>>> of the program, I'm looking for a solution like MOP or something like
>>>> that.
>>>> Any help?
>>> I've been looking around for a persistence library lately, and as far as
>>> I can tell, clsql is /probably/ good enough to do this. It has a fairly
>>> basic (but apparently sane and straight-forward) OO-relational mapping
>>> and the documentation seems to be quite good.
>>> Seehttp://clsql.b9.com/
>>> I've also had a look at cl-perec, and it looks a lot fancier, but the
>>> documentation is pretty much non-existent (or it may be just hard to find)
>>> Anyway, I can't tell you anything about the memory efficiency of either -
>>> maybe someone else can tell you more, or just try it out yourself.
>>> Joost.
>> Is Rucksack dead? kt
>
> Author doesn't have time to develop it, though concept is very
> interesthing.
uh-oh. Has Lisp open software proven that Franz is worth the <gasp>
price? [Cue the bolsheviks]
Me, I'm hot for Allegrograph. Anyone know of an open source RDF
implementation with a C API?
kzo
Ken wrote:
> Slobodan Blazeski wrote:
>> On Nov 13, 3:07 pm, Ken <·················@optonline.net> wrote:
>>> Joost Diepenmaat wrote:
>>>> On Tue, 13 Nov 2007 08:15:42 -0800, Javier wrote:
>>>>> I've got some classes, with millions of instances. So much that it is
>>>>> easy to get running out of memory.
>>>>> My simple question is:
>>>>> Is there any way to manage all of this clases through a database? I'm
>>>>> not saying just persist them, but also getting them out of memory
>>>>> so I
>>>>> can work from the database when needed? I don't want to reimplement
>>>>> all
>>>>> of the program, I'm looking for a solution like MOP or something like
>>>>> that.
>>>>> Any help?
>>>> I've been looking around for a persistence library lately, and as
>>>> far as
>>>> I can tell, clsql is /probably/ good enough to do this. It has a fairly
>>>> basic (but apparently sane and straight-forward) OO-relational mapping
>>>> and the documentation seems to be quite good.
>>>> Seehttp://clsql.b9.com/
>>>> I've also had a look at cl-perec, and it looks a lot fancier, but the
>>>> documentation is pretty much non-existent (or it may be just hard to
>>>> find)
>>>> Anyway, I can't tell you anything about the memory efficiency of
>>>> either -
>>>> maybe someone else can tell you more, or just try it out yourself.
>>>> Joost.
>>> Is Rucksack dead? kt
>>
>> Author doesn't have time to develop it, though concept is very
>> interesthing.
>
> uh-oh. Has Lisp open software proven that Franz is worth the <gasp>
> price? [Cue the bolsheviks]
>
> Me, I'm hot for Allegrograph. Anyone know of an open source RDF
> implementation with a C API?
http://librdf.org/
Mind you I /have/ AG, but there is no point in doing cells-rdf if the
rest of you food-stamp licking, government cheese eating, thrift
shopping oooh-its-gotta-be-free pikers won't be able to benefit from my
unceasing thankless toil on your behalf.
kt
On Thu, 15 Nov 2007 01:21:35 -0500, Ken wrote:
> Mind you I /have/ AG, but there is no point in doing cells-rdf if the
> rest of you food-stamp licking, government cheese eating, thrift
> shopping oooh-its-gotta-be-free pikers...
But you can't help loving us, don't you? Your get-me-a-dictionary
descriptions give you away every time. Would you really trade our
adoration of Cells for a few extra bucks you would make if it were closed
source?
Josip Gracin wrote:
> On Thu, 15 Nov 2007 01:21:35 -0500, Ken wrote:
>
>>Mind you I /have/ AG, but there is no point in doing cells-rdf if the
>>rest of you food-stamp licking, government cheese eating, thrift
>>shopping oooh-its-gotta-be-free pikers...
>
>
> But you can't help loving us, don't you? Your get-me-a-dictionary
> descriptions give you away every time.
You have a dictionary that lists "government cheese"? Cool!
kenny
--
http://www.theoryyalgebra.com/
"In the morning, hear the Way;
in the evening, die content!"
-- Confucius
On Nov 15, 7:23 pm, Ken Tilton <···········@optonline.net> wrote:
> Josip Gracin wrote:
> > On Thu, 15 Nov 2007 01:21:35 -0500, Ken wrote:
>
> >>Mind you I /have/ AG, but there is no point in doing cells-rdf if the
> >>rest of you food-stamp licking, government cheese eating, thrift
> >>shopping oooh-its-gotta-be-free pikers...
>
> > But you can't help loving us, don't you? Your get-me-a-dictionary
> > descriptions give you away every time.
>
> You have a dictionary that lists "government cheese"? Cool!
Sounds like a dictionary from the 80's. Perhaps purchaced at a thrift
shop? I've had some great finds from thrift shops, books, kitchen
equipment, suits, shoes, you name it. Indeed, there or eBay would be
where I'd expect one to find such a thing.
Ken wrote:
> uh-oh. Has Lisp open software proven that Franz is worth the <gasp>
> price? [Cue the bolsheviks]
Devil's advocate: Or did overpriced Lisp software and hardware drive
a generation of developers to cheaper (free?) alternatives such as
C/C++/Perl/Python. Given that critical mass, open-source software
flourishes across the internet. I dare say that more high-quality
free software is developed in C/C++ than the total of Lisp software.
Lisp is still stuck where Unix was some 15 years ago: fragmented
camps competing for a small high-end market. HP-UX, SunOS, IRIX, ...
all trumped by MS Windows (and MacOS and Linux)!?! Franz should be
making money by developing cool Lisp apps for new customers, not by
charging others to use the language. They have a good product, but
the market is speaking -- new developers are learning (a) the
languages already used by other software and (b) freely available
"trendy" languages where they see the opportunity to take personal
ownership. Lisp has issues in both categories.
I guess that makes me a Bolshevik.
- Daniel
On Nov 14, 10:51 pm, D Herring <········@at.tentpost.dot.com> wrote:
> Ken wrote:
> > uh-oh. Has Lisp open software proven that Franz is worth the <gasp>
> > price? [Cue the bolsheviks]
>
> Devil's advocate: Or did overpriced Lisp software and hardware drive
> a generation of developers to cheaper (free?) alternatives such as
> C/C++/Perl/Python. Given that critical mass, open-source software
> flourishes across the internet. I dare say that more high-quality
> free software is developed in C/C++ than the total of Lisp software.
>
> Lisp is still stuck where Unix was some 15 years ago: fragmented
> camps competing for a small high-end market. HP-UX, SunOS, IRIX, ...
> all trumped by MS Windows (and MacOS and Linux)!?! Franz should be
> making money by developing cool Lisp apps for new customers, not by
> charging others to use the language.
> They have a good product, but
> the market is speaking -- new developers are learning (a) the
> languages already used by other software and (b) freely available
> "trendy" languages where they see the opportunity to take personal
> ownership. Lisp has issues in both categories.
>
> I guess that makes me a Bolshevik.
>
> - Daniel
No that makes you reluctant to solve problem, prefering whining
instead.
It reminds of the old story: " Teacher hates my child."
That's a problem. It might be that you child is a lazy pain in the
ass, or maybe it's teacher really hates it. So the question is what
you gonna do about it ? You could have a word with your kid if it's a
problem or contact the principal if it's a teacher fault. But whining
won't help at all. So are you able to convince Franz to give their
staff for free ? I doubt it . So you have a choice to use some free
lisp, would you mind to say why is sbcl inferior than python
implementation. Or you could switch to c+=/python/ ruby/ whatever.
And about Franz making apps for their customers and giving away ACL
for free you should read
http://www.joelonsoftware.com/articles/SetYourPriorities.html
Slobodan
D Herring wrote:
> Ken wrote:
>> uh-oh. Has Lisp open software proven that Franz is worth the <gasp>
>> price? [Cue the bolsheviks]
>
> Devil's advocate: Or did overpriced Lisp software and hardware drive a
> generation of developers to cheaper (free?) alternatives such as
> C/C++/Perl/Python.
Whose mistake would that be?
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
D Herring wrote:
> Ken wrote:
>
>> uh-oh. Has Lisp open software proven that Franz is worth the <gasp>
>> price? [Cue the bolsheviks]
>
>
> Devil's advocate: Or did overpriced Lisp software and hardware drive a
> generation of developers to cheaper (free?) alternatives such as
> C/C++/Perl/Python.
Do you realize how huge a non sequitor that is? Probably not, you were
just looking for an excuse to talk about that issue and decided, "Bingo!
Here comes my rant about the last century!".
Hint: check the tense of the verbs in my question and your response.
jeez, now I have to teach you clowns grammar!
<sigh>
kenny
--
http://www.theoryyalgebra.com/
"In the morning, hear the Way;
in the evening, die content!"
-- Confucius
On Nov 14, 9:17 pm, Ken <·················@optonline.net> wrote:
> Slobodan Blazeski wrote:
> > On Nov 13, 3:07 pm, Ken <·················@optonline.net> wrote:
> >> Joost Diepenmaat wrote:
> >>> On Tue, 13 Nov 2007 08:15:42 -0800, Javier wrote:
> >>>> I've got some classes, with millions of instances. So much that it is
> >>>> easy to get running out of memory.
> >>>> My simple question is:
> >>>> Is there any way to manage all of this clases through a database? I'm
> >>>> not saying just persist them, but also getting them out of memory so I
> >>>> can work from the database when needed? I don't want to reimplement all
> >>>> of the program, I'm looking for a solution like MOP or something like
> >>>> that.
> >>>> Any help?
> >>> I've been looking around for a persistence library lately, and as far as
> >>> I can tell, clsql is /probably/ good enough to do this. It has a fairly
> >>> basic (but apparently sane and straight-forward) OO-relational mapping
> >>> and the documentation seems to be quite good.
> >>> Seehttp://clsql.b9.com/
> >>> I've also had a look at cl-perec, and it looks a lot fancier, but the
> >>> documentation is pretty much non-existent (or it may be just hard to find)
> >>> Anyway, I can't tell you anything about the memory efficiency of either -
> >>> maybe someone else can tell you more, or just try it out yourself.
> >>> Joost.
> >> Is Rucksack dead? kt
>
> > Author doesn't have time to develop it, though concept is very
> > interesthing.
>
> uh-oh. Has Lisp open software proven that Franz is worth the <gasp>
> price? [Cue the bolsheviks]
>
> Me, I'm hot for Allegrograph.
Allegrograph is great. Allegrograph and AllegroCache were the reasons
I bought enterprise.
The IDE is nice too, but SLIME is Ok when you get used to it, so that
was not a crucial point.
> Anyone know of an open source RDF
> implementation with a C API?
>
> kzo
Den Tue, 13 Nov 2007 21:59:36 +0000 skrev Joost Diepenmaat:
> I've also had a look at cl-perec, and it looks a lot fancier, but the
> documentation is pretty much non-existent (or it may be just hard to
> find)
Nope, cl-perec guys don't believe in docs :) You're supposed to use your
SLIME, the testsuite and optionally a highlevel overview document (which
is admittedly somewhat lacking) to work your way through the package.
Cheers,
Maciej
P� Tue, 13 Nov 2007 22:59:36 +0100, skrev Joost Diepenmaat
<·····@zeekat.nl>:
>
> Anyway, I can't tell you anything about the memory efficiency of either -
> maybe someone else can tell you more, or just try it out yourself.
>
> Joost.
CLSQL depends on on the database and server chosen. Thus it is impossible
to give a general description.
I use CLSQL with a InnoDb database. A ISAM database would be more
efficient but wouldn't support ACID.
CLSQL with PGSQL (Postgre SQL) seems a favorite here. Perhaps you should
try that.
--------------
John Thingstad