I'm interested in Lisp implementations of databases. I would also
like to know some basic things...
If you're designing a database, and you have too much data to possibly
be able to have it all loaded up into memory at once, does database
design imply having efficient ways of finding what you want in files,
and an efficient way of organizing the files, including how to divide
the data w/in the files, how many files to use, how large the files
should be, etc?
what's the deal? what changes when your implementation is Lisp-based?
Do the built-in hashtables come in handy for most people when doing
this? Any public (source available) CL db's out there? where?
dave
David Bakhash <·····@bu.edu> writes:
> I'm interested in Lisp implementations of databases. I would also
> like to know some basic things...
>
> If you're designing a database, and you have too much data to possibly
> be able to have it all loaded up into memory at once, does database
> design imply having efficient ways of finding what you want in files,
> and an efficient way of organizing the files, including how to divide
> the data w/in the files, how many files to use, how large the files
> should be, etc?
>
> what's the deal? what changes when your implementation is Lisp-based?
>
> Do the built-in hashtables come in handy for most people when doing
> this? Any public (source available) CL db's out there? where?
>
> dave
I would suggest having a look at PLOB!
http://www.lisp.de/software/plob/Welcome.html
It implements a persistent object store for lispworks and allegro, and also
some basic structures (like btrees, and I believe a transaction log) that
would be good for maintaining a database. I haven't used PLOB!, but I have
played with Wood on MCL, another persistent object store. I find this a
very intuitive way of handling data in lisp, since I can treat the
persistent store as an extension of memory. (Well, it does get a little
trickier than that at times, but it's still a lot more straightforward than
any databse would be.)
Wood is available from ftp://ftp.digitool.com/pub/mcl/contrib/
Other than this, there isn't much I have to say about CL databases.
Sunil
David Bakhash <·····@bu.edu> writes:
>I'm interested in Lisp implementations of databases. I would also
>like to know some basic things...
>If you're designing a database, and you have too much data to possibly
>be able to have it all loaded up into memory at once, does database
>design imply having efficient ways of finding what you want in files,
>and an efficient way of organizing the files, including how to divide
>the data w/in the files, how many files to use, how large the files
>should be, etc?
>what's the deal? what changes when your implementation is Lisp-based?
>Do the built-in hashtables come in handy for most people when doing
>this? Any public (source available) CL db's out there? where?
This is an interesting question, but I'm not quite sure how to
interpret it.
If you are talking about writing a DBMSesque system from scratch using
Lisp, versus interfacing with an off-the-shelf DBMS, then I honestly
don't think that the choice of Lisp over anything else would effect
the actual organization of the data very much. I say this mostly based
on the issues that the data organization inside of databases has pretty
much stabilized on some derivative of the B+Tree method. Certainly
there are other ways, and I can't fathom that innovation has stopped
cold or even taken totally divergent paths.
However, I think it is safe to say that outside of research areas,
B+Tree styles of organziation are quite dominant. Even PLOB (mentioned
elsewhere in the other reply) uses this to manage its persistent
store.
Where Lisp would make the difference is how you access the data, and
what your query and modeling "language" looks like. Using PLOB as an
example, you have CLOS effectively being your query and modeling
language. PLOB is a persistent object store.
If you wish to hunt down a recent copy of Aubrey Jaffers SLIB, a
library for Scheme, he has written a rdbms system in Scheme (a Lisp
dialect). The only issue here is that in this case, it is NOT
persistent (the database is stored purely in memory, and lost when the
image exits). However, its implemenation may be interesting as a
model.
--
Will Hartung - Rancho Santa Margarita. It's a dry heat. ······@netcom.com
1990 VFR750 - VFR=Very Red "Ho, HaHa, Dodge, Parry, Spin, HA! THRUST!"
1993 Explorer - Cage? Hell, it's a prison. -D. Duck
David Bakhash <·····@bu.edu> writes:
> I'm interested in Lisp implementations of databases. I would also
> like to know some basic things...
One thing really exites me about Lisp for database usage is to use
Lisp's flexible syntax to write down relational queries in your host
language, not as strings as in most C and Java libraries or even using
a intrisuve preprocessor like in Embedded SQL (shudder). Some
code-walking macros could even do a lot of compile-time checking. I
really don't care for this OO and persistence stuff, all I want is
rows in tables and a relational language, but preferrably integrated
into the main language and with some kind of compile-time nitpicker.
I wonder if there is more documentation or research papers on
these issue. The only thing I have is a small commercialy ad for Statice,
hardly enough. Maybe some Statice documentation is online or papers
about whatever Static was based on?
Martin
From: Will Hartung
Subject: Re: Lisp syntax for relational queries (Re: Databases in Lisp)
Date:
Message-ID: <vfr750EvDp3C.M74@netcom.com>
Martin Cracauer <········@waldstrasse.cons.org> writes:
>David Bakhash <·····@bu.edu> writes:
>> I'm interested in Lisp implementations of databases. I would also
>> like to know some basic things...
>One thing really exites me about Lisp for database usage is to use
>Lisp's flexible syntax to write down relational queries in your host
>language, not as strings as in most C and Java libraries or even using
>a intrisuve preprocessor like in Embedded SQL (shudder). Some
>code-walking macros could even do a lot of compile-time checking. I
>really don't care for this OO and persistence stuff, all I want is
>rows in tables and a relational language, but preferrably integrated
>into the main language and with some kind of compile-time nitpicker.
You may want to look over at Harlequins site, and look at their Common
SQL extensions. They're OTAY.
In their most basic form:
(select [this] [that] [other] :from [here] [there]
:where [and [= [here join_fld] [there join_fld]]
[= [this] (get-me-this)]
[= [that] (get-me-that)]]
The good news is this is converted into:
SELECT THIS,THAT,OTHER
FROM HERE,THERE
WHERE HERE.JOIN_FLD=THERE.JOIN_FLD
AND THIS="<result of (get-me-this)>"
AND THAT="<result of (get-me-that)>"
and then fed into RDBMS, returning a list of lists as a result set:
((this1 that1 other1) (this2 that2 other2))
Unfortunately, you don't have any control over the rendering of the
results from (get-me-this), it effectively just uses PRINT. (actually,
I don't know exaqctly what they use. Probably (format nil "~A" thing)).
Which is fine, 99% of the time, but I ran into a glitch going
up against an MS-ACCESS databases that wants ITS dates like this:
Select this from that where that_date=#01/01/1998#
Every OTHER database on the planet wants "01/01/1998", but Oh No, not
Access. So, I have to Intern all of my dates as symbols to use the
extended Harlequin SQL syntax. I was kind of sorta hoping that LWW was
using a generic function for its rendering of the SQL string, but
apparently they don't. Then I could just specialize the GF on
MY-ACCESS-DATE. (well, actually, they DO -- print-object, but then
they try and (read ...) it back, and #01/01/1998# can't be read... But
thanx for playing!).
And for Zeno's sake, I think I'll blame MicroSoft for this, and not
Harlequin. Just 'cuz. Harelquin went for the GCD of date formats, and
MS decided to take its ball and play elsewhere.
--
Will Hartung - Rancho Santa Margarita. It's a dry heat. ······@netcom.com
1990 VFR750 - VFR=Very Red "Ho, HaHa, Dodge, Parry, Spin, HA! THRUST!"
1993 Explorer - Cage? Hell, it's a prison. -D. Duck
On 29 Jun 1998 21:42:02 +0200, Martin Cracauer <········@waldstrasse.cons.org> wrote:
> One thing really exites me about Lisp for database usage is to use
> Lisp's flexible syntax to write down relational queries in your host
> language, not as strings as in most C and Java libraries or even using
> a intrisuve preprocessor like in Embedded SQL (shudder).
If you're simply looking for a Lisp layer on top of SQL, Franz's Allegro
Common Lisp has such an ODBC API in Lisp as a package you can add.
If you're looking more generally for "what's a good way to express queries
in a logical language with a Lisp syntax?", you might want to look into
the KIF (Knowledge Interchange Format) and KQML (Knowledge Query and
Manipulation Language). They provide Lisp-syntax definitions of first-order
logic with extensions, which is a much better query language than SQL.
More details at
http://logic.stanford.edu/
among other places.
-- Don
--
Don Geddis ······@tesserae.com
Tesserae Information Systems, Inc. http://tesserae.com
275 Shoreline Drive, Suite 505 (650) 508-7893
Redwood Shores, CA 94065 (650) 508-7891 [fax]
······@meta.Tesserae.COM (Don Geddis) writes:
> If you're simply looking for a Lisp layer on top of SQL, Franz's Allegro
> Common Lisp has such an ODBC API in Lisp as a package you can add.
>
> If you're looking more generally for "what's a good way to express queries
> in a logical language with a Lisp syntax?", you might want to look into
> the KIF (Knowledge Interchange Format) and KQML (Knowledge Query and
> Manipulation Language). They provide Lisp-syntax definitions of first-order
> logic with extensions, which is a much better query language than SQL.
> More details at
> http://logic.stanford.edu/
> among other places.
Thanks for the pointer. It is interesting. But the question should
really be: is there a "standard" syntax and semantics of SQL in CL?
(Where the definition of standard is: I run program X on ACL and on
LWW and they get interpreted and compiled without #+:ACL #+:LWW and
produce equalp results?
--
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - (0)6 - 68 80 79 23, fax. +39 - (0)6 - 68 80 79 26
http://www.parades.rm.cnr.it
On 02 Jul 1998 10:03:55 +0200, Marco Antoniotti <·······@galvani.parades.rm.cnr.it> wrote:
> But the question should
> really be: is there a "standard" syntax and semantics of SQL in CL?
That's sort of a weird question. SQL already is a standard (or, rather, a
few standards!), with a well defined syntax and semantics. It appears that
you're looking for a simple syntax translation to a Lisp-like one filled with
parentheses.
This has nothing to do with semantics; SQL already has settled that. In any
case, your new syntax won't do any good unless it gets translated to standard
SQL syntax and sent to a database somewhere.
This sounds like a pretty simple problem. I bet you could write a syntax
wrapper in just a few pages of Lisp code. It would turn calls to these Lisp
wrapper functions (which might take optional and keyword arguments) into
a big string of standard ODBC/SQL.
In fact, if you look at Franz's ODBC wrapper, they've probably written those
few pages of Common Lisp for you already...
-- Don
--
Don Geddis ······@tesserae.com
Tesserae Information Systems, Inc. http://tesserae.com
275 Shoreline Drive, Suite 505 (650) 508-7893
Redwood Shores, CA 94065 (650) 508-7891 [fax]