Frequently I use large files of S-expressions as a database for moving
information from one lisp program to another. I find that using
(read) to read these expressions back into lisp is extremely slow.
If, however, I set up the file so that it can be (load)-ed instead, it
comes in much much faster. Have other people observed this? Has
anyone got an explanation?
--
Thomas G. Dietterich
Department of Computer Science
Dearborn Hall, 303
Oregon State University
···@tesla.cs.orst.edu (Tom Dietterich) writes:
>Frequently I use large files of S-expressions as a database for moving
>information from one lisp program to another. I find that using
>(read) to read these expressions back into lisp is extremely slow.
>If, however, I set up the file so that it can be (load)-ed instead, it
>comes in much much faster. Have other people observed this? Has
>anyone got an explanation?
If you use (read) it must parse the file character by character,
tokenize it, (albeit usually by a relatively efficient FSM). If you
use (load ...) then you are presumably loading already parsed data
structures. On many implementations an array of numbers, for example,
will usually LOAD much faster than it can be READ.
Perhaps I'm misunderstanding you. Perhaps you are comparing two ascii
files whose only difference is that the latter one has forms like
(SETQ A '( ... s-expressions ...))
instead of plain s-expressions?
If so, then both have to be (READ), so it would be hard for someone to
explain why LOAD is so much faster than READ without knowing more
details about the particular files you are talking about.
>--
>Thomas G. Dietterich
>Department of Computer Science
>Dearborn Hall, 303
>Oregon State University
In article <·····················@Neon.Stanford.EDU>
········@Neon.Stanford.EDU (Michael Greenwald) writes:
>Perhaps I'm misunderstanding you. Perhaps you are comparing two ascii
>files whose only difference is that the latter one has forms like
> (SETQ A '( ... s-expressions ...))
>instead of plain s-expressions?
>
>If so, then both have to be (READ), so it would be hard for someone to
>explain why LOAD is so much faster than READ without knowing more
>details about the particular files you are talking about.
>
Yes, you are misunderstanding me. I am comparing calling READ on a
file containing a big s-expression versus calling LOAD on a nearly
identical file of the form
(setq foo s-expression)
It doesn't matter much what is in the s-expression, in my experience.
>>Thomas G. Dietterich
>>Department of Computer Science
>>Dearborn Hall, 303
>>Oregon State University
--
Thomas G. Dietterich
Department of Computer Science
Dearborn Hall, 303
Oregon State University
In article <·····@orstcs.CS.ORST.EDU> ···@tesla.cs.orst.edu (Tom Dietterich) writes:
>Frequently I use large files of S-expressions as a database for moving
>information from one lisp program to another. I find that using
>(read) to read these expressions back into lisp is extremely slow.
>If, however, I set up the file so that it can be (load)-ed instead, it
>comes in much much faster. Have other people observed this? Has
>anyone got an explanation?
>
Gee, you got me. Perhaps you might check up on the following theory?
1. Whoever wrote your Lisp system hard-coded the parser into his
"load" command without thinking about the fact that the parser
is needed separately.
2. Later, he realizes that Lisp also requires a "read" interface
to the parser.
3. Groans and slaps his head.
4. Rather than rewrite the parser, codes up "read" as follows:
a. Copy characters from input stream to a /tmp file.
b. Do quick scan to find s-expression boundaries and
edit /tmp file to change SEXPR into (setq tmpvar 'SEXPR)
c. Calls "load" to read in /tmp file.
d. Returns value of tmpvar.
o o
| (a big smiley for above paragraph)
\ | /
\_____/
P.S.: Maybe your "load" command is secretly caching a preprocessed
version of your datafiles?