From: Timothy Finin
Subject: fast print/read for transmitting Lisp structures?
Date:
Message-ID: <1u5k9aINN2sf@algol.cs.umbc.edu>
I would like to efficiently convert an a arbitrary Lisp structure into a
byte stream, transmit it to another Lisp and then recreate the Lisp
structure. I am currently using PRINT and READ, but believe that I can
achieve greater efficiency by using special purpose print-like and read-like
routines. These routines might, for example, avoid having to transmit an
atom more than once but instead use a reference number for subsequent
occurrences of an atom in the structure being sent. This might save a lot of
symbol table lookups on the read end.
My guess is that this has been done before, more than once, and probably 20
years a go at least. Does anyone have any suggestions on where I might find
such code?
Tim
this is actually an interesting problem. i worked on it a few years ago.
note that the existance of cycles and structure sharing make the problem
rather non-trivial. although i am not sure how sophisticated the current
read/print code is i would be surprised if they will do the job. if i had
any code it would be in maclisp for the old sail machine. i could given you
pointers in how to go about it, tho.
From: Timothy Finin
Subject: Re: fast print/read for transmitting Lisp structures?
Date:
Message-ID: <1u6ecaINN6vs@algol.cs.umbc.edu>
···@CS.Stanford.EDU (Ian A. Mason) writes:
>this is actually an interesting problem. i worked on it a few years ago.
>note that the existance of cycles and structure sharing make the problem
>rather non-trivial. although i am not sure how sophisticated the current
>read/print code is i would be surprised if they will do the job. if i had
>any code it would be in maclisp for the old sail machine. i could given you
>pointers in how to go about it, tho.
Well, there are some packages that tackle the problem of printing and
reading arbitrary Lisp structures that might contain cycles and such.
However, this is not the problem I was worried about. I'm afraid that my
earlier posting was poorly worded.
I can probably assume simple lists -- lists of atoms, integers, reals and
simple lists. I can stipulate that there are no cycles and I don't care
if any structure sharing is preserved. I want to have an optimized
print/read for such simple lists that will minimize total computation time
and, secondarily, bytes transmitted. The application of this is a
interprocess communication package and we want to minimize the overhead of
using it.
Tim
In article <············@algol.cs.umbc.edu> ·····@cs.umbc.edu (Timothy Finin) writes:
>I would like to efficiently convert an a arbitrary Lisp structure into a
>byte stream, transmit it to another Lisp and then recreate the Lisp
>structure. I am currently using PRINT and READ, but believe that I can
>achieve greater efficiency by using special purpose print-like and read-like
>routines. These routines might, for example, avoid having to transmit an
>atom more than once but instead use a reference number for subsequent
>occurrences of an atom in the structure being sent. This might save a lot of
>symbol table lookups on the read end.
First of all, the Common Lisp WRITE function will do this if you specify
:CIRCLE T, using the #<number>= and #<number># notation. This can be slow
for large structures, though, because it has to examine the entire
structure before it starts (it handles the cycles that have been mentioned
in some other posts in this thread).
>My guess is that this has been done before, more than once, and probably 20
>years a go at least. Does anyone have any suggestions on where I might find
>such code?
Another solution is Kerry Koitzsch's save-object.lisp, which is available
from the Lisp Repository at CMU (see the FAQ for information on how to
access the repository).
In article <············@algol.cs.umbc.edu> ·····@cs.umbc.edu (Timothy Finin) writes:
I would like to efficiently convert an a arbitrary Lisp structure into a
byte stream, transmit it to another Lisp and then recreate the Lisp
structure. I am currently using PRINT and READ, but believe that I can
achieve greater efficiency by using special purpose print-like and read-like
routines. These routines might, for example, avoid having to transmit an
atom more than once but instead use a reference number for subsequent
occurrences of an atom in the structure being sent. This might save a lot of
symbol table lookups on the read end.
My guess is that this has been done before, more than once, and probably 20
years a go at least. Does anyone have any suggestions on where I might find
such code?
I think every Common Lisp implementation needs to do this in order to
compile a file containing almost-arbitrary data structures so that, on
later loading, the structures are recreated, sharing and all. I've
often thought that a capability like DUMP-FORMS-TO-FILE should be
standardized. Indeed, I'd like to see the additional option of
dumping forms directly into a byte-vector, for easier transmission
from one running Lisp to another. Ideally, this would be both more
flexible and faster than WRITE and READ. An option to turn off
circularity-checking could prevent that feature from becoming a
performance bottleneck.
Lawrence G. Mayka
AT&T Bell Laboratories
···@iexist.att.com
Standard disclaimer.
The newer versions of AKCL contain such a feature. Here's the top of
.../c/fasdump.c:
/* Copyright William F. Schelter All Rights Reserved.
Utility for writing out lisp objects and reading them in:
Basically it attempts to write out only those things which could
be written out using princ and reread. It just uses less space
and is faster.
Primitives for dealing with a `fasd stream'.
Such a stream is really an array containing some state and a lisp file stream
.
Note that having *print-circle* == nil wil make this faster. gensyms will
still be dumped correctly in that case.
open_fasd
write_fasd_top
read_fasd_top
close_fasd
*/
Greetings,
Jens.
--
Internet: ·····@hpbeo82.bbn.hp.com HPDESK : ··············@HP1200
MausNet: Jens Kilian @ BB ·····················@hpbbi4
-------------------------------------------------------------------------------
As the air to a bird, or the sea to a fish, so is contempt to the contemptible.