I'm open to suggestions on how to process text files in Common LISP for lexical
analysis. Right now, I'm doing it character-by-character with read-char and
then using concat to combine the characters I read into tokens; is there a
better way? How much more efficient would it be to use read-line and separate
tokens from each other?
Thanks.
Bruce
In article <·················@blkcat.UUCP> ···········@f615.n109.z1.fidonet.org (Bruce Feist) writes:
>I'm open to suggestions on how to process text files in Common LISP for
>lexical analysis. Right now, I'm doing it character-by-character with
>read-char and then using concat to combine the characters I read into tokens;
>is there a better way?
If you are doing:
(setq buffer (concatentate buffer (string char)))
you would be doing better by preallocating a string buffer and growing it as
needed, then taking SUBSEQ at the end of accumulation.
>How much more efficient would it be to use read-line and separate tokens from
>each other?
It depends on the implementation's relative costs of stream operation overhead
and string allocation. I would guess that in implementations with
generational GC, the READ-LINE strategy could well be faster. Most
implementation have internal operations to read a large number of characters
into a preallocated buffer, which could get the best of both worlds at a
portability cost.
Rob