From: Jeronimo Pellegrini
Subject: Library expects string, and I need to read a huge file
Date: Thu, 14 Jun 2007 09:15:10 +0000
Message-ID: <f4r0uu$urp$1@aioe.org>

Hello.

I need to parse some very large files, so I decided to use CL-Yacc[0]
and Lexer[1].

However, Lexer expects me to give it a string (and I'd have to create a
lexer for each line, if I understand it correctly), so I'd need to split
the file (it does not fit in memory) and pass individual lines to the
lexer...
What if I have automatically generated files that do not necessarily
have newlines or other characters that I could use to split the input?

Is there a standard way to make buffered access to a stream, and then
make look like a string?
(Or, is there a lexer generator that actually operates on streams?)

Thank you!
J.

[0] http://www.pps.jussieu.fr/~jch/software/cl-yacc
[1] http://www.geocities.com/mparker762/clawk.html

From: David Lichteblau
Subject: Re: Library expects string, and I need to read a huge file
Date: Thu, 14 Jun 2007 18:11:55 +0000
Message-ID: <slrnf7317e.vu1.usenet-2006@babayaga.math.fu-berlin.de>

On 2007-06-14, Jeronimo Pellegrini <···@aleph0.info> wrote:
> (Or, is there a lexer generator that actually operates on streams?)

You could try clex.lisp from closure, it parses from a stream.

http://common-lisp.net/cgi-bin/viewcvs.cgi/*checkout*/closure/src/util/clex.lisp?rev=1.5&root=closure

There is also a hacked version of this file in cxml-rng, without the
dependency on glisp.