I want to read some lines from a text file that may or may not contain
a lisp function, sexp and then evaluate it. For example, the line
could be
"There are (princ (+ 2 2)) cats in the attic."
I've thought of substitution, but I'm loathe to add special syntax to
the strings. Any fast ideas?
i.b.
"IBMackey" <········@hotmail.com> wrote in message
···················@winny.home...
> I want to read some lines from a text file that may or may not contain
> a lisp function, sexp and then evaluate it. For example, the line
> could be
>
> "There are (princ (+ 2 2)) cats in the attic."
>
> I've thought of substitution, but I'm loathe to add special syntax to
> the strings. Any fast ideas?
>
> i.b.
I'm working on something similar, where the sexp's need to be evaluated and
inserted into the string, for that, I have the following function:
(defun generate (&key from to)
(do ((next (read-char from nil :eof t)
(read-char from nil :eof t)))
((eq next :eof))
(case next
(··@
(write (eval (read from t nil t))
:stream to
:pretty nil
:escape nil))
(otherwise (write-char next to)))))
So basically, all the normal content gets echoed to the "to" stream, and a
··@ character means that the next thing in the input stream is a lisp
expression to be read, evaluated, and written to the "to" stream. At some
point, I'd like to expand on the read-eval-print part of it, to handle read
or eval errors better. In my system, your example would be something like:
"There are @(+ 2 2) cats in the attic." => "There are 4 cats in the attic."
Incidently, this is one of my very first lisp development efforts, so if
anyone feels like commenting on my code, I'm open to suggestions or
corrections.
"Glenn Burnside" <·········@austin.rr.com> writes:
> (defun generate (&key from to)
> (do ((next (read-char from nil :eof t)
> (read-char from nil :eof t)))
> ((eq next :eof))
> (case next
> (··@
> (write (eval (read from t nil t))
> :stream to
> :pretty nil
> :escape nil))
> (otherwise (write-char next to)))))
>
> "There are @(+ 2 2) cats in the attic." => "There are 4 cats in the attic."
>
> Incidently, this is one of my very first lisp development
> efforts, so if anyone feels like commenting on my code, I'm
> open to suggestions or corrections.
Two points:
(i) Why do you set the recursive-p argument to READ and READ-CHAR
to T?
(ii) Are you sure you want to call WRITE and not PRINC?
And you might like this idiom:
(loop for char = (read-char from nil nil) while char do
...)
Regards,
--
Nils Goesche
Ask not for whom the <CONTROL-G> tolls.
PGP key ID #xD26EF2A0
* Nils Goesche <···@cartan.de>
| (ii) Are you sure you want to call WRITE and not PRINC?
I always use `write� these days, instead of the `print� family. It makes
it easier to remember exactly which printer variables are used and
affected. More often than not, being unaware of printer variables causes
bugs that are extremely hard to find, and those who blithely assume that
they have "reasonable" values have no room for reasonable disagreement
over what is reasonable. (Or I use `format�.)
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
From: Nils Goesche
Subject: Re: Extract function from strings
Date:
Message-ID: <lkpttpz654.fsf@cartan.de>
Erik Naggum <····@naggum.no> writes:
> * Nils Goesche <···@cartan.de>
> | (ii) Are you sure you want to call WRITE and not PRINC?
>
> I always use `write� these days, instead of the `print� family.
> It makes it easier to remember exactly which printer variables are
> used and affected. More often than not, being unaware of printer
> variables causes bugs that are extremely hard to find, and those
> who blithely assume that they have "reasonable" values have no
> room for reasonable disagreement over what is reasonable. (Or I
> use `format�.)
When in doubt (which admittedly is quite often) I check the HyperSpec
again. Actually, the code snippet looks like it is intended to be
used in a read macro (which would also explain the RECURSIVE-P
argument). In cases like that (a PRINT-OBJECT method being another
example), where it is very important to control the printer variables
(or not to change them) I use WRITE, too, for the same reason.
Regards,
--
Nils Goesche
"Don't ask for whom the <CTRL-G> tolls."
PGP key ID 0x0655CFA0
"Nils Goesche" <······@cartan.de> wrote in message
···················@cartan.de...
> Erik Naggum <····@naggum.no> writes:
>
> > * Nils Goesche <···@cartan.de>
> > | (ii) Are you sure you want to call WRITE and not PRINC?
> >
> > I always use `write� these days, instead of the `print� family.
> > It makes it easier to remember exactly which printer variables are
> > used and affected. More often than not, being unaware of printer
> > variables causes bugs that are extremely hard to find, and those
> > who blithely assume that they have "reasonable" values have no
> > room for reasonable disagreement over what is reasonable. (Or I
> > use `format�.)
>
> When in doubt (which admittedly is quite often) I check the HyperSpec
> again. Actually, the code snippet looks like it is intended to be
> used in a read macro (which would also explain the RECURSIVE-P
> argument). In cases like that (a PRINT-OBJECT method being another
> example), where it is very important to control the printer variables
> (or not to change them) I use WRITE, too, for the same reason.
>
> Regards,
> --
> Nils Goesche
> "Don't ask for whom the <CTRL-G> tolls."
>
> PGP key ID 0x0655CFA0
I'm actually not planning on using this in a read macro. What about the
code makes you think it's supposed to be used in that way? I used the
recursive-p argument because I found that in CLISP, not setting it to t
caused the call to READ to consume all of the whitespace following the sexp,
which I didn't want. Also, CLISP's PRINC places newlines in front of
strings when *print-pretty* is true, so it seemed simpler to use WRITE with
the appropriate arguments than to bind variables just for one call to PRINC.
Glenn B.
From: Nils Goesche
Subject: Re: Extract function from strings
Date:
Message-ID: <lk4rb1yqv8.fsf@cartan.de>
"Glenn Burnside" <·········@austin.rr.com> writes:
> "Nils Goesche" <······@cartan.de> wrote in message
> ···················@cartan.de...
> > Actually, the code snippet looks like it is intended to be
> > used in a read macro (which would also explain the RECURSIVE-P
> > argument).
> I'm actually not planning on using this in a read macro. What about
> the code makes you think it's supposed to be used in that way?
Hm, mainly because of the recursive-p argument, I guess, because it
/could/ be used in a read macro because of that :-)
> I used the recursive-p argument because I found that in CLISP, not
> setting it to t caused the call to READ to consume all of the
> whitespace following the sexp, which I didn't want.
Ah -- I think what you want is READ-PRESERVING-WHITESPACE, then.
> Also, CLISP's PRINC places newlines in front of strings when
> *print-pretty* is true, so it seemed simpler to use WRITE with the
> appropriate arguments than to bind variables just for one call to
> PRINC.
Okok, forget about the PRINC thing :)
Regards,
--
Nils Goesche
"Don't ask for whom the <CTRL-G> tolls."
PGP key ID 0x0655CFA0
From: Tim Bradshaw
Subject: Re: Extract function from strings
Date:
Message-ID: <ey3znstverf.fsf@cley.com>
* Glenn Burnside wrote:
> Incidently, this is one of my very first lisp development efforts,
> so if anyone feels like commenting on my code, I'm open to
> suggestions or corrections.
*normally* you would want to be a bit paranoid about *READ-EVAL*, but
it's not clear this matters in this case since you're about to call
EVAL anyway. In any case you probably want to wrap a
WITH-STANDARD-IO-SYNTAX + any bindings of reader variables that you
want (possibly *PACKAGE*) around things, so that you have good control
over things.
If you're dealing with lots of input, you might want to inch along the
input until you find the magic char, and then write the previous
subseq in one fell swoop with WRITE-SEQUENCE (might mean fewer IO
calls and fewer system calls, which can be slow).
Carrying this further, you might find you can win by reading and
writing to a string stream and then at some convenient point snarfing
the whole output string and smashing that out with WRITE-SEQUENCE.
Both this and the previous are things you only want to care about if
it's too slow...
As you said, you probably want better error protection (some error
protection even!).
--tim
"Tim Bradshaw" <···@cley.com> wrote in message
····················@cley.com...
> * Glenn Burnside wrote:
>
> > Incidently, this is one of my very first lisp development efforts,
> > so if anyone feels like commenting on my code, I'm open to
> > suggestions or corrections.
>
> *normally* you would want to be a bit paranoid about *READ-EVAL*, but
> it's not clear this matters in this case since you're about to call
> EVAL anyway. In any case you probably want to wrap a
> WITH-STANDARD-IO-SYNTAX + any bindings of reader variables that you
> want (possibly *PACKAGE*) around things, so that you have good control
> over things.
>
> If you're dealing with lots of input, you might want to inch along the
> input until you find the magic char, and then write the previous
> subseq in one fell swoop with WRITE-SEQUENCE (might mean fewer IO
> calls and fewer system calls, which can be slow).
>
> Carrying this further, you might find you can win by reading and
> writing to a string stream and then at some convenient point snarfing
> the whole output string and smashing that out with WRITE-SEQUENCE.
> Both this and the previous are things you only want to care about if
> it's too slow...
>
> As you said, you probably want better error protection (some error
> protection even!).
>
> --tim
>
>
>
>
>
My input isn't going to be coming from strings, but from streams (mostly
file streams). So, I don't think I can use write-sequence as you suggested.
I had started with a similar idea on how I would do this - I create a string
stream, and write to it until I find the magic character, then flush that
string to the output stream. This made things more complicated, and I was
still doing write-chars to a stream, it was just a different stream. So, I
pared it down to just write as I go. I don't know a ton about file streams
yet, but I assumed (dangerous, right?) that they would/could be implemented
with some sort of buffering, so that every call to write-char probably
doesn't generate a system call.
My little function is going to exist in a larger system for generating
output with embedded code, so I figure that the calling code can apply
package, print variable, and readtable bindings as appropriate for whatever
code is embedded in their source files, which is why I didn't put
WITH-STANDARD-IO-SYNTAX or other reader variable or printer variable
bindings inside the function, and why I use WRITE instead of PRINC.
From: Tim Bradshaw
Subject: Re: Extract function from strings
Date:
Message-ID: <ey365viaebo.fsf@cley.com>
* ibum1952 wrote:
> "There are (princ (+ 2 2)) cats in the attic."
If you are sure that the magic character that begins a sexp will never
occur otherwise in a string (which you kind of have to be), then you
can safely search for it, and then use READ to acquire the form. Thus
you can essentially tokenize the string into a bunch of string
segments together with forms. On the assumption that the forms are
all compound forms, and that they all do output to *STANDARD-OUTPUT*
(which your example would imply), then you can do something like this:
(let ((chunks (tokenize-line line)))
;; CHUNKS is a list of STRINGs or forms for evaluation.
(with-output-to-string (*standard-output*)
(loop for c in chunks
do
(etypecase c
(string (princ c *standard-output*))
(cons (eval c))))))
Obviously you want to make this much more robust, and you need to
write TOKENIZE-STRING, which is fiddly but not impossible (PEEK-CHAR
and string streams are your friend).
--tim
[ Tim Bradshaw ]
> * ibum1952 wrote:
>
> > "There are (princ (+ 2 2)) cats in the attic."
>
> [snip]
>
> (let ((chunks (tokenize-line line)))
> ;; CHUNKS is a list of STRINGs or forms for evaluation.
> (with-output-to-string (*standard-output*)
> (loop for c in chunks
> do
> (etypecase c
> (string (princ c *standard-output*))
> (cons (eval c))))))
>
> Obviously you want to make this much more robust, and you need to
> write TOKENIZE-STRING, which is fiddly but not impossible (PEEK-CHAR
> and string streams are your friend).
Perhaps you can use something like:
(defun tokenize-line (line)
(let ((result ()))
(with-input-from-string (stream line)
(with-open-stream (output (make-string-output-stream))
(loop for char = (read-char stream nil nil)
do (case char
(#\( (let ((position (file-position stream)))
(unread-char char stream)
(handler-case
(let ((r (read-preserving-whitespace stream)))
(push (get-output-stream-string output) result)
(push r result))
(error () (file-position stream position)
(write-char char output)))))
((nil) (push (get-output-stream-string output) result))
(t (write-char char output)))
while char)))
(nreverse result)))
* (tokenize-line "There are (princ (+ 2 2)) cats in the attic.")
=> ("There are " (PRINC (+ 2 2)) " cats in the attic.")
--
�rnulf