I'm trying to process each line in a file, and am wondering if there is
a simpler or more idiomatic way of doing it than the following:
;;; Simplified version of my real process-line function
(defun process-line (line)
(format t "~A~%" line))
;;; Process stream, calling per-line-fn on each line of the stream
(defun process (per-line-fn stream)
(let ((line (read-line stream nil)))
(if (not (null line))
(progn (funcall per-line-fn line)
(process per-line-fn stream)))))
And I kick off the process like so:
CL-USER> (with-open-file (stream "../../words/test.txt")
(process #'process-line stream))
asdf
zxcv
hello
world
NIL
Is there a simpler way of doing what I'm trying to do here? I've seen
some loop-based solutions, but am staying away from loop for the
moment. Are there any predefined functions or simpler idioms that I'm
overlooking (something like '(for-line-in-stream (stream
per-line-fn))")?
Thanks for any pointers.
j.k.
<·········@gmail.com> wrote:
+---------------
| I'm trying to process each line in a file, and am wondering if there is
| a simpler or more idiomatic way of doing it than the following:
|
| ;;; Simplified version of my real process-line function
| (defun process-line (line)
| (format t "~A~%" line))
|
| ;;; Process stream, calling per-line-fn on each line of the stream
| (defun process (per-line-fn stream)
| (let ((line (read-line stream nil)))
| (if (not (null line))
| (progn (funcall per-line-fn line)
| (process per-line-fn stream)))))
+---------------
This can blow up if your file is large and your CL implementation
doesn't happen to perform tail-call optimization on the tail call
to PROCESS. Note: Common Lisp is not Scheme.
+---------------
| And I kick off the process like so:
|
| CL-USER> (with-open-file (stream "../../words/test.txt")
| (process #'process-line stream))
+---------------
The standard CL idiom omits the intermediate PROCESS function
entirely, and does the whole thing in the WITH-OPEN-FILE call:
(with-open-file (stream "../../words/test.txt")
(loop for line = (read-line stream nil nil)
while line do
(per-line-fn line)))
+---------------
| Is there a simpler way of doing what I'm trying to do here?
+---------------
See above.
+---------------
| I've seen some loop-based solutions, but am staying away from
| loop for the moment.
+---------------
Why?!? It's the most natural way to code this particular task, IMHO.
You don't have to use the entire repertoire of LOOP all at once.
Just pick a few simple idiomatic templates [such as the above]
and add more refinements as you discover you really need them.
Of course, you can also use DO, but to my taste it's a good deal
clunkier:
(with-open-file (stream "../../words/test.txt")
(do ((line #1=(read-line stream nil nil) #1#))
((null line))
(per-line-fn line)))
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Martha Stewart called it a Good Thing when ····@rpw3.org (Rob Warnock) wrote:
> The standard CL idiom omits the intermediate PROCESS function
> entirely, and does the whole thing in the WITH-OPEN-FILE call:
>
> (with-open-file (stream "../../words/test.txt")
> (loop for line = (read-line stream nil nil)
> while line do
> (per-line-fn line)))
At one point, I think I created a macro to allow the above to shorten
to more like:
(with-lines-in-file (file-descriptor line)
(per-line-fn line))
That's not any fundamentally enormous improvement, of course...
--
(format nil ···@~S" "cbbrowne" "gmail.com")
http://linuxdatabases.info/info/x.html
Rules of the Evil Overlord #55. "The deformed mutants and odd-ball
psychotics will have their place in my Legions of Terror. However
before I send them out on important covert missions that require tact
and subtlety, I will first see if there is anyone else equally
qualified who would attract less attention."
<http://www.eviloverlord.com/>
Iterate (http://common-lisp.net/project/iterate/) includes something
similar:
for <var> in-file <name> &optional using <reader>
Opens the file <name> (which may be a string or pathname) for input,
and
iterates over its contents. <reader> defaults to read, so by default
<var> will be
bound to the successive forms in the file. The iterate body is wrapped
in an
unwind-protect to ensure that the file is closed no matter how the
iterate is
exited.
············@gmail.com wrote:
> Iterate (http://common-lisp.net/project/iterate/) includes something
> similar:
>
snip...
That looks like a great package! I've just looked over the manual a
little, but the macro you pointed out is exactly what I was looking
for. Using that, the code becomes:
(defun process-line (line)
(format t "~A~%" line))
(iter (for line in-file "../../words/test.txt" using #'read-line)
(process-line line))
Thanks very much...
Christopher Browne wrote:
> At one point, I think I created a macro to allow the above to shorten
> to more like:
>
> (with-lines-in-file (file-descriptor line)
> (per-line-fn line))
>
> That's not any fundamentally enormous improvement, of course...
That's sort of what I had in mind. It's good to reinvent the wheel for
practice, but I was just checking to see if there was something like
this predefined that I was overlooking.
Thanks for the response.
j.k.
Rob Warnock wrote:
> <·········@gmail.com> wrote:
> This can blow up if your file is large and your CL implementation
> doesn't happen to perform tail-call optimization on the tail call
> to PROCESS. Note: Common Lisp is not Scheme.
Thanks, I was going to check on that later. I'm using SBCL on Linux,
and I've tried it on just over a million lines, so I assume it would
have blown the stack already if it weren't optimized. Is that a safe
assumption?
And I am coming from Scheme (good guess;)). I've been reading SICP but
wanting access to CL's rich library of built-in functions and macros
for some toy projects that I'm working on.
snip...
>
> +---------------
> | I've seen some loop-based solutions, but am staying away from
> | loop for the moment.
> +---------------
>
> Why?!? It's the most natural way to code this particular task, IMHO.
>
> You don't have to use the entire repertoire of LOOP all at once.
> Just pick a few simple idiomatic templates [such as the above]
> and add more refinements as you discover you really need them.
>
> Of course, you can also use DO, but to my taste it's a good deal
> clunkier:
>
> (with-open-file (stream "../../words/test.txt")
> (do ((line #1=(read-line stream nil nil) #1#))
> ((null line))
> (per-line-fn line)))
Thanks for the suggestions. My reason for avoiding loop for the moment
is that I'm just learning CL and wanted to stick to what strikes me as
core CL (i.e., the stuff that everybody agrees belongs in CL, whereas I
read in Seibel's book that at least one of the designers of CL did not
think it belonged). 'do' type constructs or recursion seem more lispy
to me than the baroque loop, which seems like an add-on feature that
belongs outside CL itself, perhaps in some commonly used CPAN-type
archive. What do I know? Not much, but that's my reasoning, and I know
that the line between core and non-core is especially fuzzy in CL as
opposed to a language without macros, so maybe I'm not at all making
sense.
My perception is probably colored by experience with Scheme, but I find
your do version at least as easy to understand (apart from the #1 and
#1#, which I haven't seen before) as your loop version, though
familiarity breeds ease.
Thanks for your time.
j.k
<·········@gmail.com> wrote:
+---------------
| Rob Warnock wrote:
| > ...you can also use DO, but to my taste it's a good deal clunkier:
| > (with-open-file (stream "../../words/test.txt")
| > (do ((line #1=(read-line stream nil nil) #1#))
| > ((null line))
| > (per-line-fn line)))
...
| My perception is probably colored by experience with Scheme...
+---------------
I'm a former Schemer myself, and have found that one needs to
make a conscious effort to let go of "the Scheme way" and embrace
"the CL way" if one is to become fluent and idiomatic in CL,
e.g., using CL's builtin LOOP/DOTIMES/DOLIST/DO macros over
Scheme-style tail-recursive loops.
+---------------
| ...but I find your do version at least as easy to understand
| (apart from the #1 and #1#, which I haven't seen before)
+---------------
The #= and ## are reader notation for self-referential [sub]structure
in literal input [and output, if *PRINT-CIRCLE* is true], see:
http://www.lispworks.com/documentation/HyperSpec/Body/02_dho.htm
2.4.8.15 Sharpsign Equal-Sign
http://www.lispworks.com/documentation/HyperSpec/Body/02_dho.htm
2.4.8.16 Sharpsign Sharpsign
Many Schemes [MzScheme, for one] also support the #n=/#n# notation,
but I seldom saw it used there; people usually just wrote it all out:
(do ((line (read-line stream nil nil) (read-line stream nil nil)))
(...)
...)
I forget where I first saw the #n=/#n# notation used with DO
in CL -- it was a long time ago. [Hmmm... (*grep*) (*grep*)...]
Ah, yezzz... It was something Erik Naggum wrote in late 1997, see
<·························@naggum.no> and the thread around that.
Note that DO [in both Scheme & CL] has a different default for the
"step" subform than "LOOP FOR var = ..." does -- the former defaults
the "step" form to the "var" [that is, no assignment done], whereas
the latter defaults the "step" form to the "init" form. That is this:
(loop for line = (read-line stream nil nil)
...)
is just a more readable/convenient shorthand for this:
(loop for line = (read-line stream nil nil) then (read-line stream nil nil)
...)
The fact that the default in LOOP is usually what you want for
stepping through I/O is certainly one of the reasons I learned
the basics of LOOP very soon after starting to use CL.
+---------------
| ...as your loop version, though familiarity breeds ease.
+---------------
It does, but you have to make a conscious effort to practice
CL style or it never gets familiar.
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: Alexander Schmolck
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <yfs7j7omwdc.fsf@oc.ex.ac.uk>
····@rpw3.org (Rob Warnock) writes:
> <·········@gmail.com> wrote:
> +---------------
> | Rob Warnock wrote:
> | > ...you can also use DO, but to my taste it's a good deal clunkier:
> | > (with-open-file (stream "../../words/test.txt")
> | > (do ((line #1=(read-line stream nil nil) #1#))
> | > ((null line))
> | > (per-line-fn line)))
> ...
> | My perception is probably colored by experience with Scheme...
> +---------------
>
> I'm a former Schemer myself, and have found that one needs to
> make a conscious effort to let go of "the Scheme way" and embrace
> "the CL way" if one is to become fluent and idiomatic in CL,
> e.g., using CL's builtin LOOP/DOTIMES/DOLIST/DO macros over
> Scheme-style tail-recursive loops.
Not that I would generally recommend using tail calls over LOOP et al (OK,
I'll make an exception for DO) -- but is there really any significant common
lisp implementation that can't optimize self-tail calls (I thought even clisp
would)?
'as
Alexander Schmolck <··········@gmail.com> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > e.g., using CL's builtin LOOP/DOTIMES/DOLIST/DO macros over
| > Scheme-style tail-recursive loops.
|
| Not that I would generally recommend using tail calls over LOOP et al
| (OK, I'll make an exception for DO) -- but is there really any
| significant common lisp implementation that can't optimize self-tail
| calls (I thought even clisp would)?
+---------------
Even in the presence of special-variable bindings and/or UNWIND-PROTECTs?
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <87mzgffc3c.fsf@qrnik.zagroda>
····@rpw3.org (Rob Warnock) writes:
> | Not that I would generally recommend using tail calls over LOOP et al
> | (OK, I'll make an exception for DO) -- but is there really any
> | significant common lisp implementation that can't optimize self-tail
> | calls (I thought even clisp would)?
> +---------------
>
> Even in the presence of special-variable bindings and/or UNWIND-PROTECTs?
Then they are not tail calls.
--
__("< Marcin Kowalczyk
\__/ ······@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Jumping in late, it seems there are more than one ways to do it, largely
depending on preferences.
The macro I use is stolen from here:
http://kantz.com/jason/clim-primer/word.lisp
I've added substantial comments.
(defmacro do-file ((path line-variable &key (key #'identity)) &body body)
"Iterate over the lines of the file, binding the line to the
line-variable in each iterateion"
(let ((str (gensym)) ;a temporary stream variable
(var (gensym))) ;a temporary variable to hold the file input.
`(with-open-file (,str ,path :direction :input)
(do ((,var (read-line ,str nil) ;get the first line
(read-line ,str nil)) ;get the next line
((not ,var)) ;terminate when there is no text.
(let ((,line-variable (funcall ,key ,var)))
;set line-variable to var, calling our key function
;if necessary.
,@body))))) ;do the body form
Then, I usually call it with something like:
(do-file ("authors.txt" line)
(foo-function line))
--
Kirk Job-Sluder
Kirk Job Sluder <···········@jobsluder.net> wrote:
+---------------
| The macro I use is stolen from here:
| http://kantz.com/jason/clim-primer/word.lisp
...
| (defmacro do-file ((path line-variable &key (key #'identity)) &body body)
+---------------
Yes, this is useful, but I think I'd probably want to change a
few things:
1. Reverse the order of the PATH and LINE-VARIABLE arguments,
for two reasons: (a) So that it reads more in the style of
other DO-XXX and WITH-XXX macros, which list the bound variable
first; and (b) so one can naturally add additional OPEN keyword
arguments after the filename [particularly :EXTERNAL-FORMAT,
given where this thead started!].
2. [Minor] Add a :STREAM keyword arg for a variable name to bind
to the open stream, in case the user might need access to it
during the traversal [e.g., for error messages or something].
3. [Minor] Have the macro test for whether the :KEY parameter
was provided, and if not, *don't* blindly call the KEY function.
[Yes, a good compiler will optimize (FUNCALL #'IDENTITY FOO)
to just FOO, but maybe not all compilers are that good.]
4. [Minor] Use LOOP instead of DO for the iteration, mainly because
it's more concise [not that that matters much inside a macro].
5. [Minor] Change the name to DO-FILE-LINES, to better indicate
what is being iterated over. [To me, DO-FILE sounds like it does
something *to* the file.]
If it hadn't been for #2, #1(b) could have been accomodated by
simply adding ":ALLOW-OTHER-KEYS T" to the WITH-OPEN-FILE call,
but with #2 *and* #1(b), one needs to strip the :STREAM argument
out of the WITH-OPEN-FILE call, since the variable name may well
be unbound. Given that, I decided to strip out both :STREAM & :KEY.
[Though if there had been more than two arg pairs to strip, I might
have used something more efficient than multiple REMFs.] Anyway,
put it all together and you get the following:
(defmacro do-file-lines ((line path &rest open-options
&key (stream (gensym) stream-p)
(key #'identity key-p)
&allow-other-keys)
&body body)
"For each line in the file named by PATH, the BODY is executed
with the variable LINE bound to the value of the :KEY function
(default #'IDENTITY) applied to that line. If any additional
OPEN-OPTIONS are provided, they will be used when opening the
file. If a variable name is provided to :STREAM, it will be bound
to the opened stream."
(let* ((options (if (or stream-p key-p)
(let ((options (copy-list open-options)))
(when stream-p
(remf options :stream))
(when key-p
(remf options :key))
options)
open-options))
(loop-var (if key-p (gensym) line))
(let-args (if key-p `((,line (funcall ,key ,loop-var))) nil)))
`(with-open-file (,stream ,path ,@options)
(loop for ,loop-var = (read-line ,stream nil nil)
while ,loop-var do
(let ,let-args
,@body)))))
It could be used like this:
> (do-file-lines (foo "foo.tmp")
(print foo))
"The first line"
"The second line"
"And the longest and last third line"
NIL
> (do-file-lines (foo "foo.tmp" :key #'length :stream bar)
(print (list foo bar)))
(14 #<Stream for file "foo.tmp">)
(15 #<Stream for file "foo.tmp">)
(35 #<Stream for file "foo.tmp">)
NIL
>
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock <····@rpw3.org> wrote:
> 4. [Minor] Use LOOP instead of DO for the iteration, mainly because
> it's more concise [not that that matters much inside a macro].
Isn't that a bad idea? Wouldn't it cause something like this to
misbehave, for instance?:
(loop for pathname in '("a" "b" "c" "d")
do (do-file-lines (line pathname :direction :input)
(when (some-condition-met-p line)
(loop-finish))
--
Duncan Harvey
Duncan Harvey <··············@abbrvtd.org.uk> wrote:
+---------------
| Rob Warnock <····@rpw3.org> wrote:
| > 4. [Minor] Use LOOP instead of DO for the iteration, mainly because
| > it's more concise [not that that matters much inside a macro].
|
| Isn't that a bad idea? Wouldn't it cause something like this to
| misbehave, for instance?:
|
| (loop for pathname in '("a" "b" "c" "d")
| do (do-file-lines (line pathname :direction :input)
| (when (some-condition-met-p line)
| (loop-finish))
+---------------
Well, that *might* be also construed as a "feature", that is, it
gives one the ability to do an early termination on a given file. ;-}
But your point is taken -- if an iteration macro does choose
to emit a LOOP, it should state so explicitly, e.g., add the
following to the doc string previously shown: "Uses LOOP to
perform the iteration, so a (LOOP-FINISH) within BODY will
terminate the iteration and close the file."
Note that I already had some obvious qualms about declarations,
which is why the default :KEY function case wrapped a null LET
around the BODY, just so that declarations would work. Otherwise
a simple "DO ,@BODY" splice would have sufficed.
What do others think? Are these sorts of conflicts common and
serious enough to ban the emitting of LOOP from (most) iteration
macros?
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
In article <······················@speakeasy.net>,
····@rpw3.org (Rob Warnock) wrote:
> What do others think? Are these sorts of conflicts common and
> serious enough to ban the emitting of LOOP from (most) iteration
> macros?
I don't know about banning anything, especially since the macro could be
trivially re-written either way. However, I must admit that I prefer
DO-macros to LOOP when I can get away with it. I don't fully understand
the advantage for doing everything with LOOP.
>
>
> -Rob
>
> -----
> Rob Warnock <····@rpw3.org>
> 627 26th Avenue <URL:http://rpw3.org/>
> San Mateo, CA 94403 (650)572-2607
--
Kirk Job-Sluder
From: Christophe Rhodes
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <sqfym5gtb8.fsf@cam.ac.uk>
····@rpw3.org (Rob Warnock) writes:
> Note that I already had some obvious qualms about declarations,
> which is why the default :KEY function case wrapped a null LET
> around the BODY, just so that declarations would work. Otherwise
> a simple "DO ,@BODY" splice would have sufficed.
>
> What do others think? Are these sorts of conflicts common and
> serious enough to ban the emitting of LOOP from (most) iteration
> macros?
I think emitting extended LOOP in macros is poor style, but not
sufficiently poor to make me throw up my hands in horror (as long as
it's documented). I think it's important to document whatever
iteration method is used, with particular reference to block names: a
NIL block name seems to be usual.
(Also, on the subject of null LETs for declaration purposes, in this
kind of macro declarations are usually allowed, and the scope of the
declarations is usually such that they are bound declarations for any
introduced variables. This is another reason to expand into DO rather
than LOOP, because this is impossible to achieve in LOOP.)
Christophe
From: David Trudgett
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <m3slq4nvxm.fsf@rr.trudgett>
Christophe Rhodes <·····@cam.ac.uk> writes:
> ····@rpw3.org (Rob Warnock) writes:
>
>> Note that I already had some obvious qualms about declarations,
>> which is why the default :KEY function case wrapped a null LET
>> around the BODY, just so that declarations would work. Otherwise
>> a simple "DO ,@BODY" splice would have sufficed.
>>
>> What do others think? Are these sorts of conflicts common and
>> serious enough to ban the emitting of LOOP from (most) iteration
>> macros?
>
> I think emitting extended LOOP in macros is poor style, but not
> sufficiently poor to make me throw up my hands in horror (as long as
> it's documented). I think it's important to document whatever
> iteration method is used, with particular reference to block names: a
> NIL block name seems to be usual.
How about this version? How would a NIL block be added to it?
;;; A handy text file processing macro by Rob Warnock, of comp.lang.lisp.
;;; Rob modified a macro posted to c.l.l. by Kirb Job Sluder, which he
;;; in turn found at http://kantz.com/jason/clim-primer/word.lisp
;;; The macro was then modified by David Trudgett to use DO instead of
;;; LOOP for the iteration, and to add prefixes to GENSYMed symbols.
(defmacro do-file-lines ((line path &rest open-options
&key (stream (gensym "STREAM-") stream-p)
(key #'identity key-p)
&allow-other-keys)
&body body)
"For each line in the file named by PATH, the BODY is executed with
the variable LINE bound to the value of the :KEY function (default
#'IDENTITY) applied to that line. If any additional OPEN-OPTIONS are
provided, they will be used when opening the file. If a variable name
is provided to :STREAM, it will be bound to the opened stream."
(let* ((options (if (or stream-p key-p)
(let ((options (copy-list open-options)))
(when stream-p
(remf options :stream))
(when key-p
(remf options :key))
options)
open-options))
(line-var (if key-p (gensym "LINE-") line))
(let-args (if key-p `((,line (funcall ,key ,line-var))) nil)))
`(with-open-file (,stream ,path ,@options)
(do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
((eq 'eof ,line-var))
(let ,let-args
,@body)))))
>
> (Also, on the subject of null LETs for declaration purposes, in this
> kind of macro declarations are usually allowed, and the scope of the
> declarations is usually such that they are bound declarations for any
> introduced variables.
How does the above relate to that? Does something need to be done to
specifically allow declarations?
David
--
David Trudgett
http://www.zeta.org.au/~wpower/
You must be the change you wish to see in the world.
-- Mohandas Gandhi
David Trudgett <······@zeta.org.au.nospamplease> wrote:
+---------------
| How about this version? ...
| ;;; ... and to add prefixes to GENSYMed symbols.
+---------------
Small thing, perhaps, but personally I prefer *not* to do this.
In the CLs I use the system-defined macros don't do it, and I
haven't noticed that it makes debugging significantly harder.
But YMMV...
+---------------
| (do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
| ((eq 'eof ,line-var))
| ...)
+---------------
Just curious... Why did you change the NIL to 'EOF? Both are symbols,
so you don't get any more protection from the latter than the former
[given that we're using READ-LINE, which can't return either], but
using 'EOF requires the EQ test rather than simply NULL:
(do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
((null ,line-var))
...)
-Rob
p.s. Christopher Rhodes spoke adequately to the other points...
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: David Trudgett
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <m3u0ajnqsb.fsf@rr.trudgett>
····@rpw3.org (Rob Warnock) writes:
> David Trudgett <······@zeta.org.au.nospamplease> wrote:
> +---------------
> | How about this version? ...
> | ;;; ... and to add prefixes to GENSYMed symbols.
> +---------------
>
> Small thing, perhaps, but personally I prefer *not* to do this.
> In the CLs I use the system-defined macros don't do it, and I
> haven't noticed that it makes debugging significantly harder.
> But YMMV...
Yes, MMDV :-) In my short history as a Lisp programmer, I have found
it sometimes helpful when reading my macro-expansions, because I like
it to be nice and obvious what each GENSYMed symbol is supposed to
be. My brain doesn't have to work so hard! It has also been useful in
other circumstances when generating dozens of symbols for interning
and later reporting. G1784 doesn't mean much to me, but ACTIVITY-1784
or NODE-1831 gives me a clue.
System defined macros refrain from it for uber-efficiency reasons, I
imagine; whereas, I personally haven't needed to care about a
millisecond or two in my macros.
Obviously, it doesn't matter in this example, but when I mutilate code
for myself, I do it the way I like it! ;-) It's in my little UTILS
package ready for future use.
So, to answer the unspoken question, I didn't do it because I think it
is better, and other programmers should do it my way! ;-) I won't hold
it against anyone if they don't customise their generated symbols! :-)
>
> +---------------
> | (do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
> | ((eq 'eof ,line-var))
> | ...)
> +---------------
>
> Just curious... Why did you change the NIL to 'EOF? Both are symbols,
> so you don't get any more protection from the latter than the former
> [given that we're using READ-LINE, which can't return either], but
> using 'EOF requires the EQ test rather than simply NULL:
>
> (do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
> ((null ,line-var))
> ...)
Just personal preference again, Rob. Plus I knew it would get your
goat! ;-) No, not really. There is nothing wrong with your way (except
that it won't work, as you've written it above! ;-)). As for
performance, there is no appreciable difference on my system. Several
test runs showed figures varying around the following:
(time (let ((k 0)) (do-file-lines (line "text.txt") (incf k))))
TEST 1 (NULL version):
; Evaluation took:
; 0.29 seconds of real time
; 0.26 seconds of user run time
; 0.03 seconds of system run time
; 285,231,040 CPU cycles
; 0 page faults and
; 6,564,176 bytes consed.
TEST 2 (EQ version):
; Evaluation took:
; 0.28 seconds of real time
; 0.26 seconds of user run time
; 0.03 seconds of system run time
; 287,131,680 CPU cycles
; 0 page faults and
; 6,564,176 bytes consed.
wc -l text.txt
63993 text.txt
Bye for now,
David
>
> p.s. Christopher Rhodes spoke adequately to the other points...
Christophe must be resigned to that spelling, by now! ;-)
dt
--
David Trudgett
http://www.zeta.org.au/~wpower/
First they ignore you,
then they laugh at you,
then they fight you,
then you win!
-- Mohandas Gandhi
From: Christophe Rhodes
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <sq1wxnfaj2.fsf@cam.ac.uk>
David Trudgett <······@zeta.org.au.nospamplease> writes:
> ····@rpw3.org (Rob Warnock) writes:
>> p.s. Christopher Rhodes spoke adequately to the other points...
>
> Christophe must be resigned to that spelling, by now! ;-)
Heh. I think I have it easy compared to that Seible or Siebel or
whoever wrote that Portable Common Loops book...
Christophe
David Trudgett <······@zeta.org.au.nospamplease> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > (do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
| > ((null ,line-var))
| > ...)
...
| There is nothing wrong with your way (except that it won't work,
| as you've written it above! ;-)).
+---------------
Ouch! Half-done cut&paste edit.
Yes, of course, that should have been:
(do ((,line-var #1=(read-line ,stream nil nil) #1#))
((null ,line-var))
...)
+---------------
| > p.s. Christopher Rhodes spoke adequately to the other points...
|
| Christophe must be resigned to that spelling, by now! ;-)
+---------------
*Yikes!* I must stay after school & write 2000 times:
"Christophe Rhodes" != "Christopher Browne"
"Christophe Rhodes" != "Christopher Browne"
"Christophe Rhodes" != "Christopher Browne"
"Christophe Rhodes" != "Christopher Browne"
...
Apologies to all. My typos-per-posting quotient
seems to have really gone through the roof lately...
(*sigh*)
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
David Trudgett <······@zeta.org.au.nospamplease> writes:
> Yes, MMDV :-) In my short history as a Lisp programmer, I have found
> it sometimes helpful when reading my macro-expansions, because I like
> it to be nice and obvious what each GENSYMed symbol is supposed to
> be. My brain doesn't have to work so hard! It has also been useful in
> other circumstances when generating dozens of symbols for interning
> and later reporting. G1784 doesn't mean much to me, but ACTIVITY-1784
> or NODE-1831 gives me a clue.
In compiled code, most of these symbols will disappear anyways, so it
doesn't cost anything to help the debugger with longer names.
--
__Pascal Bourguignon__ http://www.informatimago.com/
"You question the worthiness of my code? I should kill you where you
stand!"
From: Christophe Rhodes
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <sqd5h8ymb0.fsf@cam.ac.uk>
David Trudgett <······@zeta.org.au.nospamplease> writes:
> Christophe Rhodes <·····@cam.ac.uk> writes:
>
>> I think emitting extended LOOP in macros is poor style, but not
>> sufficiently poor to make me throw up my hands in horror (as long as
>> it's documented). I think it's important to document whatever
>> iteration method is used, with particular reference to block names: a
>> NIL block name seems to be usual.
>
> How about this version? How would a NIL block be added to it?
There already is one: DO expands into a block named NIL.
> (defmacro do-file-lines ((line path &rest open-options
> &key (stream (gensym "STREAM-") stream-p)
> (key #'identity key-p)
> &allow-other-keys)
> &body body)
> "For each line in the file named by PATH, the BODY is executed with
> the variable LINE bound to the value of the :KEY function (default
> #'IDENTITY) applied to that line. If any additional OPEN-OPTIONS are
> provided, they will be used when opening the file. If a variable name
> is provided to :STREAM, it will be bound to the opened stream."
> (let* ((options (if (or stream-p key-p)
> (let ((options (copy-list open-options)))
> (when stream-p
> (remf options :stream))
> (when key-p
> (remf options :key))
> options)
> open-options))
> (line-var (if key-p (gensym "LINE-") line))
> (let-args (if key-p `((,line (funcall ,key ,line-var))) nil)))
> `(with-open-file (,stream ,path ,@options)
> (do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
> ((eq 'eof ,line-var))
> (let ,let-args
> ,@body)))))
>
>> (Also, on the subject of null LETs for declaration purposes, in this
>> kind of macro declarations are usually allowed, and the scope of the
>> declarations is usually such that they are bound declarations for any
>> introduced variables.
>
> How does the above relate to that? Does something need to be done to
> specifically allow declarations?
The macro above fails this criterion; the idea is that
(do-file-lines (s "/tmp/foo")
(declare (special s))
(bar))
should cause a special binding of S to be visible in BAR. With the
above macro, this won't work, as any declarations in ,@body won't be
bound declarations for the binding of LINE when a key function is not
provided. Fixing this up is quite simple; the last two lines should
be replaced by
,@(if key-p `((let ,let-args ,@body)) body)
Christophe
Christophe Rhodes <·····@cam.ac.uk> wrote:
+---------------
| David Trudgett <······@zeta.org.au.nospamplease> writes:
| > Does something need to be done to specifically allow declarations?
|
| The macro above fails this criterion; the idea is that
| (do-file-lines (s "/tmp/foo")
| (declare (special s))
| (bar))
| should cause a special binding of S to be visible in BAR. With the
| above macro, this won't work, as any declarations in ,@body won't be
| bound declarations for the binding of LINE when a key function is not
| provided. Fixing this up is quite simple; the last two lines should
| be replaced by
| ,@(if key-p `((let ,let-args ,@body)) body)
+---------------
Yup. All of which is reason enough to use DO instead of LOOP in
the expansion. So noted...
-Rob
p.s. Thanks, everyone, for the comments in re LOOP in macro expansions.
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: David Trudgett
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <m3wtff3mtw.fsf@rr.trudgett>
Christophe Rhodes <·····@cam.ac.uk> writes:
> provided. Fixing this up is quite simple; the last two lines should
> be replaced by
> ,@(if key-p `((let ,let-args ,@body)) body)
>
That's pretty nifty. Thanks for that. For the record, that makes the
modified macro:
(defmacro do-file-lines ((line path &rest open-options
&key (stream (gensym "STREAM-") stream-p)
(key #'identity key-p)
&allow-other-keys)
&body body)
"For each line in the file named by PATH, the BODY is executed with
the variable LINE bound to the value of the :KEY function (default
#'IDENTITY) applied to that line. If any additional OPEN-OPTIONS are
provided, they will be used when opening the file. If a variable name
is provided to :STREAM, it will be bound to the opened stream."
(let* ((options (if (or stream-p key-p)
(let ((options (copy-list open-options)))
(when stream-p
(remf options :stream))
(when key-p
(remf options :key))
options)
open-options))
(line-var (if key-p (gensym "LINE-") line))
(let-args (if key-p `((,line (funcall ,key ,line-var))) nil)))
`(with-open-file (,stream ,path ,@options)
(do ((,line-var #1=(read-line ,stream nil 'eof) #1#))
((eq 'eof ,line-var))
,@(if key-p
`((let ,let-args ,@body))
body)))))
A couple of test expansions (and correct output!) seem to show that it
works:
(WITH-OPEN-FILE
(#:STREAM-1781 "file" :DIRECTION :INPUT)
(DO ((#:LINE-1782 (READ-LINE #:STREAM-1781 NIL 'EOF)
(READ-LINE #:STREAM-1781 NIL 'EOF)))
((EQ 'EOF #:LINE-1782))
(LET ((LINE (FUNCALL #'(LAMBDA (X) (STRING-UPCASE X)) #:LINE-1782)))
(DECLARE (SPECIAL LINE))
(FORMAT T "~&~A~%" LINE))))
and:
(WITH-OPEN-FILE
(#:STREAM-1783 "file" :DIRECTION :INPUT)
(DO ((LINE (READ-LINE #:STREAM-1783 NIL 'EOF)
(READ-LINE #:STREAM-1783 NIL 'EOF)))
((EQ 'EOF LINE))
(DECLARE (SPECIAL LINE))
(FORMAT T "~&~A~%" LINE)))
David
--
David Trudgett
http://www.zeta.org.au/~wpower/
It is naively assumed that the fact that the majority of people share
certain ideas or feelings proves the validity of these ideas and
feelings. Nothing is further from the truth... Just as there is a
'folie a deux' there is a 'folie a millions.' The fact that millions
of people share the same vices does not make these vices virtues, the
fact that they share so many errors does not make the errors to be
truths, and the fact that millions of people share the same form of
mental pathology does not make these people sane.
-- Erich Fromm, The Sane Society, Routledge, 1955, pp.14-15
····@rpw3.org (Rob Warnock) writes:
> | (loop for pathname in '("a" "b" "c" "d")
> | do (do-file-lines (line pathname :direction :input)
> | (when (some-condition-met-p line)
> | (loop-finish))
Ouch! So no casual reader will know for sure where LOOP-FINISH jumps to.
No wonder Schemers praise referential transparency.
It also shows why
(call-with-open-file ...) may be preferrable and preferred by some.
It introduces no scoping problems.
Implementing DO-FILE-LINES as a macro is IMHO not worth the trouble it
introduces. funcall a closure will likely be fast enough.
And clever compilers could possibly inline CALL-WITH-OPEN-FILE for
bloat and hopefully speed.
Inlining causes no scoping issues, unlike macros.
Rainer Joswig (not the inexperience newbie) clearly has a point
about abuse of macros.
> But your point is taken -- if an iteration macro does choose
> to emit a LOOP, it should state so explicitly, e.g., add the
> following to the doc string previously shown: "Uses LOOP to
> perform the iteration, so a (LOOP-FINISH) within BODY will
> terminate the iteration and close the file."
Exactly. I'v strong feelings about that.
And I prefer DO-* named macros to behave like DO, and not introduce
spurious scopes:
DO-xyz => BLOCK named NIL, and space for declarations, initform
evaluated outside the scope of the new variable etc.
> What do others think? Are these sorts of conflicts common and
> serious enough to ban the emitting of LOOP from (most) iteration
> macros?
Yes. And as the current maintainer of the Iterate package, I've had
similar bad feelings about user library macros expanding to Iterate
loops, where even more can go wrong:
(iter (for pathname in '("a" ...))
(do-file-lines (line ...)
(when (foo) (COLLECT line))))
Where will COLLECTed values go, if DO-FILE-LINES expands to Iterate?!?
> | ;;; ... and to add prefixes to GENSYMed symbols.
> Small thing, perhaps, but personally I prefer *not* to do this.
Iterate may expand to many variables, it does not cost a dime to add
names to GENSYM and it's much nicer to view the expansion with
meaningful names. C-c C-m in SLIME IIRC.
Thus, I've added names almost everywhere in Iterate.
Regards,
Jorg Hohle
Telekom/T-Systems Technology Center
Hi again,
Note that my critique of do-file-lines as a macro is not so much a
critique of macros per se rather than a call for thoughtful design:
- left to right evaluation?
- evaluate once?
- consistent with other established naming conventions?
- scoping?
- declarations?
E.g. only macros can provide you with a form where you can specify all
sort of arguments to OPEN. There's little wrong with this:
(defun call-with-open-file-lines (file thunk)
;; add (when (streamp file) ? for sake of OPEN :ERROR NIL?
(iter (for x in-stream file by #'read-line)
;; IN-STREAM does the UNWIND-PROTECT+CLOSE for you!
(funcall thunk x)))
(defmacro with-file-lines ((var filename &rest options &key &allow-other-keys)
&body body)
`(call-with-open-file-lines (open filename :direction in . ,options)
#'(lambda(,var) . ,body)))
You get both the benefit of a macro and still small code.
Additionaly, binding only upon valid input has the advantage that you
can use exact declarations as in:
(with-file-lines (...)
(declare (type number my-var))
...)
-- when you can be confident to read only such objects.
There's no need to use a declaration like
(declare (type (or number <my-read-eof-values-type>) my-var))
like with an explicit
(do ((my-var (read-line file nil <eof-value>) #)#) ...)
Regards,
Jorg Hohle
European Common Lisp Meeting 2006: <http://weitz.de/eclm2006/>
On 9397 day of my life Rob Warnock wrote:
> 4. [Minor] Use LOOP instead of DO for the iteration, mainly because
> it's more concise [not that that matters much inside a macro].
Actually LOOP is bit dangerous in macroexpansions because LOOP is
anaphoric macro. For example:
(loop for it from 1 to 10
when (oddp it)
collect it)
=> (T T T T T), not (1 3 5 6 7)
But your macro if fine because LOOP-VAR is used after WHILE clause
only inside BODY.
P.S. Where is defined semantics of IT loop keyword? I have found
only single example in 6.1.8.1 and short notice in 6.1.9 "Use caution
when using a variable named IT...".
--
Ivan Boldyrev
Your bytes are bitten.
From: Edi Weitz
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <uacccpjb1.fsf@agharta.de>
On Tue, 28 Feb 2006 00:17:31 +0600, Ivan Boldyrev <···············@cgitftp.uiggm.nsc.ru> wrote:
> Where is defined semantics of IT loop keyword? I have found only
> single example in 6.1.8.1 and short notice in 6.1.9 "Use caution
> when using a variable named IT...".
6.1.6
--
European Common Lisp Meeting 2006: <http://weitz.de/eclm2006/>
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
In article <································@speakeasy.net>,
····@rpw3.org says...
> This can blow up if your file is large and your CL implementation
> doesn't happen to perform tail-call optimization on the tail call
> to PROCESS. Note: Common Lisp is not Scheme.
Rob, do you have some simple example of the code that works
more efficiently in typical Scheme implementation than in
typical Lisp implementation?
Majorinc, Kazimir <·····@email.com> wrote:
+---------------
| ····@rpw3.org says...
| > This can blow up if your file is large and your CL implementation
| > doesn't happen to perform tail-call optimization on the tail call
| > to PROCESS. Note: Common Lisp is not Scheme.
|
| Rob, do you have some simple example of the code that works
| more efficiently in typical Scheme implementation than in
| typical Lisp implementation?
+---------------
No. That's part of why I moved from Scheme to Common Lisp... ;-} ;-}
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
·········@gmail.com wrote:
> Is there a simpler way of doing what I'm trying to do here? I've seen
> some loop-based solutions, but am staying away from loop for the
> moment.
What's wrong with loop?
If you want to do more text processing, maybe something like CLAWK is an
idea:
(for-file-lines ("c:/tmp/test.txt" stream line)
(format t "~a~%" line))
test.txt:
Beth 4.00 0
Dan 3.75 0
Kathy 4.00 10
Mark 5.00 20
Mary 5.50 22
Suzie 4.25 18
(defawk sample (&aux (pay 0))
(t (incf pay ($* $2 $3)))
(END ($print *NR* "employees")
($print "total pay is" pay)
($print "average pay is " (/ pay *NR*))) )
(sample "test.txt")
6 employees
total pay is 337.5
average pay is 56.25
The homepage is at http://www.geocities.com/mparker762/clawk
I've re-packaged it as an ASDF installable package:
http://sourceforge.net/project/showfiles.php?group_id=159740
(you'll need lispbuilder-regex, too)
--
Frank Buss, ··@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de
Frank Buss wrote:
>
> What's wrong with loop?
I explained above in a little more detail, but I'm just starting with
CL, and loop seems like a second language to learn, so first things
first.
> If you want to do more text processing, maybe something like CLAWK is an
> idea:
snip...
Very nice! My needs for this particular case (each line is only 1 word)
don't warrant AWK, but I've bookmarked your site. My use of AWK
generally doesn't go much beyond cat somefile | awk "{print $3}" | sort
| uniq (:-)), but your package looks very powerful and easy to use.
Thanks for the suggestion.
j.k.
··········@gmail.com" <·········@gmail.com> writes:
> My use of AWK
> generally doesn't go much beyond cat somefile | awk "{print $3}" | sort
> | uniq (:-)), but your package looks very powerful and easy to use.
awk '{print $3}' < somefile | sort -u
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -- Georges W. Bush
From: David Trudgett
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <m38xs68f4n.fsf@rr.trudgett>
·········@gmail.com writes:
> I'm trying to process each line in a file, and am wondering if there is
> a simpler or more idiomatic way of doing it than the following:
>
> ;;; Simplified version of my real process-line function
> (defun process-line (line)
> (format t "~A~%" line))
>
> ;;; Process stream, calling per-line-fn on each line of the stream
> (defun process (per-line-fn stream)
> (let ((line (read-line stream nil)))
> (if (not (null line))
> (progn (funcall per-line-fn line)
> (process per-line-fn stream)))))
Rob and Frank have already given you some good dope, but I just
thought I'd make a small note.
Instead of constructs like:
> (if (not (null line))
> (progn (funcall per-line-fn line)
> (process per-line-fn stream)))))
I usually prefer something like:
(unless (null line)
(do x)
(do y))
Saves a double negative and an ugly progn.
Cheers,
David
--
David Trudgett
http://www.zeta.org.au/~wpower/
"Then leading him to a height, the devil showed him in a moment of
time all the kingdoms of the world and said to him, 'I will give you
all this power and the glory of these kingdoms, for it has been
committed to me and I give it to anyone I choose. Worship me, then,
and it shall all be yours.' But Jesus answered him, 'Scripture says:
You must worship the Lord your God, and serve him alone.'"
(Luke 4:5-8)
David Trudgett wrote:
> ·········@gmail.com writes:
>
>
>>I'm trying to process each line in a file, and am wondering if there is
>>a simpler or more idiomatic way of doing it than the following:
>>
>>;;; Simplified version of my real process-line function
>>(defun process-line (line)
>> (format t "~A~%" line))
>>
>>;;; Process stream, calling per-line-fn on each line of the stream
>>(defun process (per-line-fn stream)
>> (let ((line (read-line stream nil)))
>> (if (not (null line))
>> (progn (funcall per-line-fn line)
>> (process per-line-fn stream)))))
>
>
> Rob and Frank have already given you some good dope, but I just
> thought I'd make a small note.
>
> Instead of constructs like:
>
>
>> (if (not (null line))
>> (progn (funcall per-line-fn line)
>> (process per-line-fn stream)))))
>
>
>
> I usually prefer something like:
>
> (unless (null line)
> (do x)
> (do y))
>
> Saves a double negative and an ugly progn.
Wow. You argue against double negatives and then code (unless (null...? :)
Would
"(when line
(do x)
(fo y)..."
....kill anybody?
From: David Trudgett
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <m3mzglznxq.fsf@rr.trudgett>
Kenny Tilton <·············@nyc.rr.com> writes:
> David Trudgett wrote:
>> Instead of constructs like:
>>
>>> (if (not (null line))
>>> (progn (funcall per-line-fn line)
>>> (process per-line-fn stream)))))
>> I usually prefer something like:
>> (unless (null line)
>> (do x)
>> (do y))
>> Saves a double negative and an ugly progn.
>
> Wow. You argue against double negatives and then code (unless (null...? :)
>
> Would
>
> "(when line
> (do x)
> (fo y)..."
>
> ....kill anybody?
Hi Kenny,
(unless (null xyz)) is a single negative, Kenny. :-) Can I help it if
unless contains within itself a negative idea? ;-)
Your suggestion is OK if you like that sort of thing, but I have a
natural aversion to using arbitrary values as booleans (which is not
to say I never do it, especially with, say, Perl, for instance, where
there is the added danger of DWIM).
Catchya,
David
--
David Trudgett
http://www.zeta.org.au/~wpower/
Organisation, far from creating authority, is the only cure for it and
the only means whereby each of us will get used to taking an active
and conscious part in collective work, and cease being passive
instruments in the hands of leaders.
-- Errico Malatesta
David Trudgett <······@zeta.org.au.nospamplease> wrote:
+---------------
| Your suggestion is OK if you like that sort of thing, but I
| have a natural aversion to using arbitrary values as booleans...
+---------------
But remember, this is Common Lisp, not Scheme. In CL the *only*
distinguished boolean is NIL as "false" -- *everything* else is "true".
Plus, we're using (READ-LINE stream NIL NIL) here, where the outcome is
*guaranteed* [absent an I/O error throwing an exception] to be either
a [possibly zero-length] string ["true"] or NIL. So you've already got
an unambiguous boolean result, and thus (WHEN LINE ...) is IMHO a more
natural way than (UNLESS (NULL LINE) ...) to say what you're really
trying to do.
The same is true of READ-CHAR but *not* of READ, which can return NIL,
which is why people tend to use the following variation when using READ:
(with-open-file (stream "filename")
(loop with eof = (list nil)
for form = (read stream nil eof)
until (eq form eof) do
(process-one-form form)))
And of course some prefer to use the "WHILE (NOT (EQ FORM EOF))"
variant here, for parallelism with other LOOPs that use WHILE.
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
> The same is true of READ-CHAR but *not* of READ, which can return NIL,
> which is why people tend to use the following variation when using READ:
>
> (with-open-file (stream "filename")
> (loop with eof = (list nil)
> for form = (read stream nil eof)
> until (eq form eof) do
> (process-one-form form)))
Nice; I've also seen the stream object itself used as the eof value
more than a couple times.
Zach
> But remember, this is Common Lisp, not Scheme. In CL the *only*
> distinguished boolean is NIL as "false" -- *everything* else is "true".
>From R5RS, section 6.3.1 (Booleans):
Of all the standard Scheme values, only #f counts as false in
conditional expressions. Except for #f, all standard Scheme
values, including #t, pairs, the empty list, symbols, numbers,
strings, vectors, and procedures, count as true.
Also:
Welcome to scsh 0.6.6 (King Conan)
Type ,? for help.
> (if 42 'true 'false)
'true
>
Emilio Lopes <············@gmail.com> wrote:
+---------------
| > But remember, this is Common Lisp, not Scheme. In CL the *only*
| > distinguished boolean is NIL as "false" -- *everything* else is "true".
|
| From R5RS, section 6.3.1 (Booleans):
| Of all the standard Scheme values, only #f counts as false in
| conditional expressions. Except for #f, all standard Scheme
| values, including #t, pairs, the empty list, symbols, numbers,
| strings, vectors, and procedures, count as true.
+---------------
Ouch! Sorry, and apologies to the Schemers. I said almost exactly
the inverse of what I meant. (*blush*) What I was *trying* to say
is something like this: *Like* Scheme, in CL there is only one
distinguished boolean, namely NIL == "false", and *like* Scheme,
everything else is "true". Therefore there is no problem is using
the string one might get back from READ-LINE as a "true" truth value
[especially when one has done (READ-LINE stream NIL NIL) to guarantee
that EOF is "false"].
The only complication in CL is that, unlike Scheme, the sole "false"
boolean is *also* the empty list and is *also* an ordinary symbol,
whereas both of the latter two are always "true" in Scheme.
+---------------
| Also:
| Welcome to scsh 0.6.6 (King Conan)
| Type ,? for help.
| > (if 42 'true 'false)
| 'true
| >
+---------------
Yes, that works the same in both Scheme & CL. But in Scheme:
(map not (map not (list #f '() 'nil)))
==> (#f #t #t)
whereas in CL:
(mapcar #'not (mapcar #'not (list nil '() 'nil)))
==> (NIL NIL NIL)
Some see this as a good thing, some as bad. Different
languages, different styles, different communities.
Personally, now that I've been doing CL for a few years, I
*like* that fact that (EQ () NIL). On balance, it's more
convenience than confusion.
On the other hand, I *don't* especially like the fact that
(EQ 'NIL NIL) -- it requires some really bizarre contortions
in the internal implementation of CL, and even users get burned
by it once in a while [e.g., when trying to do symbolic processing
when the input comes from READ]. But I can live with it...
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: David Trudgett
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <m38xs5z95i.fsf@rr.trudgett>
····@rpw3.org (Rob Warnock) writes:
> David Trudgett <······@zeta.org.au.nospamplease> wrote:
> +---------------
> | Your suggestion is OK if you like that sort of thing, but I
> | have a natural aversion to using arbitrary values as booleans...
> +---------------
>
> But remember, this is Common Lisp, not Scheme.
Oh, yeah... ;-)
> In CL the *only* distinguished boolean is NIL as "false" --
> *everything* else is "true".
Yep. Feature or bug? :-) Whatever it is, I do take advantage of it
from time to time.
> Plus, we're using (READ-LINE stream
> NIL NIL) here, where the outcome is *guaranteed* [absent an I/O
> error throwing an exception] to be either a [possibly zero-length]
> string ["true"] or NIL. So you've already got an unambiguous boolean
> result, and thus (WHEN LINE ...) is IMHO a more natural way than
> (UNLESS (NULL LINE) ...) to say what you're really trying to do.
Your point is understood, and is a common view. After all, even C
programmers do it. From the perspective of what's natural, however, as
opposed to what's cute, or expedient, the *natural* reaction to (WHEN
LINE) is "when line what?" On the other hand, (UNLESS (NULL LINE)) is
an explicit and natural expression, and the closest you'll get to
"unless the line is null, do the following" in Lisp.
You can see that there is plenty of room for opinion here.
>
> The same is true of READ-CHAR but *not* of READ,
Oops... :-)
> which can return NIL,
> which is why people tend to use the following variation when using READ:
>
> (with-open-file (stream "filename")
> (loop with eof = (list nil)
> for form = (read stream nil eof)
> until (eq form eof) do
> (process-one-form form)))
Uhmmm....
>
> And of course some prefer to use the "WHILE (NOT (EQ FORM EOF))"
> variant here, for parallelism with other LOOPs that use WHILE.
Thanks for that, Rob. As usual, your posts contain very useful
information. I've got a print-out here of your recent post on
various ways to deliver CMUCL apps.
David
--
David Trudgett
http://www.zeta.org.au/~wpower/
Only let men cease to be hypocrites, and they would at once see that
this cruel social organization, which holds them in bondage, and is
represented to them as something stable, necessary, and ordained of
God, is already tottering and is only propped up by the falsehood of
hypocrisy, with which we, and others like us, support it.
-- Leo Tolstoy, "The Kingdom of God is Within You"
On Tue, 21 Feb 2006 14:24:25 +1100, David Trudgett
<······@zeta.org.au.nospamplease> wrote:
>opposed to what's cute, or expedient, the *natural* reaction to (WHEN
>LINE) is "when line what?" On the other hand, (UNLESS (NULL LINE)) is
That *natural* reaction is natural only to those who aren't yet fluent
in CL. To those who are fluent, the more verbose form causes them to
pause briefly to ponder the verbosity and whether they should fix it.
That makes the whole program harder to read and understand, because
the time spent on such incidental ponderings distracts the attention
from the meaning of the program.
Just as the parentheses disappear after you get used to them, forms
such as (when line ...) become patterns to recognize at a glance
without consuming any time decoding them.
It's important to follow conventions to make code clearer to everyone.
By making it possible for people to recognize patterns in code at a
glance, you make it possible for them to read and understand a program
many times faster than they otherwise could.
Those who aren't yet fluent in CL are going to take longer to read the
program no matter what you do. Instead of catering to their lack of
fluency, it's better to totally immerse them in the real CL and
encourage them to spend a lot of time with it to become fluent.
David Trudgett wrote:
> Kenny Tilton <·············@nyc.rr.com> writes:
>
>
>>David Trudgett wrote:
>>
>>>Instead of constructs like:
>>>
>>>
>>>> (if (not (null line))
>>>> (progn (funcall per-line-fn line)
>>>> (process per-line-fn stream)))))
>>>
>>>I usually prefer something like:
>>> (unless (null line)
>>> (do x)
>>> (do y))
>>>Saves a double negative and an ugly progn.
>>
>>Wow. You argue against double negatives and then code (unless (null...? :)
>>
>>Would
>>
>> "(when line
>> (do x)
>> (fo y)..."
>>
>>....kill anybody?
>
>
> Hi Kenny,
>
> (unless (null xyz)) is a single negative, Kenny. :-) Can I help it if
> unless contains within itself a negative idea? ;-)
I just remembered a programming proverb I once read: do not use
negatives in conditionals period. So never mind double negatives, this
did not want to see negatives at all. The author felt it made the code
hard to understand since people are not logic machines.
I agree and follow that rule myself, an exception being an if where the
false branch is a one-liner and the true branch a ten-liner or anything
substantial:
(if (not <condition)
(one-liner)
(big
(long
(etc etc))))
>
> Your suggestion is OK if you like that sort of thing, but I have a
> natural aversion to using arbitrary values as booleans
heh-heh, your bug is my feature.
kenny
From: David Trudgett
Subject: Re: lisp idiom for processing each line in a file?
Date:
Message-ID: <m3irr9za06.fsf@rr.trudgett>
Kenny Tilton <·············@nyc.rr.com> writes:
> I just remembered a programming proverb I once read: do not use
> negatives in conditionals period. So never mind double negatives, this
> did not want to see negatives at all. The author felt it made the code
> hard to understand since people are not logic machines.
You're killing me... ;-) (it's unethical)
>
> I agree and follow that rule myself, an exception being an if where
> the false branch is a one-liner and the true branch a ten-liner or
> anything substantial:
That's true. There are good reasons to use things like UNLESS. That's
why they would be in the language... :-)
I agree that you have to be careful about causing confusion in those
poor human readers.
>
> (if (not <condition)
> (one-liner)
> (big
> (long
> (etc etc))))
>
>
>
>> Your suggestion is OK if you like that sort of thing, but I have a
>> natural aversion to using arbitrary values as booleans
>
> heh-heh, your bug is my feature.
Perhaps it's a featureful bug.
David
--
David Trudgett
http://www.zeta.org.au/~wpower/
Today is the CAR of the CDR of your life.
-- JXM