From: Steven E. Harris
Subject: Reviving old discussion about read and recursive-p
Date:
Message-ID: <ufxdvkmj6.fsf@torus.sehlabs.com>
I'm puzzling over the relationship between functions `read',
`read-preserving-whitespace', and their recursive-p argument. Several
detailed but confusing discussions precede this one:
o RECURSIVE-P
July 27, 2002
http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/be0944fd7cfbdd76
o READ-PRESERVING-WHITESPACE in recursive calls
October 19, 2003
http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/b95149ff1732551b
I've read these threads repeatedly and don't find a satisfactory
resolution in either. On top of that, my implementation of choice --
CLISP -- defies my interpretation.
Section 23.1.3.2 of the HyperSpec discusses "The RECURSIVE-P argument".�
I read its second item consider whitespace alteration and can restate
my own interpretation of its intent as follows:
,----[ My interpretation ]
| When the recursive-p argument to `read' or `read-preserving-whitespace'
| is non-nil, that operation surrenders its whitespace-handling behavior
| to whichever operation had been invoked at top-level (supplying nil to
| the recursive-p argument).
`----
Interpreting the significance of the example shown (a reader macro for
the single quote character), I'll add:
,----[ My interpretation ]
| If the top-level call was to `read', then a nested call to `read' with
| recursive-p non-nil would /not/ preserve whitespace, but if the
| top-level call was to `read-preserving-whitespace', then a netsted call
| to `read' with recursive-p non-nil /would/ preserve whitespace.
`----
Neither of these two interpretations agree with the oft-quoted text from
the documentation for `read-preserving-whitespace':
,----[ HyperSpec on read-preserving-whitespace ]
| read-preserving-whitespace is exactly like read when the recursive-p
| argument to read-preserving-whitespace is true.
`----
I read that to say that `read-preserving-whitespace' with recursive-p
non-nil is indistinguishable from `read', suggesting that such a call
will /not/ preserve whitespace.
When I try to use the reader macro example from HyperSpec section
23.1.3.2 with CLISP, it does not produce the results one would expect
from the discussion there (using "^" rather than "'"):
,----
| CL-USER> (lisp-implementation-version)
| "2.46 (2008-07-02) (built on reini [10.0.0.5])"
| CL-USER> (set-macro-character #\^
| #'(lambda (stream char)
| (declare (ignore char))
| (list 'quote (read stream t nil t))))
|
| T
| CL-USER> (with-input-from-string (s "^foo ")
| (values (read s t :eof)
| (read-char s nil :eof)))
| 'FOO
| :EOF
| CL-USER> (with-input-from-string (s "^foo ")
| (values (read-preserving-whitespace s t :eof)
| (read-char s nil :eof)))
| 'FOO
| :EOF
`----
The HyperSpec's discussion of the example suggests that the second
attempt above should have preserved the space for `read-char' to find.
Repeating an experiment from yet another old thread on this subject, one
can see that CLISP only preserves whitespace with
`read-preserving-whitespace' when recursive-p is nil:
,----
| CL-USER> (with-input-from-string (s "foo bar")
| (read s t t nil) (read-char s))
| #\b
| CL-USER> (with-input-from-string (s "foo bar")
| (read s t t t) (read-char s))
| #\b
| CL-USER> (with-input-from-string (s "foo bar")
| (read-preserving-whitespace s t t nil) (read-char s))
| #\Space
| CL-USER> (with-input-from-string (s "foo bar")
| (read-preserving-whitespace s t t t) (read-char s))
| #\b
`----
That makes CLISP operate more like the documented behavior for
`read-preserving-whitespace', but puts it in conflict with the
discussion in HyperSpec section 23.1.3.2.
I don't understand the usefulness of the behavior as implemented
there. It suggests that nested (non-top-level) calls to
`read-preserving-whitespace' can't preserve whitespace. I'd expect that
an author usually knows when one of his calls is nested, so why would
one bother calling `read-preserving-whitespace' with recursive-p
non-nil? Is the idea there to support a non-constant recursive-p
argument, where one would be saying, "If I'm invoked at top-level, I'll
try to preserve whitespace, but will accept not doing so if I'm
nested."?
Please advise.
Footnotes:
� http://www.lispworks.com/documentation/HyperSpec/Body/23_acb.htm
--
Steven E. Harris
From: Pascal J. Bourguignon
Subject: Re: Reviving old discussion about read and recursive-p
Date:
Message-ID: <871vpeg2m5.fsf@galatea.local>
"Steven E. Harris" <···@panix.com> writes:
> I'm puzzling over the relationship between functions `read',
> `read-preserving-whitespace', and their recursive-p argument. Several
> detailed but confusing discussions precede this one:
>
> o RECURSIVE-P
> July 27, 2002
> http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/be0944fd7cfbdd76
>
> o READ-PRESERVING-WHITESPACE in recursive calls
> October 19, 2003
> http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/b95149ff1732551b
>
> I've read these threads repeatedly and don't find a satisfactory
> resolution in either. On top of that, my implementation of choice --
> CLISP -- defies my interpretation.
>
> Section 23.1.3.2 of the HyperSpec discusses "The RECURSIVE-P argument".�
> I read its second item consider whitespace alteration and can restate
> my own interpretation of its intent as follows:
>
> ,----[ My interpretation ]
> | When the recursive-p argument to `read' or `read-preserving-whitespace'
> | is non-nil, that operation surrenders its whitespace-handling behavior
> | to whichever operation had been invoked at top-level (supplying nil to
> | the recursive-p argument).
> `----
Correct.
> Interpreting the significance of the example shown (a reader macro for
> the single quote character), I'll add:
>
> ,----[ My interpretation ]
> | If the top-level call was to `read', then a nested call to `read' with
> | recursive-p non-nil would /not/ preserve whitespace, but if the
> | top-level call was to `read-preserving-whitespace', then a netsted call
> | to `read' with recursive-p non-nil /would/ preserve whitespace.
> `----
Correct.
> Neither of these two interpretations agree with the oft-quoted text from
> the documentation for `read-preserving-whitespace':
Sorry, I don't see two interpretations here. Only one. You said the
same thing twice.
> ,----[ HyperSpec on read-preserving-whitespace ]
> | read-preserving-whitespace is exactly like read when the recursive-p
> | argument to read-preserving-whitespace is true.
> `----
There's no disagreement. Indeed, both READ and
READ-PRESERVING-WHITESPACE surrender the decision of preserving
whitespaces to the toplevel call (actually the nearest call with
RECURSIVE-P set to NIL) when RECURSIVE-P is true.
> I read that to say that `read-preserving-whitespace' with recursive-p
> non-nil is indistinguishable from `read', suggesting that such a call
> will /not/ preserve whitespace.
No, it doesn't suggest such a thing. It says that preserving
whitespaces will depend on the toplevel call (actually the nearest
call with RECURSIVE-P set to NIL).
> When I try to use the reader macro example from HyperSpec section
> 23.1.3.2 with CLISP, it does not produce the results one would expect
> from the discussion there (using "^" rather than "'"):
>
> ,----
> | CL-USER> (lisp-implementation-version)
> | "2.46 (2008-07-02) (built on reini [10.0.0.5])"
> | CL-USER> (set-macro-character #\^
> | #'(lambda (stream char)
> | (declare (ignore char))
> | (list 'quote (read stream t nil t))))
> |
> | T
> | CL-USER> (with-input-from-string (s "^foo ")
> | (values (read s t :eof)
> | (read-char s nil :eof)))
> | 'FOO
> | :EOF
> | CL-USER> (with-input-from-string (s "^foo ")
> | (values (read-preserving-whitespace s t :eof)
> | (read-char s nil :eof)))
> | 'FOO
> | :EOF
> `----
These are the results I expected though...
> The HyperSpec's discussion of the example suggests that the second
> attempt above should have preserved the space for `read-char' to find.
>
> Repeating an experiment from yet another old thread on this subject, one
> can see that CLISP only preserves whitespace with
> `read-preserving-whitespace' when recursive-p is nil:
>
> ,----
> | CL-USER> (with-input-from-string (s "foo bar")
> | (read s t t nil) (read-char s))
> | #\b
> | CL-USER> (with-input-from-string (s "foo bar")
> | (read s t t t) (read-char s))
> | #\b
> | CL-USER> (with-input-from-string (s "foo bar")
> | (read-preserving-whitespace s t t nil) (read-char s))
> | #\Space
> | CL-USER> (with-input-from-string (s "foo bar")
> | (read-preserving-whitespace s t t t) (read-char s))
> | #\b
> `----
>
> That makes CLISP operate more like the documented behavior for
> `read-preserving-whitespace', but puts it in conflict with the
> discussion in HyperSpec section 23.1.3.2.
I don't see any conflict.
> I don't understand the usefulness of the behavior as implemented
> there. It suggests that nested (non-top-level) calls to
> `read-preserving-whitespace' can't preserve whitespace.
No, it suggest that that call to READ-PRESERVING-WHITESPACE will do
according to the toplevel call (actually the nearest call with
RECURSIVE-P set to NIL).
> I'd expect that
> an author usually knows when one of his calls is nested, so why would
> one bother calling `read-preserving-whitespace' with recursive-p
> non-nil?
That's exactly the point, to tell it that it's a nested call!
> Is the idea there to support a non-constant recursive-p
> argument, where one would be saying, "If I'm invoked at top-level, I'll
> try to preserve whitespace, but will accept not doing so if I'm
> nested."?
No, that's not motivating the reaseon.
> Please advise.
--
__Pascal Bourguignon__
From: Steven E. Harris
Subject: Re: Reviving old discussion about read and recursive-p
Date:
Message-ID: <uab42lmx1.fsf@torus.sehlabs.com>
···@informatimago.com (Pascal J. Bourguignon) writes:
> Sorry, I don't see two interpretations here. Only one. You said the
> same thing twice.
Yes, I agree. By "two interpretations", I should have said, "two
phrasings of the same assertion".
> There's no disagreement. Indeed, both READ and
> READ-PRESERVING-WHITESPACE surrender the decision of preserving
> whitespaces to the toplevel call (actually the nearest call with
> RECURSIVE-P set to NIL) when RECURSIVE-P is true.
I can't find any documentation about `read' that confirms this
behavior. The HyperSpec describes what `read' does, the describes how
`read-preserving-whitespace' is different, but this detail about a
recursive call to `read' respecting whitespace treatment.
> No, it doesn't suggest such a thing. It says that preserving
> whitespaces will depend on the toplevel call (actually the nearest
> call with RECURSIVE-P set to NIL).
But where in the HyperSpec is this respect for the top-level call
documented? I don't see anything there that suggests that `read' behaves
contextually.
>> ,----
>> | CL-USER> (lisp-implementation-version)
>> | "2.46 (2008-07-02) (built on reini [10.0.0.5])"
>> | CL-USER> (set-macro-character #\^
>> | #'(lambda (stream char)
>> | (declare (ignore char))
>> | (list 'quote (read stream t nil t))))
>> |
>> | T
>> | CL-USER> (with-input-from-string (s "^foo ")
>> | (values (read s t :eof)
>> | (read-char s nil :eof)))
>> | 'FOO
>> | :EOF
>> | CL-USER> (with-input-from-string (s "^foo ")
>> | (values (read-preserving-whitespace s t :eof)
>> | (read-char s nil :eof)))
>> | 'FOO
>> | :EOF
>> `----
> These are the results I expected though...
Really? In the second case, we call `read-preserving-whitespace' with
recursive-p as nil, and it invokes the reader macro for '^' which in
turn calls `read' with recursive-p set to non-nil, but the space after
the last 'o' in "foo" is eaten before the subsequent call to `read-char'
can see it. Note that `read-char' returned :EOF rather than a space.
Here, given that `read' is not a top-level call, if it is supposed to
act contextually, it should have respected the top-level call requesting
that whitespace be preserved.
By the way, I found an implementation file from Spice Lisp� on the Web
when searching for some evidence of what might be happening
internally. Note that there, `read-preserving-whitespace' always
preserves whitespace, and `read' consumes whitespace only when
recursive-p is true. That definitely doesn't agree with the contextual
model we're describing here. Is that implementation wrong?
Footnotes:
� http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/clisp/OldFiles/src/16/code/reader.lisp
--
Steven E. Harris
From: Steven E. Harris
Subject: Re: Reviving old discussion about read and recursive-p
Date:
Message-ID: <u63eqlmnm.fsf@torus.sehlabs.com>
"Steven E. Harris" <···@panix.com> writes:
> The HyperSpec describes what `read' does, the describes how
> `read-preserving-whitespace' is different, but this detail about a
> recursive call to `read' respecting whitespace treatment.
Finishing:
... but this detail about a recursive call to `read' respecting
whitespace treatment is not described explicitly; it's only hinted at
obliquely in section 23.1.3.2.
Perhaps this should be good enough:
,----[ HyperSpec section 23.1.3.2 ]
| A recursive call does not alter whether the reading process is to
| preserve whitespace[2] or not (as determined by whether the outermost
| call was to read or read-preserving-whitespace).
`----
If that's the full statement of this feature, then just reading the page
on `read' and `read-preserving-whitespace' is not sufficient to
understand how this is supposed to work.
--
Steven E. Harris
From: Pascal J. Bourguignon
Subject: Re: Reviving old discussion about read and recursive-p
Date:
Message-ID: <87d48yekd7.fsf@galatea.local>
"Steven E. Harris" <···@panix.com> writes:
> "Steven E. Harris" <···@panix.com> writes:
>
>> The HyperSpec describes what `read' does, the describes how
>> `read-preserving-whitespace' is different, but this detail about a
>> recursive call to `read' respecting whitespace treatment.
>
> Finishing:
>
> ... but this detail about a recursive call to `read' respecting
> whitespace treatment is not described explicitly; it's only hinted at
> obliquely in section 23.1.3.2.
>
> Perhaps this should be good enough:
>
> ,----[ HyperSpec section 23.1.3.2 ]
> | A recursive call does not alter whether the reading process is to
> | preserve whitespace[2] or not (as determined by whether the outermost
> | call was to read or read-preserving-whitespace).
> `----
>
> If that's the full statement of this feature, then just reading the page
> on `read' and `read-preserving-whitespace' is not sufficient to
> understand how this is supposed to work.
Indeed, CLHS pages are not self sufficient. The authors of the
standard tried to avoid any redundancy. If an element of
specification is written somewhere, it isn't replicated anywhere else.
So you have to read CLHS in whole.
--
__Pascal Bourguignon__
From: Steven E. Harris
Subject: Re: Reviving old discussion about read and recursive-p
Date:
Message-ID: <uws71jfjd.fsf@torus.sehlabs.com>
"Steven E. Harris" <···@panix.com> writes:
> Really? In the second case, we call `read-preserving-whitespace' with
> recursive-p as nil, and it invokes the reader macro for '^' which in
> turn calls `read' with recursive-p set to non-nil, but the space after
> the last 'o' in "foo" is eaten before the subsequent call to `read-char'
> can see it. Note that `read-char' returned :EOF rather than a space.
>
> Here, given that `read' is not a top-level call, if it is supposed to
> act contextually, it should have respected the top-level call requesting
> that whitespace be preserved.
I'd appreciate some comment on this observed behavior with CLISP. Does
my example demonstrate a lack of conformance?
--
Steven E. Harris