I'm somewhat confused by the way some implementations of the Lisp
pretty-printer behave. Maybe I just don't understand the pretty
printer.
This whole exercise started when I tried to investigate some (perceived)
bugs in CLisp. In the course of this, I found three small snippets
which produce behaviour surprising to me. Throughout, CMU CL, SBCL, ECL
and Allegro behave consistently, both with respect to each other and my
expectations. However, CLisp, LispWorks and ABCL all surprise me. (GCL
just hasn't implemented the pretty printer at all.)
The calls to FRESH-LINE are there for the benefit of CMU CL and SBCL;
without them, the fact that the prompt `* ' has been written to
*STANDARD-OUTPUT* without a newline (which came from echoing
*STANDARD-INPUT*) throws off the pretty-printer's idea of the initial
cursor position.
(progn
(fresh-line)
(pprint-logical-block (*standard-output* nil :prefix "hello ")
(princ (format nil "~A" "there"))
(terpri)))
What I expected:
hello there
=> NIL
I get this from CMU CL, SBCL, ECL, ABCL, Allegro and LispWorks. However CLisp
gives me:
hello there
=> NIL
I'm pretty sure that this is just a CLisp bug: some stuff is probably
being kept in special variables rather than associated with the PP
stream. I get the expected result if I don't use FORMAT, or if the
FORMAT control string just contains "there", rather than using ~A.
Second case.
(progn
(fresh-line)
(pprint-logical-block (*standard-output* nil)
(pprint-indent :block 5)
(princ "hello")
(pprint-newline :mandatory)
(princ "there")
(terpri)))
What I expected:
hello
there
=> NIL
I get this from CMU CL, SBCL, ECL and Allegro. LispWorks and ABCL both
give me
hellothere
=> NIL
which is very surprising. In particular, I thought (pprint-newline
:mandatory) meant that it had to insert a line break there.
CLisp gives me
hello
there
=> NIL
which raises the question of whether PPRINT-INDENT is meant to produce
output immediately or just for subsequent lines of the block.
While investigating LispWorks's behaviour, I tried the following.
(progn
(fresh-line)
(pprint-logical-block (*standard-output* nil)
(pprint-indent :block 5)
(format t ········@_there~%")))
This does the same as the previous snippet, except on LispWorks, which
gives me
hello
there
=> NIL
In particular, ABCL still fails to break the line at all. Now I don't
understand why ··@_ is different from (pprint-newline :mandatory).
Anyone capable of easing my confusion? Is the pretty-printer a
well-known source of bugs and/or incompatible interpretations of the
standard?
-- [mdw]
Mark Wooding wrote:
> I'm somewhat confused by the way some implementations of the Lisp
> pretty-printer behave. Maybe I just don't understand the pretty
> printer.
A familiar feeling.
> CLisp gives me
>
> hello
> there
>
> => NIL
>
> which raises the question of whether PPRINT-INDENT is meant to produce
> output immediately or just for subsequent lines of the block.
The standard states:
"Changes in indentation caused by pprint-indent do not take effect
until after the next line break."
I have no good explanation for clisp's behavior (other than a possible
bug). In particular, the following works as expected:
(progn
(terpri)
(pprint-logical-block (nil nil)
(write-string "hello")
(pprint-indent :block 5)
(pprint-newline :mandatory)
(write-string "there")))
clisp output:
hello
there
> While investigating LispWorks's behaviour, I tried the following.
>
> (progn
> (fresh-line)
> (pprint-logical-block (*standard-output* nil)
> (pprint-indent :block 5)
> (format t ········@_there~%")))
Note that t designates *terminal-io*, not *standard-output*.
Michael
Michael Weber <·········@foldr.org> wrote:
> "Changes in indentation caused by pprint-indent do not take effect
> until after the next line break."
Right, good. So that's another CLisp bug, presumably.
> Note that t designates *terminal-io*, not *standard-output*.
Really? CLHS 22.3:
: format sends the output to destination. [...] If destination is t,
: the output is sent to standard output.
For (almost?) all other output functions, specifying the destination as
NIL means *STANDARD-OUTPUT* and T means *TERMINAL-IO*; but since NIL as
a format destination means `return me a string', I guess they thought
the inconsistency was worth it.
(I think I'd rather have seen a destination of :STRING or something to
indicate that a string was wanted, but I suspect FORMAT predates
keywords.)
-- [mdw]
From: Kent M Pitman
Subject: Re: Pretty printing experts, pretty please?
Date:
Message-ID: <uej92pr83.fsf@nhplace.com>
Mark Wooding <···@distorted.org.uk> writes:
> Michael Weber <·········@foldr.org> wrote:
> [...]
> > Note that t designates *terminal-io*, not *standard-output*.
>
> Really?
Heh. T _is_ a stream designator for terminal I/O, but FORMAT does not
take a stream designator as its first argument. :) Mostly all of the
other I/O functions do take stream designators. (Which means they do
not accept NIL as an argument to do what a lot of people sometimes
request but would be messy to implement.)
> CLHS 22.3:
>
> : format sends the output to destination. [...] If destination is t,
> : the output is sent to standard output.
Right.
> For (almost?) all other output functions, specifying the destination as
> NIL means *STANDARD-OUTPUT* and T means *TERMINAL-IO*; but since NIL as
> a format destination means `return me a string', I guess they thought
> the inconsistency was worth it.
I think that was the argument, yes.
Random walk through historical recollections follows.
Understand that in MACLISP, the NIL = standard output and T = terminal
I/O convention arose on systems [PDP-10] where you might really want
to use the real terminal (there was no window system). There was a
long and complicated transition in MACLISP from Old I/O (which was not
stream-based) to New I/O (which was, but which didn't have
programmable streams, only the terminal and files and very much later
Software File Arrays (SFA's) which were finally more like streams. But
the _real_ traditional function of the NIL/T distinction was to grandfather
in the Old I/O mechanism (which defaulted the stream and hence fit better
in the model of thinking of using standard input and standard output) and
the New I/O mechanism, with NIL (or omitted) selecting the old way and
T selecting the new way. But T was really just shorthand for either TYI
on input or TYO on output. (The problem was that Maclisp had no thing
which was both TYI and TYO, so T was able to pun between them... or you
could use (status ttycons tyi) to get tyo or (status ttycons tyo) to get
to tyi... In general, an input stream was trying to have a ttycons'd other
as a way of secretly allowing bidirectional streams. [Note: TYI and TYO also
were the names of READ-CHAR and WRITE-CHAR in MACLISP, when taken as function
names rather than variable names... so don't get confused there.]
Then on the LispM they tried to allow all of this even though they'd never
had Old I/O and they instead added standard-input and standard-output (later
*standard-input* and *standard-output* when CL arrived). Normally on the
LispM, *standard-input* and *standard-output* were set up to be synonym
streams for *terminal-io*, and because the LispM had windowing, every window
came with *terminal-io* bound to the window itself and *standard-input* and
*standard-output* bound to synonym streams for the window, which you could
rebind to other things as needed (but you were meant to leave *terminal-io*
alone). But this made *terminal-io* a rational thing to output to at least
a bit more often than it now is (since some implementations like LispWorks
bind *terminal-io* to what the Lisp Machine would have called the "cold
load stream"--a pre-window system bootstrap stream on the LispM but in
LispWorks a stream to the DOS or CommandPrompt window that is creating the
Lisp and that ordinarily the user would never see... so outputing to it is
mysterious... at least on the LispM if something did I/O to the cold load,
the system would figure that no one was seeing it and would arrange to bug
you about looking at it in a pop-up window instead of having it fall silent).
But FORMAT was not a Maclisp function originally. It was probably added
backported to Maclisp later in the day with reduced functionality but mostly
people just used lots of princ's, prin1's, tyo's, and terpri's. (Trivia:
terpri returns NIL in MACLISP and PRINT/PRIN1/TYO all returned T. So if you
were putting debugging info in conditionals, you'd want to do
(AND (NOT (TERPRI)) (PRIN1 X) (TYO 32.) X)
for example... I used to love those (NOT (TERPRI)) things since it made it
look like you weren't going to TERPRI... Btw, in case anyone is curious,
the reason Maclisp did not return the argument was that it would require
heap-consing integers in number-declared code, and they wanted not to create
a lot of unnecessary garbage... not that the GC was all that slow in a
no-virtual-memory system, but it was still extra space, even temporarily,
given MACLISP had a fixed upper limit on storage on the PDP-10 of 256K words
(about 1.25 megabytes, since those were 36 bit words). Ah, the good old days,
when life was so much simpler. Amazing we got any work done at all, given
that the chief programming tool was the shoehorn.
To be honest, I actually think T as the argument for FORMAT _might_
have been a misunderstanding--I'm not sure, though. I think a lot of
people thought PRIN1 of something to T actually did mean print to
standard output... I'm not sure which came first--whether people
thought that because FORMAT did it or FORMAT did it because it didn't
understand the convention. But you're right that the NIL case was
pretty much "don't do output" and kind of had to be NIL, so it was not
open to negotiation. And, really, no one does output directly to
*TERMINAL-IO*; the function of *TERMINAL-IO* (at least on the LispM)
was more of a bootstrapping issue for a window to decide if you wanted
to do raw output to the window or to do I/O calls efficiently).
> (I think I'd rather have seen a destination of :STRING or something to
> indicate that a string was wanted, but I suspect FORMAT predates
> keywords.)
Keywords were there in 1981 as illustrated here:
http://www.unlambda.com/lmman/lmman_1.html
Search for "(:)". My older LispM manuals are not at the location
I'm posting from so I couldn't search hardcopy to see if they were
there in, say, 1979 when the black lispm was out, but I think that
version used names like tv-beep rather than tv:beep later, so you
might be right that format predates that.
:string would have looked ugly though. Frankly, I never understood
why they didn't just have (format-string ...) which would have been
fewer chars that (format :string ...). But the REAL reason to do it
as a separate function is the dataflow in/out. You've got to implement
it totally differently inside for the string case and the non-string case,
and the caller has to be equally aware of the difference because one
is for-value and one is for-side-effect.
Kent M Pitman <······@nhplace.com> wrote:
> Random walk through historical recollections follows.
[Snipped, somewhat regretfully.]
Thank you for this. I find historical perspectives on computing to be
both useful and fascinating, so your recollections are most valuable.
> :string would have looked ugly though.
True.
> Frankly, I never understood why they didn't just have (format-string
> ...) which would have been fewer chars that (format :string ...). But
> the REAL reason to do it as a separate function is the dataflow
> in/out. You've got to implement it totally differently inside for the
> string case and the non-string case, and the caller has to be equally
> aware of the difference because one is for-value and one is
> for-side-effect.
Yes, of course. I'd sort of noticed that before, but not really thought
too deeply about how strange it actually is. I suppose it would make a
bit more sense if functions constructed by FORMATTER had the same
interface, but they just accept a raw stream.
Besides, by the looks of things FORMATTER was introduced as part of the
pretty-printer, so this would have been at best a retrospective
justification, and requiring FORMATTER functions to obey FORMAT's
strange interface rules would presumably have been seen as contrary to
the former's performance goals.
-- [mdw]
P� Fri, 18 Apr 2008 21:17:57 +0200, skrev Michael Weber
<·········@foldr.org>:
>
> (progn
> (terpri)
> (pprint-logical-block (nil nil)
> (write-string "hello")
> (pprint-indent :block 5)
> (pprint-newline :mandatory)
> (write-string "there")))
>
> clisp output:
> hello
> there
>
Just curious. Why use the pprint primitives instead of it's format
operators?
For the record (you have probably figured this out) the pprint spec is
available at
ftp://publications.ai.mit.edu/ai-publications/pdf/AIM-1102a.pdf
--------------
John Thingstad
John Thingstad <·······@online.no> wrote:
> Just curious. Why use the pprint primitives instead of it's format
> operators?
Because I was trying to isolate bugs (or misunderstandings), and didn't
want FORMAT potentially confusing matters. Since LispWorks ··@_ appears
to behave differently from (pprint-newline :mandatory), I think I was
justified. ;-)
> ftp://publications.ai.mit.edu/ai-publications/pdf/AIM-1102a.pdf
I had seen that (a while) before, but thanks for the reminder.
-- [mdw]
Mark Wooding <···@distorted.org.uk> wrote:
+---------------
| John Thingstad <·······@online.no> wrote:
| > ftp://publications.ai.mit.edu/ai-publications/pdf/AIM-1102a.pdf
|
| I had seen that (a while) before, but thanks for the reminder.
+---------------
You might also want to look at Chapter 1 of one of Waters's
earlier papers (August 1993):
http://www.merl.com/reports/docs/TR93-17.pdf
http://www.merl.com/papers/TR93-17/
Richard Waters, "Some Useful Lisp Algorithms: Part 2"
Chapter 1 "Using the New Common Lisp Pretty Printer"
Chapter 2 "Macroexpand-All: An Example of a Simple Lisp Code Walker"
Chapter 3 "To NReverse When Consing a List or By Pointer
Manipulation, To Avoid It; That Is the Question"
The blurb for Chapter 1 [in the 2nd URL] says:
Chapter 1 "Using the New Common Lisp Pretty Printer" explains
how the pretty printing facilities that have been adopted as
part of the forthcoming Common Lisp standard can be used to gain
detailed control over the printing of lists. As an example, it
shows how the pretty printer can be used to print a subset of
Lisp as Pascal.
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607