From: Damien Kick
Subject: read, print, and macro characters
Date: 
Message-ID: <8y79an63.fsf@email.mot.com>
I'm wondering acout reading/printing in the presence of modified
readtable, i.e. one has used MAKE-DISPATCH-MACRO-CHARACTER to add a
new family of macros.  For example, using ACL 6.2.

    PG-USER(5): '!symbol-name
    !SYMBOL-NAME
    PG-USER(6): '#symbol-name
    Error: Non-list following #S: YMBOL-NAME [file position = 13]
    [condition type: READER-ERROR]
    [changing package from "COMMON-GRAPHICS-USER" to "PLAYGROUND-USER"]
    PG-USER(7): '\#symbol-name
    |#SYMBOL-NAME|
    PG-USER(8): (make-dispatch-macro-character #\!)
    T
    PG-USER(9): (defun |!` READER| (stream char)
      (declare (ignore char))
      (let ((*readtable* (copy-readtable *readtable*)))
        (setf (readtable-case *readtable*) :preserve)
        (funcall (get-macro-character #\`)
                 stream
                 #\`)))
    |!` READER|
    PG-USER(10): (set-dispatch-macro-character #\! #\` #'|!` READER|)
    T
    PG-USER(11): '!symbol-name
    Error: No dispatch function defined for #\s.
    [condition type: SIMPLE-ERROR]
    PG-USER(12): '|!symbol-name|
    |!symbol-name|
    PG-USER(13): *
    |!symbol-name|
    PG-USER(14): 

After adding "!`" as a macro character, "!symbol-name" is printed
escaped.  This seems to me to be a good thing.  However, with CMUCL, I
find the following after adding the same macro character sequence.

    * '\!symbol-name
    !SYMBOL-NAME
    * *
    !SYMBOL-NAME
    * 

Which is correct?

From: Steven M. Haflich
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <MnpCd.6512$5R.2759@newssvr21.news.prodigy.com>
Damien Kick wrote:

> I'm wondering acout reading/printing in the presence of modified
> readtable, i.e. one has used MAKE-DISPATCH-MACRO-CHARACTER to add a
> new family of macros.  For example, using ACL 6.2.

>     PG-USER(12): '|!symbol-name|
>     |!symbol-name|
>     PG-USER(13): *
>     |!symbol-name|
>     PG-USER(14): 
> 
> After adding "!`" as a macro character, "!symbol-name" is printed
> escaped.  This seems to me to be a good thing.  However, with CMUCL, I
> find the following after adding the same macro character sequence.
> 
>     * '\!symbol-name
>     !SYMBOL-NAME
>     * *
>     !SYMBOL-NAME
>     * 

Both are correct!  Your ACL and CMUCL examples are quite different,
and neither relfects on the modified readtable.

The |!symbol-name| you passed to ACL is a symbol with a flock of
lower-case characters in its name.  The printer is required to
escape it.  The symbol you passed to CMUCL is a symbol with no
lower-case or other irregular characters in its name.  The printer
correctly prints it without escaping.

In neither case is the #\! affecting how the symbol prints, and
there is nothing in the ANS that specifies or even suggests that
changes to the readtable (other than readtable-case) affect what
the printer produces.  Now, it might have been nice if the language
had been defined so that the printer would produce output rereadable
under the current readtable, but can you spell `unkomputable'?
From: Damien Kick
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <oeg57ya6.fsf@email.mot.com>
"Steven M. Haflich" <·················@alum.mit.edu> writes:

> Damien Kick wrote:
> 
> > I'm wondering acout reading/printing in the presence of modified
> > readtable, [...]
> 
> The |!symbol-name| you passed to ACL is a symbol with a flock of
> lower-case characters in its name.  [...]

Doh!  I meant for that to be "\\!symbol-name".

> In neither case is the #\! affecting how the symbol prints, and
> there is nothing in the ANS that specifies or even suggests that
> changes to the readtable (other than readtable-case) affect what
> the printer produces.  Now, it might have been nice if the language
> had been defined so that the printer would produce output rereadable
> under the current readtable, but can you spell `unkomputable'?

I would think that it would be beneficial to have both of the
following work without error.

    PG-USER(16): (read-from-string (prin1-to-string '\#symbol))
    |#SYMBOL|
    9
    PG-USER(17): (read-from-string (prin1-to-string '\!symbol))
    Error: No dispatch function defined for #\S.
    [condition type: SIMPLE-ERROR]
    PG-USER(18): 

Why would that be unkomputable?  Instead of hardcoding special
treatment for #\# in the printer, just test if GET-MACRO-CHARACTER
returned something that indicated that it was a character for which
one needed to use an escape when printing a symbol name.  Something
like that doesn't seem to be that far fetched to me.  What am I not
considering that would make this untenable?

But regardless of whether or not it would be possible, does that fact
that the spec doesn't call for special printing of newly annointed
macro characters mean that one should be wary about doing so?  I've
been toying with the idea of using #\! as a dispatching macro
character but it makes me a little uncomfortable that the second
READ-FROM-STRING example signals an error when this is the case.  Is
this something about which I really shouldn't be worried?  It makes me
think that even though the standard reserves #\! for use by the
programmer
<http://www.franz.com/support/documentation/6.2/ansicl/subsecti/characte.htm>
one might not want to use it because it breaks other assumptions,
i.e. being able to print symbol names rereadably.
From: Damien Kick
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <zmzowwu2.fsf@email.mot.com>
Damien Kick <······@email.mot.com> writes:

> "Steven M. Haflich" <·················@alum.mit.edu> writes:
> 
> > In neither case is the #\! affecting how the symbol prints, and
> > there is nothing in the ANS that specifies or even suggests that
> > changes to the readtable (other than readtable-case) affect what
> > the printer produces.  Now, it might have been nice if the language
> > had been defined so that the printer would produce output rereadable
> > under the current readtable, but can you spell `unkomputable'?

Actually, in section 22.1.3.3, "Printing Symbols",
<http://www.franz.com/support/documentation/6.2/ansicl/subsubse/printin8.htm>,
one finds the following:

        When printing a symbol, the printer inserts enough single
        escape and/or multiple escape characters (backslashes and/or
        vertical-bars) so that if read were called with the same
        *readtable* and with *read-base* bound to the current output
        base, it would return the same symbol (if it is not apparently
        uninterned) or an uninterned symbol with the same print name
        (otherwise).

>     PG-USER(16): (read-from-string (prin1-to-string '\#symbol))
>     |#SYMBOL|
>     9
>     PG-USER(17): (read-from-string (prin1-to-string '\!symbol))
>     Error: No dispatch function defined for #\S.
>     [condition type: SIMPLE-ERROR]
>     PG-USER(18): 

Section 22.1.3.3 would suggest to me that one should expect that both
of the above uses of READ-FROM-STRING/PRIN1-TO-STRING should work.
Unfortuantely, 22.1.3.3 also tells us:

        For additional restrictions concerning characters with
        nonstandard syntax types in the current readtable, see the
        variable *print-readably*

So, if I understand everything correctly, making #\! be a dispatch
macro character is to make it have a nonstandard syntax type, even
though it is reserved for the programmer to use.  Section 22.4.27,
"*print-readably*",
<http://www.franz.com/support/documentation/6.2/ansicl/dictentr/print-re.htm>
tells us:

        Specifically, printing any object O1 produces a printed
        representation that, when seen by the Lisp reader while the
        standard readtable is in effect, will produce an object O2
        that is similar to O1.

It only specifically mentions the Lisp reader working with the
standard readtable.  But why does 22.1.3.3 even bother mentioning
PRINT/READ equivalence with "the same readtable" when 22.4.27 then
mentions only "the standard readtable"?  Presumably, since 22.1.3.3
only mentions 22.4.27 for nonstandard syntax types, this should be
read that changes to the current readtable that don't alter a
character's syntax types, i.e. with readtable-case, should exhibit
PRINT/READ equivalence.

> Why would that be unkomputable?  Instead of hardcoding special
> treatment for #\# in the printer, just test if GET-MACRO-CHARACTER
> returned something that indicated that it was a character for which
> one needed to use an escape when printing a symbol name.  Something
> like that doesn't seem to be that far fetched to me.  What am I not
> considering that would make this untenable?

    ;; Assuming something has been done to #\! in *readtable*
    (loop for char across (symbol-name '\!scrod-la-dod) do
          (cond ((eql (get-macro-character char (copy-readtable nil))
                      (get-macro-character char))
                 (princ char))
                (t
                 (princ #\\)
                 (princ char))))

I know this is too simplistic but doesn't it demonstrate that there
should be some way to notice that a character needs to be escaped
besides hardcoding recognition of a character's normal syntax type
into the printer?  Is the problem that not all of the default cases
should be escaped; i.e. someone has altered the syntax type of a
character with something other than the macro character functions:
MAKE-DISPATCH-MACRO-CHARACTER, etc.?
From: Steven M. Haflich
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <07MCd.7406$yV1.4542@newssvr14.news.prodigy.com>
Damien Kick wrote:
> Damien Kick <······@email.mot.com> writes:

>>"Steven M. Haflich" <·················@alum.mit.edu> writes:
>>
>>>In neither case is the #\! affecting how the symbol prints, and
>>>there is nothing in the ANS that specifies or even suggests that
>>>changes to the readtable (other than readtable-case) affect what
>>>the printer produces.  Now, it might have been nice if the language
>>>had been defined so that the printer would produce output rereadable
>>>under the current readtable, but can you spell `unkomputable'?
> 
> Actually, in section 22.1.3.3, "Printing Symbols",
> <http://www.franz.com/support/documentation/6.2/ansicl/subsubse/printin8.htm>,
> one finds the following:

Apparently I was wrong whether the ANS anywhere implies readtable
dispatches should affect the printer, but apparently neither you nor X3J13
thought seriously about the correct spelling for "unkomputable".

>         ... so that if read were called with the SAME *readtable* ...

But "same" is a term of art in the ANS.  See the Glossary, and the only
one of the three meanings for "same" that could apply here is meaning #2,
which is basically the eql test.  Unfortunately, readtables are
first-class objects with first-class-object eql identity, but nearly
everything about a readtable is dynamically modifiable -- readtable-case
as well as the dispatch and/or macro-dispatch for every character.  Things
that are the same(2) don't necessarily _always_ operate the same.

Why is Turing involved in this discussion?  Well, suppose I define a
readtable dispatch function that is sensitive to the day of the week...
Whether the print-time readtable is the "same" as the read-time readtable
when I decide to reread the printed file is hard for Alan to compute in
advance.  It depends when I wake up...

Seriously, I've put forth a comical extreme example here, but there are
very real and serious issues here if one believes the printer can figure
out what a readtable will do, even if one diallows my "same" argument
above.  Google Goedel to find out more.

It's clear that X3J13 didn't think through to the bottom of this issue.
Mea culpa -- I was a member...
From: Damien Kick
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <652bbkhn.fsf@email.mot.com>
"Steven M. Haflich" <·················@alum.mit.edu> writes:

> Damien Kick wrote:
> > Damien Kick <······@email.mot.com> writes:
> 
> >>"Steven M. Haflich" <·················@alum.mit.edu> writes:
> >>
> >>> [...] there is nothing in the ANS that specifies or even
> >>> suggests that changes to the readtable (other than
> >>> readtable-case) affect what the printer produces.  [...]
> > 
> > Actually, in section 22.1.3.3, "Printing Symbols", [...]
> 
> Apparently I was wrong whether the ANS anywhere implies readtable
> dispatches should affect the printer, but apparently neither you nor
> X3J13 thought seriously about the correct spelling for
> "unkomputable".

Well, I wouldn't dare put myself in the same company as X3J13 when
slipshod spelling is involved.

> >         ... so that if read were called with the SAME *readtable* ... 
> 
> But "same" is a term of art in the ANS.  [...] Things that are the
> same(2) don't necessarily _always_ operate the same.

Ah, one of those "depends on what the meaning of 'is' is" things...

> Why is Turing involved in this discussion?  [...]

Because Lisp isn't really all that special since one can do with C
anything that one can do with Lisp?

> [...] Well, suppose I define a readtable dispatch function that is
> sensitive to the day of the week...  [...] Seriously, I've put forth
> a comical extreme example here, but there are very real and serious
> issues here if one believes the printer can figure out what a
> readtable will do, even if one diallows my "same" argument above.

Okay, in the general case, this is true.  But for the specific example
of whether or not a #\! in a symbol name needs to be escaped; it
shouldn't be that difficult to figure, should it?  I mean, if a
character had already been included in the symbol name, then one
already knows that it doesn't do anything more than make up part of
the name of the symbol.  It need not be all of nothing; i.e. if one
can't figure out how to get all possible PRINT/READ pairings to be
equivalent then it need not be the case that none of them can be made
workable.  Of course, I also haven't tried to think about what such a
subset of potential modifications and a protocol for working within
that subset would be such that PRINTER/READER equivalence is
maintained.  Perhaps that is when one falls into the spelling trap.
From: Steven M. Haflich
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <TC2Dd.8561$5R.467@newssvr21.news.prodigy.com>
Damien Kick wrote:

>>[...] Well, suppose I define a readtable dispatch function that is
>>sensitive to the day of the week...  [...] Seriously, I've put forth
>>a comical extreme example here, but there are very real and serious
>>issues here if one believes the printer can figure out what a
>>readtable will do, even if one diallows my "same" argument above.
> 
> Okay, in the general case, this is true.  But for the specific example
> of whether or not a #\! in a symbol name needs to be escaped; it
> shouldn't be that difficult to figure, should it?  I mean, if a
> character had already been included in the symbol name, then one
> already knows that it doesn't do anything more than make up part of
> the name of the symbol.  It need not be all of nothing; i.e. if one
> can't figure out how to get all possible PRINT/READ pairings to be
> equivalent then it need not be the case that none of them can be made
> workable.  Of course, I also haven't tried to think about what such a
> subset of potential modifications and a protocol for working within
> that subset would be such that PRINTER/READER equivalence is
> maintained.  Perhaps that is when one falls into the spelling trap.

Suppose for the sake of argument we replace the incautious ANS wording
"same readtable" with "current readtable" implying the readtable will
not be changed before the rereading occurs.  Now, since no effective
procedure can reason about what an arbitrary procedure will do (the
Turing Theorem) that means the printer must escape any symbol name
component character whose readtable dispatch function has been altered
(either for leading characters or all characters, depending on
get-macro-character terminatingp).

That would _almost_ work, but what if the syntax of all the single-
and multiple-escape characters have been changed in the current
readtable and there are no escape characters available?
From: Damien Kick
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <d5wi1cgy.fsf@email.mot.com>
"Steven M. Haflich" <·················@alum.mit.edu> writes:

> Damien Kick wrote:
> 
> > Okay, in the general case, this is true.  But for the specific
> > example of whether or not a #\! in a symbol name needs to be
> > escaped; it shouldn't be that difficult to figure, should it?
> > [...]
> 
> Suppose for the sake of argument we replace the incautious ANS
> wording "same readtable" with "current readtable" implying the
> readtable will not be changed before the rereading occurs.  Now,
> [...] that means the printer must escape any symbol name component
> character whose readtable dispatch function has been altered (either
> for leading characters or all characters, depending on
> get-macro-character terminatingp).
> 
> That would _almost_ work, but what if the syntax of all the single-
> and multiple-escape characters have been changed in the current
> readtable and there are no escape characters available?

<nod> I hadn't considered this possibility; i.e. changing the syntax
of the escape characters.  However, I was also attempting to limit the
scope of this discussion from any "arbitrary procedure" to a limited
subset of potential modifications; specifically, assuming changes
having been made to #\! but I should also generalize this to be only
those characters which have been labeled by the spec in 2.1.4 as being
made available to the programmer: (#\! #\?  #\{ #\} #\[ #\]).  What if
one limited the allowable changes present in the "current readtable"
for which PRINT/READ equivalence was to be maintained to only these
characters; i.e. the changes are no longer completely arbitrary.  Or
even if the restriction was even stronger, one can only use these
reserved characters with MAKE-DISPATCH-MACRO-CHARACTER and still
expect PRINT/READ equivalence of symbols with the "current
readtable".  And even if none of this held true, wouldn't it be nice
if there was some kind of a print-table that one could use to specify
what to do instead of attempting to determine the Right Thing from the
readtable?

Of course, the discussion about whether or not one could conceivably
construct such a PRINT/READ world is moot.  Apparently it is not the
case because of the description of *PRINT-READABLY* in the spec only
allowing for PRINT/READ equivalence with the standard readtable;
regardless of whatever Turing might have to say about how its
impossible to actually spell unkomputable.

But let me switch gears a bit...

The reason I wondering about all of this in the first place is that
I'm pondering using #\! and #\? as dispatching macro characters.
Using #\# as the only dispatching macro character seems to be too
limiting as this means that one can only use "#!", "#?", "#{", etc.,
and still be safely within the gaurantess of 2.1.4, AFAICT.  In
otherwords, (again, AFAICT) those macro sequences not specifically
reserved for programmers might be used as extensions by ones Lisp
implementation and still have it be a conforming implementation.  It
seems to me that breaking the PRINT/READ equivalence of symbols that
use these reserved characters by changing their character syntax is
something of a too-big-of-a-disadvantage or am I just being paranoid?
For example, Norvig's Prolog-in-Lisp uses "?x" for variables and it
would be nice if his "?x" could "coexist" as "\?x" with my "?x" as a
reader macro.  If one only uses the reserved characters for dispatch
marco sub-characters, then PRINT/READ (of symbol names, anyway) is not
affected.  Would these factors lead one to the conclusion that, at the
risk of repeating myself, as a style guideline, one should not use the
reserved characters as dispatching marco characters, even though
they're reserved for such usage?
From: Damien Kick
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <4qhoevrv.fsf@email.mot.com>
Damien Kick <······@email.mot.com> writes:

> But let me switch gears a bit...
> 
> The reason I wondering about all of this in the first place is that
> I'm pondering using #\! and #\? as dispatching macro characters.
> [...] Would these factors [about PRINT/READ equivalence breaking]
> lead one to the conclusion that, [...] as a style guideline, one
> should not use the reserved characters as dispatching marco
> characters, even though they're reserved for such usage?

So quiet.  So very quiet.  Am I asking such a kooky question?  Once
more into the breach...

I've been wanting to write some programs in CMUCL which make frequent
use of RUN-PROGRAM.  However, I find myself wishing that I did not
have to use a list of strings to specify the program and parameters to
be used.

    (run-program "cleartool"
                 '("find" "-version"
                   "{ version(.../squib/LATEST) }"))

However, one could not simply do

    (run-program* '(cleartool find -version
                    |{ version(.../squib/LATEST) }|))

because of the case conversion being done by the reader.  The only way
that I can think of avoiding this only for the arguments being passed
to RUN-PROGRAM* would be a reader macro.  Consider, for example,
something like the following:

    (lambda (stream char)
      (declare (ignore char))
      (let ((*readtable* (copy-readtable *readtable*)))
        (setf (readtable-case *readtable*) :preserve)
        (funcall (get-macro-character #\')
                 stream
                 #\')))

With this, I would be able to write something like the following:

    (defun run-program* (command)
      (with-output-to-string (output)
        (let ((command-strings (mapcar (lambda (element)
                                         (format nil "~A" element))
                                       command)))
          (run-program (car command-strings)
                       (cdr command-strings)
                       :output output))))

And not have to write:

    (run-program* '(|cleartool| |find| ...))

But then I started thinking that a case preserving reader macro for
QUOTE might be too limiting.  What if I didn't want to simply quote a
literal list but rather also have a backquote.  And perhaps a reader
macro for RUN-PROGRAM* itself?  It seemed like a whole family of
readtable related macros might come in handy for interacting with
environments that did not share Lisp's LOVE OF SHOUTING CODE OUT LOUD
AND PROUD.  This is what lead me to think that perhaps a new
dispatching macro character would be the best thing.  This way, one
could define "!'" to be a case preserving version of QUOTE and also
"!`" as a case preserving version of backquote.

    (defun |!` READER| (stream char)
      (declare (ignore char))
      (let ((*readtable* (copy-readtable *readtable*)))
        (setf (readtable-case *readtable*) :preserve)
        (funcall (get-macro-character #\`)
                 stream
                 #\`)))

And then one could also write a reader macro that would allow one to
simply code the following:

    ;; Would allow intermixing of strings and symbols as command text
    !$(cleartool find "." -version |{ version(.../squib/LATEST) }|)

as if one had written:

    (run-program* !`(cleartool find ...))

However, I'm not sure if it is good Common Lisp coding style to do
something like this because, for example, it seems to break the normal
PRINT/READ equivalence for symbol names.  To be honest, this
PRINT/READ business is the only disadvantage that immediately occurs
to me.  So, I would appreciate any comments/thoughts about the subject
of passing #\! to MAKE-DISPATCH-MACRO-CHARACTER but also any feedback
regarding my desire for a concise way to use RUN-PROGRAM; i.e. does
anyone have a better approach for a shorthand notation that doesn't
involve reader macros.  I didn't like the idea of

    ;; Using a format control string to be passed to (FORMAT NIL ...)
    ;; because 'tis very flexible/powerful...
    (run-program "cleartool find . -version ~
                  \"{ version(.../squib/LATEST) }\")

because I would rather avoid the need to deal with the nested quoting
of arguments with spaces, etc., business.
From: Damien Kick
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <vf9tauwc.fsf@email.mot.com>
Damien Kick <······@email.mot.com> writes:

> So quiet.  So very quiet.  Am I asking such a kooky question?

Well, I'm having a converstaion with myself, so...

I found in another thread in c.l.l. that Steven Haflich has used "!"
as a macro character for Lisp code related to HTML generation
<http://groups-beta.google.com/group/comp.lang.lisp/browse_frm/thread/cf3430d10e17ca67/4a02209e5d3659d8?q=steven+haflich+HTML+macro&_done=%2Fgroups%3Fhl%3Den%26lr%3D%26q%3Dsteven+haflich+HTML+macro%26&_doneTitle=Back+to+Search&&d#4a02209e5d3659d8>
and so he apparently doesn't consider it bad style to change the
syntax type of the character.  Unless someone else chimes in with the
opinion that PRINT/READ equivalence is important enough to warrant
shying away from fiddling with "!" ... anyone? ... anyone? .. Bueller?
... I'll just forge ahead and use it.  Anyway, thanks, Steven, for
clarifying the CLHS issues with printing symbols.
From: Trent Buck
Subject: Re: read, print, and macro characters
Date: 
Message-ID: <20050120234817.080dc2fc@harpo.marx>
Up spake Damien Kick:
> <http://groups-beta.google.com/group/comp.lang.lisp/browse_frm/thread/cf3430d10e17ca67/4a02209e5d3659d8?q=steven+haflich+HTML+macro&_done=%2Fgroups%3Fhl%3Den%26lr%3D%26q%3Dsteven+haflich+HTML+macro%26&_doneTitle=Back+to+Search&&d#4a02209e5d3659d8>

Please make an effort to shorten google links.  We don't need to know
what your search query was!

························································@newssvr25.news.prodigy.com

-- 
-trent
<foo> This is very strange.
<bar> What?
<foo> Non CS classes.
<foo> They're like forty percent chicks.
<foo> When I walked in, I thought I was in a bar.