From: ······@gmail.com
Subject: how to write a ''multiple characters' macro character' ?
Date: 
Message-ID: <1167251960.962339.159480@a3g2000cwd.googlegroups.com>
Hi,

In CLtL, there is an example of how to write the colon (comment)
character macro.
In place of the colon, I'd like to use, say, two dashes, i.e. '--'.
My idea is to use a dispatch macro character on the first dash. If the
following character is also a dash, then it's a comment, else, treat
the whole thing 'normally'.

1/ Is-it the correct way to go ?
2/ What if it was three dashes (and not only two) ? The statement 'the
following' is now 'the following two'...
3/ How to treat the 'whole thing normally' ?

Thanks, and happy new year :)
--thu
From: Pascal Bourguignon
Subject: Re: how to write a ''multiple characters' macro character' ?
Date: 
Message-ID: <874prht0p3.fsf@thalassa.informatimago.com>
······@gmail.com writes:
> In CLtL, there is an example of how to write the colon (comment)
> character macro.
> In place of the colon, I'd like to use, say, two dashes, i.e. '--'.
> My idea is to use a dispatch macro character on the first dash. If the
> following character is also a dash, then it's a comment, else, treat
> the whole thing 'normally'.
>
> 1/ Is-it the correct way to go ?

What about (let ((abc--def 21)) (* 2 abc--def)) ?
What about (let ((abc-def 21)) (* 2 abc-def)) ?

The problem is that if you want ABC--COMMENT to parse as ABC followed
by a comment, then #\- must be a terminating macro character, but then
ABC-DEF woud be parsed as two tokens ABC and -DEF so you'd have to add
macro character functions to all the constituent characters!

This is independent of the choice whether #\- is bound to a simple
reader macro or a dispatching reader macro.  


> 2/ What if it was three dashes (and not only two) ? The statement 'the
> following' is now 'the following two'...

In some cases, with only two characters, you could envisage to
UNREAD-CHAR the second one to let the normal reader process it
normally.  But in the case where you want to have this second
character scanned along with the first character, or when you want to
look ahead more, it wouldn't work.  You'll need a more complex
scanner.


> 3/ How to treat the 'whole thing normally' ?

That sounds to be far enough to the lisp syntax that it would be
easier to write a specific scanner instead of trying to hack reader
macros for this.

Theorically, it's possible to do anything with reader macros: just put
reader macros on each character, and do your own scanning from them.
But then the standard reader algorithms becomes useless overhead.



Now if we say that -- is to be interpreted as comment-start only when
it appears at the start of a token, we could write:

(SET-MACRO-CHARACTER #\- 
   (lambda (stream char)
      (if (char= char (peek-char nil stream))
          (progn ;; a comment
              (read-line stream nil nil)
              (values))
          ;; Here problems begin:
          ;; We cannot UNREAD the #\- because we've peek'ed a character.
          ;; We cannot READ because we don't know what #\- is part of:
          ;; a package name? a symbol name? a number?
          ;; So we must write our own implementation of the lisp reader
          ;; algorithm here, and when we know what kind of token it is, 
          ;; since the tokenization function of the lisp reader is not
          ;; exported either, we have to implement it too, or
          ;; call READ-FROM-STRING and have reparse the token reparsed.
          (good-luck)))
   :non-terminating)


Another problem to implement the lisp reader algorithm, in a way that
is compatible with the current implementation, is that the constituent
traits of the character is not accessible (in a standard way at
least); we only know the constituent traits of the standard
characters, but for the rest of the characters, you can only specify
your own traits, and hope they're the same in the current
implementation.  (That means that using READ-FROM-STRING to tokenize
the read characters could give unexpected results, if non standard
characters are used). 


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

"Klingon function calls do not have "parameters" -- they have
"arguments" and they ALWAYS WIN THEM."