From: D Herring
Subject: ANN: read-macros
Date: 
Message-ID: <4945c841$0$17067$6e1ede2f@read.cnntp.org>
Introducing a new package: read-macros.
CL's always had them; now there are a few more to choose from.

Initial release at
http://www.cliki.net/read-macros

Features:
* an extensible syntax (similar to #S)
* conditional reads
* conditional reads based on *features* (similar to #+ and #-)
* stateful block reads (similar to CPP's #if/#elif/#else/#endif)
* lexical read manipulation (e.g. bind *read-base*)
* here documents


Example:
#?(read-cond-features
     sbcl sb-ext:*core-pathname*
     ecl si:*help-message*
     t "Nothing special")

is equivalent to

#+sbcl sb-ext:*core-pathname*
#+(and ecl (not sbcl)) si:*help-message*
#-(or sbcl ecl) "Nothing special"


I would appreciate any comments, especially with regards to
* choice of dispatch character (#?)
* other read macros to implement
* any bugs

Thanks for looking,
Daniel

From: budden
Subject: Re: ANN: read-macros
Date: 
Message-ID: <743e114b-760a-49a0-8f44-6985f2b6c949@q26g2000prq.googlegroups.com>
Hi Daniel!
  Sorry, I got no time to look at it now, maybe later. The common idea
is clear from older topic. But:
- be sure to check prior art. It is hardly possible to invent
something now. Reinventing more and more wheels looks like
a creating a chaos. E.g. there is about dozen build-related libraries
in CL! Checking prior art might be much harder to do than to implement
everything from the scratch, but if we do not want lisp to die with
entropy death, we need to do it any time we invent something.
- someone have already mentioned that you might redefine #e to do
#else. I'd suggest to implement
#w which allows to assign reader macros to any symbol (not character)
starting from "w"
E.g., #with-read-time-package read-eval-form-to-package-designator
form-to-read-in-the-scope-of-the-package
#with-feature read-eval-form-to-form form-to-read-in-scope-of-feature
etc.
#1wpackage::symbol might be then used to get read-macro symbol from
another-package.
This would improve readmacros scalability.

This way, we might have several [ ] readmacros:
(set-symbol-readmacro sql::[ (get-macro-character #\[ *sql-
readtable*))
(in-package :my-package)
(set-symbol-readmacro '|[| #'my-reader)
(set-macro-character #\[ #'my-reader)

[read some custom stuff]
#1wsql:[ select * from foo]

this is sligthly ugly (and fills the code with unhappy emoticons), but
it is less ugly than
readmacros conflicts.

Or, maybe even
#2wreadtable-designator:char

These are just the ideas still, I have no feeling that they get ripe.
I'd like to have more
elegant syntax, but I afraid occuping extra characters would hurt
portability.
From: Helmut Eller
Subject: Re: ANN: read-macros
Date: 
Message-ID: <m2r649mush.fsf@common-lisp.net>
* D Herring [2008-12-15 04:01+0100] writes:

> I would appreciate any comments, especially with regards to
> * choice of dispatch character (#?)
> * other read macros to implement
> * any bugs

It would be useful if the #? dispatch char could be used in a region
without clobbering the global readtable.

Here is a little idea to implement that.  We are using a "readtable
stack".  E.g. assume we have two functions like:

(let ((stack '()))
  (defun push-readtable (readtable)
    (push *readtable* stack)
    (setq *readtable* readtable)
    (values))
  (defun pop-readtable ()
    (setq *readtable* (pop stack))
    (values)))

then we could enable your dispatch character with something like this:

#.(push-readtable special-readtable-for-#?)
... use #? ...
#.(pop-readtable)

we could also turn this idiom into symbol-macros so that we can write:

#.enable-#?
... use #? ...
#.disable-#?

Which looks quite readable and doesn't modify the global readtable.

Helmut.
From: Kaz Kylheku
Subject: Re: ANN: read-macros
Date: 
Message-ID: <20081231093344.152@gmail.com>
On 2008-12-15, Helmut Eller <······@common-lisp.net> wrote:
> * D Herring [2008-12-15 04:01+0100] writes:
>
>> I would appreciate any comments, especially with regards to
>> * choice of dispatch character (#?)
>> * other read macros to implement
>> * any bugs
>
> It would be useful if the #? dispatch char could be used in a region
> without clobbering the global readtable.
>
> Here is a little idea to implement that.  We are using a "readtable
> stack".  E.g. assume we have two functions like:

I think he already has such an idea in there. That's how the
#ifdef/#else/#endif-inspired readers work.

> (let ((stack '()))
>   (defun push-readtable (readtable)
>     (push *readtable* stack)
>     (setq *readtable* readtable)
>     (values))
>   (defun pop-readtable ()
>     (setq *readtable* (pop stack))
>     (values)))

One small problem is, there is no unwind-protect for this kind of ad-hoc
stack.

(read-from-string "#.(push-readtable ...)")

Now the *readtable* remains replaced, and the original is on that stack.

Another problem is that you're pushing and popping over what may be the global
binding of *readtable* for all threads.
From: Helmut Eller
Subject: Re: ANN: read-macros
Date: 
Message-ID: <m2hc55dzzn.fsf@common-lisp.net>
* Kaz Kylheku [2008-12-15 18:49+0100] writes:

>
> I think he already has such an idea in there. That's how the
> #ifdef/#else/#endif-inspired readers work.

AFAICT, the original code installs #? as global reader macro and the #?
dispatches to the others.

>
>> (let ((stack '()))
>>   (defun push-readtable (readtable)
>>     (push *readtable* stack)
>>     (setq *readtable* readtable)
>>     (values))
>>   (defun pop-readtable ()
>>     (setq *readtable* (pop stack))
>>     (values)))
>
> One small problem is, there is no unwind-protect for this kind of ad-hoc
> stack.
>
> (read-from-string "#.(push-readtable ...)")
>
> Now the *readtable* remains replaced, and the original is on that stack.

It's still better than clobbering the global readtable.  Because its
easy to write UNDO-READTABLE-STACK but it's not easy to revert random
changes to the global readtable.

> Another problem is that you're pushing and popping over what may be
> the global binding of *readtable* for all threads.

Installing a global reader macro has the same problem (and every other
global variable or shared mutable datastructure).  The programmer has to
take certain responsibilities.  One of those is to handle all those
thread related issues.  Threads are just like FFIs in this regard.

Helmut.
From: D Herring
Subject: Re: ANN: read-macros
Date: 
Message-ID: <49472d52$0$17069$6e1ede2f@read.cnntp.org>
Helmut Eller wrote:
> * D Herring [2008-12-15 04:01+0100] writes:
> 
>> I would appreciate any comments, especially with regards to
>> * choice of dispatch character (#?)
>> * other read macros to implement
>> * any bugs
> 
> It would be useful if the #? dispatch char could be used in a region
> without clobbering the global readtable.
...
> #.(push-readtable special-readtable-for-#?)
> ... use #? ...
> #.(pop-readtable)

That's an interesting idea; but I feel its outside the scope of my 
library.  Also note that any readtable changes inside this push/pop 
may be unintentionally discarded...  The EDITOR-HINTS project (IMHO 
misnamed in its current implementation) provides readtable 
manipulations which may do what you want.

http://common-lisp.net/project/editor-hints/
http://rpgoldman.real-time.com/lisp/editor-hints.html

See my other post or the new read-macros package for how I addressed 
this issue.

Thanks,
Daniel

P.S.  Thanks to Lu�s Oliveira for pointing me to EDITOR-HINTS.
From: budden
Subject: Re: ANN: read-macros
Date: 
Message-ID: <453eaf8d-bdd6-4c4b-86e9-b657176813a7@w1g2000prk.googlegroups.com>
Hi!
  I think it is reasonable to note here that SLIME has package-
readtable alist:
swank::*readtable-alist*
  This might be useful for something.
-----------
$5/hour lisp freelancer.
From: Tobias C. Rittweiler
Subject: Re: ANN: read-macros
Date: 
Message-ID: <87skonm75y.fsf@freebits.de>
D Herring <········@at.tentpost.dot.com> writes:

> Helmut Eller wrote:
>> * D Herring [2008-12-15 04:01+0100] writes:
>>
>>> I would appreciate any comments, especially with regards to
>>> * choice of dispatch character (#?)
>>> * other read macros to implement
>>> * any bugs
>>
>> It would be useful if the #? dispatch char could be used in a region
>> without clobbering the global readtable.
> ...
>> #.(push-readtable special-readtable-for-#?)
>> ... use #? ...
>> #.(pop-readtable)
>
> That's an interesting idea; but I feel its outside the scope of my
> library.  Also note that any readtable changes inside this push/pop
> may be unintentionally discarded...  The EDITOR-HINTS project (IMHO
> misnamed in its current implementation) provides readtable
> manipulations which may do what you want.

The current version is at

  http://common-lisp.net/~trittweiler/darcs/editor-hints/

  -T.
From: D Herring
Subject: Re: ANN: read-macros
Date: 
Message-ID: <49472bdd$0$17067$6e1ede2f@read.cnntp.org>
D Herring wrote:
> Introducing a new package: read-macros.
> CL's always had them; now there are a few more to choose from.
> 
> Initial release at
> http://www.cliki.net/read-macros

I've uploaded a new version which should address the concerns about 
automatically installing read-macros:dispatch as #? when loading.

This is still the default behavior; but it can be suppressed by adding 
:delay-read-macros-install to *features*.  The dispatcher can be 
installed at any time via a simple
(read-macros:install :subchar #\? :readtable *readtable*)

Thus I sidestepped the question of complex readtable maintenance.

Also new: HTML documentation.

Please let me know if any other issues arise.

Thanks,
Daniel Herring