From: Vladimir Zolotykh
Subject: 10.3 Non-functional expanders
Date: 
Message-ID: <3C6810A0.F433CFD3@eurocom.od.ua>
Reading On Lisp by Paul Graham I've found the following (10.3
Non-functional Expanders):

  (defmacro nil! (x)	; wrong
    (incf *nil!s*)
    `(setf ,x nil))

  "With this definition, the global *nil!s* will be incremented each
  time a call to nil! is expanded. However, we are mistaken if we expect
  the value of this variable to tell us how often nil! was called. A
  give call can be, and often is, expanded more than once.  For
  example, a preprocessor which performed transformations on your source
  code might have to expand the macro calls in an expression before it
  could decide whether or not to transform it.

(pp 136).

Is 'preprocessor' here means some part of CL implementation ? Or is it
supposed to be some imagined tools someone could write ? Anyway macro
(as I understand it) should be expanded beyond any conditions or I
mistaken ?

Would you mind to give me some hints what speach is going on here ?

-- 
Vladimir Zolotykh

From: Kent M Pitman
Subject: Re: 10.3 Non-functional expanders
Date: 
Message-ID: <sfwwuxjvn0g.fsf@shell01.TheWorld.com>
Vladimir Zolotykh <······@eurocom.od.ua> writes:

> Reading On Lisp by Paul Graham I've found the following (10.3
> Non-functional Expanders):
> 
>   (defmacro nil! (x)	; wrong
>     (incf *nil!s*)
>     `(setf ,x nil))
> 
>   "With this definition, the global *nil!s* will be incremented each
>   time a call to nil! is expanded. However, we are mistaken if we expect
>   the value of this variable to tell us how often nil! was called. A
>   give call can be, and often is, expanded more than once.  For
>   example, a preprocessor which performed transformations on your source
>   code might have to expand the macro calls in an expression before it
>   could decide whether or not to transform it.
> 
> (pp 136).
> 
> Is 'preprocessor' here means some part of CL implementation ? Or is it
> supposed to be some imagined tools someone could write ? Anyway macro
> (as I understand it) should be expanded beyond any conditions or I
> mistaken ?
> 
> Would you mind to give me some hints what speach is going on here ?

See the definition of "minimal compilation" in the compiler chapter of
CLHS.  You do have to fully expand all macros in the compiler, but not
necessarily in the interpreter.

But, even if you're using a compiler, this doesn't mean you call each
macro function once per macro form.  For better or worse, some macros
tree-walk their enclosed forms before expanding in order to figure out
how to expand, even though they leave it to the system to "expand for
real".  I've seen both implementation-defined and user-defined macros
do this.  I recommend trying to avoid it where one can, but it is in the
nature of some macros that they do this.  e.g., consider a macro
 (with-constant-folding ...)
that tries to do pre-processing on its contents to work around some problem
with some compilers not doing so.  It might expand a macro looking for
things to operate on, and seeing none, it might leave the original form
to be compiled anew.  You might think it "obvious" that it should leave
the expanded form in place, but this is often not good since sometimes a
compiler will compile an unexpanded form better than an expanded form.
(This fact, in turn, makes it unclear whether with-constant-folding is
"worth it", but I'll leave that issue aside.  The fact is that the language
design permits such operations, so it's worth knowing that your code may
not expand just once.)

I recall for Symbolics Macsyma trying to implement the LISTARRAY and
FILLARRAY constructs from Maclisp before we had ROW-MAJOR-AREF in CL.
I wrote a program that would iterate from 0 to array-dimension-limit,
making proper loops for iterating over each kind of array.  But then I
found an implementation with 65,000 or some such as its array
dimension limit and that didn't work so well, so I went with 7 as an
artificial limit.  That would seem ok, but under CLTL, macros could
expand to declarations, so things like DOTIMES would have to
macroexpand the first element of the body to see if it was a
declaration.  If it was, it did one thing; if not, it descended the
tree with no declarations.  Well, with problem was that it was doing
this recursively, so with 7 nested DOTIMES, it took about 1/3 of the
total compilation time for the entire Macsyma algebra/calculus system
to compile these two functions (that is, about 10 out of 30 minutes in
the processor speeds of the day). And that time was reduced to
"negligible" by changing (dotimes (i nx) (dotimes (j ny) ...)) to
(dotimes (i nx) nil (dotimes (j ny) nil ...)) so that the first form
was easily determined to be a non-declaration without recursively
expanding all enclosed DOTIMES.  It is for that reason that we got rid
of macros expanding into declarations; it was too easy for an
implementation to make this mistake.  But we did not get rid of the
ability of a macro to macroexpand its enclosed forms, we only got rid
of the requirement that it do so in these pareticular cases.
From: Vladimir Zolotykh
Subject: Re: 10.3 Non-functional expanders
Date: 
Message-ID: <3C6910E2.F19315B7@eurocom.od.ua>
Thank you Kent, Michael.

This was completely new point for me. I've never imagined something 
similar. Rather a pragmatic conclusion I could make 'Write macros
in pure functional style'.

-- 
Vladimir Zolotykh
From: Michael Parker
Subject: Re: 10.3 Non-functional expanders
Date: 
Message-ID: <AB3EEB85CD248994.026B782568759DE1.A65134B1D7A151F4@lp.airnews.net>
Vladimir Zolotykh wrote:
> 
> Reading On Lisp by Paul Graham I've found the following (10.3
> Non-functional Expanders):
> 
>   (defmacro nil! (x)    ; wrong
>     (incf *nil!s*)
>     `(setf ,x nil))
> 
>   "With this definition, the global *nil!s* will be incremented each
>   time a call to nil! is expanded. However, we are mistaken if we expect
>   the value of this variable to tell us how often nil! was called. A
>   give call can be, and often is, expanded more than once.  For
>   example, a preprocessor which performed transformations on your source
>   code might have to expand the macro calls in an expression before it
>   could decide whether or not to transform it.
> 
> (pp 136).
> 
> Is 'preprocessor' here means some part of CL implementation ? Or is it
> supposed to be some imagined tools someone could write ? Anyway macro
> (as I understand it) should be expanded beyond any conditions or I
> mistaken ?
> 
> Would you mind to give me some hints what speach is going on here ?

Sure.  Since macroexpand and macroexpand-1 are available to user
programs, it's possible that a macro may use this to look inside one
of the forms, and make sure that he's seeing cooked CL.  For example,
if you were writing a macro to emulate original-Dylan's "bind" form in
CL, you might want to macroexpand the value side to see if it's a VALUES
form, and if so compile it into a let (dylan's bind uses values to
implement parallel-bind semantics) instead of a multiple-value-bind.

Or if you're doing some serious code grovelling, like writing a version
of scheme's named let that is trying very hard to figure out the tail
and non-tail call sites and whether or not the function value is
escaping,
you would be doing lots of macroexpands that aren't really directly
related to compilation.

So, since it's being called by system-level code, and may be called
unpredictably, it needs to be coded in a functional manner or things
could get wierd.