From: Kaelin Colclasure
Subject: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <CfWF3.328$4z.38961@newsin1.ispchannel.com>
I'm puzzled here... Consider the following transcript:
---8<---
USER(500): (defmacro m1 (&body forms)
     `(em-1 ,@forms 1-em))
M1
USER(501): (defmacro m2 (&body forms)
     `(em-2 ,@forms 2-em))
M2
USER(502): (macroexpand '(m1 (m2 1 2)))
(EM-1 (M2 1 2) 1-EM)
T
USER(503): (macroexpand *)
(EM-1 (M2 1 2) 1-EM)
NIL
USER(504): (macroexpand (second *))
(EM-2 1 2 2-EM)
T
---8<---

I thought the difference between #'macroexpand and #'macroexpand-1
was that the former recursively expanded nested macros... Or is it
that it recursively expands forms that expand to macro forms? If it's
the latter, is there a convenient way to get recursive expansion?

-- Kaelin

From: Christopher R. Barry
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <87ogeu3ij0.fsf@2xtreme.net>
"Kaelin Colclasure" <······@everest.com> writes:

> I'm puzzled here... Consider the following transcript:
> ---8<---
> USER(500): (defmacro m1 (&body forms)
>      `(em-1 ,@forms 1-em))
> M1
> USER(501): (defmacro m2 (&body forms)
>      `(em-2 ,@forms 2-em))
> M2
> USER(502): (macroexpand '(m1 (m2 1 2)))
> (EM-1 (M2 1 2) 1-EM)
> T
> USER(503): (macroexpand *)
> (EM-1 (M2 1 2) 1-EM)
> NIL
> USER(504): (macroexpand (second *))
> (EM-2 1 2 2-EM)
> T
> ---8<---
> 
> I thought the difference between #'macroexpand and #'macroexpand-1
> was that the former recursively expanded nested macros... Or is it
> that it recursively expands forms that expand to macro forms? If it's
> the latter, is there a convenient way to get recursive expansion?

A macro form must expand to another macro form to be recursively
expanded by MACROEXPAND. MACROEXPAND-1 will only macroexpand the form
one level regardless of whether it expands to another macro form. The
fi:lisp-macroexpand-recursively command will actually walk your code
finding and expanding every nested macro form that's not quoted, but
it doesn't have an ANSI CL equivalent.

The ELI sources show that this command uses CLOS::WALK-FORM as the
"expander", which is not documented in the Allegro CL documentation.

But, FWIW:

  (clos::walk-form '(m1 (m2 1 2)))
 => (EM-1 (EM-2 1 2 2-EM) 1-EM)

Christopher
From: Kent M Pitman
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <sfw6713vdts.fsf@world.std.com>
"Kaelin Colclasure" <······@everest.com> writes:

> I'm puzzled here... Consider the following transcript:
> ---8<---
> USER(500): (defmacro m1 (&body forms)
>      `(em-1 ,@forms 1-em))
> M1
> USER(501): (defmacro m2 (&body forms)
>      `(em-2 ,@forms 2-em))
> M2
> USER(502): (macroexpand '(m1 (m2 1 2)))
> (EM-1 (M2 1 2) 1-EM)
> T
> USER(503): (macroexpand *)
> (EM-1 (M2 1 2) 1-EM)
> NIL
> USER(504): (macroexpand (second *))
> (EM-2 1 2 2-EM)
> T
> ---8<---

The above behavior is correct.

> 
> I thought the difference between #'macroexpand and #'macroexpand-1
> was that the former recursively expanded nested macros... Or is it
> that it recursively expands forms that expand to macro forms? If it's
> the latter, is there a convenient way to get recursive expansion?

No, iteratively at the same level (i.e., checking over and over that
the car of the outer form isn't a macro form, or expanding it if it
is), NOT recursively descending into 
subforms.  

There is nothing in CL that descends subforms, fully macroexpanding
everything, though other dialects have offered this.  It's kind of a
shame in some regards, actually.  Though there are some arguments why
it's a bad idea.

Forcing full macroexpansion can be handy in debugging and that's
why many people ask for it.

But if macros were to call this mythical macroexpand-all function when
expanding for real, they could make it seriously harder for the
compiler to do a good job since some macros may have
implementation-defined special form definitions that can only trigger
if you've not forced the macro to expand, and such special form
definitions may compile notably better than the expanded form.
From: Liam Healy
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <51emfpa15p.fsf@corkie.nrl.navy.mil>
Kent M Pitman <······@world.std.com> writes:

> There is nothing in CL that descends subforms, fully macroexpanding
> everything, though other dialects have offered this.  It's kind of a
> shame in some regards, actually.  Though there are some arguments why
> it's a bad idea.
> 
> Forcing full macroexpansion can be handy in debugging and that's
> why many people ask for it.
> 
> But if macros were to call this mythical macroexpand-all function when
> expanding for real, they could make it seriously harder for the
> compiler to do a good job since some macros may have
> implementation-defined special form definitions that can only trigger
> if you've not forced the macro to expand, and such special form
> definitions may compile notably better than the expanded form.

Hmmm.  I always wondered why this wasn't a standard form
because it struck me that a compiler *must* use the equivalent.
But couldn't you have a "user interface" macroexpand-all that was
possibly different from what the compiler uses (or used different
options)? 

Sometimes, I've wanted a selective-macroexpand-all to just expand my
macros, and not system macros.  So I hacked one up, and it works, but
it's not right proper because it's dumb about code walking.  What would
be really nice would be a standard code walker.

Two points of reference for the original poster:
 - ELI's recursive-macroexpand calls clos::walk-form which is a
   macroexpand-all, but of course is ACL dependent.
 - There was a macroexpand-all given by Waters
   in Lisp Pointers (vol VI, no. 1, p. 25 Jan.-Mar. 1993).  
   I think it had some non-portable dependence, however.


-- 
Liam Healy
··········@nrl.navy.mil
From: Kent M Pitman
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <sfwzoyd4b1w.fsf@world.std.com>
Liam Healy <··········@nrl.navy.mil> writes:

> What would be really nice would be a standard code walker.

Chorus:  The Lisp Machine had this.

I vaguely recall that maybe someone (Moon, maybe?) even tried to get
Symbolics to release the code into the public domain.  Can't remember
if that succeeded.  Maybe I'm misremembering the event entirely.

 
> Two points of reference for the original poster:
>  - ELI's recursive-macroexpand calls clos::walk-form which is a
>    macroexpand-all, but of course is ACL dependent.
>  - There was a macroexpand-all given by Waters
>    in Lisp Pointers (vol VI, no. 1, p. 25 Jan.-Mar. 1993).  
>    I think it had some non-portable dependence, however.

It's hard to write it so it works through MACROLET since there's no 
portable way to extend the set of environments and since user-defined macros
might use those environments to expand macros before giving back control
to the code-walker.

I came up with a kludge that I think worked in most or all implementations
even though it was horribly violative of the rules the standard said.  But
I can't remember what I did with the code.  I'll dredge it up some day.
It involves some awful kludge of consing up a strange looking MACROLET
form to EVAL and then finishing your code-walk from within the expander
of the contrived MACROLET...

It would be better of ANSI CL had the environment manipulation primitives
we'd wanted to put in at the time CLTL2 was done.  But those were found to
have lots of design bugs and we ran out of resources to fix their definition
before the time ANSI CL was sealed, so we withdrew that support from the
language rather than have everyone pester us about why the spec would have
said it should work but implementations (rightly) had problems carrying 
through.  With such environmental manipulation primitives, a codewalker
could be written portably.
From: Chuck Fry
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <37ea9331$0$202@nntp1.ba.best.com>
In article <···············@world.std.com>,
Kent M Pitman  <······@world.std.com> wrote:
>It would be better of ANSI CL had the environment manipulation primitives
>we'd wanted to put in at the time CLTL2 was done.  But those were found to
>have lots of design bugs and we ran out of resources to fix their definition
>before the time ANSI CL was sealed, so we withdrew that support from the
>language rather than have everyone pester us about why the spec would have
>said it should work but implementations (rightly) had problems carrying 
>through.  With such environmental manipulation primitives, a codewalker
>could be written portably.

Has any further design work been done on this?  I would like to see
environment access in a future standard.

 -- Chuck
--
	    Chuck Fry -- Jack of all trades, master of none
 ······@chucko.com (text only please)  ········@home.com (MIME enabled)
Lisp bigot, mountain biker, car nut, sometime guitarist and photographer
The addresses above are real.  All spammers will be reported to their ISPs.
From: Kent M Pitman
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <sfwd7v98fs6.fsf@world.std.com>
······@best.com (Chuck Fry) writes:

> In article <···············@world.std.com>,
> Kent M Pitman  <······@world.std.com> wrote:
> >It would be better of ANSI CL had the environment manipulation [...] But
> >those were found to have lots of design bugs and we ran out of
> >resources to fix their definition [...]
> Has any further design work been done on this?  I would like to see
> environment access in a future standard.

Your guess is as good as mine.  I think if someone had they'd have
mentioned it.  Then again, I might have missed the mention.  Certainly
nothing at J13, but that's hardly surprising.  J13 is a body merely to
standardize, not to do language research, and has been effectively 
dormant since the creation of the work that you are familiar with.  
Maybe some vendor has done something with it...?
From: Rainer Joswig
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <joswig-2309992250020001@194.163.195.67>
In article <···············@world.std.com>, Kent M Pitman <······@world.std.com> wrote:

> It would be better of ANSI CL had the environment manipulation primitives
> we'd wanted to put in at the time CLTL2 was done.  But those were found to
> have lots of design bugs and we ran out of resources to fix their definition
> before the time ANSI CL was sealed, so we withdrew that support from the
> language rather than have everyone pester us about why the spec would have
> said it should work but implementations (rightly) had problems carrying 
> through.  With such environmental manipulation primitives, a codewalker
> could be written portably.

And a type inferencer?
From: Howard R. Stearns
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <37EBC440.CAF320F4@elwood.com>
Liam Healy wrote:
> 
> Kent M Pitman <······@world.std.com> writes:
> 
> > There is nothing in CL that descends subforms, fully macroexpanding
> > everything, though other dialects have offered this.  It's kind of a
> > shame in some regards, actually.  Though there are some arguments why
> > it's a bad idea.
> >
> > Forcing full macroexpansion can be handy in debugging and that's
> > why many people ask for it.
> >
> > But if macros were to call this mythical macroexpand-all function when
> > expanding for real, they could make it seriously harder for the
> > compiler to do a good job since some macros may have
> > implementation-defined special form definitions that can only trigger
> > if you've not forced the macro to expand, and such special form
> > definitions may compile notably better than the expanded form.
> 
> Hmmm.  I always wondered why this wasn't a standard form
> because it struck me that a compiler *must* use the equivalent.
> But couldn't you have a "user interface" macroexpand-all that was
> possibly different from what the compiler uses (or used different
> options)?
> 
> Sometimes, I've wanted a selective-macroexpand-all to just expand my
> macros, and not system macros.  So I hacked one up, and it works, but
> it's not right proper because it's dumb about code walking.  What would
> be really nice would be a standard code walker.

Well, I think you answered your own question about why there isn't a
standard user interace version.  It's pretty hard to do right.

> 
> Two points of reference for the original poster:
>  - ELI's recursive-macroexpand calls clos::walk-form which is a
>    macroexpand-all, but of course is ACL dependent.
>  - There was a macroexpand-all given by Waters
>    in Lisp Pointers (vol VI, no. 1, p. 25 Jan.-Mar. 1993).
>    I think it had some non-portable dependence, however.

I would add that the PCL implementation of CLOS has a general purpose
code walker as well.  I don't recall it being very compact, but it's
been a while.

Chucko asked about other work on environment stuff.
I think Duanne at Franz is giving a lot of good thought to this.

I have been told by some damn smart people that there's no such thing as
a general purpose code walker -- depending on what you need, eveyone
writes their own and they don't really share a body of code.

I'm too naive to believe that, personally.  My view is that using
MOP/open-implementation techniques, one can write a general
code-walking/compiling/interpreting framework that allows itslef to be
pulled in different directions about the design space:
  multiple language semantics on one axis,
  multiple target architectures for output on another asis, 
  multiple purposes, such as compilation, interpretation,
macro-expanding or who-calls-database-producing on another axis.

Alas, I'm not working on this right now, and I don't have anything worth
separating and sharing at the point I suspended.
From: David Combs
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <7seq31$8e7@dfw-ixnews14.ix.netcom.com>
In article <··············@corkie.nrl.navy.mil>,
....
>   in Lisp Pointers (vol VI, no. 1, p. 25 Jan.-Mar. 1993).  


What ever happened to lisp pointers.  It's been a Long Time since
I've gotten one.

What replaced it?  Sigplan notices?  Usenix "read me"?

David Combs
From: Kent M Pitman
Subject: Re: macroexpand vs. macroexpand-1 vs. ELI's recursive-macroexpand
Date: 
Message-ID: <sfw3dw4c314.fsf@world.std.com>
·······@netcom.com (David Combs) writes:

> In article <··············@corkie.nrl.navy.mil>,
> ....
> >   in Lisp Pointers (vol VI, no. 1, p. 25 Jan.-Mar. 1993).  
> 
> 
> What ever happened to lisp pointers.  It's been a Long Time since
> I've gotten one.

ACM cancelled it.
 
> What replaced it?  Sigplan notices?  Usenix "read me"?

No real replacement.