From: Peter Seibel
Subject: Question about compilation/evaluation environments
Date:
Message-ID: <m3brfnbo84.fsf@javamonkey.com>
Suppose I have a .lisp with the following contents:
(defmacro foo (form) `(progn (print 'foo) ,form))
(defun bar () (foo (print 'x)))
(defmacro baz (form) (foo `(progn ,form)))
(defmacro biff (form) `(progn ,(foo form)))
(defmacro quux (form) (baz `(progn ,form)))
When I try to COMPILE-FILE this file, Allegro (6.2 on GNU/Linux)
complains at me about "attempt to call `FOO' which is an undefined
function." If I remove the definition of QUUX it compiles cleanly.
Other Lisp's I tried (CMUCL and SBCL) have no problem with it, even
with QUUX there. The folks at Franz tried to explain to me why this is
non-portable (separation of the compilation of evaluation environment)
but I'm still confused so I'm hoping someone here can help me out.
Here's my analysis of how this file will be processed by COMPILE-FILE
and the effect it will have on the compilation and evaluation
environment (which we will assume are separate.) Maybe someone can
point out where I go astray, if I do.
So we start at the top:
(defmacro foo (form) `(progn (print 'foo) ,form))
On seeing this form the compiler compiles its body to produce a
macroexpander function and stores the definition in the compilation
environment.
(defun bar () (foo (print 'x)))
I assume this is legal since otherwise the requirement that DEFMACRO
store the macro definition in the compilation environment doesn't mean
much of anything. More particularly, in order to compile this the
compiler must look up FOO in the compilation environment and run its
macroexpander (in the evaluation environment) and then compile the
resulting form.
(defmacro baz (form) (foo `(progn ,form)))
This seems to me no different from compiling a function: the compiler
looks up FOO in the compilation environment and runs the
macroexpanderand and then compiles the result. In this case the result
is BAZ's macroexpander which is then also stored in the compilation
environment.
(defmacro biff (form) `(progn ,(foo form)))
Should be the same as compiling BAZ: to compile the call to FOO it uses
the compilation environment to find the macroexpander, expands it and
compiles the resulting form.
FWIW, a file containing only the forms mentioned so far will in fact
compile cleanly in Allegro 6.2. Now we get to the case that Allegro
won't compile, complaining about no function definition for FOO:
(defmacro quux (form) (baz `(progn ,form)))
So this one I still don't understand. It seems that it's no different
than the earlier ones--the compiler uses the compilation environment
to find the definition of BAZ. Having found it it runs its
macroexpander in the evaluation environment. Since BAZ has already
been compiled it should no longer contain any reference to FOO so the
fact that FOO isn't in the evaluation environment shouldn't matter.
If, for some reason, the definition of BAZ that the compiler stored in
the compilation environment is not even minimally-compiled I guess I
can see why this would fail. Is it legal for the compiler to store a
non-compiled definition in the compilation environment? Or is my whole
analysis busted in some other way?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Suppose I have a .lisp with the following contents:
>
> (defmacro foo (form) `(progn (print 'foo) ,form))
>
> (defun bar () (foo (print 'x)))
>
> (defmacro baz (form) (foo `(progn ,form)))
>
> (defmacro biff (form) `(progn ,(foo form)))
>
> (defmacro quux (form) (baz `(progn ,form)))
>
> When I try to COMPILE-FILE this file, Allegro (6.2 on GNU/Linux)
> complains at me about "attempt to call `FOO' which is an undefined
> function." If I remove the definition of QUUX it compiles cleanly.
> Other Lisp's I tried (CMUCL and SBCL) have no problem with it, even
> with QUUX there. The folks at Franz tried to explain to me why this is
> non-portable (separation of the compilation of evaluation environment)
> but I'm still confused so I'm hoping someone here can help me out.
I see you've had several repsonses, but none have gotten down to the
crux of the problem. Let's zero in on that.
The problematic form is this macro form:
(defmacro baz (form) (foo `(progn ,form)))
Note that if I change the definition to
(defmacro baz (form) `(foo (progn ,form)))
your test file compiles with no problem in Allegro CL.
Now, let's get back to the problem form:
(defmacro baz (form) (foo `(progn ,form)))
and break down what it is trying to do, and where it must
do what it is trying to do.
Consider section 3.1.2.1.2.2: Macro Forms: Part of the
major paragraph describing how macros are expanded says
"The function returned by macro-function is a function
of two arguments, called the expansion function. The
expansion function is invoked by calling the macroexpand
==================================^^^^^^^
hook with the expansion function as its first argument,
the entire macro form as its second argument, and an
environment object (corresponding to the current lexical
environment) as its third argument. The macroexpand hook,
in turn, calls the expansion function with the form as its
==========^^^^^
first argument and the environment as its second argument."
Note that there are two calls, here, but the macroexpand
hook usually acts as just a pass-through, so one could
simplify by saying that the expansion function is called
with the form as the first argument and the environment as
the second environment.
It is clear in this situation that the environment being
passed is the compiler environment. It is that environment
which the macro-function will consult in order to find
definitions for the expansion of its form. However, just
what is "its form?" Is it the form "(foo `(progn ,form))"?
No, that is the argument to the macro-function, which, as
a function, accepts arguments that have already been
evaluated. So we are looking at needing to evaluate
(foo `(progn ,form))
before sending it to baz's macro-function.
Now, where do we get foo's definition? Out of the compilation
environment? No, evaluation has nothing to do with the compilation
environment. Instead, it is the "evaluation environment", which
is defined by the glossary as "run-time environment in which macro
expanders and code specified by eval-when to be evaluated are
evaluated. All evaluations initiated by the compiler take place
in the evaluation environment."
So is foo's definition in the run-time environment? One might
intuitively think that is should, and apparently other CL
implementations choose to do this. But 3.2.3.1.1, last
paragraph, says "It is not specified whether definitions
made available in the compilation environment are available
in the evaluation environment ..."
so in fact whether foo is defined or not at the time
of baz'z macroexpansion is explicitly unspecified, which
means that your definition is non-portable.
Another way to say this (and perhaps to make it easier to
understand what is happening) is to rewrite the macro definition
into an almost equivalent:
(defmacro baz (form) (foo `(progn ,form)))
~=
(defmacro baz (form &environment env)
(macroexpand (foo `(progn ,form)) env))
This is obviously not a correct translation, because it ignores
macroexpand-hook, and the fact that baz is a macro means that
two macroexpansions occur instead of one. But it does illustrate
the difference (by separating them) between the macroexpansion
of baz (which occurs in the compilation environment) and the
macroexpansion of foo, which occurs as the result of an evaluation
and which thus is not part of the compilation environment.
It also becomes clear when I perform a similar translation on
the corrected form:
(defmacro baz (form) `(foo (progn ,form)))
~=
(defmacro baz (form &environment env)
(macroexpand `(foo (progn ,form)) env))
that now the entire code (including the macroexpansion of foo)
is done within the current lexical environment (i.e. the compile-file
environment)
I have more to say on the "why" of our own decision to implement
the way we do, but this message is getting too long, and is best
held to an explanation in terms of what the ANS actually says.
I'll try to work up a "why" response in another message.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3fz4y8dho.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Suppose I have a .lisp with the following contents:
>>
>> (defmacro foo (form) `(progn (print 'foo) ,form))
>>
>> (defun bar () (foo (print 'x)))
>>
>> (defmacro baz (form) (foo `(progn ,form)))
>>
>> (defmacro biff (form) `(progn ,(foo form)))
>>
>> (defmacro quux (form) (baz `(progn ,form)))
>>
>> When I try to COMPILE-FILE this file, Allegro (6.2 on GNU/Linux)
>> complains at me about "attempt to call `FOO' which is an undefined
>> function." If I remove the definition of QUUX it compiles cleanly.
>> Other Lisp's I tried (CMUCL and SBCL) have no problem with it, even
>> with QUUX there. The folks at Franz tried to explain to me why this
>> is non-portable (separation of the compilation of evaluation
>> environment) but I'm still confused so I'm hoping someone here can
>> help me out.
>
> I see you've had several repsonses, but none have gotten down to the
> crux of the problem. Let's zero in on that.
>
> The problematic form is this macro form:
>
> (defmacro baz (form) (foo `(progn ,form)))
Why is this the problematic form. The file compiles fine with this
form. It's the (defmacro quux ...) that causes the problem.
> Note that if I change the definition to
>
> (defmacro baz (form) `(foo (progn ,form)))
>
> your test file compiles with no problem in Allegro CL.
Yes. Of course.
> Now, let's get back to the problem form:
>
> (defmacro baz (form) (foo `(progn ,form)))
>
> and break down what it is trying to do, and where it must
> do what it is trying to do.
>
> Consider section 3.1.2.1.2.2: Macro Forms: Part of the
> major paragraph describing how macros are expanded says
>
>
> "The function returned by macro-function is a function
> of two arguments, called the expansion function. The
> expansion function is invoked by calling the macroexpand
> ==================================^^^^^^^
> hook with the expansion function as its first argument,
> the entire macro form as its second argument, and an
> environment object (corresponding to the current lexical
> environment) as its third argument. The macroexpand hook,
> in turn, calls the expansion function with the form as its
> ==========^^^^^
> first argument and the environment as its second argument."
>
> Note that there are two calls, here, but the macroexpand
> hook usually acts as just a pass-through, so one could
> simplify by saying that the expansion function is called
> with the form as the first argument and the environment as
> the second environment.
>
> It is clear in this situation that the environment being
> passed is the compiler environment. It is that environment
> which the macro-function will consult in order to find
> definitions for the expansion of its form. However, just
> what is "its form?" Is it the form "(foo `(progn ,form))"?
> No, that is the argument to the macro-function, which, as
> a function, accepts arguments that have already been
> evaluated. So we are looking at needing to evaluate
>
> (foo `(progn ,form))
What macro are we talking about here? Won't the form that will be
passed to BAZ's expansion function always be a list whose CAR is BAZ?
I.e "the entire macro form" as it says in the section you quoted
above.
So you've already lost me. If you can stand it, let my try one more
time to step through what I think should be happening as the forms in
this file are compiled with COMPILE-FILE. If you can show me where in
this I go wrong--make an unwarrented assumption or get some fact
wrong--that will probabably be the quickest way for me to see the
error of my ways. I'll start at the top:
(defmacro foo (form) `(progn (print 'foo) ,form))
This definition is compiled into an expansion function that takes a
form of the form (FOO ...) and parses out part of it and binds it to
the variable FORM and then runs the code `(progn (print 'foo) ,form)
to generate the macro expansion.
Next BAR:
(defun bar () (foo (print 'x)))
To compile BAR the macro expansion function for FOO is found in the
compilation environment and invoked with the form:
(foo (print 'x)).
FOO's expansion function binds FORM to (print 'x) and returns:
(progn (print 'foo) (print 'x))
which is then compiled, becoming the body of BAR.
Now to the macro BAZ, which seems to me will be compiled in
essentially the same way as the function BAR:
(defmacro baz (form) (foo `(progn ,form)))
To compile BAZ the compiler must compile the body. Since the CAR of
the first expression in the body is a macro (found in the compilation
environment) it must invoke the expansion function for FOO with the
form:
(foo `(progn ,form))
The expansion function for FOO binds FORM to `(progn ,form) and returns:
(progn (print 'foo) `(progn ,form))
which is then compiled, becoming the definition of BAR's expansion
function. This definition is stored in the compilation environment for
later use in the same file.
Now BIFF:
(defmacro biff (form) `(progn ,(foo form)))
To compile BIFF the compiler must compile the expression
`(progn ,(foo form)
or, perhaps more clearly, an expresson like:
(list 'progn (foo form))
In order to compile this expression it needs to compile all the
subforms. When it hits (foo form) it again finds that FOO is a macro
and invokes its expansion function, this time with the form:
(foo form)
The parameter FORM in FOO's expansion function is bound to the symbol
FORM and FOO returns:
(progn (print 'foo) (print form))
which is then compiled. The complete compiled LIST expression is made
the body of BIFF's expansion function which is squirreled away in the
compilation environment.
Now finally to the form that actually causes Allegro problems:
(defmacro quux (form) (baz `(progn ,form)))
It still seems to me that when the compiler hits the form:
(baz `(progn ,form))
it's going to look up BAZ's expansion function in the compilation
environment and invoke it with the form:
(baz `(progn ,form))
If we're using the definition of BAZ that we compiled a few steps
back, that function will take apart this form, binding BAZ's parameter
FORM to:
`(progn ,form)
and then running the code:
(progn (print 'foo) `(progn ,form))
or:
(progn (print 'foo) (list 'progn form))
which will print 'foo and return
(progn `(progn ,form))
which will then be compiled and installed as the body of QUUX's macro
expansion function. I don't see why anything had to be evaluated in
the run-time environment. *Unless* the definition of BAZ used to
compile QUUX was not even minimally compiled. And I can't find
anything that says one way or another whether the definitions of
macros stored in the compilation environment have to be minimally
compiled or not. So is that it? Or am I missing something else?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > Peter Seibel <·····@javamonkey.com> writes:
> >
> >> Suppose I have a .lisp with the following contents:
> >>
> >> (defmacro foo (form) `(progn (print 'foo) ,form))
> >>
> >> (defun bar () (foo (print 'x)))
> >>
> >> (defmacro baz (form) (foo `(progn ,form)))
> >>
> >> (defmacro biff (form) `(progn ,(foo form)))
> >>
> >> (defmacro quux (form) (baz `(progn ,form)))
> >>
> >> When I try to COMPILE-FILE this file, Allegro (6.2 on GNU/Linux)
> >> complains at me about "attempt to call `FOO' which is an undefined
> >> function." If I remove the definition of QUUX it compiles cleanly.
> >> Other Lisp's I tried (CMUCL and SBCL) have no problem with it, even
> >> with QUUX there. The folks at Franz tried to explain to me why this
> >> is non-portable (separation of the compilation of evaluation
> >> environment) but I'm still confused so I'm hoping someone here can
> >> help me out.
> >
> > I see you've had several repsonses, but none have gotten down to the
> > crux of the problem. Let's zero in on that.
> >
> > The problematic form is this macro form:
> >
> > (defmacro baz (form) (foo `(progn ,form)))
>
> Why is this the problematic form. The file compiles fine with this
> form. It's the (defmacro quux ...) that causes the problem.
That is true a priori, and in fact it is not the compilation of baz
which is the problem, but the expansion of baz. And what I am trying
to say here is that baz is not a normative macro; macros usually return
a form for evaluation or compilation, often after performing some
macroexpand-time evaluation. What is not normative here is that
your macro is counting on functionality (i.e. foo's macro definition)
being available that the spec clearly says cannot be counted on to
be available.
> > It is clear in this situation that the environment being
> > passed is the compiler environment. It is that environment
> > which the macro-function will consult in order to find
> > definitions for the expansion of its form. However, just
> > what is "its form?" Is it the form "(foo `(progn ,form))"?
> > No, that is the argument to the macro-function, which, as
> > a function, accepts arguments that have already been
> > evaluated. So we are looking at needing to evaluate
> >
> > (foo `(progn ,form))
>
> What macro are we talking about here? Won't the form that will be
> passed to BAZ's expansion function always be a list whose CAR is BAZ?
> I.e "the entire macro form" as it says in the section you quoted
> above.
Sorry, yes, this was confusing; I skipped a step. In fact, baz's
macro-function receives the whole form, e.g.:
[1] CL-USER(10): (pprint *)
(BAZ (BAZ `(PROGN ,FORM))
#<Augmentable COMPILATION environment 1 1 1 1 2 2 1 4 2 1 5 6 @ #x10bedd0a>)
[1] CL-USER(11):
but at this point we are calling a function, and the function processes
its body as normal forms to evaluate:
[1] CL-USER(15): (pprint *)
(BLOCK BAZ
(LET* ()
(LET* ((#:G130 (CDR EXCL::**MACROARG**))
(FORM (EXCL::CAR-FUSSY #:G130 'FORM))
(#:G131 (EXCL::LAMBDASCAN-MAXARGS 0 (CDR #:G130) '(FORM))))
(DECLARE (IGNORABLE #:G131))
(FOO `(PROGN ,FORM)))))
[1] CL-USER(16):
Note that one of these forms is (FOO `(PROGN ,FORM)), which is
evaluated in the evaluation environment.
> So you've already lost me. If you can stand it, let my try one more
> time to step through what I think should be happening as the forms in
> this file are compiled with COMPILE-FILE. If you can show me where in
> this I go wrong--make an unwarrented assumption or get some fact
> wrong--that will probabably be the quickest way for me to see the
> error of my ways. I'll start at the top:
>
> (defmacro foo (form) `(progn (print 'foo) ,form))
>
> This definition is compiled into an expansion function that takes a
> form of the form (FOO ...) and parses out part of it and binds it to
> the variable FORM and then runs the code `(progn (print 'foo) ,form)
> to generate the macro expansion.
Yes.
Note that it is specified that foo's definition is also stored into the
compilation environment for later use, but it is unspecified as to
whether that definition is compiled or interpreted. Implementations
are free to compile these definitions, but they will have been compiled
with the COMPILE function, in the evaluation environment.
> Next BAR:
>
> (defun bar () (foo (print 'x)))
>
> To compile BAR the macro expansion function for FOO is found in the
> compilation environment and invoked with the form:
>
> (foo (print 'x)).
>
> FOO's expansion function binds FORM to (print 'x) and returns:
>
> (progn (print 'foo) (print 'x))
>
> which is then compiled, becoming the body of BAR.
Yes, and note that all definitions that bar uses are available
for that macroexpansion. The print function is available in the
global environment, but even if it had been something undefined,
it would be treated simply as a function and a call to that
function generated. But this is fine, since the body form of foo
hasn't been _called_ yet, only code generated that will later be
loaded. Note also that 'bar doesn't even yet have a definition;
that occurs at load time.
> Now to the macro BAZ, which seems to me will be compiled in
> essentially the same way as the function BAR:
>
> (defmacro baz (form) (foo `(progn ,form)))
>
> To compile BAZ the compiler must compile the body. Since the CAR of
> the first expression in the body is a macro (found in the compilation
> environment) it must invoke the expansion function for FOO with the
> form:
>
> (foo `(progn ,form))
>
> The expansion function for FOO binds FORM to `(progn ,form) and returns:
>
> (progn (print 'foo) `(progn ,form))
>
> which is then compiled, becoming the definition of BAR's expansion
> function.
I think you mean BAZ's here, rather than BAR's.
Yes, but it is not required that baz actually receive this
(compiled) expansion function. It is only required that
the macro's definition be stored for later use, and that
definition might be interpreted.
> This definition is stored in the compilation environment for
> later use in the same file.
Here's where we go bad; it is not "this" definition (i.e. the
one created by the file compiler which will eventually become
the definition for BAZ when the file is loaded). It is, in fact,
unspecified whether the definition stored for later use is
compiled or not.
It is critical that one understand the entirety of section
3.2.3.1.1, and especially _why_ the example of nonportable code
on that page is in fact nonportable.
> Now BIFF:
>
> (defmacro biff (form) `(progn ,(foo form)))
>
> To compile BIFF the compiler must compile the expression
>
> `(progn ,(foo form)
>
> or, perhaps more clearly, an expresson like:
>
> (list 'progn (foo form))
>
> In order to compile this expression it needs to compile all the
> subforms. When it hits (foo form) it again finds that FOO is a macro
> and invokes its expansion function, this time with the form:
>
> (foo form)
>
> The parameter FORM in FOO's expansion function is bound to the symbol
> FORM and FOO returns:
>
> (progn (print 'foo) (print form))
>
> which is then compiled. The complete compiled LIST expression is made
> the body of BIFF's expansion function which is squirreled away in the
> compilation environment.
All correct here.
> Now finally to the form that actually causes Allegro problems:
>
> (defmacro quux (form) (baz `(progn ,form)))
>
> It still seems to me that when the compiler hits the form:
>
> (baz `(progn ,form))
>
> it's going to look up BAZ's expansion function in the compilation
> environment and invoke it with the form:
>
> (baz `(progn ,form))
This is true. But what exactly _is_ baz's expansion function?
Is it compiled, or is it interpreted?
Note that 3.2.3.1, item 6, is very specific about what happens with
a toplevel form that is not a special case (a defmacro form is itself
a macro form, so its expansion would be macroexpanded, which might
result in a progn of at least a (setf (macro-function ...) ...)
form in either a regular form or an eval-when (load eval) situation, and
some other style of association between the name and the definition in
an eval-when (compile) situation) these non-special-case forms are simply
minimally compiled, or if in compile-time-too mode they are first evaluated
in the evaluation environment and then minimally compiled. There is
nothing that says that the definitions for each of these subforms must be
the same definition, nor does it say that the definition is the one that
has been compiled.
> If we're using the definition of BAZ that we compiled a few steps
> back,
We're not, necessarily.
that function will take apart this form, binding BAZ's parameter
> FORM to:
>
> `(progn ,form)
>
> and then running the code:
>
> (progn (print 'foo) `(progn ,form))
>
> or:
>
> (progn (print 'foo) (list 'progn form))
>
> which will print 'foo and return
>
> (progn `(progn ,form))
>
> which will then be compiled and installed as the body of QUUX's macro
> expansion function. I don't see why anything had to be evaluated in
> the run-time environment.
Not "run-time", "evaluation". See 3.2.3.1.1
> *Unless* the definition of BAZ used to
> compile QUUX was not even minimally compiled. And I can't find
> anything that says one way or another whether the definitions of
> macros stored in the compilation environment have to be minimally
> compiled or not. So is that it?
Yes, this is part of it. You see nothing that says one way or the
other that it's true, and yet you assume it's true. Why?
In fact, the spec is explicit that you _can't_ assume one way or
the other.
Or am I missing something else?
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3brfm6rv4.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
>> *Unless* the definition of BAZ used to compile QUUX was not even
>> minimally compiled. And I can't find anything that says one way or
>> another whether the definitions of macros stored in the compilation
>> environment have to be minimally compiled or not. So is that it?
>
> Yes, this is part of it.
Seems to me like that's all of it. Unless I'm still missing something,
the answer to my original question could well have been: "The reason
this file can't necessarily be compiled with COMPILE-FILE is that
there is no guarantee that the definition of BAZ that is stored in the
compile time environment is minimally compiled and therefore uses of
the BAZ that need to be expanded at compile time (such as when we
compile QUUX) may require us be able to find all the definitions,
including macros, that occur in the body of BAZ in the evaluation
environment. And the evaluation environment does *not* contain the
definition of FOO when the compilation and evaluation environments are
not the same."
> You see nothing that says one way or the other that it's true, and
> yet you assume it's true. Why?
Jeez, in my very first post (to say nothing of earlier messages to
·······@franz.com) I noted:
If, for some reason, the definition of BAZ that the compiler stored
in the compilation environment is not even minimally-compiled I
guess I can see why this would fail. Is it legal for the compiler to
store a non-compiled definition in the compilation environment?
If someone had just said--"yes, it is legal for the compiler to store
a non-compiled definition in the compilation environment" we'd have
been done a long time ago.
> In fact, the spec is explicit that you _can't_ assume one way or the
> other.
Okay. So I still don't know where in the spec it talks *explicitly*
about whether the definition stored in the compilation environment is
minimally compiled or not. I'd be grateful for a pointer.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> >> *Unless* the definition of BAZ used to compile QUUX was not even
> >> minimally compiled. And I can't find anything that says one way or
> >> another whether the definitions of macros stored in the compilation
> >> environment have to be minimally compiled or not. So is that it?
> >
> > Yes, this is part of it.
>
> Seems to me like that's all of it.
No. I think we're talking past each other as to what "it" is.
Clearly, you've been asking the same questions about whether the macro
is compiled or not, and the answers you've been getting from us
all through the process are "it doesn't matter". I've taken a
different approach, by trying to help you to understand what happens
when the form is not compiled. Once you see that uncompiled code
necessarily precludes one from assuming that foo is defined when baz
is run, then it is a slightly smaller step to see that one can't
assume that foo is defined even when baz is compiled. In other words,
understanding that a macro might not be minimally compiled is part
of "it" (i.e. understanding where the confusion is coming from,
which was what you asked me to help you to step through, if I could
stand it. Well, I can stand it, and I'd like for you to be able to
come to an understanding of what the spec allows and disallows in
this context).
> Jeez, in my very first post (to say nothing of earlier messages to
> ·······@franz.com) I noted:
>
> If, for some reason, the definition of BAZ that the compiler stored
> in the compilation environment is not even minimally-compiled I
> guess I can see why this would fail. Is it legal for the compiler to
> store a non-compiled definition in the compilation environment?
>
> If someone had just said--"yes, it is legal for the compiler to store
> a non-compiled definition in the compilation environment" we'd have
> been done a long time ago.
Consider the boolean logic expression (a v b) ^ (a v ~b). I.e.
"a and b, or a and not b". This expression simplifies to the
much simpler expression "a", and it does not at all depend on b.
Your question has been, even through the spr conversatin earlier,
"is it because ~b?" to which we answered "no, it is because of a".
That might have seemed like an evasion of your question, but what it
really was was a statement that b doesn't matter; b or ~b, it's all
about a.
> > In fact, the spec is explicit that you _can't_ assume one way or the
> > other.
>
> Okay. So I still don't know where in the spec it talks *explicitly*
> about whether the definition stored in the compilation environment is
> minimally compiled or not. I'd be grateful for a pointer.
Start with 3.2.1 ("Compiler Terminology"): A couple of paragraphs
are apropos:
"The compilation environment is maintained by the compiler and is used
to hold definitions and declarations to be used internally by the
compiler. Only those parts of a definition needed for correct
compilation are saved. The compilation environment is used as the
environment argument to macro expanders called by the compiler. It is
unspecified whether a definition available in the compilation
environment can be used in an evaluation initiated in the startup
environment or evaluation environment.
The evaluation environment is a run-time environment in which macro
expanders and code specified by eval-when to be evaluated are
evaluated. All evaluations initiated by the compiler take place in
the evaluation environment."
Now whether the implementation evaluates by using an interpreter,
or by compiling and then executing the compiled code, that evaluation
is still run in the same environment. In this case, the macro-function
is run in the evaluation environment. So even if baz's macro-function
is compiled for the purposes of evaluation at compile-time, that
compilation must be as if it is being done in the evaluation environment.
Now, for implementations that equate the evaluation and compilation
environments, it might be the case that an optimization could take
the output from the file compiler (which compiles in the compilation
environment) and use it _as_ _if_ it had been compiled in the evaluation
environment as part of the evaluation of the macro function. However,
that optimization cannot be assumed, because for any implementation
that evaluates by compilation _and_ separates the compilation and
evaluation environments (I know of none of these, but the spec allows
for it) there will likely be two different compilation steps.
Now, if you can see the issues involved here, the burning question
might be "but why _not_ merge the compilation and evaluation
environments, and thus allow macros to be compiled as they appear
in the file?" and the answer is simply: "late binding". I don't
have time to work up a practical example, but consider that compiling
a baz definition and expanding out the foo macro forces you to recompile
that definition if you redefine foo but intend to use baz again. Keeping
things uncompiled allows a minimum of (eval-when (:compile-toplevel ...) ...)
forms wrapping macros, or a minimal ordering requirement on the macro
definitions.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m31xgh7qhk.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Duane Rettig <·····@franz.com> writes:
>>
>> >> *Unless* the definition of BAZ used to compile QUUX was not even
>> >> minimally compiled. And I can't find anything that says one way or
>> >> another whether the definitions of macros stored in the compilation
>> >> environment have to be minimally compiled or not. So is that it?
>> >
>> > Yes, this is part of it.
>>
>> Seems to me like that's all of it.
>
> No. I think we're talking past each other as to what "it" is.
> Clearly, you've been asking the same questions about whether the macro
> is compiled or not, and the answers you've been getting from us
> all through the process are "it doesn't matter". I've taken a
> different approach, by trying to help you to understand what happens
> when the form is not compiled. Once you see that uncompiled code
> necessarily precludes one from assuming that foo is defined when baz
> is run, then it is a slightly smaller step to see that one can't
> assume that foo is defined even when baz is compiled. In other words,
> understanding that a macro might not be minimally compiled is part
> of "it" (i.e. understanding where the confusion is coming from,
> which was what you asked me to help you to step through, if I could
> stand it. Well, I can stand it, and I'd like for you to be able to
> come to an understanding of what the spec allows and disallows in
> this context).
>
>> Jeez, in my very first post (to say nothing of earlier messages to
>> ·······@franz.com) I noted:
>>
>> If, for some reason, the definition of BAZ that the compiler stored
>> in the compilation environment is not even minimally-compiled I
>> guess I can see why this would fail. Is it legal for the compiler to
>> store a non-compiled definition in the compilation environment?
>>
>> If someone had just said--"yes, it is legal for the compiler to store
>> a non-compiled definition in the compilation environment" we'd have
>> been done a long time ago.
>
> Consider the boolean logic expression (a v b) ^ (a v ~b). I.e.
> "a and b, or a and not b". This expression simplifies to the
> much simpler expression "a", and it does not at all depend on b.
> Your question has been, even through the spr conversatin earlier,
> "is it because ~b?" to which we answered "no, it is because of a".
> That might have seemed like an evasion of your question, but what it
> really was was a statement that b doesn't matter; b or ~b, it's all
> about a.
So you're telling me that if the definition of BAZ that the compiler
stored in the compilation environment was minimally compiled (i.e. no
longer contained a FOO form) that compiling QUUX would still complain
about FOO not being defined. That's strange.
>> > In fact, the spec is explicit that you _can't_ assume one way or
>> > the other.
>>
>> Okay. So I still don't know where in the spec it talks *explicitly*
>> about whether the definition stored in the compilation environment is
>> minimally compiled or not. I'd be grateful for a pointer.
>
> Start with 3.2.1 ("Compiler Terminology"): A couple of paragraphs
> are apropos:
>
> "The compilation environment is maintained by the compiler and is used
> to hold definitions and declarations to be used internally by the
> compiler. Only those parts of a definition needed for correct
> compilation are saved. The compilation environment is used as the
> environment argument to macro expanders called by the compiler. It is
> unspecified whether a definition available in the compilation
> environment can be used in an evaluation initiated in the startup
> environment or evaluation environment.
>
> The evaluation environment is a run-time environment in which macro
> expanders and code specified by eval-when to be evaluated are
> evaluated. All evaluations initiated by the compiler take place in
> the evaluation environment."
>
> Now whether the implementation evaluates by using an interpreter, or
> by compiling and then executing the compiled code, that evaluation
> is still run in the same environment. In this case, the
> macro-function is run in the evaluation environment. So even if
> baz's macro-function is compiled for the purposes of evaluation at
> compile-time, that compilation must be as if it is being done in the
> evaluation environment.
But what if the definition of BAZ that is stored in the compilation
environment is the *compiled* one. Then when the macro expander is run
in the evaluation environment it doesn't matter that FOO isn't found
in that environment because it's already been compiled away. That is
there are (at least) three ways BAZ could be evaluated:
- BAZ is stored as a not-even minimally compiled sexps and is
evaluated at macroexpansion time by pure interpretation in the
evaluation environment. Under this strategy QUUX won't compile
because FOO isn't in the evaluation environment.
- BAZ is stored as a not-even minimally compiled sexps and is
evaluated at macroexpansion time by being compiled (in the
evaluation environment) and the newly compiled expansion function
being run. Under this strategy QUUX also won't compile because FOO
isn't in the evaluation environment and thus the compilation of
BAZ fails.
- BAZ is (minimally) compiled in the compilation environment and
that compiled function is stored as the definition in the
compilation environment for later use. (As well as being turned
into the code that will be emitted to the .fasl file.) In this
case QUUX should compile fine since the evaluation of BAZ is no
longer dependent on the definition of FOO being available.
It seems to me that all three of these strategies are compatible with
separating the compilation and evalution environments. And the first
two are really the same--just different strategies for "interpreting"
BAZ. But with the later strategy, QUUX compiles fine. So perhaps you
see why I think the key difference, given that the compilation and
evalution environment are separate, is whether or not the definition
of BAZ that is stored in the compilation environment is minimally
compiled.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> So you're telling me that if the definition of BAZ that the compiler
> stored in the compilation environment was minimally compiled (i.e. no
> longer contained a FOO form) that compiling QUUX would still complain
> about FOO not being defined. That's strange.
No, it wouldn't get that far. Baz is compiled in the compilation
environment for the purposes of storing into the fasl file. But it
is compiled in the evaluation environment, if at all, in order to be
evaluated at compile time, so if it would have otherwise complained
that foo was not defined under non-compled situation, then it would
have also complained when attempting to compile baz; i.e. the
compilation of baz would have failed with an undefined function error,
before even getting to the compilation of quux.
> >> > In fact, the spec is explicit that you _can't_ assume one way or
> >> > the other.
> >>
> >> Okay. So I still don't know where in the spec it talks *explicitly*
> >> about whether the definition stored in the compilation environment is
> >> minimally compiled or not. I'd be grateful for a pointer.
> >
> > Start with 3.2.1 ("Compiler Terminology"): A couple of paragraphs
> > are apropos:
> >
> > "The compilation environment is maintained by the compiler and is used
> > to hold definitions and declarations to be used internally by the
> > compiler. Only those parts of a definition needed for correct
> > compilation are saved. The compilation environment is used as the
> > environment argument to macro expanders called by the compiler. It is
> > unspecified whether a definition available in the compilation
> > environment can be used in an evaluation initiated in the startup
> > environment or evaluation environment.
> >
> > The evaluation environment is a run-time environment in which macro
> > expanders and code specified by eval-when to be evaluated are
> > evaluated. All evaluations initiated by the compiler take place in
> > the evaluation environment."
> >
> > Now whether the implementation evaluates by using an interpreter, or
> > by compiling and then executing the compiled code, that evaluation
> > is still run in the same environment. In this case, the
> > macro-function is run in the evaluation environment. So even if
> > baz's macro-function is compiled for the purposes of evaluation at
> > compile-time, that compilation must be as if it is being done in the
> > evaluation environment.
>
> But what if the definition of BAZ that is stored in the compilation
> environment is the *compiled* one.
There are two compilations here. One, for the fasl file to be loaded
at a later time, is compiled in the compilation environment. The other,
in service to the evaluation of the macro at compile-time, _must_ be
compiled in the evaluation environment. When the two environments are
identical, the compilations can probably be optimized into one, but if
the two environments are separate, and if foo was not stored into the
evaluation environment, then the compilations must be separate, and
the latter compilation must fail.
> Then when the macro expander is run
> in the evaluation environment it doesn't matter that FOO isn't found
> in that environment because it's already been compiled away.
This assumes that the compilation succeeded in the first place.
That is
> there are (at least) three ways BAZ could be evaluated:
>
> - BAZ is stored as a not-even minimally compiled sexps and is
> evaluated at macroexpansion time by pure interpretation in the
> evaluation environment. Under this strategy QUUX won't compile
> because FOO isn't in the evaluation environment.
Correct.
> - BAZ is stored as a not-even minimally compiled sexps and is
> evaluated at macroexpansion time by being compiled (in the
> evaluation environment) and the newly compiled expansion function
> being run. Under this strategy QUUX also won't compile because FOO
> isn't in the evaluation environment and thus the compilation of
> BAZ fails.
Correct also.
> - BAZ is (minimally) compiled in the compilation environment and
> that compiled function is stored as the definition in the
> compilation environment for later use.
This can't be done if the compilation and evaluation environments
are distinct, because the evaluation _must_ be done in the evaluation
environment.
(As well as being turned
> into the code that will be emitted to the .fasl file.) In this
> case QUUX should compile fine since the evaluation of BAZ is no
> longer dependent on the definition of FOO being available.
The only way for this to work is if the compilation and evaluation
environments are equivalent. But that creates a fourth case:
- BAZ is stored as not minimally compiled (i.e. interpreted) in
the compilation (evaluation) environment and is evaluated in the
evaluation (compilation) environment. Remember that FOO also had
to be stored this way earlier, so it thus must be found.
So now you have two cases (one interpreted and one compiled) that have
a single behavior, for separate environments, and two cases (one compiled
and one interpreted) which have a single behavior, for the same-environment
strategy. As I said, whether the forms are compiled or not makes no
difference.
> It seems to me that all three of these strategies are compatible with
> separating the compilation and evalution environments.
Incorrect.
> And the first
> two are really the same--just different strategies for "interpreting"
> BAZ.
Correct.
But with the later strategy, QUUX compiles fine.
Not with separated environments.
So perhaps you
> see why I think the key difference, given that the compilation and
> evalution environment are separate, is whether or not the definition
> of BAZ that is stored in the compilation environment is minimally
> compiled.
It is the mistakes I've pointed out above that lead to these incorrect
conclusions.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
I've been writing some code to help me follow this
very high level technical discussion. It is a toy compiler
that recognises defmacro and does minimal compilation. It
reads and writes lists, not files.
I think that I have developed the code to the point that it
illuminates the issues.
(defvar *output-list* nil)
(defvar *compilation-environment* nil) ;assoc list of macros
;;; compile the forms one by one
;;; side-effect on the global variable *compilation-environment
;;; allow each form to influence the compilation of subsequent forms
(defun compile-list(forms)
(dolist (form forms)
(compile-form form)))
(defun expand-list (forms)
(if (consp forms)
(mapcar #'expand-form forms)
forms))
(defun expand-form(form)
(cond ((atom form) form) ;no symbol macros
((assoc (car form) *compilation-environment*)
;; notice the unusual form of recursion
;; with expanded forms being resubmitted
;; to the expander so that it runs until
;; quiescent
(expand-form (run-expander (cdr (assoc (car form)
*compilation-environment*))
(cdr form))))
(t (cons (car form)
(expand-list (cdr form))))))
(defun run-expander (macro-definition arg-list)
(progv (car macro-definition) arg-list
(eval (cons 'progn (cdr macro-definition)))))
;;; some accessors
(defun function-name (defun) (second defun))
(defun macro-name (defmacro)(second defmacro))
(defun arg-list (defun) (third defun))
(defun body-forms (defun) (cdddr defun))
(defun compile-form (f)
(cond ((eql (car f) 'defun)
(push (list* 'code-for
'defun
(function-name f)
(arg-list f)
(expand-list (body-forms f)))
*output-list*))
((eql (car f) 'defmacro)
(push (list* 'code-for
'defmacro
(macro-name f)
(arg-list f)
(expand-list (body-forms f)))
*output-list*)
(push (cdr f) *compilation-environment*))
(t (push (cons 'code-for (expand-form f))
*output-list*))))
(defun driver (code)
(setf *output-list* nil
*compilation-environment* nil)
(compile-list code)
(values (reverse *output-list*)
*compilation-environment*))
I've converted the backquotes to LIST etc, to avoid
confusing myself. My code reproduces the behaviour in
question.
(driver '((defmacro foo (form)(list 'progn '(print 'foo) form))
(defun bar () (foo (print 'x)))
(defmacro baz (form) (foo (list 'progn form)))
(defmacro biff (form) (list 'progn (foo form)))))
((CODE-FOR DEFMACRO FOO (FORM)
(LIST 'PROGN '(PRINT 'FOO) FORM))
(CODE-FOR DEFUN BAR NIL
(PROGN (PRINT 'FOO) (PRINT 'X)))
(CODE-FOR DEFMACRO BAZ (FORM)
(PROGN
(PRINT 'FOO)
(LIST 'PROGN FORM)))
(CODE-FOR DEFMACRO BIFF (FORM)
(LIST 'PROGN
(PROGN (PRINT 'FOO) FORM))))
((BIFF (FORM) (LIST 'PROGN (FOO FORM)))
(BAZ (FORM) (FOO (LIST 'PROGN FORM)))
(FOO (FORM)
(LIST 'PROGN '(PRINT 'FOO) FORM)))
(driver '((defmacro foo (form)(list 'progn '(print 'foo) form))
(defun bar () (foo (print 'x)))
(defmacro baz (form) (foo (list 'progn form)))
(defmacro biff (form) (list 'progn (foo form)))
(defmacro quux (form) (baz (list 'progn form)))))
Error in KERNEL:%COERCE-TO-FUNCTION: the function FOO is undefined.
It is clear what has happened. BAZ has been expanded, ready
for generating machine code, in the output file, but not in
the *compilation-environment*. So when RUN-EXPANDER calls
EVAL, the code for BAZ has FOO in it. This matters because
EVAL does not know about FOO.
Well, that is conforming behaviour so we could stop there.
If we want to make the example run, by changing the
implementation, not by adding eval-when to the source, we
have a choice of three options.
1)Add an extra step to defmacro processing that evals the
defmacro form. This puts it into the image, so that the
eval in run-expander can use it.
2&3)Macroexpand the code of the macroexpanders before
storing them in the compilation environment. There are two
distinct versions of this idea.
2)Use run-expander to macroexpand the macro bodies.
3)Use cl:macroexpand (recursively) to macroexpand the macro
bodies. This doesn't work, unless we tell the lisp image
about the macros by evaling the defmacro forms, in which
case it is not necessary.
I hope these three claims are true, I've run out of time to
change the code and run them to check. Call it an exercise
for lurkers :-)
Alan Crowe
Edinburgh
Scotland
Alan Crowe <····@cawtech.freeserve.co.uk> writes:
> I've been writing some code to help me follow this
> very high level technical discussion. It is a toy compiler
> that recognises defmacro and does minimal compilation. It
> reads and writes lists, not files.
>
> I think that I have developed the code to the point that it
> illuminates the issues.
It's a good start; thanks for taking the time to understand this
and to try to help others to do so as well. Your code needs
a little bit of work in order to:
1. Correctly emulate what is going on (you have no
*evaluation-environment* defined or used anywhere, nor
is there the necessary environment arguments to the various
functions that need them.
2. Demonstrate Peter's suggested optimization for only doing
the compilation once, and to then demonstrate why it is
impossible to assume that this ptimization can always
occur.
To start, I'd suggest a feature - let's call it :merged-env,
which can be either pushed onto the *features* list or not
(or deleted, if already pushed). Then, prepend the following
to your code:
#+merged-env
(define-symbol-macro *evaluation-environment* *compilation-environment*)
#-merged-env
(defvar *evaluation-environment* nil) ;assoc list of macros
then modify the functions that need them to accept an environment
argument, passing the correct argument to each based on what is
being done at the time.
I assume that compile-list is roughly equivalent to compile-file
in your toy compiler, and expand-form is roughly macroexpand,
and run-expander is similarly macroexpand-1. These latter two
need to accept an environment arg, and expand-form needs not to
hardwire *compilation-environment*, but instead use its argument.
Your comment about the unusual nature of the recusrion in expand-form
is incorrect; it is in fact the normal operation of macroexpand.
I haven't analyzed it thoroughly, but I think you can get away
with using eval in run-expander, because expand-form is doing all
of the work in getting information out of the (soon to be correct)
environment. But it may also be that a different approach closer
to macroexpand-1 must be taken, since the walk may have to consult
the given environment in order to know how to expand a macro when
that environment _does_ have the definition in it.
I'm sure there are a few other things that need to be tweaked,
but it seems you're very close to something that truly illustrates
the issues.
> It is clear what has happened. BAZ has been expanded, ready
> for generating machine code, in the output file, but not in
> the *compilation-environment*. So when RUN-EXPANDER calls
> EVAL, the code for BAZ has FOO in it. This matters because
> EVAL does not know about FOO.
I was going to respond to this, but I'll leave this until
after you change your code (if you choose to do so).
> Well, that is conforming behaviour so we could stop there.
>
> If we want to make the example run, by changing the
> implementation, not by adding eval-when to the source, we
> have a choice of three options.
>
> 1)Add an extra step to defmacro processing that evals the
> defmacro form. This puts it into the image, so that the
> eval in run-expander can use it.
This extra step would be the equivalent to wrapping the defmacro
in (eval-when (compile)...)
> 2&3)Macroexpand the code of the macroexpanders before
> storing them in the compilation environment. There are two
> distinct versions of this idea.
These macroexpansions must be done in the correct environment.
If evaluation and compilation environments are separate, then
they will consult different environments, possibly generating
different results.
> 2)Use run-expander to macroexpand the macro bodies.
I think that's what you're doing already. Perhaps you mean
"earlier".
> 3)Use cl:macroexpand (recursively) to macroexpand the macro
> bodies. This doesn't work, unless we tell the lisp image
> about the macros by evaling the defmacro forms, in which
> case it is not necessary.
>
> I hope these three claims are true, I've run out of time to
> change the code and run them to check. Call it an exercise
> for lurkers :-)
It would be interesting if you or others found some more time to
carry this toy example to conclusion. It would make a great
book example :-).
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3wty86exf.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
>> - BAZ is (minimally) compiled in the compilation environment and
>> that compiled function is stored as the definition in the
>> compilation environment for later use.
>
> This can't be done if the compilation and evaluation environments
> are distinct, because the evaluation _must_ be done in the
> evaluation environment.
What evaluation? The only evaluation that I see that needs to be done
to *compile* BAZ is evaluating FOO's expansion function after it is
found in the compilation environment. That is, if FOO's expansion
function uses other functions they must be available in the evaluation
environment. Because in my example FOO contain only calls to PROGN and
PRINT, both of which are available in the evaluation environment, this
should work fine. I understand that this may not be the way Allegro
does it but why is this a non-conforming implementation strategy.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> >> - BAZ is (minimally) compiled in the compilation environment and
> >> that compiled function is stored as the definition in the
> >> compilation environment for later use.
> >
> > This can't be done if the compilation and evaluation environments
> > are distinct, because the evaluation _must_ be done in the
> > evaluation environment.
>
> What evaluation?
This one from 3.2.1 (emphasis mine):
The evaluation environment is a run-time environment in
which macro expanders and code specified by eval-when to be
========^^^^^^^^^^^^^^^
evaluated are evaluated. All evaluations initiated by the compiler
============^^^^^^^^^^^^^==================^^^^^^^^^^^^^^^^^^^^^^^^^
take place in the evaluation environment.
When the compiler compiles a form, it calls macroexpand directly with
the compilation environment. When the compiler has to perform evaluation
itself, it must use the evaluation environment. Another way to look at
it: back to the original definition and, for the purpose of discussion:
(defmacro baz (form) `(foo (progn ,form)))
(defmacro baz (form) (foo `(progn ,form)))
In the first definition, which is the way most people are used to seeing
macros, the body of the macro is "data", and it is processed wholely by
the compiler and is macroexpanded in the compiler environment. The second
form (which, though not as common, is stll seen quite often in various
forms) has two components; a data component, and what I'll call a preprocessing
component. Another of the same kind of macro might be something like
(defmacro decide (&whole whole x y)
(if (symbolp x)
`(do-it-faster ,x ,y)
`,whole))
Note that this macro also has a data component (made up of two possible
result forms) and a preprocessing component, which decides which form to
use for the final compilation or evaluation. The (if (symbolp x) ... part
is being run in at macroexpand time, and if it is the compiler encountering
the macro then that preprocessing part is run in the evaluation environment.
> The only evaluation that I see that needs to be done
> to *compile* BAZ is evaluating FOO's expansion function after it is
> found in the compilation environment.
No, in your version of baz (the second one, above) the call to foo is
part of the preprocessing part of the macro, and so is expanded at
macroexpand time in an evaluation initiated by the compiler.
But also, as code to be eventually loaded into a lisp from a fasl file,
the whole form is also processed by the compiler and is macroexpanded
in the compilation environment for the purposes of storing into that
fasl file. Note that the former (the evaluation) is initiated not by the
compilation of baz, but by the use of baz in a compile-time macroexpansion
of quux, which also references baz as part of this "preprocessing" part
of the macro.
That is, if FOO's expansion
> function uses other functions they must be available in the evaluation
> environment.
FOO was not necessarily found when the macroexpansion for evaluation
occurred. The spec expressly prohibits this assumption. See again:
3.2.3.1.1
> Because in my example FOO contain only calls to PROGN and
> PRINT, both of which are available in the evaluation environment, this
> should work fine. I understand that this may not be the way Allegro
> does it but why is this a non-conforming implementation strategy.
Because you're assuming something that the spec expressly prohibits you
from assuming. What FOO references is irrelevant, if FOO itself is
not found.
I think that you've got yourself stuck into the mode of thinking that
there is only one macroexpansion of baz involved here. Well, there
may in fact be only one call to macroexpand, but it is conceptually
two macroexpansions, and they may have to be separate. I think once
you understand this, everything else will fall into place.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3fz4wsahu.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Duane Rettig <·····@franz.com> writes:
[snip]
> (defmacro baz (form) `(foo (progn ,form)))
>
> (defmacro baz (form) (foo `(progn ,form)))
[snip]
>> The only evaluation that I see that needs to be done to *compile*
>> BAZ is evaluating FOO's expansion function after it is found in the
>> compilation environment.
>
> No, in your version of baz (the second one, above) the call to foo
> is part of the preprocessing part of the macro, and so is expanded
> at macroexpand time in an evaluation initiated by the compiler.
Are you talking about when BAZ is compiled or when QUUX is compiled? I
assume, since Allegro has no problem with it, that it is legal for the
compiler to be able to macroexpand (foo `(progn ,form)) when compiling
BAZ. So you must be talking about when we compile QUUX.
I agree that the macro expansion function for BAZ is evaluated in the
evaluation environment when BAZ is expanded as part of the compilation
of QUUX. What I don't see is why this evaluation *must* itself involve
FOO. Why can't the definition of BAZ that is stored in the compilation
environment (and thus that well be used to macroexpand the BAZ form
that occurs in QUUX) be at least minimally compiled?
According to the definition of DEFMACRO:
If a defmacro form appears as a top level form, the compiler must
store the macro definition at compile time, so that occurrences of
the macro later on in the file can be expanded correctly.
It seems to me that the key question here is what "macro definition
means". I assume "the macro definition" means the macro expansion
function that will be run (in the evaluation environment) in order to
expand a cons whose CAR is BAZ. Assuming that's right, my further
contention is that this function *can legally be* a minimally compiled
version of BAZ, i.e. one in which the FOO form was expanded when BAZ
was compiled. I.e. no different than if I had written:
(defmacro baz (form) (progn (print 'foo) `(progn ,form)))
Clearly if I had written BAZ this way, then there would be no problem
compiling QUUX: the the compiler would evaluate BAZ's macro expansion
function in the evaluation environment, finding PROGN, PRINT, and
whatever the backquote is read as, and everything would be fine.
> FOO was not necessarily found when the macroexpansion for evaluation
> occurred. The spec expressly prohibits this assumption. See again:
> 3.2.3.1.1
Which macroexpansion are you talking about,
>> Because in my example FOO contain only calls to PROGN and
>> PRINT, both of which are available in the evaluation environment, this
>> should work fine. I understand that this may not be the way Allegro
>> does it but why is this a non-conforming implementation strategy.
>
> Because you're assuming something that the spec expressly prohibits you
> from assuming.
Okay, all I'm assuming is that the definition of BAZ that is stored in
the compilation environment can be minimially compiled. So where is
that "expressly prohibit[ed]".
> I think that you've got yourself stuck into the mode of thinking that
> there is only one macroexpansion of baz involved here.
Okay, maybe I'm stuck in that mode because I *do* think there is only
one macro expansion of BAZ. When QUUX is compiled the compiler looks
up the macro expansion function named BAZ in the compilation
environment and invokes it (via the macro expander machinery) passing
it the whole BAZ form. That function runs some code and eventually
returns a form which is then compiled. When does the other macro
expansion occur?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
On 2004-10-02 20:43, in article ·············@franz.com, "Duane Rettig"
<·····@franz.com> wrote:
>> Because in my example FOO contain only calls to PROGN and
>> PRINT, both of which are available in the evaluation environment, this
>> should work fine. I understand that this may not be the way Allegro
>> does it but why is this a non-conforming implementation strategy.
>
> Because you're assuming something that the spec expressly prohibits you
> from assuming. What FOO references is irrelevant, if FOO itself is
> not found.
It seems to me that this is the crux of the misunderstanding.
Peter's reference to "implementation strategy" refers to how one might
implement a Lisp *compiler*.
Duane's response is in reference to the "implementation strategy" employed
by Peter's for (baz ) and (quux ).
--jon
In article <··············@javamonkey.com>,
Peter Seibel <·····@javamonkey.com> wrote:
> Suppose I have a .lisp with the following contents:
>
> (defmacro foo (form) `(progn (print 'foo) ,form))
>
> (defun bar () (foo (print 'x)))
>
> (defmacro baz (form) (foo `(progn ,form)))
>
> (defmacro biff (form) `(progn ,(foo form)))
>
> (defmacro quux (form) (baz `(progn ,form)))
>
> When I try to COMPILE-FILE this file, Allegro (6.2 on GNU/Linux)
> complains at me about "attempt to call `FOO' which is an undefined
> function." If I remove the definition of QUUX it compiles cleanly.
> Other Lisp's I tried (CMUCL and SBCL) have no problem with it, even
> with QUUX there. The folks at Franz tried to explain to me why this is
> non-portable (separation of the compilation of evaluation environment)
> but I'm still confused so I'm hoping someone here can help me out.
>
> Here's my analysis of how this file will be processed by COMPILE-FILE
> and the effect it will have on the compilation and evaluation
> environment (which we will assume are separate.) Maybe someone can
> point out where I go astray, if I do.
Looks okay to me.
OpenMCL and LispWorks seem to have no problem with it, too.
Peter Seibel <·····@javamonkey.com> writes:
> Suppose I have a .lisp with the following contents:
>
> (defmacro foo (form) `(progn (print 'foo) ,form))
>
> (defun bar () (foo (print 'x)))
>
> (defmacro baz (form) (foo `(progn ,form)))
>
> (defmacro biff (form) `(progn ,(foo form)))
>
> (defmacro quux (form) (baz `(progn ,form)))
>
> When I try to COMPILE-FILE this file, Allegro (6.2 on GNU/Linux)
> complains at me about "attempt to call `FOO' which is an undefined
> function." If I remove the definition of QUUX it compiles cleanly.
> Other Lisp's I tried (CMUCL and SBCL) have no problem with it, even
> with QUUX there. The folks at Franz tried to explain to me why this is
> non-portable (separation of the compilation of evaluation environment)
> but I'm still confused so I'm hoping someone here can help me out.
clisp, cmucl, sbcl all compile and load this file with no
problem. Perhaps you should switch to professional tools? ;-)
> Here's my analysis of how this file will be processed by COMPILE-FILE
> and the effect it will have on the compilation and evaluation
> environment (which we will assume are separate.) Maybe someone can
> point out where I go astray, if I do.
>
> So we start at the top:
>
> (defmacro foo (form) `(progn (print 'foo) ,form))
>
> On seeing this form the compiler compiles its body to produce a
> macroexpander function and stores the definition in the compilation
> environment.
Why should it store it in the compilation environment?
Try:
(EVAL-WHEN (:COMPILE-TOPLEVEL :LOAD-TOPLEVEL :EXECUTE)
(defmacro foo (form) `(progn (print 'foo) ,form)))
Or perhaps more precisely, why shouldn't it garbage collect the
compilation environment once that top-level form has been compiled?
Try:
(WITH-COMPILATION-UNIT ()
(defmacro ...)
(defun ...)
...)
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we.
Pascal Bourguignon wrote:
>
> Why should it store it in the compilation environment?
Because the language spec says it should. It ought to work to say
;;; start of file
;;;
(defmacro with-grob ...)
(defun foo (...)
(with-grob ...))
;;; end of file
and compile that, and have it work. And lo, the language is specified
so it does.
--tim
"Tim Bradshaw" <··········@tfeb.org> writes:
> Pascal Bourguignon wrote:
> >
> > Why should it store it in the compilation environment?
>
> Because the language spec says it should. It ought to work to say
I expressed myself incorrectly.
"DEFUN is to required to perform any compile-time side effects."
If a macro use a function, the programmer has to ensure that this
function is in the compilation environment with EVAL-WHEN.
The macro QUUX uses the macro BAZ which uses the function FOO, but at
macro-expansion time invoked at compilation time, FOO is not defined,
hence the error detected by this particular implementation.
I'd say either change of implementation or write your own DEFUN:
(defpackage "COMMON-LISP-CORRECTED"
(:use "COMMON-LISP")
(:shadow "DEFUN")
#.(cons :export (let ((syms '()))
(do-external-symbols (s "COMMON-LISP")
(push s syms)) syms)))
(in-package "COMMON-LISP-CORRECTED")
(defmacro defun (&rest args)
`(eval-when (:compile-toplevel :load-toplevel :execute)
(common-lisp:defun ,@args)))
(in-package "COMMON-LISP-USER")
(defpackage "MY-APP"
(:use "COMMON-LISP-CORRECTED"))
(in-package "MY-APP")
(defmacro foo (form) `(progn (print 'foo) ,form))
(defun bar () (foo (print 'x)))
(defmacro baz (form) (foo `(progn ,form)))
(defmacro biff (form) `(progn ,(foo form)))
(defmacro quux (form) (baz `(progn ,form)))
This should work even on the OP's implementation.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we.
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3k6ua8g84.fsf@javamonkey.com>
Pascal Bourguignon <····@mouse-potato.com> writes:
> "Tim Bradshaw" <··········@tfeb.org> writes:
>
>> Pascal Bourguignon wrote:
>> >
>> > Why should it store it in the compilation environment?
>>
>> Because the language spec says it should. It ought to work to say
>
> I expressed myself incorrectly.
>
> "DEFUN is to required to perform any compile-time side effects."
I assume that first "to" is supposed to be a "not". Anyway, here's the
quote cut-n-pasted from the Hyperspec:
"defun is not required to perform any compile-time side effects. In
particular, defun does not make the function definition available at
compile time." (emphasis mine)
On the other hand:
"If a defmacro form appears as a top level form, the compiler must
store the macro definition at compile time, so that occurrences of
the macro later on in the file can be expanded correctly. Users must
ensure that the body of the macro can be evaluated at compile time
if it is referenced within the file being compiled."
> If a macro use a function, the programmer has to ensure that this
> function is in the compilation environment with EVAL-WHEN.
Yes. But none of my macros use functions; only other macros.
> The macro QUUX uses the macro BAZ which uses the function FOO,
^^^^^^^^
Look again, FOO is a macro. The only function was BAR which is never
used--I just threw it in so I could talk about compiling a function.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Pascal Bourguignon <····@mouse-potato.com> writes:
>
> > "Tim Bradshaw" <··········@tfeb.org> writes:
> >
> >> Pascal Bourguignon wrote:
> >> >
> >> > Why should it store it in the compilation environment?
> >>
> >> Because the language spec says it should. It ought to work to say
> >
> > I expressed myself incorrectly.
> >
> > "DEFUN is to required to perform any compile-time side effects."
>
> I assume that first "to" is supposed to be a "not". Anyway, here's the
> quote cut-n-pasted from the Hyperspec:
>
> "defun is not required to perform any compile-time side effects. In
> particular, defun does not make the function definition available at
> compile time." (emphasis mine)
>
> On the other hand:
>
> "If a defmacro form appears as a top level form, the compiler must
> store the macro definition at compile time, so that occurrences of
> the macro later on in the file can be expanded correctly. Users must
> ensure that the body of the macro can be evaluated at compile time
> if it is referenced within the file being compiled."
And so, in fact, foo's definition is indeed stored at compile-time,
in the compiler-environment. The compiler-environment is the one which
is used to generate forms for later loading for such forms as
(defmacro bas2 (form) `(foo (progn ,form)))
or even
(defun basfunc (form) (foo (progn form)))
(note that in either of these situations, compile-time evaluation is
not occurring, all macroexpansion is being done with the compiler
environment with the explicit purpose of generating code for load-time)
But the spec explicitly leaves unspecified whether the definiton is
to be saved in the _evaluation_ environment, which is what the
macro-function actually runs in. See my previous message elsewhere
in this thread.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
I find this discussion very interesting, but I'm having a hard time
following it. I went back to the Hyperspec and compared Peter's
original description to what I could understand in there. I have a few
comments/questions which may or may not bear directly on Peter's
question.
Peter Seibel <·····@javamonkey.com> writes:
> (defun bar () (foo (print 'x)))
>
> I assume this is legal since otherwise the requirement that DEFMACRO
> store the macro definition in the compilation environment doesn't mean
> much of anything. More particularly, in order to compile this the
> compiler must look up FOO in the compilation environment and run its
> macroexpander (in the evaluation environment) and then compile the
> resulting form.
>
> (defmacro baz (form) (foo `(progn ,form)))
>
> This seems to me no different from compiling a function: the compiler
> looks up FOO in the compilation environment and runs the
> macroexpanderand and then compiles the result. In this case the result
> is BAZ's macroexpander which is then also stored in the compilation
> environment.
defmacro associates baz with an expansion function, the body of which
is (foo `(progn ,form)). I can't see any requirement that the
expansion function be compiled or otherwise processed, so the compiler
may not even look at the symbol foo, and isnt' required to expand it
before storing the expansion function. Ergo there's no guarantee that
foo would be looked up in the compilation environment. Is that right?
(I.e. 3.2.2.2 says that macro calls in source code being compiled are
expanded, but the body of baz doesn't necessarily qualify as "source
code being compiled", does it?)
If baz is used by code that is compiled, the compiler finds the
definition in the compilation environment, but the expander function
is evaluated in the evaluation environment, which might not have the
definition of foo required by the evaluator.
>
> (defmacro biff (form) `(progn ,(foo form)))
>
> Should be the same as compiling BAZ: to compile the call to FOO it uses
> the compilation environment to find the macroexpander, expands it and
> compiles the resulting form.
This is different. If biff is subsequently called by code being
compiled, then the call is expanded to a form that references
foo. When the expansion is then compiled, foo is defined in the
compilation environment.
> FWIW, a file containing only the forms mentioned so far will in fact
> compile cleanly in Allegro 6.2. Now we get to the case that Allegro
> won't compile, complaining about no function definition for FOO:
>
> (defmacro quux (form) (baz `(progn ,form)))
If I was right about my earlier assumptions, then should associate an
(uncompiled?) expander function with quux in the compilation
environment. If the expander function is not compiled, then the macro
form "(baz ...)" won't be expanded.
I don't understand why adding the macro definition of quux triggers
the error, but I think I do understand why baz doesn't work. I would
think that you would actually have to use quux somewhere to see the
problem, so that its macro expander function is evaluated. Perhaps
expander functions _are_ being compiled, but in a different
compilation environment from the source code being compiled?
How does one deal with this problem? If macro A is used by other
macros in the same compilation, do you need to wrap the defmacro form
with (eval-when (:execute) ...)? (Did I miss a discussion of this in
the other replies?)
3.2.3.1.1 seems to address a similar problem. This raises more
questions for me. If a source file that contains a simple defmacro
form at top-level is loaded, the definition of LOAD requires that each
top-level form be executed. But if you compile that source file, does
the compiler necessarily have to leave any trace of the defmacro form
in the compiled version of the file? If I load the compiled file, is
the macro necessarily be defined in my runtime environment? I think it
must be defined, because defmacro is itself a macro, and 3.2.3.1 says
that the expansion of a top-level macro form gets compiled. OK, that's
a relief.
Thanks in advance for further clarifications on this complex topic.
Dan Muller <·········@sneakemail.com> writes:
> I find this discussion very interesting, but I'm having a hard time
> following it. I went back to the Hyperspec and compared Peter's
> original description to what I could understand in there. I have a few
> comments/questions which may or may not bear directly on Peter's
> question.
It's hard to follow because it is a complex subject. This is one
of he places where all that's different between Lisp and other languages
comes together. And because Common Lisp is borne of many different
Lisp dialects, and when CLtL1 was first implemented there were many
different interpretations of what eval-when meant, and there was no
concept of "top-level processing"; most of the defining forms did
what they needed to by "magic", as a black box. So for the spec,
the X3J13 committee worked out the details for eval-when processing,
and defining forms were mandated to be macros. A lot of what goes
on in a defining form can be seen explicitly by doing soemthing
like (pprint (macroexpand '...)) on that form. However, since the
magic has been exposed and demystified, what is left is complex.
Fortunately, the complexity that Peter has come across is not
often seen, because the evalution portion of macro definitions
don't tend to be defined in terms of other macros; they are usually
in terms of CL operators, or in terms of user-defined functions
already in the compiling lisp's global environment or else macros
that have been wrapped in an eval-when to activate them early.
> Peter Seibel <·····@javamonkey.com> writes:
>
> > (defun bar () (foo (print 'x)))
> >
> > I assume this is legal since otherwise the requirement that DEFMACRO
> > store the macro definition in the compilation environment doesn't mean
> > much of anything. More particularly, in order to compile this the
> > compiler must look up FOO in the compilation environment and run its
> > macroexpander (in the evaluation environment) and then compile the
> > resulting form.
> >
> > (defmacro baz (form) (foo `(progn ,form)))
> >
> > This seems to me no different from compiling a function: the compiler
> > looks up FOO in the compilation environment and runs the
> > macroexpanderand and then compiles the result. In this case the result
> > is BAZ's macroexpander which is then also stored in the compilation
> > environment.
>
> defmacro associates baz with an expansion function, the body of which
> is (foo `(progn ,form)). I can't see any requirement that the
> expansion function be compiled or otherwise processed, so the compiler
> may not even look at the symbol foo, and isnt' required to expand it
> before storing the expansion function. Ergo there's no guarantee that
> foo would be looked up in the compilation environment. Is that right?
Absolutely correct. It _will_ do a macroexpansion on foo, in order to
compile it for storage in the fasl file, and _this_ macroexpansion
is done in the compilation environment. But because the compiler is
not trying to "call" bar, only to macroexpand its definition for the
compilation process, there is no problem finding foo (because it looks
in the compilation environment) and the definition of baz in the fasl
file will have foo already pre-macroexpanded.
> (I.e. 3.2.2.2 says that macro calls in source code being compiled are
> expanded, but the body of baz doesn't necessarily qualify as "source
> code being compiled", does it?)
There's a difference between the definition of baz and "call" to it;
i.e. the form with the macro's name in the car.
> If baz is used by code that is compiled, the compiler finds the
> definition in the compilation environment, but the expander function
> is evaluated in the evaluation environment, which might not have the
> definition of foo required by the evaluator.
It depends on who invoked the expander function. If the compiler,
operating on code it is processing for the file it is compiling,
macroexpands the form, then that macroexpansion takes place in
the compiler environment. But if during the execution of a macro
on behalf of the compiler the macro is expanded, it is done in the
evaluation environment.
> > (defmacro biff (form) `(progn ,(foo form)))
> >
> > Should be the same as compiling BAZ: to compile the call to FOO it uses
> > the compilation environment to find the macroexpander, expands it and
> > compiles the resulting form.
>
> This is different. If biff is subsequently called by code being
> compiled, then the call is expanded to a form that references
> foo. When the expansion is then compiled, foo is defined in the
> compilation environment.
>
> > FWIW, a file containing only the forms mentioned so far will in fact
> > compile cleanly in Allegro 6.2. Now we get to the case that Allegro
> > won't compile, complaining about no function definition for FOO:
> >
> > (defmacro quux (form) (baz `(progn ,form)))
>
> If I was right about my earlier assumptions, then should associate an
> (uncompiled?) expander function with quux in the compilation
> environment. If the expander function is not compiled, then the macro
> form "(baz ...)" won't be expanded.
>
> I don't understand why adding the macro definition of quux triggers
> the error, but I think I do understand why baz doesn't work. I would
> think that you would actually have to use quux somewhere to see the
> problem, so that its macro expander function is evaluated. Perhaps
> expander functions _are_ being compiled, but in a different
> compilation environment from the source code being compiled?
> How does one deal with this problem? If macro A is used by other
> macros in the same compilation, do you need to wrap the defmacro form
> with (eval-when (:execute) ...)? (Did I miss a discussion of this in
> the other replies?)
Likely an (eval-when (:compile-toplevel ...) ...) form around any macro
that is needed later in this file at an eval-time position. This will
place the definition into the evaluation environment, because of 3.2.3.1.
> 3.2.3.1.1 seems to address a similar problem. This raises more
> questions for me. If a source file that contains a simple defmacro
> form at top-level is loaded, the definition of LOAD requires that each
> top-level form be executed. But if you compile that source file, does
> the compiler necessarily have to leave any trace of the defmacro form
> in the compiled version of the file?
No, it leaves the expansion. Note that the expansion can itself be
multiple forms, including other macro expansions or even other defining
forms.
> If I load the compiled file, is
> the macro necessarily be defined in my runtime environment?
As long as the defmacro is not wrapped in (eval-when (:compile-toplevel)...)
it is. Try putting your defmacro form within (pprint (macroexpand '...))
to see how (how much the implementation shows you depends on how much of
the magic it has removed for you).
I think it
> must be defined, because defmacro is itself a macro, and 3.2.3.1 says
> that the expansion of a top-level macro form gets compiled. OK, that's
> a relief.
Yes, of course.
> Thanks in advance for further clarifications on this complex topic.
Hopefully this clarifies things and doesn't muddy them up even
further :-)
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m37jq8s8ik.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
> Dan Muller <·········@sneakemail.com> writes:
>
>> defmacro associates baz with an expansion function, the body of which
>> is (foo `(progn ,form)). I can't see any requirement that the
>> expansion function be compiled or otherwise processed, so the compiler
>> may not even look at the symbol foo, and isnt' required to expand it
>> before storing the expansion function. Ergo there's no guarantee that
>> foo would be looked up in the compilation environment. Is that right?
>
> Absolutely correct. It _will_ do a macroexpansion on foo, in order
> to compile it for storage in the fasl file, and _this_
> macroexpansion is done in the compilation environment. But because
> the compiler is not trying to "call" bar, only to macroexpand its
> definition for the compilation process, there is no problem finding
> foo (because it looks in the compilation environment) and the
> definition of baz in the fasl file will have foo already
> pre-macroexpanded.
So where does the standard prohibit the compiler from storing the
compiled version of BAZ's expansion function in the compilation
environment?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
>> Dan Muller <·········@sneakemail.com> writes:
>>
>>> defmacro associates baz with an expansion function, the body of which
>>> is (foo `(progn ,form)). I can't see any requirement that the
>>> expansion function be compiled or otherwise processed, so the compiler
>>> may not even look at the symbol foo, and isnt' required to expand it
>>> before storing the expansion function. Ergo there's no guarantee that
>>> foo would be looked up in the compilation environment. Is that right?
>>
>> Absolutely correct. It _will_ do a macroexpansion on foo, in order
>> to compile it for storage in the fasl file, and _this_
>> macroexpansion is done in the compilation environment. But because
>> the compiler is not trying to "call" bar, only to macroexpand its
>> definition for the compilation process, there is no problem finding
>> foo (because it looks in the compilation environment) and the
>> definition of baz in the fasl file will have foo already
>> pre-macroexpanded.
>
> So where does the standard prohibit the compiler from storing the
> compiled version of BAZ's expansion function in the compilation
> environment?
>
It doesn't prohibit it. But it doesn't require it, either. There are
typically _two_ copies of the expansion function at issue here. One
goes in the fasl (if the defmacro wasn't wrapped in "(eval
(:comile-toplevel) ...)"), and one is stored in the compilation
environment. There's no requirement that the latter be compiled -- it
could be stored as a sexpr, for instance. If it is compiled, and the
evaluation and compilation environments are different, then I suspect
that the copy for the compilation environment would have to be
compiled in the _evaluation_ environment to be fully conforming,
because when that copy is evaluated by the compiler, that's the
environment that this should happen in.
I hope I got that right...
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m33c0vsjkq.fsf@javamonkey.com>
Dan Muller <·········@sneakemail.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Duane Rettig <·····@franz.com> writes:
>>
>>> Dan Muller <·········@sneakemail.com> writes:
>>>
>>>> defmacro associates baz with an expansion function, the body of which
>>>> is (foo `(progn ,form)). I can't see any requirement that the
>>>> expansion function be compiled or otherwise processed, so the compiler
>>>> may not even look at the symbol foo, and isnt' required to expand it
>>>> before storing the expansion function. Ergo there's no guarantee that
>>>> foo would be looked up in the compilation environment. Is that right?
>>>
>>> Absolutely correct. It _will_ do a macroexpansion on foo, in order
>>> to compile it for storage in the fasl file, and _this_
>>> macroexpansion is done in the compilation environment. But because
>>> the compiler is not trying to "call" bar, only to macroexpand its
>>> definition for the compilation process, there is no problem finding
>>> foo (because it looks in the compilation environment) and the
>>> definition of baz in the fasl file will have foo already
>>> pre-macroexpanded.
>>
>> So where does the standard prohibit the compiler from storing the
>> compiled version of BAZ's expansion function in the compilation
>> environment?
>>
>
> It doesn't prohibit it. But it doesn't require it, either.
That's what I think. But Duane keeps telling me that it is prohibited.
Or at least that's what I understand him to be saying.
> There are typically _two_ copies of the expansion function at issue
> here. One goes in the fasl (if the defmacro wasn't wrapped in "(eval
> (:comile-toplevel) ...)"), and one is stored in the compilation
> environment. There's no requirement that the latter be compiled --
> it could be stored as a sexpr, for instance. If it is compiled, and
> the evaluation and compilation environments are different, then I
> suspect that the copy for the compilation environment would have to
> be compiled in the _evaluation_ environment to be fully conforming,
> because when that copy is evaluated by the compiler, that's the
> environment that this should happen in.
>
> I hope I got that right...
It's that last bit that I'm not convinced about. The spec says (Sec
3.2.1):
The evaluation environment is a run-time environment in which macro
expanders and code specified by eval-when to be evaluated are
evaluated. All evaluations initiated by the compiler take place in
the evaluation environment.
But it's not clear to me that there's any "evaluation" that takes
place when compiling BAZ other than the invocation of FOO's macro
expander. I.e. to compile BAZ the we can find the definition of FOO in
the compilation environment but then evaluate it (i.e. invoke its
macro expander) in the evaluation environment. Which means that any
functions and variables used in FOO to compute its expansion must be
available in the evaluation environment. But after FOO's macro
expander has been run, and its expansion compiled, I claim we have a
"definition" of BAZ that can be stored in the compilation environment
that has no further dependency on FOO. So the fact that BAZ's macro
expander will later be invoked in the evaluation environment is fine
because it is no longer dependent on the definition of FOO.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter,
I appreciate all of the effort that you've put into elucidating the darker
corners of these issues for the rest of us.
On 2004-10-03 15:16, in article ··············@javamonkey.com, "Peter
Seibel" <·····@javamonkey.com> wrote:
> That's what I think. But Duane keeps telling me that it is prohibited.
> Or at least that's what I understand him to be saying.
Actually, I think what he's saying is that it is prohibited for you to
*assume* that things will be done the way you expect them to be done. While
they can be done that way, they don't *have* to be done that way. And
that's why your definition is non-portable.
This thread has been super interesting. I hate to say it, but I think
I've gotten bitten by the "time to write my own lisp compiler/interpreter"
but. Just so that I can understand this issue better.
--jon
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3acv3qvmt.fsf@javamonkey.com>
Jon Boone <········@comcast.net> writes:
> Peter,
>
> I appreciate all of the effort that you've put into elucidating the
> darker corners of these issues for the rest of us.
>
>
> On 2004-10-03 15:16, in article ··············@javamonkey.com, "Peter
> Seibel" <·····@javamonkey.com> wrote:
>
>> That's what I think. But Duane keeps telling me that it is
>> prohibited. Or at least that's what I understand him to be saying.
>
> Actually, I think what he's saying is that it is prohibited for you
> to *assume* that things will be done the way you expect them to be
> done. While they can be done that way, they don't *have* to be done
> that way. And that's why your definition is non-portable.
If that's it, then we're done. But I think Duanne and I had moved on
to much more pedantic and theoretical point. I agree that Allegro's
implementation is legit. I disagree with the claim--the one I think
Duanne is making--that the failure to compile my original file is an
inevitable consequence of the decision to keep the compilation and
evaluation environments separate. I claim that an implementation that
kept those two environments separate but which also stored minimally
(at least) compiled versions of macros in the compilation environment
would be conforming *and* would compile my original test file.
So there are three possible outcomes to this debate (other than one or
the other of us just giving up):
- Someone (maybe me) figures out a way to help me understand why it's
*illegal* for a Common Lisp compiler to store minimally compiled
macro definitions in the compilation environment.
- Someone (maybe me) figures out a way to help me understand why even
if the compiler *does* store a minimally compiled macro definition
for BAZ in the compilation environment the compilation of QUUX
*still* fails.
- Duane conceeds that storing minimally compiled macro definitions in
the compilation environment is conforming and would allow my
original file to compile even in an implementation with separate
compilation and evaluation environments.
Note also that I am *not* making any claim as to whether such an
implementation strategy (for a Common Lisp compiler) is better in any
way than either Allegro's or the implementations that simply merge the
compilation and evaluation environments. Just saying it's legal.
> This thread has been super interesting. I hate to say it, but I
> think I've gotten bitten by the "time to write my own lisp
> compiler/interpreter" but. Just so that I can understand this issue
> better.
Heh. Well, at least write it in Common Lisp.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Jon Boone <········@comcast.net> writes:
>
> > Peter,
> >
> > I appreciate all of the effort that you've put into elucidating the
> > darker corners of these issues for the rest of us.
> >
> >
> > On 2004-10-03 15:16, in article ··············@javamonkey.com, "Peter
> > Seibel" <·····@javamonkey.com> wrote:
> >
> >> That's what I think. But Duane keeps telling me that it is
> >> prohibited. Or at least that's what I understand him to be saying.
> >
> > Actually, I think what he's saying is that it is prohibited for you
> > to *assume* that things will be done the way you expect them to be
> > done. While they can be done that way, they don't *have* to be done
> > that way. And that's why your definition is non-portable.
>
> If that's it, then we're done. But I think Duanne and I had moved on
> to much more pedantic and theoretical point.
This is the case; we're not yet done. I had to take the rest of the
weekend off to gather my thoughts and to enlist the help of my colleague
Steve Haflich, who was on the X3J13 committee and had a large part in
designing the current eval-when strategy. To reiterate the pedantic
and theoretical point, I believe that it is illegal for a compiler to
macroexpand once for the two purposes of the macroexpansion, and thus,
if the implementation provides separate compilation and evaluation
environments (i.e. it chooses not to store a macro definition into the
evaluation environment), then your example _must_ fail. I think that
this is the only issue left on the table.
> I agree that Allegro's
> implementation is legit. I disagree with the claim--the one I think
> Duanne is making--that the failure to compile my original file is an
> inevitable consequence of the decision to keep the compilation and
> evaluation environments separate. I claim that an implementation that
> kept those two environments separate but which also stored minimally
> (at least) compiled versions of macros in the compilation environment
> would be conforming *and* would compile my original test file.
>
> So there are three possible outcomes to this debate (other than one or
> the other of us just giving up):
>
> - Someone (maybe me) figures out a way to help me understand why it's
> *illegal* for a Common Lisp compiler to store minimally compiled
> macro definitions in the compilation environment.
Hopefully Steve's response to me (with footnoted commentd by me) will
help to do just that. Note that it is not really the storing of compiled
macro definitions that is at issue; the problem is what environment the
definition was compiled in that eventually is used for the evaluation.
In the case where the compilation and evaluation environments are
separate, the definition compiled in the compilation environment cannot
be used in place of one which was supposed to be evaluated (or compiled
and executed) in the evaluation environment.
> - Someone (maybe me) figures out a way to help me understand why even
> if the compiler *does* store a minimally compiled macro definition
> for BAZ in the compilation environment the compilation of QUUX
> *still* fails.
I don't know where you've seen this; obviously there is no
implementation that does this (for separated environments), so I
assume you are theorizing here.
> - Duane conceeds that storing minimally compiled macro definitions in
> the compilation environment is conforming and would allow my
> original file to compile even in an implementation with separate
> compilation and evaluation environments.
No way. Never happen. You can't make me do it!
:-)
> Note also that I am *not* making any claim as to whether such an
> implementation strategy (for a Common Lisp compiler) is better in any
> way than either Allegro's or the implementations that simply merge the
> compilation and evaluation environments. Just saying it's legal.
>
> > This thread has been super interesting. I hate to say it, but I
> > think I've gotten bitten by the "time to write my own lisp
> > compiler/interpreter" but. Just so that I can understand this issue
> > better.
>
> Heh. Well, at least write it in Common Lisp.
Agreed. Best language in the world to write a language in.
Including itself.
Here is Steve's response. I've also added footnotes in []
to underscore some of our current conversations.
=====
In all this, has anyone yet commented on the two examples in
3.2.3.1.1, which directly show the case in point? [1] Unfortunately
these paragraphs are botched because they speak in terms of "the
interpreter" which is not something semantically defined or required
to be present in ANSI CL. It should have written about the
definitions being "available in the evaluation environment" but this
inconsistency apparently slipped through proofreading.
[[1]: Note that I had mentioned this early on in the thread, and
have assumed that you had understood it. However, on the surface
it might appear that this section doesn't pertain to your question
about why there must be two macroexpansions. But, in fact, it does;
read on...]
[this is quoted from 3.2.3.1.1]:
In particular, the information stored by the defining macros at
compile time might or might not be available to the interpreter
(either during or after compilation), or during subsequent calls to
the compiler. For example, the following code is nonportable
because it assumes that the compiler stores the macro definition of
foo where it is available to the interpreter:
(defmacro foo (x) `(car ,x))
(eval-when (:execute :compile-toplevel :load-toplevel)
(print (foo '(a b c))))
A portable way to do the same thing would be to include the macro
definition inside the eval-when form, as in:
(eval-when (:execute :compile-toplevel :load-toplevel)
(defmacro foo (x) `(car ,x))
(print (foo '(a b c))))
In any case, the argument that there be only a single definition of a
macro created by defmacro at compile-file time is obviously bogus:
Also from 3.2.3.1.1:
A convenient model for explaining how these side effects happen is
that the defining macro expands into one or more eval-when forms,
and that the calls which cause the compile-time side effects to
happen appear in the body of an (eval-when (:compile-toplevel) ...)
form.
Further the definition of eval-when 3.2.3.1 requires that:
6. Otherwise, the form is a top level form that is not one of the
special cases. In compile-time-too mode, the compiler first evaluates
the form in the evaluation environment and then minimally compiles it.
In not-compile-time mode, the is simply minimally compiled. All are
treated as non-top-level forms.
Now, the macroexpansion of defmacro must result in one or the other of
these schemes:
(progn (eval-when (compile) ...mumble-macro-function...)
(eval-when (load eval) ...mooble-macro-function...)
or
(progn (eval-when (compile load eval) ...mumble-macro-function...)
[Relating this part in his treatise to section 3.2.3.1, item 6 above
may seem like distant logic, but it really is a simple following of
the spec, step by step; whatever is in the eval-when form is going
to be either one of the kinds of form in the first 5 steps the same
page, or it's not (it's likely to be either some kind of
(setf (macro-function ...) ...) form, or else a call to some internal
function to do some magic). In either case, we eventually get to
step 6 which now becomes apropos, and we're either in a compile-time-too
mode or we're not, from any surrounding eval-when wrappings]
Either way, the definition of eval-when results in the macro-function
creation code be executed twice. It first evaluates the form that
creates the macro, and _then) minimally compiles _the_ _form_ [2]. It
would make no sense to say that the result of evaluating the form is
somehow compiled -- that result for defmacro is the symbol that names
the macro, not the macro-function.
[[3] Note that the actual wording in item 6 says "it", but it is
clear what the "it" stands for in this case. So whatever is _stored_
into the macro definition at any particular time, the actual
compile-time-too macroexpansion _must_ occur from the form itself,
not from some preprocessed code.]
=====
Saturday I had gotten myself mired in your question of potential
compiler optimizations for the copmile-time-too evaluation, and was
considering pointing out the glossary term for "evaluation" (which
states that the two-step option of compiling and then executing be
not detectable fro the one-step option of using an interpreter),
and this might contributes to the "why" of the explicit spec, but
it is within itself a weak argument in answering your question. I
see that someone else saw the same definition, but I believe that
Steve's logic provides the definitive answer.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3y8im1iu3.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
[Quoting Steve Haflich, I think]
> Now, the macroexpansion of defmacro must result in one or the other
> of these schemes:
>
> (progn (eval-when (compile) ...mumble-macro-function...)
> (eval-when (load eval) ...mooble-macro-function...)
>
> or
>
> (progn (eval-when (compile load eval) ...mumble-macro-function...)
Okay, I think I'm almost there. Before I post another big step-by-step
analysis, can someone confirm that the following is a reasonable
simplification of what this:
(defmacro foo (form) `(progn (print 'foo) ,form))
would expand into:
(progn
(eval-when (:compile-toplevel)
(setf (macro-function 'foo)
#'(lambda (whole env)
(declare (ignore env))
(with-parsed-macro-lambda-list (form) whole
`(progn ,form)))))
(eval-when (:load-toplevel :execute)
(setf (macro-function 'foo)
#'(lambda (whole env)
(declare (ignore env))
(with-parsed-macro-lambda-list (form) whole
`(progn ,form)))))
'foo)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3u0ta1iq0.fsf@javamonkey.com>
Peter Seibel <·····@javamonkey.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> [Quoting Steve Haflich, I think]
>
>> Now, the macroexpansion of defmacro must result in one or the other
>> of these schemes:
>>
>> (progn (eval-when (compile) ...mumble-macro-function...)
>> (eval-when (load eval) ...mooble-macro-function...)
>>
>> or
>>
>> (progn (eval-when (compile load eval) ...mumble-macro-function...)
>
>
> Okay, I think I'm almost there. Before I post another big step-by-step
> analysis, can someone confirm that the following is a reasonable
> simplification of what this:
>
> (defmacro foo (form) `(progn (print 'foo) ,form))
>
> would expand into:
>
> (progn
> (eval-when (:compile-toplevel)
> (setf (macro-function 'foo)
> #'(lambda (whole env)
> (declare (ignore env))
> (with-parsed-macro-lambda-list (form) whole
> `(progn ,form)))))
>
> (eval-when (:load-toplevel :execute)
> (setf (macro-function 'foo)
> #'(lambda (whole env)
> (declare (ignore env))
> (with-parsed-macro-lambda-list (form) whole
> `(progn ,form)))))
> 'foo)
BTW, that is intended for a Lisp that separates the compilation and
evaluation environments.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
> > Duane Rettig <·····@franz.com> writes:
> >
> > [Quoting Steve Haflich, I think]
Correct.
> >> Now, the macroexpansion of defmacro must result in one or the other
> >> of these schemes:
> >>
> >> (progn (eval-when (compile) ...mumble-macro-function...)
> >> (eval-when (load eval) ...mooble-macro-function...)
> >>
> >> or
> >>
> >> (progn (eval-when (compile load eval) ...mumble-macro-function...)
> >
> >
> > Okay, I think I'm almost there. Before I post another big step-by-step
> > analysis, can someone confirm that the following is a reasonable
> > simplification of what this:
> >
> > (defmacro foo (form) `(progn (print 'foo) ,form))
> >
> > would expand into:
> >
> > (progn
> > (eval-when (:compile-toplevel)
> > (setf (macro-function 'foo)
> > #'(lambda (whole env)
> > (declare (ignore env))
> > (with-parsed-macro-lambda-list (form) whole
> > `(progn ,form)))))
> >
> > (eval-when (:load-toplevel :execute)
> > (setf (macro-function 'foo)
> > #'(lambda (whole env)
> > (declare (ignore env))
> > (with-parsed-macro-lambda-list (form) whole
> > `(progn ,form)))))
> > 'foo)
>
> BTW, that is intended for a Lisp that separates the compilation and
> evaluation environments.
Understood.
Depending on where you're going with this, the answer in spirit
to your question is "yes" (but not quite precisely; that's why I
qualify the affirmative answer).
However, why ask me? Why not ask the lisp itself, by using the
magic demystifying formula:
(pprint (macroexpand '(defmacro foo (form) `(progn (print 'foo) ,form))))
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3pt3y1e0k.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Peter Seibel <·····@javamonkey.com> writes:
>>
>> > Duane Rettig <·····@franz.com> writes:
>> >
>> > [Quoting Steve Haflich, I think]
>
> Correct.
>
>> >> Now, the macroexpansion of defmacro must result in one or the other
>> >> of these schemes:
>> >>
>> >> (progn (eval-when (compile) ...mumble-macro-function...)
>> >> (eval-when (load eval) ...mooble-macro-function...)
>> >>
>> >> or
>> >>
>> >> (progn (eval-when (compile load eval) ...mumble-macro-function...)
>> >
>> >
>> > Okay, I think I'm almost there. Before I post another big step-by-step
>> > analysis, can someone confirm that the following is a reasonable
>> > simplification of what this:
>> >
>> > (defmacro foo (form) `(progn (print 'foo) ,form))
>> >
>> > would expand into:
>> >
>> > (progn
>> > (eval-when (:compile-toplevel)
>> > (setf (macro-function 'foo)
>> > #'(lambda (whole env)
>> > (declare (ignore env))
>> > (with-parsed-macro-lambda-list (form) whole
>> > `(progn ,form)))))
>> >
>> > (eval-when (:load-toplevel :execute)
>> > (setf (macro-function 'foo)
>> > #'(lambda (whole env)
>> > (declare (ignore env))
>> > (with-parsed-macro-lambda-list (form) whole
>> > `(progn ,form)))))
>> > 'foo)
>>
>> BTW, that is intended for a Lisp that separates the compilation and
>> evaluation environments.
>
> Understood.
>
> Depending on where you're going with this, the answer in spirit to
> your question is "yes" (but not quite precisely; that's why I
> qualify the affirmative answer).
I think I know why. See below.
> However, why ask me? Why not ask the lisp itself, by using the magic
> demystifying formula:
>
> (pprint (macroexpand '(defmacro foo (form) `(progn (print 'foo) ,form))))
Key word: "simplification". There's a lot of gorp in any actual
implementation of DEFMACRO; I'm trying to get down to the essence.
Anyway, I think the answer to my question is "no". And better news, I
think I finally understand what you and Steve have been trying to
hammer into my thick skull.
I'll try another step-by-step analysis, this time showing why you are
correct.
To start with, here's why the expansion of DEFMACRO I gave before
can't be quite right for an implementation that separates evaluation
and compilation environments. Assume we are compiling a file. We are
in not-compile-time mode. We hit:
(defmacro foo (form) '(progn (print 'foo) ,form))
Per rule 2 (from Sec. 3.2.3.1) it is macro expanded. Suppose its
expansion is as I proposed. Per rule 3 the forms in the PROGN are
processed in not-compile-time mode. So we process the (eval-when
(:compile-toplevel) ...) form in not-compile-time mode.
Per rule 5 we evaluate the body of the eval-when in the evaluation
environment. That means we evaluate the (setf (macro-function 'foo)
...) in the evaluation environment which means that the definition of
FOO has been stored in the evaluation environment, not the compilation
environment. Thus my earlier expansion can't be right.
In an implementation, such as Allegro, that separates the compilation
and evaluation environments, a simplified version has to look more
like:
(progn
(eval-when (:compile-toplevel)
(impl:store-in-compilation-env 'macro-function 'foo
#'(lambda (whole env)
(declare (ignore env))
(with-parsed-macro-lambda-list (form) whole
`(progn ,form)))))
(eval-when (:load-toplevel :execute)
(setf (macro-function 'foo)
#'(lambda (whole env)
(declare (ignore env))
(with-parsed-macro-lambda-list (form) whole
`(progn ,form)))))
'foo)
And, in fact, this is more or less what that DEFMACRO macroexpands to
in Allegro. (Please let me know if I've removed any thing crucial to
this discussion.) With this expansion the implementation uses an
implementation-specifc mechanism to save a definition of FOO in the
compilation environment without putting it in the (separate)
evaluation environment. Since nothing is ever evaluated in the
compilation environment the only way that definition is used is by the
compiler itself which, as part of the implementation, can use the
hidden compilation environment to find macro definitions it needs when
compiling.
So now let's take a look at how that definition might be used. I think
we all agree that given the DEFMACRO of FOO above it is legal and
portable to use it in the same file like this:
(defun bar () (foo (+ 1 2)))
So let's look at why that works. The DEFUN expands into something
along the lines of:
(progn
(setf (fdefinition 'bar) #'(lambda () (block bar (foo (+ 1 2)))))
'bar)
According to the rules of 3.2.3.1 this top-level form is processed in
not-compile-time mode. Again the forms in the PROGN are themselves
processed as top-level forms in not-compile-time. So the
(setf (fdefinition 'bar) ...)
form is processed in not-compile-time mode which, per rule 6 means it
"is simply minimally compiled". It is during this compilation that the
compiler accesses the compilation environment to find the definition
of FOO in order to macroexpand the FOO form.
Now, finally, to the macro BAZ:
(defmacro baz (form) (foo `(progn ,form)))
This expands into something like:
(progn
(eval-when (:compile-toplevel)
(impl:store-in-compilation-env 'macro-function 'baz
#'(lambda (whole env)
(declare (ignore env))
(with-parsed-macro-lambda-list (form) whole
(foo `(progn ,form)))))
(eval-when (:load-toplevel :execute)
(setf (macro-function 'baz)
#'(lambda (whole env)
(declare (ignore env))
(with-parsed-macro-lambda-list (form) whole
(foo `(progn ,form))))))
'foo)
As before we start in not-compile-time mode, process the forms in the
PROGN in the same mode which gets us to processing the
(eval-when (:compile-toplevel) ...)
in not-compile-time mode. Per rule 5 that means we evaluate the
expression in the evaluation environment. Which eventually leads to
evaluating:
#'(lambda (whole env)
(declare (ignore env))
(with-parsed-macro-lambda-list (form) whole
(foo `(progn ,form))))
in the evaluation environment. Now depending on how the FUNCTION
special operator works that may simply return a SEXP for later
interpretation (as Dan Muller would like) or may compile the LAMBDA
form. But in the later case it is compiling in the evaluation
environment in which FOO is not defined.
The root of my confusion, it seems, was the misaprehension that all
compilation performed during file compilation was performed using the
compilation environment which is *not* true. In particular *implicit*
compilation that happens as a result of compile-time evaluation
doesn't use the compilation environment.
Finally, the key sentence in the spec--for this discussion--is
perhaps:
"In compile-time-too mode, the compiler first evaluates the form in
the evaluation environment AND THEN minimally compiles it."
(emphasis mine)
That is, if the order of evaluation and minimal compilation were
reversed or left unspecified, then the form could be minimally
compiled (by the file compiler, using the compilation environment) and
the resulting form then evaluated in the evaluation environment. In
that case the evaluation of BAZ would not be dependent on the
definition of FOO and and everything else would have worked the way I
originally proposed.
Anyway, thanks for bearing with me.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> I'll try another step-by-step analysis, this time showing why you are
> correct.
[ ... ]
> Anyway, thanks for bearing with me.
No problem; looks like you got it.
For those who didn't catch the reason for this:
> > Depending on where you're going with this, the answer in spirit to
> > your question is "yes" (but not quite precisely; that's why I
> > qualify the affirmative answer).
>
> I think I know why. See below.
Note that Peter's original question had the following:
...
(eval-when (:compile-toplevel)
(setf (macro-function 'foo)
...))
(eval-when (:load-toplevel :execute)
(setf (macro-function 'foo)
...))
which is hard to accomplish when (eval-when (compile load eval) ...)
is wrapped around the form (because it's hard to have two different
definitions in the same place at the same time...)
But his corrected style:
...
(eval-when (:compile-toplevel)
(impl:store-in-compilation-env 'macro-function 'foo
...))
(eval-when (:load-toplevel :execute)
(setf (macro-function 'foo)
...))
works around this problem, by storing one in an implementation-specific
place.
Note also that there should be no reason why the following
style couldn't eventually be implemented:
...
(eval-when (:compile-toplevel)
(setf (macro-function 'foo *compilation-environment*)
...))
(eval-when (:load-toplevel :execute)
(setf (macro-function 'foo)
...))
or to instead use an augment-environment form to do this storing.
Either approach would remove the implementation-dependent
nature from the macroexpansion and make the defmacro definition
pseudo-portable.
We didn't implement this in 7.0 (appearing soon), because we ran
out of time and would have to deal with bootstrapping issues in
order to get it working (along with the other defining forms that
are candidates for the eat-our-own-dog-food philosophy that will
continue to validate our environments-access approach). It
shouldn't be a hard change for a future release; just tedious.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3655q1a24.fsf@javamonkey.com>
Duane Rettig <·····@franz.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> I'll try another step-by-step analysis, this time showing why you
>> are correct.
>
> [ ... ]
>
>> Anyway, thanks for bearing with me.
>
> No problem; looks like you got it.
Cool. Just out of curiosity, I wonder if you have any thoughts on the
notion that in a different universe the spec could have allowed forms
processed in compile-time-too mode to be minimally compiled (using the
compilation environment) and the resulting form evaluated (in the
evaluation environment) rather than evaluating the uncompiled form.
That is, would a language so specified be obviously worse than Common
Lisp or just slightly different.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > Peter Seibel <·····@javamonkey.com> writes:
> >
> >> I'll try another step-by-step analysis, this time showing why you
> >> are correct.
> >
> > [ ... ]
> >
> >> Anyway, thanks for bearing with me.
> >
> > No problem; looks like you got it.
>
> Cool. Just out of curiosity, I wonder if you have any thoughts on the
> notion that in a different universe the spec could have allowed forms
> processed in compile-time-too mode to be minimally compiled (using the
> compilation environment) and the resulting form evaluated (in the
> evaluation environment) rather than evaluating the uncompiled form.
> That is, would a language so specified be obviously worse than Common
> Lisp or just slightly different.
I don't know - It might have resulted in a Common Lisp that was harder
to get code to run portably in various implementations (assuming you had
gotten all of the then X3J13 committee members to buy into it).
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Peter Seibel <·····@javamonkey.com> writes:
> Dan Muller <·········@sneakemail.com> writes:
>
>> Peter Seibel <·····@javamonkey.com> writes:
>>
>>> Duane Rettig <·····@franz.com> writes:
>>>
>>>> Dan Muller <·········@sneakemail.com> writes:
>>>>
>>>>> defmacro associates baz with an expansion function, the body of which
>>>>> is (foo `(progn ,form)). I can't see any requirement that the
>>>>> expansion function be compiled or otherwise processed, so the compiler
>>>>> may not even look at the symbol foo, and isnt' required to expand it
>>>>> before storing the expansion function. Ergo there's no guarantee that
>>>>> foo would be looked up in the compilation environment. Is that right?
>>>>
>>>> Absolutely correct. It _will_ do a macroexpansion on foo, in order
>>>> to compile it for storage in the fasl file, and _this_
>>>> macroexpansion is done in the compilation environment. But because
>>>> the compiler is not trying to "call" bar, only to macroexpand its
>>>> definition for the compilation process, there is no problem finding
>>>> foo (because it looks in the compilation environment) and the
>>>> definition of baz in the fasl file will have foo already
>>>> pre-macroexpanded.
>>>
>>> So where does the standard prohibit the compiler from storing the
>>> compiled version of BAZ's expansion function in the compilation
>>> environment?
>>>
>>
>> It doesn't prohibit it. But it doesn't require it, either.
>
> That's what I think. But Duane keeps telling me that it is prohibited.
> Or at least that's what I understand him to be saying.
>
>> There are typically _two_ copies of the expansion function at issue
>> here. One goes in the fasl (if the defmacro wasn't wrapped in "(eval
>> (:comile-toplevel) ...)"), and one is stored in the compilation
>> environment. There's no requirement that the latter be compiled --
>> it could be stored as a sexpr, for instance. If it is compiled, and
>> the evaluation and compilation environments are different, then I
>> suspect that the copy for the compilation environment would have to
>> be compiled in the _evaluation_ environment to be fully conforming,
>> because when that copy is evaluated by the compiler, that's the
>> environment that this should happen in.
>>
>> I hope I got that right...
>
> It's that last bit that I'm not convinced about. The spec says (Sec
> 3.2.1):
>
> The evaluation environment is a run-time environment in which macro
> expanders and code specified by eval-when to be evaluated are
> evaluated. All evaluations initiated by the compiler take place in
> the evaluation environment.
>
> But it's not clear to me that there's any "evaluation" that takes
> place when compiling BAZ other than the invocation of FOO's macro
> expander. I.e. to compile BAZ the we can find the definition of FOO in
> the compilation environment but then evaluate it (i.e. invoke its
> macro expander) in the evaluation environment.
No, the spec says that expansion functions are invoked in the
evaluation environment. I take that to mean that they must act as if
you started from a sexpr and interpreted it in that environment. If
compiling it first in the compilation environment would give you a
different result, then I'd say that it would be a violation to do
so. If the compilation and evaluation environments are provably
different, then it's possible to get a different result by compiling
it first in the compilation environment. (And since "compiling in the
evaluation environment" almost sounds like an oxymoron, I wonder if
it's _ever_ OK to compile expander functions when the two environments
are different.)
> Which means that any
> functions and variables used in FOO to compute its expansion must be
> available in the evaluation environment. But after FOO's macro
> expander has been run, and its expansion compiled, I claim we have a
> "definition" of BAZ that can be stored in the compilation environment
> that has no further dependency on FOO. So the fact that BAZ's macro
> expander will later be invoked in the evaluation environment is fine
> because it is no longer dependent on the definition of FOO.
And the very fact that you can do so, _and_ the evaluation environment
doesn't have a definition for FOO (in this implementation), shows that
BAZ's expansion function is not acting as if evaluated in the
evaluation environment. Ergo, violation?
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3ekkfqw5w.fsf@javamonkey.com>
Dan Muller <·········@sneakemail.com> writes:
> No, the spec says that expansion functions are invoked in the
> evaluation environment. I take that to mean that they must act as if
> you started from a sexpr and interpreted it in that environment.
Why do you read it that way? I agree that's a legitimate way to do it
but I don't see why it is required.
> If compiling it first in the compilation environment would give you
> a different result, then I'd say that it would be a violation to do
> so. If the compilation and evaluation environments are provably
> different, then it's possible to get a different result by compiling
> it first in the compilation environment. (And since "compiling in
> the evaluation environment" almost sounds like an oxymoron, I wonder
> if it's _ever_ OK to compile expander functions when the two
> environments are different.)
I'm not sure I understand what "compiling in the evaluation
environment" means. When code is compiled, macros that must be
expanded as part of the compilation are *found* in the compilation
environment. The macro expansion function is then run, with the
compilation environment as the &environment argument but the
evaluation happens in the evaluation environment. Thus functions
called by the macro expansion function must be available in the
evaluation environment. Now the key question--it seems to me--is what
restrictions there are on the the form of the macro expansion function
that is stored in the compilation environment. You maintain that the
definition *must be* a raw sexp, not even minimally compiled. This of
course means that invoking the function will cause it to be
interpreted. This interpretation may be either literal interpretation
or may proceed by first compiling the body and then running the just
compiled code. I agree that in the later case *that* compilation may
use the evaluation environment, just the way a non-compiling
interpreter would, to find macro definitions. So those two methods of
intepretation have the same result as far as finding or not finding
definitions that are only available in the compilation environment.
>
>> Which means that any functions and variables used in FOO to compute
>> its expansion must be available in the evaluation environment. But
>> after FOO's macro expander has been run, and its expansion
>> compiled, I claim we have a "definition" of BAZ that can be stored
>> in the compilation environment that has no further dependency on
>> FOO. So the fact that BAZ's macro expander will later be invoked in
>> the evaluation environment is fine because it is no longer
>> dependent on the definition of FOO.
>
> And the very fact that you can do so, _and_ the evaluation
> environment doesn't have a definition for FOO (in this
> implementation), shows that BAZ's expansion function is not acting
> as if evaluated in the evaluation environment. Ergo, violation?
Okay, we agree that the spec says that macro expansion functions are
evaluated in the evaluation environment. You take that to mean
something like: "macro expansion functions are evaluated as if by
interpretation of the original macro body in the evaluation
environment." I take it mean, "macro expansion functions are evaluated
by invoking whatever definition was stored in the compilation
environment in the evaluation environment." In both case they are
"evaluated in the evaluation environment". I'm just saying that no one
has shown me anything that say the definition that is stored in the
compilation environment (and thus found to be invoked) can't be
invoked. Now, I agree that if "evaluation" of a macro expansion
function necessarily means "interpretation of the original sexp" than
you are right. But I don't see that that is required. (And it would
seem strange to me that it mean that because that's not the case
elsewhere in the language. For instance, it's perfectly conforming for
an implementation to take every function or macro definition typed at
the REPL and immediately compile it--there is no interpreter. So why
should that same technique be illegal in the compiler?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Dan Muller <·········@sneakemail.com> writes:
>
>> No, the spec says that expansion functions are invoked in the
>> evaluation environment. I take that to mean that they must act as if
>> you started from a sexpr and interpreted it in that environment.
>
> Why do you read it that way? I agree that's a legitimate way to do it
> but I don't see why it is required.
Well, I guess it depends on what you mean by evaluating some
code. According to the glossary, it means executing the code
represented by a form. In this case, the form is the macro function,
aka macro expander, aka expansion function... That's not the same as
"evaluating the compiled representation of the form which was modified
by virtue of being compiled".
Macro expanding is done _as part of_ evaluating or compiling a
form. Replacing code with a version of the same code after expanding
macro forms in it does not produce code that behave identically; if
the macro definitions subsequently change, then evaluating the
expanded code can give different results from evaluating the
unexpanded code. So if the spec says that a macro function is
evaluated at some point, I assume that it means what it
says. Evaluating a previously-expanded form of the macro function is
not necessarily the same thing. More on this in a moment.
>> If compiling it first in the compilation environment would give you
>> a different result, then I'd say that it would be a violation to do
>> so. If the compilation and evaluation environments are provably
>> different, then it's possible to get a different result by compiling
>> it first in the compilation environment. (And since "compiling in
>> the evaluation environment" almost sounds like an oxymoron, I wonder
>> if it's _ever_ OK to compile expander functions when the two
>> environments are different.)
>
> I'm not sure I understand what "compiling in the evaluation
> environment" means. When code is compiled, macros that must be
> expanded as part of the compilation are *found* in the compilation
> environment. The macro expansion function is then run, with the
> compilation environment as the &environment argument but the
> evaluation happens in the evaluation environment. Thus functions
> called by the macro expansion function must be available in the
> evaluation environment. Now the key question--it seems to me--is what
> restrictions there are on the the form of the macro expansion function
> that is stored in the compilation environment. You maintain that the
> definition *must be* a raw sexp, not even minimally compiled.
No, just that when evaluated, it acts as if it were. As I described
above, it follows from the precise language in the spec, from what
I've seen so far. It says that the _macro functions_ are evaluated, not
the compiled macro functions, or the macro functions with their macro
forms previously expanded. Compilation changes code -- otherwise this
discussion would be pointless, because there'd be no detectable
difference in the alternatives that we're focusing on.
> This of
> course means that invoking the function will cause it to be
> interpreted. This interpretation may be either literal interpretation
> or may proceed by first compiling the body and then running the just
> compiled code. I agree that in the later case *that* compilation may
> use the evaluation environment, just the way a non-compiling
> interpreter would, to find macro definitions. So those two methods of
> intepretation have the same result as far as finding or not finding
> definitions that are only available in the compilation environment.
Right. As I said above, the Lisp system can do things any way it
wants, as long as the results are indistinguishable from the processes
required by the standard.
>>> Which means that any functions and variables used in FOO to compute
>>> its expansion must be available in the evaluation environment. But
>>> after FOO's macro expander has been run, and its expansion
>>> compiled, I claim we have a "definition" of BAZ that can be stored
>>> in the compilation environment that has no further dependency on
>>> FOO. So the fact that BAZ's macro expander will later be invoked in
>>> the evaluation environment is fine because it is no longer
>>> dependent on the definition of FOO.
>>
>> And the very fact that you can do so, _and_ the evaluation
>> environment doesn't have a definition for FOO (in this
>> implementation), shows that BAZ's expansion function is not acting
>> as if evaluated in the evaluation environment. Ergo, violation?
>
> Okay, we agree that the spec says that macro expansion functions are
> evaluated in the evaluation environment. You take that to mean
> something like: "macro expansion functions are evaluated as if by
> interpretation of the original macro body in the evaluation
> environment." I take it mean, "macro expansion functions are evaluated
> by invoking whatever definition was stored in the compilation
> environment in the evaluation environment." In both case they are
> "evaluated in the evaluation environment".
Agreed: "macro expansion functions are evaluated in the evaluation environment"
OK, what's a macro expansion function, as produced by defmacro?
Looking at DEFMACRO, we have:
The body of the expansion function is specified by forms. [Where
"forms" refers to the arguments to DEFMACRO.] Forms are executed in
order. The value of the last form executed is returned as the
expansion of the macro. The body forms of the expansion function
(but not the lambda-list) are implicitly enclosed in a block whose
name is name.
OK, there's a little leeway here: "specified by" is a little bit
vague. But I think it's reasonable to assume the intent that the macro
function body should act just like the forms would, when it's invoked.
If you agree that the macro function form shouldn't make detectable
changes to the forms, then BAZ's macro function is defined in terms of
a form that references FOO as an operator. The definition of DEFMACRO
does not call for evaluating or compiling the macro function at this
time, and these are, I think, the only two times that the spec
explicitly calls for macro expansion to occur.
Macro expansion is a _detectable_ change to code, because the
definition of a macro can change over time and between contexts. So I
would argue that macro expansion should only occur when the standard
explicitly calls for it. Between the time that a defmacro form is
evaluated, and the resulting macro function is stored, I don't see an
opportunity for macro expansion in the standard. Unless you wanted to
give yourself a lot of leeway with that "specified by" phrase.
> I'm just saying that no one
> has shown me anything that say the definition that is stored in the
> compilation environment (and thus found to be invoked) can't be
> invoked. Now, I agree that if "evaluation" of a macro expansion
> function necessarily means "interpretation of the original sexp" than
> you are right. But I don't see that that is required. (And it would
> seem strange to me that it mean that because that's not the case
> elsewhere in the language. For instance, it's perfectly conforming for
> an implementation to take every function or macro definition typed at
> the REPL and immediately compile it--there is no interpreter. So why
> should that same technique be illegal in the compiler?
Because of timing. In the REPL, you can certainly
compile-then-execute. This will be atomic, from the user's point of
view. But if the spec says that a form will be stored somewhere, then
executed later, it doesn't seem right to assume that it's OK to
compile the form when stored. In the intervening time, all kinds of
things can happen to macros referenced by the form. "Storing" doesn't
imply "expand macros"; "evaluate" explicitly requires it.
In fact, I've now convinced myself that compiling the forms provided
to defmacro before storing them is _always_ wrong, according to the
language in the standard. (Assuming "compiling" as described in the
standard, including expansion of macros. Again, internally, one might
have a special mode of compiling that doesn't expand macros.) For
instance, the following behavior in CLisp is wrong:
CL-USER> (defmacro ber (x) `(* x 2))
BER
CL-USER> (defmacro blub (x) (ber x))
BLUB
CL-USER> (blub 5)
10
CL-USER> (defmacro ber (x) `(* x 3))
BER
CL-USER> (blub 5)
10
The last call to blub should return 15. To get the above behavior, I
should have to force the body of BLUB to macroexpanded, or compiled
(using SETF and MACRO-FUNCTION).
Since the spec doesn't explicitly give the system the latitude to
modify the forms handed to DEFMACRO, the above should behave more like
this sequence:
CL-USER> (setq blub (lambda (x) (ber x)))
#<CLOSURE :LAMBDA (X) (BER X)>
CL-USER> (defmacro ber (x) `(* x 2))
BER
CL-USER> (apply blub '(5))
10
CL-USER> (defmacro ber (x) `(* x 3))
BER
CL-USER> (apply blub '(5))
15
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3655rqms5.fsf@javamonkey.com>
Dan Muller <·········@sneakemail.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Dan Muller <·········@sneakemail.com> writes:
>>
>>> No, the spec says that expansion functions are invoked in the
>>> evaluation environment. I take that to mean that they must act as
>>> if you started from a sexpr and interpreted it in that
>>> environment.
>>
>> Why do you read it that way? I agree that's a legitimate way to do
>> it but I don't see why it is required.
>
> Well, I guess it depends on what you mean by evaluating some code.
> According to the glossary, it means executing the code represented
> by a form. In this case, the form is the macro function, aka macro
> expander, aka expansion function... That's not the same as
> "evaluating the compiled representation of the form which was
> modified by virtue of being compiled".
>
> Macro expanding is done _as part of_ evaluating or compiling a
> form. Replacing code with a version of the same code after expanding
> macro forms in it does not produce code that behave identically; if
> the macro definitions subsequently change, then evaluating the
> expanded code can give different results from evaluating the
> unexpanded code.
That is true of *any* compilation. So what you're really saying is if
BAZ is defined in a file and QUUX contains a BAZ form, the code that
runs when compiling that file *must* be interpreted while subsequent
uses of BAZ (in other files) may use the compiled version. So please
show me where in the spec this requirement comes from. In the meantime
it might be useful to consider section "3.1 Evaluation" (emphasis mine)
Execution of code CAN BE ACCOMPLISHED BY A VARIETY OF MEANS ranging
from direct interpretation of a form representing a program to
invocation of compiled code produced by a compiler.
Evaluation is the process by which a program is executed in Common
Lisp. The mechanism of evaluation is manifested both implicitly
through the effect of the Lisp read-eval-print loop, and explicitly
through the presence of the functions eval, compile, compile-file,
and load. ANY OF THESE FACILITIES MIGHT SHARE THE SAME EXECUTION
STRATEGY, OR EACH MIGHT USE A DIFFERENT ONE.
> So if the spec says that a macro function is evaluated at some
> point, I assume that it means what it says. Evaluating a
> previously-expanded form of the macro function is not necessarily
> the same thing. More on this in a moment.
It would help me if we could stop using the phrase "a macro function
is evaluated". I'm not sure whether you're talking about evaluating a
macro form, i.e. a cons whose CAR is the name of a macro, and invoking
a macro function, i.e. calling the function that computes the
expansion of a macro. To get back to the spec, Section "3.1.2.1.2.2
Macro Forms" discusses how macro *forms* are evaluated:
If the operator names a macro, its associated macro function is
applied to the entire form and the result of that application is
used in place of the original form.
So, as I read that, the macro's expansion function is just another
function. I don't see anything that says that that function can not be
compiled. You're essentially saying that when used by the file
compiler it *must* be interpreted. I haven't seen anything in the spec
that says that.
> OK, what's a macro expansion function, as produced by defmacro?
> Looking at DEFMACRO, we have:
>
> The body of the expansion function is specified by forms. [Where
> "forms" refers to the arguments to DEFMACRO.] Forms are executed in
> order. The value of the last form executed is returned as the
> expansion of the macro. The body forms of the expansion function
> (but not the lambda-list) are implicitly enclosed in a block whose
> name is name.
>
> OK, there's a little leeway here: "specified by" is a little bit
> vague. But I think it's reasonable to assume the intent that the macro
> function body should act just like the forms would, when it's invoked.
Why we don't expect that of other functions. The dictionary entry for
DEFUN says:
The body of the function defined by defun consists of forms; they
are executed as an implicit progn when the function is called.
Does that mean functions can't be compiled either? Presumably not.
> Macro expansion is a _detectable_ change to code, because the
> definition of a macro can change over time and between contexts.
Yes. There are in fact detectable differences between compiled code
and uncompiled code. But there's no guarantee--that I'm aware of--that
code is ever *not* compiled. You seem to be operating on the
assumption that the spec delineates a specific set of situations in
which it is acceptable to compile code and otherwise it must interpret
it. But that--again as far as I know--not true.
> So I would argue that macro expansion should only occur when the
> standard explicitly calls for it.
When does it *explicitly* call for it?
>> I'm just saying that no one has shown me anything that say the
>> definition that is stored in the compilation environment (and thus
>> found to be invoked) can't be invoked. Now, I agree that if
>> "evaluation" of a macro expansion function necessarily means
>> "interpretation of the original sexp" than you are right. But I
>> don't see that that is required. (And it would seem strange to me
>> that it mean that because that's not the case elsewhere in the
>> language. For instance, it's perfectly conforming for an
>> implementation to take every function or macro definition typed at
>> the REPL and immediately compile it--there is no interpreter. So
>> why should that same technique be illegal in the compiler?
>
> Because of timing. In the REPL, you can certainly
> compile-then-execute. This will be atomic, from the user's point of
> view.
I don't think so. If, when you type a function in at the REPL, Lisp
compiles it, any uses of that macro will be expanded once.
Contrariwise, if Lisp stores the raw sexps and interprets the function
each time it is called then the macro will be expanded each time it is
called. There may be UI reasons to use one strategy or the other but
both are legal. For instance try this sequence in a couple Lisp's:
(defmacro foo (form) (print "Expanding FOO") form)
(defun bar () (foo (+ 1 2)))
(bar)
(bar)
Some Lisp's (Allegro for instance) will print "Expanding FOO" once,
after you enter the DEFUN of BAR. Others (CLISP and SBCL) will print
it each time you invoke BAR. And others (CMUCL) will print it after
the first time you invoke BAR but not subsequently. You're saying only
Allegro is conformant.
> But if the spec says that a form will be stored somewhere, then
> executed later, it doesn't seem right to assume that it's OK to
> compile the form when stored.
So how do you explain all the implementations that do exactly that
will all DEFUNs (and DEFMACROs)?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Dan Muller <·········@sneakemail.com> writes:
>
>> Peter Seibel <·····@javamonkey.com> writes:
>>
>>> Dan Muller <·········@sneakemail.com> writes:
>>>
>>>> No, the spec says that expansion functions are invoked in the
>>>> evaluation environment. I take that to mean that they must act as
>>>> if you started from a sexpr and interpreted it in that
>>>> environment.
>>>
>>> Why do you read it that way? I agree that's a legitimate way to do
>>> it but I don't see why it is required.
>>
>> Well, I guess it depends on what you mean by evaluating some code.
>> According to the glossary, it means executing the code represented
>> by a form. In this case, the form is the macro function, aka macro
>> expander, aka expansion function... That's not the same as
>> "evaluating the compiled representation of the form which was
>> modified by virtue of being compiled".
>>
>> Macro expanding is done _as part of_ evaluating or compiling a
>> form. Replacing code with a version of the same code after expanding
>> macro forms in it does not produce code that behave identically; if
>> the macro definitions subsequently change, then evaluating the
>> expanded code can give different results from evaluating the
>> unexpanded code.
>
> That is true of *any* compilation. So what you're really saying is if
> BAZ is defined in a file and QUUX contains a BAZ form, the code that
> runs when compiling that file *must* be interpreted while subsequent
> uses of BAZ (in other files) may use the compiled version. So please
> show me where in the spec this requirement comes from. In the meantime
> it might be useful to consider section "3.1 Evaluation" (emphasis mine)
>
> Execution of code CAN BE ACCOMPLISHED BY A VARIETY OF MEANS ranging
> from direct interpretation of a form representing a program to
> invocation of compiled code produced by a compiler.
>
> Evaluation is the process by which a program is executed in Common
> Lisp. The mechanism of evaluation is manifested both implicitly
> through the effect of the Lisp read-eval-print loop, and explicitly
> through the presence of the functions eval, compile, compile-file,
> and load. ANY OF THESE FACILITIES MIGHT SHARE THE SAME EXECUTION
> STRATEGY, OR EACH MIGHT USE A DIFFERENT ONE.
Right, _execution_ of code can be accomplished by compiling first, or
not. I thought I made it clear that I understand that.
But time and context may matter. There are two things going on here:
storing the definition, and later on evaluating it. I'm arguing that
the requirement to _store_ the definition of a macro expansion
function for _later_ evaluation may not give the sytem latitude to
modify the definition in between those two times. Compilation
modifies.
>> So if the spec says that a macro function is evaluated at some
>> point, I assume that it means what it says. Evaluating a
>> previously-expanded form of the macro function is not necessarily
>> the same thing. More on this in a moment.
>
> It would help me if we could stop using the phrase "a macro function
> is evaluated". I'm not sure whether you're talking about evaluating a
> macro form, i.e. a cons whose CAR is the name of a macro, and invoking
> a macro function, i.e. calling the function that computes the
> expansion of a macro.
Since we're talking fine points of the standard, I was sticking
closely to its terminology. See the Glossary entry for "macro
function". I tried to use "macro form" and "macro function"
appropriately. Sorry if you found this confusing.
> To get back to the spec, Section "3.1.2.1.2.2
> Macro Forms" discusses how macro *forms* are evaluated:
>
> If the operator names a macro, its associated macro function is
> applied to the entire form and the result of that application is
> used in place of the original form.
>
> So, as I read that, the macro's expansion function is just another
> function. I don't see anything that says that that function can not be
> compiled. You're essentially saying that when used by the file
> compiler it *must* be interpreted. I haven't seen anything in the spec
> that says that.
True, it doesn't say that explicitly. But it specifies a certain
behavior, and I'm arguing that compiling prior to storing can result
in different behavior. I guess the question really is whether it's an
acceptable difference in behavior or not.
>> OK, what's a macro expansion function, as produced by defmacro?
>> Looking at DEFMACRO, we have:
>>
>> The body of the expansion function is specified by forms. [Where
>> "forms" refers to the arguments to DEFMACRO.] Forms are executed in
>> order. The value of the last form executed is returned as the
>> expansion of the macro. The body forms of the expansion function
>> (but not the lambda-list) are implicitly enclosed in a block whose
>> name is name.
>>
>> OK, there's a little leeway here: "specified by" is a little bit
>> vague. But I think it's reasonable to assume the intent that the macro
>> function body should act just like the forms would, when it's invoked.
>
> Why we don't expect that of other functions. The dictionary entry for
> DEFUN says:
>
> The body of the function defined by defun consists of forms; they
> are executed as an implicit progn when the function is called.
>
> Does that mean functions can't be compiled either? Presumably not.
Huh. Good point. I looked really hard, but I still think that a strict
reading of the standard implies that macro expansion shouldn't occur
at this point -- and this would appear to apply to DEFUN, too. That's
disturbing. I must be missing something, but I don't see what it
is. Perhaps the standard's lack of specificity on this point has just
allowed convention to supersede the implications.
>> Macro expansion is a _detectable_ change to code, because the
>> definition of a macro can change over time and between contexts.
>
> Yes. There are in fact detectable differences between compiled code
> and uncompiled code. But there's no guarantee--that I'm aware of--that
> code is ever *not* compiled. You seem to be operating on the
> assumption that the spec delineates a specific set of situations in
> which it is acceptable to compile code and otherwise it must interpret
> it. But that--again as far as I know--not true.
>
>> So I would argue that macro expansion should only occur when the
>> standard explicitly calls for it.
>
> When does it *explicitly* call for it?
During evaluation of code, and at compile time; 3.1.2.1.2.2 and
3.2.2.2, respectively.
>>> I'm just saying that no one has shown me anything that say the
>>> definition that is stored in the compilation environment (and thus
>>> found to be invoked) can't be invoked. Now, I agree that if
>>> "evaluation" of a macro expansion function necessarily means
>>> "interpretation of the original sexp" than you are right. But I
>>> don't see that that is required. (And it would seem strange to me
>>> that it mean that because that's not the case elsewhere in the
>>> language. For instance, it's perfectly conforming for an
>>> implementation to take every function or macro definition typed at
>>> the REPL and immediately compile it--there is no interpreter. So
>>> why should that same technique be illegal in the compiler?
>>
>> Because of timing. In the REPL, you can certainly
>> compile-then-execute. This will be atomic, from the user's point of
>> view.
>
> I don't think so. If, when you type a function in at the REPL, Lisp
> compiles it, any uses of that macro will be expanded once.
> Contrariwise, if Lisp stores the raw sexps and interprets the function
> each time it is called then the macro will be expanded each time it is
> called. There may be UI reasons to use one strategy or the other but
> both are legal. For instance try this sequence in a couple Lisp's:
>
> (defmacro foo (form) (print "Expanding FOO") form)
>
> (defun bar () (foo (+ 1 2)))
>
> (bar)
>
> (bar)
>
My bad, I didn't read the end of your paragraph carefully enough. I
thought you were talking about evaluation versus compile-and-execute
of the forms given to the REPL, but you were discussing the
compilation of function and macro definitions.
> Some Lisp's (Allegro for instance) will print "Expanding FOO" once,
> after you enter the DEFUN of BAR. Others (CLISP and SBCL) will print
> it each time you invoke BAR. And others (CMUCL) will print it after
> the first time you invoke BAR but not subsequently. You're saying only
> Allegro is conformant.
>
>> But if the spec says that a form will be stored somewhere, then
>> executed later, it doesn't seem right to assume that it's OK to
>> compile the form when stored.
>
> So how do you explain all the implementations that do exactly that
> will all DEFUNs (and DEFMACROs)?
>
Actually, I would say that for this behavior at the REPL, CLISP and
SBCL sound conforming.
<shrug> Maybe I'm assuming more accuracy than the standard
has. Certainly, compiling on the fly is common. There's also no
argument that expanding macros early or late changes behavior. The
standard isn't really explicit about whether the effects of early
macro expansion are an _acceptable_ change in the behavior of forms
provided by the user in some contexts. However, it goes to great
lengths to explain some points in time when macro expansion
_must_ occur.
I don't think the standard explicitly prohibits early expansion, but I
think it arguably implies the prohibition, as I've explained. Can you
point out where it explicitly or implicitly allows it, without
referring to behavior of implementations? The bits you've quoted so
far just allow compilation in lieu of evaluation, but don't say when
the compilation can occur.
Taking the interpretation that a behavior-changing modification is
allowed when not expressly forbidden is dangerous. For example: if a
system is smart enough to recognize that some data will be used as
code later, why not expand macro calls in that data early?
(defmacro ber (x) `(* x 2))
(setq blub '(lambda (x) (ber x)) ; Note the quote this time.
(apply (eval blub) '(5))
(defmacro ber (x) `(* x 3))
(apply (eval blub) '(5))
Would a system be justified in returning 10 from both calls to APPLY?
Does the standard explicitly prohibit the expansion of macros in a
list stored as a symbol's value? How about if the system arranged to
do the early expansion without it being detectable by code that might
dissect the contents of BLUB as data? (Perhaps it could store two
versions and return the expanded list when it knows the value will be
passed to eval.)
I'm being somewhat facetious, of course. But in fact nothing seems to
explicitly _prohibit_ this.
P.S.
I made a mistake in the last example from my previous
message:
(setq blub (lambda (x) (ber x)))
(defmacro ber (x) `(* x 2))
(apply blub '(5))
(defmacro ber (x) `(* x 3))
(apply blub '(5))
The first two forms should be swapped, otherwise the invocation of BER
in BLUB is treated as a function call. If BER is defined first, then
the invocation of LAMBDA results in early macro expansion, just like
DEFUN, and both invocations of BLUB return 10.
In the first form, the lambda macro is just shorthand for (function
(lambda (x) (ber x))). FUNCTION's definition in the standard lacks any
explicit mention of macro expansion or compilation, just like DEFUN
and DEFMACRO. (It's a special operator, unlike the other two which are
macros.) Would it be legitimate for an implementation to return 10
from both of the invocations of BLUB? (Actually,
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m3oejipm5x.fsf@javamonkey.com>
Dan Muller <·········@sneakemail.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Dan Muller <·········@sneakemail.com> writes:
>>
>>> Peter Seibel <·····@javamonkey.com> writes:
>>>
>>>> Dan Muller <·········@sneakemail.com> writes:
>>>>
>>>>> No, the spec says that expansion functions are invoked in the
>>>>> evaluation environment. I take that to mean that they must act as
>>>>> if you started from a sexpr and interpreted it in that
>>>>> environment.
>>>>
>>>> Why do you read it that way? I agree that's a legitimate way to do
>>>> it but I don't see why it is required.
>>>
>>> Well, I guess it depends on what you mean by evaluating some code.
>>> According to the glossary, it means executing the code represented
>>> by a form. In this case, the form is the macro function, aka macro
>>> expander, aka expansion function... That's not the same as
>>> "evaluating the compiled representation of the form which was
>>> modified by virtue of being compiled".
>>>
>>> Macro expanding is done _as part of_ evaluating or compiling a
>>> form. Replacing code with a version of the same code after expanding
>>> macro forms in it does not produce code that behave identically; if
>>> the macro definitions subsequently change, then evaluating the
>>> expanded code can give different results from evaluating the
>>> unexpanded code.
>>
>> That is true of *any* compilation. So what you're really saying is if
>> BAZ is defined in a file and QUUX contains a BAZ form, the code that
>> runs when compiling that file *must* be interpreted while subsequent
>> uses of BAZ (in other files) may use the compiled version. So please
>> show me where in the spec this requirement comes from. In the meantime
>> it might be useful to consider section "3.1 Evaluation" (emphasis mine)
>>
>> Execution of code CAN BE ACCOMPLISHED BY A VARIETY OF MEANS ranging
>> from direct interpretation of a form representing a program to
>> invocation of compiled code produced by a compiler.
>>
>> Evaluation is the process by which a program is executed in Common
>> Lisp. The mechanism of evaluation is manifested both implicitly
>> through the effect of the Lisp read-eval-print loop, and explicitly
>> through the presence of the functions eval, compile, compile-file,
>> and load. ANY OF THESE FACILITIES MIGHT SHARE THE SAME EXECUTION
>> STRATEGY, OR EACH MIGHT USE A DIFFERENT ONE.
>
> Right, _execution_ of code can be accomplished by compiling first, or
> not. I thought I made it clear that I understand that.
>
> But time and context may matter. There are two things going on here:
> storing the definition, and later on evaluating it. I'm arguing that
> the requirement to _store_ the definition of a macro expansion
> function for _later_ evaluation may not give the sytem latitude to
> modify the definition in between those two times. Compilation
> modifies.
I think we may be reaching the point we we'll just have to agree to
disagree. You take "evaluation" to mean "interpreting of unmodified
Lisp data". I take it to mean, "execution of the code in a way
compatible with 3.1" which may include executing an already compiled
version. Since our differences seem to boil down to how we interpret
the exact same language in the spec it's not clear what either of us
can say to convince the other. Maybe some kind third party can point
out something that one or the other of us is missing.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Dan Muller <·········@sneakemail.com> writes:
>
>> Peter Seibel <·····@javamonkey.com> writes:
>>
>>> Dan Muller <·········@sneakemail.com> writes:
>>>
>>>> Peter Seibel <·····@javamonkey.com> writes:
>>>>
>>>>> Dan Muller <·········@sneakemail.com> writes:
>>>>>
>>>>>> No, the spec says that expansion functions are invoked in the
>>>>>> evaluation environment. I take that to mean that they must act as
>>>>>> if you started from a sexpr and interpreted it in that
>>>>>> environment.
>>>>>
>>>>> Why do you read it that way? I agree that's a legitimate way to do
>>>>> it but I don't see why it is required.
>>>>
>>>> Well, I guess it depends on what you mean by evaluating some code.
>>>> According to the glossary, it means executing the code represented
>>>> by a form. In this case, the form is the macro function, aka macro
>>>> expander, aka expansion function... That's not the same as
>>>> "evaluating the compiled representation of the form which was
>>>> modified by virtue of being compiled".
>>>>
>>>> Macro expanding is done _as part of_ evaluating or compiling a
>>>> form. Replacing code with a version of the same code after expanding
>>>> macro forms in it does not produce code that behave identically; if
>>>> the macro definitions subsequently change, then evaluating the
>>>> expanded code can give different results from evaluating the
>>>> unexpanded code.
>>>
>>> That is true of *any* compilation. So what you're really saying is if
>>> BAZ is defined in a file and QUUX contains a BAZ form, the code that
>>> runs when compiling that file *must* be interpreted while subsequent
>>> uses of BAZ (in other files) may use the compiled version. So please
>>> show me where in the spec this requirement comes from. In the meantime
>>> it might be useful to consider section "3.1 Evaluation" (emphasis mine)
>>>
>>> Execution of code CAN BE ACCOMPLISHED BY A VARIETY OF MEANS ranging
>>> from direct interpretation of a form representing a program to
>>> invocation of compiled code produced by a compiler.
>>>
>>> Evaluation is the process by which a program is executed in Common
>>> Lisp. The mechanism of evaluation is manifested both implicitly
>>> through the effect of the Lisp read-eval-print loop, and explicitly
>>> through the presence of the functions eval, compile, compile-file,
>>> and load. ANY OF THESE FACILITIES MIGHT SHARE THE SAME EXECUTION
>>> STRATEGY, OR EACH MIGHT USE A DIFFERENT ONE.
>>
>> Right, _execution_ of code can be accomplished by compiling first, or
>> not. I thought I made it clear that I understand that.
>>
>> But time and context may matter. There are two things going on here:
>> storing the definition, and later on evaluating it. I'm arguing that
>> the requirement to _store_ the definition of a macro expansion
>> function for _later_ evaluation may not give the sytem latitude to
>> modify the definition in between those two times. Compilation
>> modifies.
>
> I think we may be reaching the point we we'll just have to agree to
> disagree. You take "evaluation" to mean "interpreting of unmodified
> Lisp data". I take it to mean, "execution of the code in a way
> compatible with 3.1" which may include executing an already compiled
> version. Since our differences seem to boil down to how we interpret
> the exact same language in the spec it's not clear what either of us
> can say to convince the other. Maybe some kind third party can point
> out something that one or the other of us is missing.
Agreed. To clarify further, though: I'm just describing how I would
tend to interpret the standard, based on prior experience with other
standards documents. Years of staring at inscrutable parts of the C++
standard may have damaged my brain somewhat. If a few people come
along and say "That's nice Dan, but _this_ is how the Lisp community
interprets this part of the standard", I'll be fine with that. Points
of view from Lisp implementors will be particularly interesting, of
course.
I think everyone knows that redefining macros that are already in use
is problematic anyway. I think, though, that my more strict
interpretation would be more convenient when doing exploratory work at
the REPL. Would you agree?
From: Peter Seibel
Subject: Re: Question about compilation/evaluation environments
Date:
Message-ID: <m37jq631cf.fsf@javamonkey.com>
Dan Muller <·········@sneakemail.com> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Dan Muller <·········@sneakemail.com> writes:
>>
>>> Peter Seibel <·····@javamonkey.com> writes:
>>>
>>>> Dan Muller <·········@sneakemail.com> writes:
>>>>
>>>>> Peter Seibel <·····@javamonkey.com> writes:
>>>>>
>>>>>> Dan Muller <·········@sneakemail.com> writes:
>>>>>>
>>>>>>> No, the spec says that expansion functions are invoked in the
>>>>>>> evaluation environment. I take that to mean that they must act as
>>>>>>> if you started from a sexpr and interpreted it in that
>>>>>>> environment.
>>>>>>
>>>>>> Why do you read it that way? I agree that's a legitimate way to do
>>>>>> it but I don't see why it is required.
>>>>>
>>>>> Well, I guess it depends on what you mean by evaluating some code.
>>>>> According to the glossary, it means executing the code represented
>>>>> by a form. In this case, the form is the macro function, aka macro
>>>>> expander, aka expansion function... That's not the same as
>>>>> "evaluating the compiled representation of the form which was
>>>>> modified by virtue of being compiled".
>>>>>
>>>>> Macro expanding is done _as part of_ evaluating or compiling a
>>>>> form. Replacing code with a version of the same code after expanding
>>>>> macro forms in it does not produce code that behave identically; if
>>>>> the macro definitions subsequently change, then evaluating the
>>>>> expanded code can give different results from evaluating the
>>>>> unexpanded code.
>>>>
>>>> That is true of *any* compilation. So what you're really saying is if
>>>> BAZ is defined in a file and QUUX contains a BAZ form, the code that
>>>> runs when compiling that file *must* be interpreted while subsequent
>>>> uses of BAZ (in other files) may use the compiled version. So please
>>>> show me where in the spec this requirement comes from. In the meantime
>>>> it might be useful to consider section "3.1 Evaluation" (emphasis mine)
>>>>
>>>> Execution of code CAN BE ACCOMPLISHED BY A VARIETY OF MEANS ranging
>>>> from direct interpretation of a form representing a program to
>>>> invocation of compiled code produced by a compiler.
>>>>
>>>> Evaluation is the process by which a program is executed in Common
>>>> Lisp. The mechanism of evaluation is manifested both implicitly
>>>> through the effect of the Lisp read-eval-print loop, and explicitly
>>>> through the presence of the functions eval, compile, compile-file,
>>>> and load. ANY OF THESE FACILITIES MIGHT SHARE THE SAME EXECUTION
>>>> STRATEGY, OR EACH MIGHT USE A DIFFERENT ONE.
>>>
>>> Right, _execution_ of code can be accomplished by compiling first, or
>>> not. I thought I made it clear that I understand that.
>>>
>>> But time and context may matter. There are two things going on here:
>>> storing the definition, and later on evaluating it. I'm arguing that
>>> the requirement to _store_ the definition of a macro expansion
>>> function for _later_ evaluation may not give the sytem latitude to
>>> modify the definition in between those two times. Compilation
>>> modifies.
>>
>> I think we may be reaching the point we we'll just have to agree to
>> disagree. You take "evaluation" to mean "interpreting of unmodified
>> Lisp data". I take it to mean, "execution of the code in a way
>> compatible with 3.1" which may include executing an already compiled
>> version. Since our differences seem to boil down to how we interpret
>> the exact same language in the spec it's not clear what either of us
>> can say to convince the other. Maybe some kind third party can point
>> out something that one or the other of us is missing.
>
> Agreed. To clarify further, though: I'm just describing how I would
> tend to interpret the standard, based on prior experience with other
> standards documents. Years of staring at inscrutable parts of the C++
> standard may have damaged my brain somewhat. If a few people come
> along and say "That's nice Dan, but _this_ is how the Lisp community
> interprets this part of the standard", I'll be fine with that. Points
> of view from Lisp implementors will be particularly interesting, of
> course.
>
> I think everyone knows that redefining macros that are already in
> use is problematic anyway. I think, though, that my more strict
> interpretation would be more convenient when doing exploratory work
> at the REPL. Would you agree?
I agree that there are UI benefits provided by the REPL storing
non-compiled versions of functions and macros that you enter
interactively. I just don't agree that there's anything in the spec
that requires that behavior. (And if there is SBCL and CLISP are
non-conformant.)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Dan Muller <·········@sneakemail.com> writes:
>
> > I think everyone knows that redefining macros that are already in
> > use is problematic anyway. I think, though, that my more strict
> > interpretation would be more convenient when doing exploratory work
> > at the REPL. Would you agree?
>
> I agree that there are UI benefits provided by the REPL storing
> non-compiled versions of functions and macros that you enter
> interactively. I just don't agree that there's anything in the spec
> that requires that behavior. (And if there is SBCL and CLISP are
> non-conformant.)
I've now answered the fundamental question elsehwere in this thread.
And it has nothing to do with the choice that implementations make
as to whether they compile their definitions by default. It has
everything to do with what environment the definitions are actually
defined and evaluated in. At the REPL, there is only one environment,
but in a compile-file, there are more than one, and that poses
restrictions on how the forms are processed and the macro definitions
are evaluated at compile-time, as opposed to how they are compiled
for later loading.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182