The docs for MACROLET say:
The macro-expansion functions defined by macrolet are defined in the
lexical environment in which the macrolet form appears. Declarations
and macrolet and symbol-macrolet definitions affect the local macro
definitions in a macrolet, but the consequences are undefined if the
local macro definitions reference any local variable or function
bindings that are visible in that lexical environment.
So my question is: is the new macro definition supposed to be in the
environment that is passed to the local macro. In other words, if I do
this:
(defmacro foo () ''done)
(macrolet ((foo (&environment env) (macroexpand '(foo) env)))
(foo))
should the expansion recurse forever or should the call to MACROEXPAND
find the DEFMACRO'd version of FOO in env? In Allegro 6.2 it appears
to go into infinite recursion; I was sort of hoping for it to find the
global definition of FOO, i.e. be more like FLET than LABELS. Of
course in this case I could leave out the &environment parameter all
together and it would find the global definition. But what I really
want is for this to work, as in not recurse forever:
(macrolet ((foo () ''locally-done))
(macrolet ((foo (&environment env) (macroexpand '(foo) env)))
(foo)))
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> wrote:
+---------------
| The docs for MACROLET say:
| The macro-expansion functions defined by macrolet are defined in the
| lexical environment in which the macrolet form appears. Declarations
| and macrolet and symbol-macrolet definitions affect the local macro
| definitions in a macrolet...
...
| bindings that are visible in that lexical environment.
|
| So my question is: is the new macro definition supposed to be in the
| environment that is passed to the local macro.
+---------------
I think so. Looking in the "Discussion" section of "Issue
DECLARATION-SCOPE:NO-HOISTING Summary" (which, yes, I know
is not normative), one finds this:
-- for LABELS and MACROLET, a function name's scope includes all the
code forms for the functions being defined by the special form
[a compiler writer must know how not to get into an infinite loop
of substitutions when there are 'in-line' declarations on these
mutually recursive names];
+---------------
| In Allegro 6.2 it [his example] appears to go into infinite recursion;
+---------------
In CMUCL as well.
+---------------
| I was sort of hoping for it to find the global definition of FOO,
| i.e. be more like FLET than LABELS.
+---------------
I understand, but the above "Issue" quote suggests the opposite.
Also consider that DEFMACRO definitions are allowed to be self-recursive
(and if so, must be careful to terminate), so it seems reasonable that
MACROLET definitions would also be allowed to be self-recursive.
But that's just IMHO...
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
> Peter Seibel <·····@javamonkey.com> wrote:
> +---------------
> | The docs for MACROLET say:
> | The macro-expansion functions defined by macrolet are defined in the
> | lexical environment in which the macrolet form appears. Declarations
> | and macrolet and symbol-macrolet definitions affect the local macro
> | definitions in a macrolet...
> ...
> | bindings that are visible in that lexical environment.
> |
> | So my question is: is the new macro definition supposed to be in the
> | environment that is passed to the local macro.
> +---------------
>
> I think so. Looking in the "Discussion" section of "Issue
> DECLARATION-SCOPE:NO-HOISTING Summary" (which, yes, I know
> is not normative), one finds this:
>
> -- for LABELS and MACROLET, a function name's scope includes all the
> code forms for the functions being defined by the special form
> [a compiler writer must know how not to get into an infinite loop
> of substitutions when there are 'in-line' declarations on these
> mutually recursive names];
>
> +---------------
> | In Allegro 6.2 it [his example] appears to go into infinite recursion;
> +---------------
>
> In CMUCL as well.
>
> +---------------
> | I was sort of hoping for it to find the global definition of FOO,
> | i.e. be more like FLET than LABELS.
> +---------------
>
> I understand, but the above "Issue" quote suggests the opposite.
>
> Also consider that DEFMACRO definitions are allowed to be self-recursive
> (and if so, must be careful to terminate), so it seems reasonable that
> MACROLET definitions would also be allowed to be self-recursive.
>
> But that's just IMHO...
Well, that issue is good evidence that MACROLET was expected to be
more like LABELS than FLET. I was just wondering.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel wrote:
> Well, that issue is good evidence that MACROLET was
> expected to be more like LABELS than FLET. I was just
> wondering.
Recursive macros can appear very naturally, for example
(defmacro dobits ((bn &rest bn-1...b0) &body code)
(if bn-1...b0
`(dotimes (,bn 2)
;; recursive invocation
(dobits ,bn-1...b0 ,@code))
`(dotimes (,bn 2) ,@code)))
=> DOBITS
(dobits (a b c)(print (list a b c))) =>
(0 0 0)
(0 0 1)
(0 1 0)
(0 1 1)
(1 0 0)
(1 0 1)
(1 1 0)
(1 1 1)
NIL
I would have found it a nasty shock if
(macrolet((do-bits((1st-var &rest more) &body code)
(if more
`(dotimes (,1st-var 2)
(do-bits ,more ,@code))
`(dotimes (,1st-var 2) ,@code))))
(do-bits (a b c)
(print (list a b c))))
=>
(0 0 0)
(0 0 1)
(0 1 0)
(0 1 1)
(1 0 0)
(1 0 1)
(1 1 0)
(1 1 1)
hadn't worked.
Alan Crowe
Edinburgh
Scotland
In article <··············@javamonkey.com>,
Peter Seibel <·····@javamonkey.com> wrote:
> The docs for MACROLET say:
>
> The macro-expansion functions defined by macrolet are defined in the
> lexical environment in which the macrolet form appears. Declarations
> and macrolet and symbol-macrolet definitions affect the local macro
> definitions in a macrolet, but the consequences are undefined if the
> local macro definitions reference any local variable or function
> bindings that are visible in that lexical environment.
>
> So my question is: is the new macro definition supposed to be in the
> environment that is passed to the local macro. In other words, if I do
> this:
>
> (defmacro foo () ''done)
>
> (macrolet ((foo (&environment env) (macroexpand '(foo) env)))
> (foo))
>
> should the expansion recurse forever or should the call to MACROEXPAND
> find the DEFMACRO'd version of FOO in env? In Allegro 6.2 it appears
> to go into infinite recursion; I was sort of hoping for it to find the
> global definition of FOO
Why? The whole point of &environment is to give you a handle on the
lexical environment of the form being expanded. In this case, the
lexical environment of the last (foo) is within the macrolet that
defines FOO, so it should recurse forever.
> i.e. be more like FLET than LABELS.
I think you're confused about what &environment does. &environment,
unlike all the other lambda-list keywords, does what it does when the
macro it defines is expanded, not when it is defined. In your example,
&environment operates when the last (foo) is expanded, not when the
containing macrolet is evaluated.
> Of
> course in this case I could leave out the &environment parameter all
> together and it would find the global definition. But what I really
> want is for this to work, as in not recurse forever:
>
> (macrolet ((foo () ''locally-done))
> (macrolet ((foo (&environment env) (macroexpand '(foo) env)))
> (foo)))
Why do you want that to "work"? That example is not enough to divine
your real intention. What are you really trying to do?
E.
Erann Gat <·········@flownet.com> writes:
> In article <··············@javamonkey.com>,
> Peter Seibel <·····@javamonkey.com> wrote:
>
>> The docs for MACROLET say:
>>
>> The macro-expansion functions defined by macrolet are defined in the
>> lexical environment in which the macrolet form appears. Declarations
>> and macrolet and symbol-macrolet definitions affect the local macro
>> definitions in a macrolet, but the consequences are undefined if the
>> local macro definitions reference any local variable or function
>> bindings that are visible in that lexical environment.
>>
>> So my question is: is the new macro definition supposed to be in the
>> environment that is passed to the local macro. In other words, if I do
>> this:
>>
>> (defmacro foo () ''done)
>>
>> (macrolet ((foo (&environment env) (macroexpand '(foo) env)))
>> (foo))
>>
>> should the expansion recurse forever or should the call to MACROEXPAND
>> find the DEFMACRO'd version of FOO in env? In Allegro 6.2 it appears
>> to go into infinite recursion; I was sort of hoping for it to find the
>> global definition of FOO
>
> Why? The whole point of &environment is to give you a handle on the
> lexical environment of the form being expanded. In this case, the
> lexical environment of the last (foo) is within the macrolet that
> defines FOO, so it should recurse forever.
I might put that slightly differently: "the point of &environment is
to give you a handle on the lexical environment *in which* the form
*is* being expanded." If you think that and what you said are
equivalent that's fine; but maybe you see the distinction I'm trying
to draw.
>> i.e. be more like FLET than LABELS.
>
> I think you're confused about what &environment does. &environment,
> unlike all the other lambda-list keywords, does what it does when the
> macro it defines is expanded, not when it is defined. In your example,
> &environment operates when the last (foo) is expanded, not when the
> containing macrolet is evaluated.
How is an &environment different from other parameters? Don't they all
"do what they do" when the macro actually runs--i.e. they are all
bound to arguments in various ways. The &environment parameter just
happens to be bound to a piece of data that wasn't explicitly passed.
That piece of data represents the lexical environment in which the
macro is being expanded. For instance I could use it to get ahold of
the lexically bound version of FOO instead of the global one in this
expression:
(defmacro foo () ''global-foo)
(macrolet ((foo () ''inner-foo))
(macrolet ((bar (&environment env) (macroexpand-1 '(foo) env)))
(bar))) ==> INNER-FOO
My question is: does that lexical environment include the definition
being created by the MACROLET to which the &environment is being
passed. It seems clear now that the anwer in Common Lisp is yes. But
it also seems like it could have gone either way just the same way the
lexical environment of a function defined with FLET doesn't contain
its own definition so this doesn't recurse:
(flet ((foo () (foo))) (foo)) ;; no infinite recursion
while the same function defined with LABELS does:
(labels ((foo () (foo))) (foo)) ;; infinite recursion.
>> Of course in this case I could leave out the &environment parameter
>> all together and it would find the global definition. But what I
>> really want is for this to work, as in not recurse forever:
>>
>> (macrolet ((foo () ''locally-done))
>> (macrolet ((foo (&environment env) (macroexpand '(foo) env)))
>> (foo)))
>
> Why do you want that to "work"? That example is not enough to divine
> your real intention. What are you really trying to do?
Nothing too serious--just playing around. But I was on the trail of
something along the following lines, a macro to introduce new
"lexical" bindings which if used incorrectly would be detected at
compile time:
(defmacro lookup-binding (name type)
(error "No binding for ~a in ~a namespace" name type))
(defmacro bind ((&rest bindings) &body body)
`(macrolet ((lookup-binding (name type &environment env)
(or (loop for (bname btype form) in ',bindings
when (and (eql name bname) (eql type btype)) return form)
(macroexpand-1 `(lookup-binding ,name ,type) env))))
,@body))
(defun foo ()
(bind ((x foo 10) (x bar 20) (y foo 30) (z bar 40))
(list (lookup-binding x foo)
(lookup-binding x bar)
(lookup-binding y foo)
(bind ((z baz 100) (x foo 200))
(list (lookup-binding z baz)
(lookup-binding x foo)
(lookup-binding y foo))))))
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
In article <··············@javamonkey.com>,
Peter Seibel <·····@javamonkey.com> wrote:
> Erann Gat <·········@flownet.com> writes:
>
> > In article <··············@javamonkey.com>,
> > Peter Seibel <·····@javamonkey.com> wrote:
> >
> >> The docs for MACROLET say:
> >>
> >> The macro-expansion functions defined by macrolet are defined in the
> >> lexical environment in which the macrolet form appears. Declarations
> >> and macrolet and symbol-macrolet definitions affect the local macro
> >> definitions in a macrolet, but the consequences are undefined if the
> >> local macro definitions reference any local variable or function
> >> bindings that are visible in that lexical environment.
> >>
> >> So my question is: is the new macro definition supposed to be in the
> >> environment that is passed to the local macro. In other words, if I do
> >> this:
> >>
> >> (defmacro foo () ''done)
> >>
> >> (macrolet ((foo (&environment env) (macroexpand '(foo) env)))
> >> (foo))
> >>
> >> should the expansion recurse forever or should the call to MACROEXPAND
> >> find the DEFMACRO'd version of FOO in env? In Allegro 6.2 it appears
> >> to go into infinite recursion; I was sort of hoping for it to find the
> >> global definition of FOO
> >
> > Why? The whole point of &environment is to give you a handle on the
> > lexical environment of the form being expanded. In this case, the
> > lexical environment of the last (foo) is within the macrolet that
> > defines FOO, so it should recurse forever.
>
> I might put that slightly differently: "the point of &environment is
> to give you a handle on the lexical environment *in which* the form
> *is* being expanded." If you think that and what you said are
> equivalent that's fine; but maybe you see the distinction I'm trying
> to draw.
No, sorry, they seem the same to me.
> How is an &environment different from other parameters? Don't they all
> "do what they do" when the macro actually runs--i.e. they are all
> bound to arguments in various ways.
Only &environment gives you access to the lexical environment of the
form being expanded. All other elements of the macro have as their
lexical environment the lexical environment of the macro's definition.
e.g.:
(let ( (x 1) )
(defmacro foo (&aux (y x)) y))
(let ( (x 2) ) (foo)) --> 1
(macrolet ( (foo () 1) )
(defmacro baz () (foo)))
(macrolet ( (foo () 2) ) (baz)) --> 1
> The &environment parameter just
> happens to be bound to a piece of data that wasn't explicitly passed.
That is true, but it misses the crucial point: the &environment
parameter is bound to a piece of data THAT DOESNT AND CANNOT EXIST until
the macroexpansion takes place.
> That piece of data represents the lexical environment in which the
> macro is being expanded. For instance I could use it to get ahold of
> the lexically bound version of FOO instead of the global one in this
> expression:
>
> (defmacro foo () ''global-foo)
>
> (macrolet ((foo () ''inner-foo))
> (macrolet ((bar (&environment env) (macroexpand-1 '(foo) env)))
> (bar))) ==> INNER-FOO
>
> My question is: does that lexical environment include the definition
> being created by the MACROLET to which the &environment is being
> passed. It seems clear now that the anwer in Common Lisp is yes.
Yes, but not for the reason you think. :-) It's not that macrolet is
like LABELS and not like FLET, it's that the scope of a macrolet is
lexical, so necessarily any use of a macro defined by macrolet must, by
definition, be within the lexical scope of the macrolet, and therefore
the lexical environment of the expansion must include the macro being
defined by the macrolet.
> But it also seems like it could have gone either way
No, that's not right, and if you think it is you're still missing the
point.
> just the same way the
> lexical environment of a function defined with FLET doesn't contain
> its own definition so this doesn't recurse:
>
> (flet ((foo () (foo))) (foo)) ;; no infinite recursion
>
> while the same function defined with LABELS does:
>
> (labels ((foo () (foo))) (foo)) ;; infinite recursion.
But that is a false analogy. The lexical scope of the recursive (or
not) call to FOO can be determined at the time FOO is compiled. By
contrast, the lexical scope of a macro expansion that uses an
&environment parameter cannot be determined at the time the macro is
compiled, it can only be determined at the time the macro is expanded.
Consider this:
(flet ((foo () (foo))) #'foo)
and contrast it with:
(macrolet ((foo (&environment e) e)) ???)
How would you fill in the question marks to get an analogous behavior?
Answer: you can't. The reason you can't is that information is missing.
That missing information is the crucial difference between macrolet and
FLET/LABELS, and the reason that your comparison is a false analogy.
> > Why do you want that to "work"? That example is not enough to divine
> > your real intention. What are you really trying to do?
>
> Nothing too serious--just playing around. But I was on the trail of
> something along the following lines, a macro to introduce new
> "lexical" bindings which if used incorrectly would be detected at
> compile time:
>
> (defmacro lookup-binding (name type)
> (error "No binding for ~a in ~a namespace" name type))
>
> (defmacro bind ((&rest bindings) &body body)
> `(macrolet ((lookup-binding (name type &environment env)
> (or (loop for (bname btype form) in ',bindings
> when (and (eql name bname) (eql type btype))
> return form)
> (macroexpand-1 `(lookup-binding ,name ,type) env))))
> ,@body))
>
> (defun foo ()
> (bind ((x foo 10) (x bar 20) (y foo 30) (z bar 40))
> (list (lookup-binding x foo)
> (lookup-binding x bar)
> (lookup-binding y foo)
> (bind ((z baz 100) (x foo 200))
> (list (lookup-binding z baz)
> (lookup-binding x foo)
> (lookup-binding y foo))))))
Ah. What you want is not to redefine the lookup-binding macro, what you
want is a lexically scoped environment (probably chained so that nested
bind macros will work properly). Try something like this:
(defvar *bindings* (make-empty-bindings))
(defmacro %current-bindings () *bindings*)
(defmacro with-bindings (b &body body)
`(macrolet ( (%current-bindings () ,b) ) ,@body))
(defun env-bindings (env) (macroexpand '(%current-bindings) env))
(defmacro current-compiletime-bindings (&environment env)
(env-bindings env))
(defmacro bind (...)
`(let ( (new-bindings (augment-bindings
(current-compiletime-bindings) ...)) )
(with-bindings ,new-bindings ...))
I obviously haven't tested this so I may have left out a backquote or a
comma somewhere.
E.
Erann Gat <·········@flownet.com> writes:
> In article <··············@javamonkey.com>,
> Peter Seibel <·····@javamonkey.com> wrote:
>> The &environment parameter just happens to be bound to a piece of
>> data that wasn't explicitly passed.
>
> That is true, but it misses the crucial point: the &environment
> parameter is bound to a piece of data THAT DOESNT AND CANNOT EXIST
> until the macroexpansion takes place.
I'm not sure what I said that makes you think I think the &environment
exists before macroexpansion. Isn't macroexpansion time and compile
time the same time? Isn't that the whole point of macros? Anyway, let
me try to explain my thinking one more time in greater detail so you
can point more precisely to where you think I go astray:
(defmacro foo () ''outer-foo)
(macrolet ((foo () ''inner-foo))
(macrolet ((bar (&environment env)
`(list
,(macroexpand-1 '(foo))
,(macroexpand-1 '(foo) env))))
(bar)))
==> (OUTER-FOO INNER-FOO)
Note how the second MACROEXPAND-1 which is passed the &environment
object "sees" the lexically bound definition of FOO instead of the
global definition. That is, at the time BAR's expansion code runs it
exists in a specific lexical environment, namely one in which the
macro FOO is lexically bound to code that generates the expansion
'INNER-FOO. So far so good, I hope.
So here's my mental model of what's going on. When the outer MACROLET
compiled the current lexical environment is augmented with a binding
for FOO (to the macro that expands to 'INNER-FOO). Call this
environment E1.
Then the compiler compiles the body of the outer MACROLET, namely the
inner MACROLET. First E1 is augmented with the binding for BAR, giving
us a new environment E2. Then it compiles the body of the inner
MACROLET. When it compiles the call to BAR it looks in the current
environment, E2, finds the definiton of BAR and invokes it to generate
the expansion.
Because BAR declares an &environment parameter the compiler has to
pass an environment object when it invokes it. My point is, the
compiler could just as well pass E1 as E2. In Common Lisp it passed E2
which is why if we were to change the name of BAR to FOO the second
call to MACROEXPAND-1 will go into infinite recursion but I don't see
any in prinicple reason why it *must* pass E2 rather than E1. They
certainly both exist at macroexpansion time.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
In article <··············@javamonkey.com>,
Peter Seibel <·····@javamonkey.com> wrote:
> Erann Gat <·········@flownet.com> writes:
>
> > In article <··············@javamonkey.com>,
> > Peter Seibel <·····@javamonkey.com> wrote:
>
> >> The &environment parameter just happens to be bound to a piece of
> >> data that wasn't explicitly passed.
> >
> > That is true, but it misses the crucial point: the &environment
> > parameter is bound to a piece of data THAT DOESNT AND CANNOT EXIST
> > until the macroexpansion takes place.
>
> I'm not sure what I said that makes you think I think the &environment
> exists before macroexpansion. Isn't macroexpansion time and compile
> time the same time?
There are two different compile times in play here. There's the time at
which a macro definition is compiled, and the time at which in
invocation of a macro is compiled. These are two different times.
> Isn't that the whole point of macros? Anyway, let
> me try to explain my thinking one more time in greater detail so you
> can point more precisely to where you think I go astray:
>
> (defmacro foo () ''outer-foo)
>
> (macrolet ((foo () ''inner-foo))
> (macrolet ((bar (&environment env)
> `(list
> ,(macroexpand-1 '(foo))
> ,(macroexpand-1 '(foo) env))))
> (bar)))
>
> ==> (OUTER-FOO INNER-FOO)
>
> Note how the second MACROEXPAND-1 which is passed the &environment
> object "sees" the lexically bound definition of FOO instead of the
> global definition. That is, at the time BAR's expansion code runs it
> exists in a specific lexical environment, namely one in which the
> macro FOO is lexically bound to code that generates the expansion
> 'INNER-FOO. So far so good, I hope.
>
> So here's my mental model of what's going on. When the outer MACROLET
> compiled the current lexical environment is augmented with a binding
> for FOO (to the macro that expands to 'INNER-FOO). Call this
> environment E1.
>
> Then the compiler compiles the body of the outer MACROLET, namely the
> inner MACROLET. First E1 is augmented with the binding for BAR, giving
> us a new environment E2. Then it compiles the body of the inner
> MACROLET. When it compiles the call to BAR it looks in the current
> environment, E2, finds the definiton of BAR and invokes it to generate
> the expansion.
>
> Because BAR declares an &environment parameter the compiler has to
> pass an environment object when it invokes it. My point is, the
> compiler could just as well pass E1 as E2. In Common Lisp it passed E2
> which is why if we were to change the name of BAR to FOO the second
> call to MACROEXPAND-1 will go into infinite recursion but I don't see
> any in prinicple reason why it *must* pass E2 rather than E1. They
> certainly both exist at macroexpansion time.
The reason is that E1 is already available to BAR by default. If
&environment passed E1 (the lexical environment of the definition of
BAR) instead of E2 (the lexical environment of the invocation of BAR)
then &environment would have no utility. There would be nothing you
could do with &environment if it behaved that way that you could not do
if &environment didn't exist at all.
E.
Peter Seibel <·····@javamonkey.com> writes:
> My question is: does that lexical environment include the definition
> being created by the MACROLET to which the &environment is being
> passed. It seems clear now that the anwer in Common Lisp is yes.
Consider also:
(macrolet ((foo (form &environment env) `',(macroexpand-1 form env)))
(symbol-macrolet ((bar result))
(foo bar)))
If the environment passed to the macro expansion function of FOO
didn't include the macro binding of FOO, then it presumably
wouldn't include the symbol macro binding of BAR either. That
would hamper the ability of macros to analyze their arguments.
> But I was on the trail of something along the following lines,
> a macro to introduce new "lexical" bindings which if used
> incorrectly would be detected at compile time:
I've been wondering of this. Is it appropriate for such a macro
to signal an error if it detects the code being compiled is
lexically incorrect? Or should it instead signal a warning and
expand to (error 'SOME-SUBCLASS-OF-PROGRAM-ERROR ...), which
might or might not be eventually executed?
Kalle Olavi Niemitalo <···@iki.fi> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> My question is: does that lexical environment include the definition
>> being created by the MACROLET to which the &environment is being
>> passed. It seems clear now that the anwer in Common Lisp is yes.
>
> Consider also:
>
> (macrolet ((foo (form &environment env) `',(macroexpand-1 form env)))
> (symbol-macrolet ((bar result))
> (foo bar)))
>
> If the environment passed to the macro expansion function of FOO
> didn't include the macro binding of FOO, then it presumably
> wouldn't include the symbol macro binding of BAR either. That
> would hamper the ability of macros to analyze their arguments.
Yes. That's a good point. Now I think I see what Erann was getting at.
The point he was making, I think--and that I was missing until now--is
that the &environment parameter passed to the call to FOO is the
lexical environment of the *macro call site* not the environment in
which the MACROLET appears. Thus, to expand on your example:
;; At this point we have environment E0
(macrolet ((foo (&environment env) (macroexpand-1 'bar env)))
;; Now we have E1 which is E0 + FOO
(symbol-macrolet ((bar 'bar-one))
;; Now we have E2, E1 + BAR
(foo)) ;; &environment is E2
(symbol-macrolet ((bar 'bar-two))
;; Now we have E3, E1 + BAR but a different binding for BAR
(foo))) ;; &environment is E3
Erann, if that was your point, I get it now. Though I think I still
might argue that there could be some use for a version of MACROLET
that passes E0 (in this example) as the &environment rather than E2
and E3. Or better yet, first class environments that supports
functions that the code in FOO could use to get at the bindings from
the environment in which it was defined rather than called.
> I've been wondering of this. Is it appropriate for such a macro to
> signal an error if it detects the code being compiled is lexically
> incorrect? Or should it instead signal a warning and expand to
> (error 'SOME-SUBCLASS-OF-PROGRAM-ERROR ...), which might or might
> not be eventually executed?
No doubt a matter of taste. For me, it seems silly for a macro that
*can* detect that it is generating code that will do nothing but
generate an error to wait until runtime. Though if you didn't want to
distrurb the flow of development you could have it both ways--signal
an error at compiletime with CERROR and in then if the user chooses to
continue, generate an expansion as you suggest, that signals an error
at runtime. At the very least, IMHO, the macro out to signal a warning
at compile time.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel wrote:
> No doubt a matter of taste. For me, it seems silly for a macro that
> *can* detect that it is generating code that will do nothing but
> generate an error to wait until runtime.
This is not silly at all. What if the macro form is in dead code?
Paul
"Paul F. Dietz" <·····@dls.net> writes:
> Peter Seibel wrote:
>
>> No doubt a matter of taste. For me, it seems silly for a macro that
>> *can* detect that it is generating code that will do nothing but
>> generate an error to wait until runtime.
>
> This is not silly at all. What if the macro form is in dead code?
Like I said, a matter of taste. If I was writing a macro that could
detect in certain situations it was being used incorrectly I'd rather
take the hit of getting an error the few times when it was used
incorrectly *and* in dead code in order to get told about the problem
in the times when it was being used incorrectly in live code.
Obviously my estimation of which I'd prefer is based on my gut feeling
of how likely it is that I'll use such a (hypothetical) macro
incorrectly versus in dead code.
Plus if it's in dead code then shouldn't it have been eliminated an
not run at all and thus not signaled an error? (Hmmm, is that legal
for the compiler to do?)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel wrote:
> Plus if it's in dead code then shouldn't it have been eliminated an
> not run at all and thus not signaled an error? (Hmmm, is that legal
> for the compiler to do?)
Well, yes, but the compiler doesn't get a chance to do this if the
macro throws a fit at macro expansion time.
Paul
"Paul F. Dietz" <·····@dls.net> writes:
> Peter Seibel wrote:
>
>> Plus if it's in dead code then shouldn't it have been eliminated an
>> not run at all and thus not signaled an error? (Hmmm, is that legal
>> for the compiler to do?)
>
> Well, yes, but the compiler doesn't get a chance to do this if the
> macro throws a fit at macro expansion time.
Sure. But macros (and special forms) are expanded from the outside in.
So if I write:
(if nil (some-macro-that-barfs) (something-else))
in theory at least the processing of the IF can do dead-code
elimination and not bother invoking some-macro-that-barfs.
But if dead-code elimination happens in a separate pass after
everything is minimally compiled, then yes, whiny macros will cause
trouble in cases where they technically needn't.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
In article <··············@javamonkey.com>,
Peter Seibel <·····@javamonkey.com> wrote:
> Kalle Olavi Niemitalo <···@iki.fi> writes:
>
> > Peter Seibel <·····@javamonkey.com> writes:
> >
> >> My question is: does that lexical environment include the definition
> >> being created by the MACROLET to which the &environment is being
> >> passed. It seems clear now that the anwer in Common Lisp is yes.
> >
> > Consider also:
> >
> > (macrolet ((foo (form &environment env) `',(macroexpand-1 form env)))
> > (symbol-macrolet ((bar result))
> > (foo bar)))
> >
> > If the environment passed to the macro expansion function of FOO
> > didn't include the macro binding of FOO, then it presumably
> > wouldn't include the symbol macro binding of BAR either. That
> > would hamper the ability of macros to analyze their arguments.
>
> Yes. That's a good point. Now I think I see what Erann was getting at.
> The point he was making, I think--and that I was missing until now--is
> that the &environment parameter passed to the call to FOO is the
> lexical environment of the *macro call site* not the environment in
> which the MACROLET appears. Thus, to expand on your example:
>
> ;; At this point we have environment E0
> (macrolet ((foo (&environment env) (macroexpand-1 'bar env)))
> ;; Now we have E1 which is E0 + FOO
> (symbol-macrolet ((bar 'bar-one))
> ;; Now we have E2, E1 + BAR
> (foo)) ;; &environment is E2
> (symbol-macrolet ((bar 'bar-two))
> ;; Now we have E3, E1 + BAR but a different binding for BAR
> (foo))) ;; &environment is E3
>
> Erann, if that was your point, I get it now.
Yes, that's it exactly.
> Though I think I still
> might argue that there could be some use for a version of MACROLET
> that passes E0 (in this example) as the &environment rather than E2
> and E3.
You have that. E0 is what you get by default for all code in the body
of FOO.
> Or better yet, first class environments that supports
> functions that the code in FOO could use to get at the bindings from
> the environment in which it was defined rather than called.
That's what lexical closures are for.
> > I've been wondering of this. Is it appropriate for such a macro to
> > signal an error if it detects the code being compiled is lexically
> > incorrect? Or should it instead signal a warning and expand to
> > (error 'SOME-SUBCLASS-OF-PROGRAM-ERROR ...), which might or might
> > not be eventually executed?
>
> No doubt a matter of taste. For me, it seems silly for a macro that
> *can* detect that it is generating code that will do nothing but
> generate an error to wait until runtime.
In a language as dynamic as Lisp there are very few circumstances under
which it can be known definitively that the only possible outcome from
any given code fragment is an error. But lexical references are one of
the notable exceptions, so IMO a compile-time error (or warning) is
appropriate.
E.
Erann Gat <·········@flownet.com> writes:
> In article <··············@javamonkey.com>,
> Peter Seibel <·····@javamonkey.com> wrote:
>> Yes. That's a good point. Now I think I see what Erann was getting
>> at. The point he was making, I think--and that I was missing until
>> now--is that the &environment parameter passed to the call to FOO
>> is the lexical environment of the *macro call site* not the
>> environment in which the MACROLET appears. Thus, to expand on your
>> example:
>>
>> ;; At this point we have environment E0
>> (macrolet ((foo (&environment env) (macroexpand-1 'bar env)))
>> ;; Now we have E1 which is E0 + FOO
>> (symbol-macrolet ((bar 'bar-one))
>> ;; Now we have E2, E1 + BAR
>> (foo)) ;; &environment is E2
>> (symbol-macrolet ((bar 'bar-two))
>> ;; Now we have E3, E1 + BAR but a different binding for BAR
>> (foo))) ;; &environment is E3
>>
>> Erann, if that was your point, I get it now.
>
> Yes, that's it exactly.
Cool. FWIW, I wouldn't explain that in terms of *time* so much as
where the lexical environment is taken from.
>> Though I think I still might argue that there could be some use for
>> a version of MACROLET that passes E0 (in this example) as the
>> &environment rather than E2 and E3.
>
> You have that. E0 is what you get by default for all code in the
> body of FOO.
Not quite because if we call MACROEXPAND-1 without passing an
environment object we get the global lexical environment. When E0
happens to be the global environment it works but not in general. That
is:
(defmacro foo () ''global-foo)
(macrolet ((foo () ''macrolet-foo))
(macrolet ((bar () (macroexpand '(foo))))
(bar))) ==> GLOBAL-FOO
If you'll recall, this is essentially my original case, where BAR was
also called FOO and I was trying to avoid infinite recursion.
>> Or better yet, first class environments that supports functions
>> that the code in FOO could use to get at the bindings from the
>> environment in which it was defined rather than called.
>
> That's what lexical closures are for.
I don't know what you mean. How can I use lexical closures to close
over macrolet bindings? Well, maybe it'll come to me in my sleep. Good
night.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
In article <··············@javamonkey.com>,
Peter Seibel <·····@javamonkey.com> wrote:
> Erann Gat <·········@flownet.com> writes:
>
> > In article <··············@javamonkey.com>,
> > Peter Seibel <·····@javamonkey.com> wrote:
>
> >> Yes. That's a good point. Now I think I see what Erann was getting
> >> at. The point he was making, I think--and that I was missing until
> >> now--is that the &environment parameter passed to the call to FOO
> >> is the lexical environment of the *macro call site* not the
> >> environment in which the MACROLET appears. Thus, to expand on your
> >> example:
> >>
> >> ;; At this point we have environment E0
> >> (macrolet ((foo (&environment env) (macroexpand-1 'bar env)))
> >> ;; Now we have E1 which is E0 + FOO
> >> (symbol-macrolet ((bar 'bar-one))
> >> ;; Now we have E2, E1 + BAR
> >> (foo)) ;; &environment is E2
> >> (symbol-macrolet ((bar 'bar-two))
> >> ;; Now we have E3, E1 + BAR but a different binding for BAR
> >> (foo))) ;; &environment is E3
> >>
> >> Erann, if that was your point, I get it now.
> >
> > Yes, that's it exactly.
>
> Cool. FWIW, I wouldn't explain that in terms of *time* so much as
> where the lexical environment is taken from.
OK.
> >> Though I think I still might argue that there could be some use for
> >> a version of MACROLET that passes E0 (in this example) as the
> >> &environment rather than E2 and E3.
> >
> > You have that. E0 is what you get by default for all code in the
> > body of FOO.
>
> Not quite because if we call MACROEXPAND-1 without passing an
> environment object we get the global lexical environment.
Yes quite. You forget that there is more than one way to macroexpand a
macro. In fact, there are three ways, each of which gives you a
different lexical environment:
(macroexpand '(foo)) ; Uses the global lexical environment
(macroexpand '(foo) env) ; Uses env
(foo) ; Uses the lexical environment at the point where the call appears
> When E0
> happens to be the global environment it works but not in general. That
> is:
>
> (defmacro foo () ''global-foo)
>
> (macrolet ((foo () ''macrolet-foo))
> (macrolet ((bar () (macroexpand '(foo))))
> (bar))) ==> GLOBAL-FOO
(macrolet ((foo () ''macrolet-foo))
(macrolet ((bar () `',(foo)))
(bar))) ==> MACROLET-FOO
> >> Or better yet, first class environments that supports functions
> >> that the code in FOO could use to get at the bindings from the
> >> environment in which it was defined rather than called.
> >
> > That's what lexical closures are for.
>
> I don't know what you mean. How can I use lexical closures to close
> over macrolet bindings? Well, maybe it'll come to me in my sleep. Good
> night.
It's very simple: don't forget that macros are functions. The
difference between a macro and a non-macro function is that a macro is
called at compile time and its result is interpreted as code. But other
than that they are functions just like any other:
? (macrolet ((foo () ''macrolet-foo)) (lambda () (foo)))
#<Anonymous Function #xDCD096>
? (funcall *)
MACROLET-FOO
?
E.
Erann Gat <·········@flownet.com> writes:
> > >> Though I think I still might argue that there could be some use for
> > >> a version of MACROLET that passes E0 (in this example) as the
> > >> &environment rather than E2 and E3.
> > >
> > > You have that. E0 is what you get by default for all code in the
> > > body of FOO.
> >
> > Not quite because if we call MACROEXPAND-1 without passing an
> > environment object we get the global lexical environment.
>
> Yes quite. You forget that there is more than one way to macroexpand a
> macro. In fact, there are three ways, each of which gives you a
> different lexical environment:
>
> (macroexpand '(foo)) ; Uses the global lexical environment
> (macroexpand '(foo) env) ; Uses env
======================^^^
> (foo) ; Uses the lexical environment at the point where the call appears
Note also that the flagged line could include a globally-saved
environment at a random time, as well as the variable that is
sometime accepted by a macro or a compiler-macro definition. I
wonder what havoc could be wreaked by some programmer trying to
force his will on an otherwise understandable program? Ah, yes,
this is CL, which gives us all a lot of rope...
> > I don't know what you mean. How can I use lexical closures to close
> > over macrolet bindings? Well, maybe it'll come to me in my sleep. Good
> > night.
>
> It's very simple: don't forget that macros are functions. The
> difference between a macro and a non-macro function is that a macro is
> called at compile time and its result is interpreted as code.
This is technically correct, but not complete, and since you are
talking to the writer of a book, I'd like to complete the information
(I notice that this is not the first time people have said that
macroexpansion occurs at compile time, and I would not want people
to misunderstand that it can occur at other times as well, including
in your example below, which is really occurring at evaluate time
if you have a lisp EVAL implemetation that doesn't compile its forms)
So to be precise, macroexpansion occurs at macroexpand-time, which
occurs in one of three places:
- compile time
- eval time
- due to an explicit call to macroexpand or macroexpand-1
> But other
> than that they are functions just like any other:
>
> ? (macrolet ((foo () ''macrolet-foo)) (lambda () (foo)))
> #<Anonymous Function #xDCD096>
> ? (funcall *)
> MACROLET-FOO
> ?
Note that with an EVAL that doesn't compile forms that it is
interpreting, the closure is interpreted (the effects are
the same, of course):
CL-USER(1): (macrolet ((foo () ''macrolet-foo)) (lambda () (foo)))
#<Interpreted Closure (unnamed) @ #x717c2e7a>
CL-USER(2): (funcall *)
MACROLET-FOO
CL-USER(3):
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
In article <·············@franz.com>, Duane Rettig <·····@franz.com>
wrote:
> > (macroexpand '(foo)) ; Uses the global lexical environment
> > (macroexpand '(foo) env) ; Uses env
> ======================^^^
> > (foo) ; Uses the lexical environment at the point where the call appears
>
> Note also that the flagged line could include a globally-saved
> environment at a random time, as well as the variable that is
> sometime accepted by a macro or a compiler-macro definition. I
> wonder what havoc could be wreaked by some programmer trying to
> force his will on an otherwise understandable program? Ah, yes,
> this is CL, which gives us all a lot of rope...
Indeed:
? (defmacro call-with-current-lexical-environment
(thunk &environment env)
`(funcall ,thunk ',env))
CALL-WITH-CURRENT-LEXICAL-ENVIRONMENT
? (let ( (x 1) ) (call-with-current-lexical-environment #'identity))
;Compiler warnings :
; Unused lexical variable X, in an anonymous lambda form.
#<CCL::LEXICAL-ENVIRONMENT #xDDAB6E>
Oh, the horror! :-)
> > > I don't know what you mean. How can I use lexical closures to close
> > > over macrolet bindings? Well, maybe it'll come to me in my sleep. Good
> > > night.
> >
> > It's very simple: don't forget that macros are functions. The
> > difference between a macro and a non-macro function is that a macro is
> > called at compile time and its result is interpreted as code.
>
> This is technically correct, but not complete, and since you are
> talking to the writer of a book, I'd like to complete the information
> (I notice that this is not the first time people have said that
> macroexpansion occurs at compile time, and I would not want people
> to misunderstand that it can occur at other times as well, including
> in your example below, which is really occurring at evaluate time
> if you have a lisp EVAL implemetation that doesn't compile its forms)
Yes, I'm chronically careless about this. Mea culpa. Thanks for the
clarification.
E.
I had wondered about this. The index of the CLHS for &environment shows
several entries. IIRC, only find-class contains a statement that the
&environment argument has dynamic scope and that references to it outside
that scope are undefined. Lispworks for Windows 4.3.7 seems to assume
dynamic scope in more places than just find-class:
CL-USER 1 > (let ((x 1) (y 2)) (macrolet ((foo (&environment e) e)) (foo)))
#S(LEXICAL::ENVIRONMENT LEXICAL::VARIABLES ((Y . #:Y) (X . #:X))
LEXICAL::FUNCTIONS ((FOO . #'(LAMBDA (DSPEC::%%MACROARG%% #:&ENVIRONMENT1076
&AUX (#:&WHOLE1077 DSPEC::%%MACROARG%%) (E #:&ENVIRONMENT1076) (#:NIL1078
(CDR #:&WHOLE1077)) (#:CHECK-LAMBDA-LIST-TOP-LEVEL1079
(DSPEC::CHECK-LAMBDA-LIST-TOP-LEVEL (QUOTE NIL) #:&WHOLE1077 #:NIL1078 0 0
(QUOTE NIL) :MACRO))) (DECLARE (LAMBDA-LIST &ENVIRONMENT E)) (BLOCK FOO
E)))) LEXICAL::REMOTE-ENVIRONMENT NIL)
CL-USER 2 > (setq *e* *)
Notice ((Y . #:Y) (X . #:X)). It seems that #:Y is out of dynamic scope
after (1) has completed and so having access to the &environment structure
isn't interesting for variables, other than seeing what was in lexical scope
at the time the structure was created. It is more interesting that the foo
macro definition is saved in the &environment structure:
CL-USER 3 > (defmacro baz () '(foo))
BAZ
CL-USER 4 > (macroexpand '(baz))
(FOO)
T
CL-USER 5 > (macroexpand-1 '(baz) *e*)
(FOO)
T
CL-USER 6 > (macroexpand '(baz) *e*)
#S(LEXICAL::ENVIRONMENT LEXICAL::VARIABLES ((Y . #:Y) (X . #:X))
LEXICAL::FUNCTIONS ((FOO . #'(LAMBDA (DSPEC::%%MACROARG%% #:&ENVIRONMENT1076
&AUX (#:&WHOLE1077 DSPEC::%%MACROARG%%) (E #:&ENVIRONMENT1076) (#:NIL1078
(CDR #:&WHOLE1077)) (#:CHECK-LAMBDA-LIST-TOP-LEVEL1079
(DSPEC::CHECK-LAMBDA-LIST-TOP-LEVEL (QUOTE NIL) #:&WHOLE1077 #:NIL1078 0 0
(QUOTE NIL) :MACRO))) (DECLARE (LAMBDA-LIST &ENVIRONMENT E)) (BLOCK FOO
E)))) LEXICAL::REMOTE-ENVIRONMENT NIL)
T
Having foo work differently given the &environment argument reminds me of
Erann's locales.
Jeff
"Erann Gat" <·········@flownet.com> wrote in message
····································@nntp1.jpl.nasa.gov...
> In article <·············@franz.com>, Duane Rettig <·····@franz.com>
> wrote:
>
> > > (macroexpand '(foo)) ; Uses the global lexical environment
> > > (macroexpand '(foo) env) ; Uses env
> > ======================^^^
> > > (foo) ; Uses the lexical environment at the point where the call
appears
> >
> > Note also that the flagged line could include a globally-saved
> > environment at a random time, as well as the variable that is
> > sometime accepted by a macro or a compiler-macro definition. I
> > wonder what havoc could be wreaked by some programmer trying to
> > force his will on an otherwise understandable program? Ah, yes,
> > this is CL, which gives us all a lot of rope...
>
> Indeed:
>
> ? (defmacro call-with-current-lexical-environment
> (thunk &environment env)
> `(funcall ,thunk ',env))
> CALL-WITH-CURRENT-LEXICAL-ENVIRONMENT
> ? (let ( (x 1) ) (call-with-current-lexical-environment #'identity))
> ;Compiler warnings :
> ; Unused lexical variable X, in an anonymous lambda form.
> #<CCL::LEXICAL-ENVIRONMENT #xDDAB6E>
>
> Oh, the horror! :-)
>
...
In article <····················@sea-read.news.verio.net>,
"Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
> I had wondered about this. The index of the CLHS for &environment shows
> several entries. IIRC, only find-class contains a statement that the
> &environment argument has dynamic scope and that references to it outside
> that scope are undefined.
Heh, good point. CLHS section 3.4.4 does indeed say, "The object that
is bound to the environment parameter has dynamic extent."
Oh well.
> Having foo work differently given the &environment argument reminds me of
> Erann's locales.
It should. Locales are just first-class lexical environments (with
indefinite extent).
E.
Erann Gat <·········@flownet.com> writes:
> In article <····················@sea-read.news.verio.net>,
> "Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
>
> > I had wondered about this. The index of the CLHS for &environment shows
> > several entries. IIRC, only find-class contains a statement that the
> > &environment argument has dynamic scope and that references to it outside
> > that scope are undefined.
>
> Heh, good point. CLHS section 3.4.4 does indeed say, "The object that
> is bound to the environment parameter has dynamic extent."
>
> Oh well.
Well...
Dynamic-extent can be extended by implementation (though of course
portable programs would not be able to use such extensions). For
example, if my compiler didn't optimize a dynamic-extent call to
list, and the list were thus allocated in the lisp heap instead of the
stack, then my debugger could, if it knew about this, rummage around in
the entrails of the heap to "see" the remains of the list whose extent
is out of scope. The same extendability can be done for environments,
as well, and that is one of the features of my environments-access
extension.
> > Having foo work differently given the &environment argument reminds me of
> > Erann's locales.
>
> It should. Locales are just first-class lexical environments (with
> indefinite extent).
Yes, I recognized a huge similarity in the problems you were solving
with your ILC presentation and the problems I've been working on in
environments-access. Note that our interpreter cleans up interpreted
environments for the most part:
CL-USER(2): (let ((x 1) (y 2))
(macrolet ((foo (&environment e) e))
(list (foo)
(macrolet ((bar (&environment e) e))
(bar)))))
(#<Augmentable EVALUATION environment (1 1) @ #x10b08762>
#<Augmentable EVALUATION environment (1 1 1) @ #x10b08d7a>)
CL-USER(3): :i *
A NEW proper list @ #x10b3ff89 with 2 elements
0-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08762>
1-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08d7a>
CL-USER(4): :i 0 1
A NEW SYS::AUGMENTABLE-ENVIRONMENT-BASE struct @ #x10b081b2 = #<# @ #x10b081b2>
0 Class --------> #<STRUCTURE-CLASS SYS::AUGMENTABLE-ENVIRONMENT-BASE>
1 VARIABLE-HASHTABLE -> #<EQ hash-table with 2 entries @ #x10b07daa>
2 FUNCTION-HASHTABLE -> #<EQUAL hash-table with 0 entries @ #x10b07ee2>
3 BLOCK-HASHTABLE -> #<EQ hash-table with 0 entries @ #x10b0801a>
4 TAG-HASHTABLE -> #<EQL hash-table with 0 entries @ #x10b080a2>
5 DECLARATION-HASHTABLE -> #<EQ hash-table with 0 entries @ #x10b0812a>
6 ID -----------> fixnum 48 [#x000000c0]
7 SYMBOL-PROPS -> The symbol NIL
8 NON-SYMBOL-PROPS -> The symbol NIL
9 DISABLED-PROPS -> The symbol NIL
CL-USER(5):
(note how the function namespaces have been cleaned out, as well as most
of the internal variables (gensyms, etc) other than the topmost let
variables - I actually don't recall why the let special-operator is
leaving them in...)
But compilation environments, though still dynamic extent to some
degree, are more controlled in the dynamicity of their extent:
CL-USER(5): (defun foo ()
(let ((x 1) (y 2))
(macrolet ((foo (&environment e) e))
(list (foo)
(macrolet ((bar (&environment e) e))
(bar))))))
FOO
CL-USER(6): (compile *)
; While compiling FOO:
Warning: Variable Y is never used.
Warning: Variable X is never used.
FOO
T
NIL
CL-USER(7): (setq envs (foo))
(#<Augmentable COMPILATION environment (2 2 1 1 1 1 1) @ #x10b13b3a>
#<Augmentable COMPILATION environment (1 2 3 2 1 1 1 1 1) @ #x10b1428a>)
CL-USER(8): (sys:function-information 'foo (car envs))
:MACRO
(#<Interpreted Closure FOO @ #x10b13982>)
NIL
T
CL-USER(9): (sys:function-information 'bar (car envs))
NIL
CL-USER(10): (sys:function-information 'bar (cadr envs))
:MACRO
(#<Interpreted Closure BAR @ #x10b141ba>)
NIL
T
CL-USER(11):
Note how these environment objects (very lightweight objects which
refer into a larger base environment object which has a sense of
contour) can still access their contours, at the proper levels.
Note also that the descriptions of the environments (literally,
each one's "index") describes the shape of the contours to which
these lightweight environment objects refer. Theoretically, one
would need not save actual environment objects after the compilation
in order to do some post-compilation analysis on the environment -
one could define functionality that said "rebuild me an environment
object for contour (2 2 1 1 1 1 1), and one for (1 2 3 2 1 1 1 1 1),
and then use those to perform access operations on the total
environment. This kind of analysis is probably a ways off, though.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
In article <·············@franz.com>, Duane Rettig <·····@franz.com>
wrote:
> Erann Gat <·········@flownet.com> writes:
>
> > In article <····················@sea-read.news.verio.net>,
> > "Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
> >
> > > I had wondered about this. The index of the CLHS for &environment shows
> > > several entries. IIRC, only find-class contains a statement that the
> > > &environment argument has dynamic scope and that references to it outside
> > > that scope are undefined.
> >
> > Heh, good point. CLHS section 3.4.4 does indeed say, "The object that
> > is bound to the environment parameter has dynamic extent."
> >
> > Oh well.
>
> Well...
>
> Dynamic-extent can be extended by implementation (though of course
> portable programs would not be able to use such extensions).
I wonder why the standard mentions this at all. Why not just leave it
up to the user to (declare (dynamic-extent e)) if they want to? Feel
free to treat that as a rhetorical question.
> > > Having foo work differently given the &environment argument reminds me of
> > > Erann's locales.
> >
> > It should. Locales are just first-class lexical environments (with
> > indefinite extent).
>
> Yes, I recognized a huge similarity in the problems you were solving
> with your ILC presentation and the problems I've been working on in
> environments-access.
There's nothing new under the sun. I guess I should have also
reiterated that there really isn't any such thing as "Erann's locales".
At best there is "Erann's implementation of locales in Common Lisp", but
the idea of locales comes from T (at least that's where I stole it from).
> Note that our interpreter cleans up interpreted
> environments for the most part:
>
> CL-USER(2): (let ((x 1) (y 2))
> (macrolet ((foo (&environment e) e))
> (list (foo)
> (macrolet ((bar (&environment e) e))
> (bar)))))
> (#<Augmentable EVALUATION environment (1 1) @ #x10b08762>
> #<Augmentable EVALUATION environment (1 1 1) @ #x10b08d7a>)
> CL-USER(3): :i *
> A NEW proper list @ #x10b3ff89 with 2 elements
> 0-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08762>
> 1-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08d7a>
> CL-USER(4): :i 0 1
> A NEW SYS::AUGMENTABLE-ENVIRONMENT-BASE struct @ #x10b081b2 = #<# @
> #x10b081b2>
> 0 Class --------> #<STRUCTURE-CLASS SYS::AUGMENTABLE-ENVIRONMENT-BASE>
> 1 VARIABLE-HASHTABLE -> #<EQ hash-table with 2 entries @ #x10b07daa>
> 2 FUNCTION-HASHTABLE -> #<EQUAL hash-table with 0 entries @ #x10b07ee2>
> 3 BLOCK-HASHTABLE -> #<EQ hash-table with 0 entries @ #x10b0801a>
> 4 TAG-HASHTABLE -> #<EQL hash-table with 0 entries @ #x10b080a2>
> 5 DECLARATION-HASHTABLE -> #<EQ hash-table with 0 entries @ #x10b0812a>
> 6 ID -----------> fixnum 48 [#x000000c0]
> 7 SYMBOL-PROPS -> The symbol NIL
> 8 NON-SYMBOL-PROPS -> The symbol NIL
> 9 DISABLED-PROPS -> The symbol NIL
> CL-USER(5):
>
> (note how the function namespaces have been cleaned out, as well as most
> of the internal variables (gensyms, etc) other than the topmost let
> variables - I actually don't recall why the let special-operator is
> leaving them in...)
What happened to the macro definitions of FOO and BAR? Shouldn't they
be in there somewhere? Or am I missing something?
E.
Erann Gat <·········@flownet.com> writes:
> In article <·············@franz.com>, Duane Rettig <·····@franz.com>
> wrote:
>
> > Erann Gat <·········@flownet.com> writes:
> >
> > > In article <····················@sea-read.news.verio.net>,
> > > "Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
> > >
> > > > I had wondered about this. The index of the CLHS for &environment shows
> > > > several entries. IIRC, only find-class contains a statement that the
> > > > &environment argument has dynamic scope and that references to it outside
> > > > that scope are undefined.
> > >
> > > Heh, good point. CLHS section 3.4.4 does indeed say, "The object that
> > > is bound to the environment parameter has dynamic extent."
> > >
> > > Oh well.
> >
> > Well...
> >
> > Dynamic-extent can be extended by implementation (though of course
> > portable programs would not be able to use such extensions).
>
> I wonder why the standard mentions this at all. Why not just leave it
> up to the user to (declare (dynamic-extent e)) if they want to? Feel
> free to treat that as a rhetorical question.
It shouldn't be rhetorical. The standard mentions it as a promise to
the implementor (not to have to keep them around) and as a warning
to the user (not to assume that they will be kept around). This
comes up in your later question, below.
> > > > Having foo work differently given the &environment argument reminds me of
> > > > Erann's locales.
> > >
> > > It should. Locales are just first-class lexical environments (with
> > > indefinite extent).
> >
> > Yes, I recognized a huge similarity in the problems you were solving
> > with your ILC presentation and the problems I've been working on in
> > environments-access.
>
> There's nothing new under the sun. I guess I should have also
> reiterated that there really isn't any such thing as "Erann's locales".
> At best there is "Erann's implementation of locales in Common Lisp", but
> the idea of locales comes from T (at least that's where I stole it from).
But the fact that someone else invented/stole/uses such similar
functionality provides comfort to me that what I am doing will
eventually satisfy the needs of more than just me as an
implementor.
> > Note that our interpreter cleans up interpreted
> > environments for the most part:
> >
> > CL-USER(2): (let ((x 1) (y 2))
> > (macrolet ((foo (&environment e) e))
> > (list (foo)
> > (macrolet ((bar (&environment e) e))
> > (bar)))))
> > (#<Augmentable EVALUATION environment (1 1) @ #x10b08762>
> > #<Augmentable EVALUATION environment (1 1 1) @ #x10b08d7a>)
> > CL-USER(3): :i *
> > A NEW proper list @ #x10b3ff89 with 2 elements
> > 0-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08762>
> > 1-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08d7a>
> > CL-USER(4): :i 0 1
> > A NEW SYS::AUGMENTABLE-ENVIRONMENT-BASE struct @ #x10b081b2 = #<# @
> > #x10b081b2>
> > 0 Class --------> #<STRUCTURE-CLASS SYS::AUGMENTABLE-ENVIRONMENT-BASE>
> > 1 VARIABLE-HASHTABLE -> #<EQ hash-table with 2 entries @ #x10b07daa>
> > 2 FUNCTION-HASHTABLE -> #<EQUAL hash-table with 0 entries @ #x10b07ee2>
> > 3 BLOCK-HASHTABLE -> #<EQ hash-table with 0 entries @ #x10b0801a>
> > 4 TAG-HASHTABLE -> #<EQL hash-table with 0 entries @ #x10b080a2>
> > 5 DECLARATION-HASHTABLE -> #<EQ hash-table with 0 entries @ #x10b0812a>
> > 6 ID -----------> fixnum 48 [#x000000c0]
> > 7 SYMBOL-PROPS -> The symbol NIL
> > 8 NON-SYMBOL-PROPS -> The symbol NIL
> > 9 DISABLED-PROPS -> The symbol NIL
> > CL-USER(5):
> >
> > (note how the function namespaces have been cleaned out, as well as most
> > of the internal variables (gensyms, etc) other than the topmost let
> > variables - I actually don't recall why the let special-operator is
> > leaving them in...)
>
> What happened to the macro definitions of FOO and BAR? Shouldn't they
> be in there somewhere? Or am I missing something?
Perhaps the parenthesized comment? Permission is given to me to clean
out the macro definitions by that statement in the spec you questioned
calling out the dynamic-extent nature of the environment. The fact that
the variables (x and y) are not cleaned out is a slight puzzle to me,
and I'll probably go back as a low-priority optimization task to find
out why, and to possibly clean those out as well.
Note that in the compiled-version example in my previous post, the
definitions are not cleaned out - the information is kept.
Perhaps the real question should be "is there a reason to make
different decisions as to whether to save dynamic-extent information
for the interpreter, vs whether to save it for the compiler?"
The answer is "yes":
The interpreter is not intended to do any analysis; it instead blindly
follows instructions in its execution of special forms and in its
expansion of macros and calling of functions. Thus, there is no
reason to save the information for later analysis that will never be
done anyway, and there is _great_ reason to not save it (since saving
the data makes them thus non-ephemeral and clogs up the
garbage-collector). When I had first implemented this
cleanup-on-contour-exit when I first implemented our interpreter with
these environments back in version 6.0, I had been concerned that such
artificial work of removing data in this structure would slow the
evaluator to unacceptable levels. Instead, it turned out to be _much_
faster to do the clean-ups, rather than to waste time trying to gc all
of that data. The newer interpreter was indeed slower than the older
one (which saved environmental data by binding global variables that
held alists), but not by very much.
The compiler, on the other hand, does analysis on the results of
the environments it saves, and so the more information it can save
about its environment, the better job it can do of analyzing. Much
more data is saved than in an interpreter, but the data that is
saved is in fact static (it describes a compilation tree, not the
current dynamic state of an execution), and so it is not as hard
on the gc as un-cleaned interpreted environments would be.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
In article <·············@franz.com>, Duane Rettig <·····@franz.com>
wrote:
> Erann Gat <·········@flownet.com> writes:
>
> > In article <·············@franz.com>, Duane Rettig <·····@franz.com>
> > wrote:
> >
> > > Erann Gat <·········@flownet.com> writes:
> > >
> > > > In article <····················@sea-read.news.verio.net>,
> > > > "Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
> > > >
> > > > > I had wondered about this. The index of the CLHS for &environment
> > > > > shows
> > > > > several entries. IIRC, only find-class contains a statement that the
> > > > > &environment argument has dynamic scope and that references to it
> > > > > outside
> > > > > that scope are undefined.
> > > >
> > > > Heh, good point. CLHS section 3.4.4 does indeed say, "The object that
> > > > is bound to the environment parameter has dynamic extent."
> > > >
> > > > Oh well.
> > >
> > > Well...
> > >
> > > Dynamic-extent can be extended by implementation (though of course
> > > portable programs would not be able to use such extensions).
> >
> > I wonder why the standard mentions this at all. Why not just leave it
> > up to the user to (declare (dynamic-extent e)) if they want to? Feel
> > free to treat that as a rhetorical question.
>
> It shouldn't be rhetorical. The standard mentions it as a promise to
> the implementor (not to have to keep them around) and as a warning
> to the user (not to assume that they will be kept around). This
> comes up in your later question, below.
>
> > > > > Having foo work differently given the &environment argument reminds
> > > > > me of
> > > > > Erann's locales.
> > > >
> > > > It should. Locales are just first-class lexical environments (with
> > > > indefinite extent).
> > >
> > > Yes, I recognized a huge similarity in the problems you were solving
> > > with your ILC presentation and the problems I've been working on in
> > > environments-access.
> >
> > There's nothing new under the sun. I guess I should have also
> > reiterated that there really isn't any such thing as "Erann's locales".
> > At best there is "Erann's implementation of locales in Common Lisp", but
> > the idea of locales comes from T (at least that's where I stole it from).
>
> But the fact that someone else invented/stole/uses such similar
> functionality provides comfort to me that what I am doing will
> eventually satisfy the needs of more than just me as an
> implementor.
>
> > > Note that our interpreter cleans up interpreted
> > > environments for the most part:
> > >
> > > CL-USER(2): (let ((x 1) (y 2))
> > > (macrolet ((foo (&environment e) e))
> > > (list (foo)
> > > (macrolet ((bar (&environment e) e))
> > > (bar)))))
> > > (#<Augmentable EVALUATION environment (1 1) @ #x10b08762>
> > > #<Augmentable EVALUATION environment (1 1 1) @ #x10b08d7a>)
> > > CL-USER(3): :i *
> > > A NEW proper list @ #x10b3ff89 with 2 elements
> > > 0-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08762>
> > > 1-> SYS::AUGMENTABLE-ENVIRONMENT struct = #<# @ #x10b08d7a>
> > > CL-USER(4): :i 0 1
> > > A NEW SYS::AUGMENTABLE-ENVIRONMENT-BASE struct @ #x10b081b2 = #<# @
> > > #x10b081b2>
> > > 0 Class --------> #<STRUCTURE-CLASS SYS::AUGMENTABLE-ENVIRONMENT-BASE>
> > > 1 VARIABLE-HASHTABLE -> #<EQ hash-table with 2 entries @ #x10b07daa>
> > > 2 FUNCTION-HASHTABLE -> #<EQUAL hash-table with 0 entries @
> > > #x10b07ee2>
> > > 3 BLOCK-HASHTABLE -> #<EQ hash-table with 0 entries @ #x10b0801a>
> > > 4 TAG-HASHTABLE -> #<EQL hash-table with 0 entries @ #x10b080a2>
> > > 5 DECLARATION-HASHTABLE -> #<EQ hash-table with 0 entries @
> > > #x10b0812a>
> > > 6 ID -----------> fixnum 48 [#x000000c0]
> > > 7 SYMBOL-PROPS -> The symbol NIL
> > > 8 NON-SYMBOL-PROPS -> The symbol NIL
> > > 9 DISABLED-PROPS -> The symbol NIL
> > > CL-USER(5):
> > >
> > > (note how the function namespaces have been cleaned out, as well as most
> > > of the internal variables (gensyms, etc) other than the topmost let
> > > variables - I actually don't recall why the let special-operator is
> > > leaving them in...)
> >
> > What happened to the macro definitions of FOO and BAR? Shouldn't they
> > be in there somewhere? Or am I missing something?
>
> Perhaps the parenthesized comment? Permission is given to me to clean
> out the macro definitions by that statement in the spec you questioned
> calling out the dynamic-extent nature of the environment. The fact that
> the variables (x and y) are not cleaned out is a slight puzzle to me,
> and I'll probably go back as a low-priority optimization task to find
> out why, and to possibly clean those out as well.
Ah. So if I wrote the following instead:
(let ((x 1) (y 2))
(macrolet ((foo (&environment e) e))
(list (foo)
(macrolet ((bar (&environment e) e))
(f (bar)))))
then I would expect to see two items in the function hashtable, is that
right?
> The interpreter is not intended to do any analysis; it instead blindly
> follows instructions in its execution of special forms and in its
> expansion of macros and calling of functions.
Yes, this is what puzzles me. I would expect to see the environment
"cleaned out" in the compiled version, not the interpreted version, but
the exact opposite seems to be happening. I feel like I'm missing
something really basic here.
E.
Erann Gat <·········@flownet.com> writes:
> > > > (note how the function namespaces have been cleaned out, as well as most
> > > > of the internal variables (gensyms, etc) other than the topmost let
> > > > variables - I actually don't recall why the let special-operator is
> > > > leaving them in...)
> > >
> > > What happened to the macro definitions of FOO and BAR? Shouldn't they
> > > be in there somewhere? Or am I missing something?
> >
> > Perhaps the parenthesized comment? Permission is given to me to clean
> > out the macro definitions by that statement in the spec you questioned
> > calling out the dynamic-extent nature of the environment. The fact that
> > the variables (x and y) are not cleaned out is a slight puzzle to me,
> > and I'll probably go back as a low-priority optimization task to find
> > out why, and to possibly clean those out as well.
>
> Ah. So if I wrote the following instead:
>
> (let ((x 1) (y 2))
> (macrolet ((foo (&environment e) e))
> (list (foo)
> (macrolet ((bar (&environment e) e))
> (f (bar)))))
>
> then I would expect to see two items in the function hashtable, is that
> right?
I'm not sure what this code should do (parens are mismatched, and
f is undefined), but here is one way you can see the un-cleaned-out
environment:
CL-USER(2): (let ((x 1) (y 2))
(macrolet ((foo (&environment e) e))
(list (foo)
(macrolet ((bar (&environment e) e))
(setq *x* (bar))
(break "foo")))))
Break: foo
Restart actions (select using :continue):
0: return from break.
1: Return to Top Level (an "abort" restart).
2: Abort entirely from this (lisp) process.
[1c] CL-USER(3): (sys:function-information 'bar *x*)
:MACRO
(#<Interpreted Closure BAR @ #x10b0d2ca>)
NIL
T
[1c] CL-USER(4):
[Note that this does not work on 6.2; it only works on 7.0 versions]
> > The interpreter is not intended to do any analysis; it instead blindly
> > follows instructions in its execution of special forms and in its
> > expansion of macros and calling of functions.
>
> Yes, this is what puzzles me. I would expect to see the environment
> "cleaned out" in the compiled version, not the interpreted version, but
> the exact opposite seems to be happening. I feel like I'm missing
> something really basic here.
Perhaps if you tell me why you would expect the compiled version to
clean out the environment and not the interpreted version, we can
then figure out what you're missing.
Note that since environments are dynamic-extent, it is perfectly
legal to clean any parts of them that go out of scope, as soon as
they go out of scope. Also, since the extension to the environments
module causes environments to be indefinite-extent, it is fine if
those parts are _not_ cleaned out, ever. And finally, consider that
unlike pushed-down alist style environments, these environments are
not automatically cleaned out; some program construct has to explicitly
do this clean-out. So with that in mind, whether the compiler and/or
interpreter cleans out parts of the environment would depend on
whether it is _desireable_ to do so. So we would look at what the pros
and cons of cleaning out the environment are...
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
In article <·············@franz.com>, Duane Rettig <·····@franz.com>
wrote:
> Erann Gat <·········@flownet.com> writes:
>
> > > > > (note how the function namespaces have been cleaned out, as well as
> > > > > most
> > > > > of the internal variables (gensyms, etc) other than the topmost let
> > > > > variables - I actually don't recall why the let special-operator is
> > > > > leaving them in...)
> > > >
> > > > What happened to the macro definitions of FOO and BAR? Shouldn't they
> > > > be in there somewhere? Or am I missing something?
> > >
> > > Perhaps the parenthesized comment? Permission is given to me to clean
> > > out the macro definitions by that statement in the spec you questioned
> > > calling out the dynamic-extent nature of the environment. The fact that
> > > the variables (x and y) are not cleaned out is a slight puzzle to me,
> > > and I'll probably go back as a low-priority optimization task to find
> > > out why, and to possibly clean those out as well.
> >
> > Ah. So if I wrote the following instead:
> >
> > (let ((x 1) (y 2))
> > (macrolet ((foo (&environment e) e))
> > (list (foo)
> > (macrolet ((bar (&environment e) e))
> > (f (bar)))))
> >
> > then I would expect to see two items in the function hashtable, is that
> > right?
>
> I'm not sure what this code should do
Upon reflection, neither am I. :-/
> Perhaps if you tell me why you would expect the compiled version to
> clean out the environment and not the interpreted version, we can
> then figure out what you're missing.
I guess it's because of just what you went on to say:
> Note that since environments are dynamic-extent, it is perfectly
> legal to clean any parts of them that go out of scope, as soon as
> they go out of scope. Also, since the extension to the environments
> module causes environments to be indefinite-extent, it is fine if
> those parts are _not_ cleaned out, ever. And finally, consider that
> unlike pushed-down alist style environments, these environments are
> not automatically cleaned out; some program construct has to explicitly
> do this clean-out. So with that in mind, whether the compiler and/or
> interpreter cleans out parts of the environment would depend on
> whether it is _desireable_ to do so. So we would look at what the pros
> and cons of cleaning out the environment are...
I would have expected a priori that it would be more desirable for the
compiler than for the interpreter to do the analysis needed to figure
out when cleaning was a net win. But IANALI (I Am Not A Lisp
Implementor).
E.
Erann Gat <·········@flownet.com> writes:
> In article <·············@franz.com>, Duane Rettig <·····@franz.com>
> wrote:
>
>
> > Perhaps if you tell me why you would expect the compiled version to
> > clean out the environment and not the interpreted version, we can
> > then figure out what you're missing.
>
> I guess it's because of just what you went on to say:
>
> > Note that since environments are dynamic-extent, it is perfectly
> > legal to clean any parts of them that go out of scope, as soon as
> > they go out of scope. Also, since the extension to the environments
> > module causes environments to be indefinite-extent, it is fine if
> > those parts are _not_ cleaned out, ever. And finally, consider that
> > unlike pushed-down alist style environments, these environments are
> > not automatically cleaned out; some program construct has to explicitly
> > do this clean-out. So with that in mind, whether the compiler and/or
> > interpreter cleans out parts of the environment would depend on
> > whether it is _desireable_ to do so. So we would look at what the pros
> > and cons of cleaning out the environment are...
>
> I would have expected a priori that it would be more desirable for the
> compiler than for the interpreter to do the analysis needed to figure
> out when cleaning was a net win. But IANALI (I Am Not A Lisp
> Implementor).
OK, I think we're getting closer to where the missed assumptions lie.
Note then further that there is no analysis involved in deciding
when and/or what to clean out - it's just a decision to do it or
not. In-scope or out-of-scope requires no analysis; it is easily
calculated from the index stored with the data. The analysis I'm
talking about is what the compiler generally does in more than one
pass, the first pass being the macroexpand/environment-building pass,
and further passes might involve further type propagation, register
coloring, lap code generation, etc. Each of those passes (i.e. the
further analysis) can make use of whatever environment information
is available, so it makes no sense to remove the information until
the entire lexical unit is compiled.
The interpreter, on the other hand, executes a form, and then its job
is done. So the environment information that has gone out of scope
is no longer needed, and can be cleaned out.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
In article <·············@franz.com>, Duane Rettig <·····@franz.com>
wrote:
> Erann Gat <·········@flownet.com> writes:
>
> > In article <·············@franz.com>, Duane Rettig <·····@franz.com>
> > wrote:
> >
> >
> > > Perhaps if you tell me why you would expect the compiled version to
> > > clean out the environment and not the interpreted version, we can
> > > then figure out what you're missing.
> >
> > I guess it's because of just what you went on to say:
> >
> > > Note that since environments are dynamic-extent, it is perfectly
> > > legal to clean any parts of them that go out of scope, as soon as
> > > they go out of scope. Also, since the extension to the environments
> > > module causes environments to be indefinite-extent, it is fine if
> > > those parts are _not_ cleaned out, ever. And finally, consider that
> > > unlike pushed-down alist style environments, these environments are
> > > not automatically cleaned out; some program construct has to explicitly
> > > do this clean-out. So with that in mind, whether the compiler and/or
> > > interpreter cleans out parts of the environment would depend on
> > > whether it is _desireable_ to do so. So we would look at what the pros
> > > and cons of cleaning out the environment are...
> >
> > I would have expected a priori that it would be more desirable for the
> > compiler than for the interpreter to do the analysis needed to figure
> > out when cleaning was a net win. But IANALI (I Am Not A Lisp
> > Implementor).
>
> OK, I think we're getting closer to where the missed assumptions lie.
>
> Note then further that there is no analysis involved in deciding
> when and/or what to clean out - it's just a decision to do it or
> not. In-scope or out-of-scope requires no analysis; it is easily
> calculated from the index stored with the data. The analysis I'm
> talking about is what the compiler generally does in more than one
> pass, the first pass being the macroexpand/environment-building pass,
> and further passes might involve further type propagation, register
> coloring, lap code generation, etc. Each of those passes (i.e. the
> further analysis) can make use of whatever environment information
> is available, so it makes no sense to remove the information until
> the entire lexical unit is compiled.
>
> The interpreter, on the other hand, executes a form, and then its job
> is done. So the environment information that has gone out of scope
> is no longer needed, and can be cleaned out.
Ah, that makes sense. Maybe that's why the let-bound variables didn't
get cleaned up? Their scope is potentially indefinite.
E.
Erann Gat <·········@flownet.com> writes:
> In article <·············@franz.com>, Duane Rettig <·····@franz.com>
> wrote:
> > The interpreter, on the other hand, executes a form, and then its job
> > is done. So the environment information that has gone out of scope
> > is no longer needed, and can be cleaned out.
>
> Ah, that makes sense. Maybe that's why the let-bound variables didn't
> get cleaned up? Their scope is potentially indefinite.
Only if they're specials. Again, I haven't had time to look at why
they do this yet - my guess is that it is simply an oversight in the
interpreter, and thus a potential optimization opportunity.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Erann Gat <·········@flownet.com> writes:
> There's nothing new under the sun. I guess I should have also
> reiterated that there really isn't any such thing as "Erann's locales".
> At best there is "Erann's implementation of locales in Common Lisp", but
> the idea of locales comes from T (at least that's where I stole it from).
Talking about lightweight stuff:
IIRC, unlike Common Lisp's simulation of locales based on SETF, a
locative to a an element of an array would not maintain that array
alive. The array can be GC'ed, and only that one element may remain,
accessible via the locative. I'm pretty sure Oaklisp (a descendent of
T) implements it this way.
Regards,
Jorg Hohle
Telekom/T-Systems Technology Center
Duane,
Would you please help me to understand what's going on with:
(interpreted)
> CL-USER(2): (let ((x 1) (y 2))
> (macrolet ((foo (&environment e) e))
> (list (foo)
> (macrolet ((bar (&environment e) e))
> (bar)))))
> (#<Augmentable EVALUATION environment (1 1) @ #x10b08762>
> #<Augmentable EVALUATION environment (1 1 1) @ #x10b08d7a>)
and
(compiled)
> CL-USER(5): (defun foo ()
> (let ((x 1) (y 2))
> (macrolet ((foo (&environment e) e))
> (list (foo)
> (macrolet ((bar (&environment e) e))
> (bar))))))
...
> (#<Augmentable COMPILATION environment (2 2 1 1 1 1 1) @ #x10b13b3a>
> #<Augmentable COMPILATION environment (1 2 3 2 1 1 1 1 1) @ #x10b1428a>)
In particular, I'm trying to understand (1 1) and (1 1 1), and (2 2 1 1 1 1
1) and (1 2 3 2 1 1 1 1 1).
I think you're saying the lists essentially point to bindings in a chain of
environment objects. What is the structure of the data being indexed? Only
one number was added in the interpreted environment while two were added in
the compiled environment (and one changed, if the lists are aligned right
justified). What is the difference between the interpreted and compiled
structures being indexed?
I've seen references to indexing into fields within lexical contours using
<depth><position> pairs but I can't make that fit with what I'm seeing
above.
Also, I have seen references to lexical contours as a graphical
representation of a chain of environment objects, with rectangles drawn
within rectangles representing each new scope/environment object. (I am
supposing the graphical representation reminds one of contours on a
geographic contour map.) I get the feeling that understanding is off base
given your statement about "the shape of the contours to which
these lightweight environment objects refer". Do you mind defining contours,
and the shapes of contours, for me?
Thanks for your help,
Jeff
...
> Note how these environment objects (very lightweight objects which
> refer into a larger base environment object which has a sense of
> contour) can still access their contours, at the proper levels.
> Note also that the descriptions of the environments (literally,
> each one's "index") describes the shape of the contours to which
> these lightweight environment objects refer. Theoretically, one
> would need not save actual environment objects after the compilation
> in order to do some post-compilation analysis on the environment -
> one could define functionality that said "rebuild me an environment
> object for contour (2 2 1 1 1 1 1), and one for (1 2 3 2 1 1 1 1 1),
> and then use those to perform access operations on the total
> environment. This kind of analysis is probably a ways off, though.
...
[ I think I figuerd out the problem with my post ... Sorry for the
resend, but the first was incomplete... Duane]
"Jeff Caldwell" <·········@yaSPhooAM.com> writes:
> Duane,
>
> Would you please help me to understand what's going on with:
Heh ...
A complete explanation of this whole thing would probably require
me to write up a tutorial for the implementation of our
environments-access module, which I probably will eventually do
at some ILC conference, especially when we prepare to actually
release the module into opensource. It is quite a clever system,
invented by Steve Haflich, and I've been improving on it for quite
a while now, ever since Allegro 6.0 or 6.1, when we first defined
an alternate interpreter using this module as its implementation,
and then when we switched to using that implementation exclusively.
For 7.0, we have mostly completed the integration of this module
into the compiler, and will continue to improve on it after that.
I don't think I have time for a complete explanation, but since
you have asked pointed questions about a very small portion of the
module, I can attempt to answer those specifically.
> (interpreted)
> > CL-USER(2): (let ((x 1) (y 2))
> > (macrolet ((foo (&environment e) e))
> > (list (foo)
> > (macrolet ((bar (&environment e) e))
> > (bar)))))
> > (#<Augmentable EVALUATION environment (1 1) @ #x10b08762>
> > #<Augmentable EVALUATION environment (1 1 1) @ #x10b08d7a>)
>
> and
>
> (compiled)
> > CL-USER(5): (defun foo ()
> > (let ((x 1) (y 2))
> > (macrolet ((foo (&environment e) e))
> > (list (foo)
> > (macrolet ((bar (&environment e) e))
> > (bar))))))
> ...
> > (#<Augmentable COMPILATION environment (2 2 1 1 1 1 1) @ #x10b13b3a>
> > #<Augmentable COMPILATION environment (1 2 3 2 1 1 1 1 1) @ #x10b1428a>)
>
> In particular, I'm trying to understand (1 1) and (1 1 1), and (2 2 1 1 1 1
> 1) and (1 2 3 2 1 1 1 1 1).
>
> I think you're saying the lists essentially point to bindings in a chain of
> environment objects.
Not really. Chains are explicily what we want to avoid, because we want to
get as close to O(1) time as possible when accessing these environments.
I think that the traditional approach to tracking bindings in an interpreter
(and even in a compiler) is to maintain global specials that house bindings
for each namespace (we used to use %venv% and %fenv% for variables and
functions, respectively) and to push new bindings onto these variables as
necessary, lambda-binding these at various points where code in the interpreter
might take alternative branches. When the interpreter enters a special form
such as LET, it enters a new variable contour, binding %venv% to itself.
When the LET is exitted, %venv% gets unbound, thus popping off the contour.
You get the idea, hopefully.
The elegance of this approach is that it uses lisp to implement itself
with extremely little mechanism, because the binding mechanism and the
recursion implents the binding/recursion of the code being interpreted.
However, there are several problems with this approach, including these:
1. As the alists used to associate names with their bindings get longer
due to more complexity, searches through these lists take longer, and
even looking up a nonexistent name requires a search through the entire
alist as it is currently bound.
2. The automatic nature of the contour entry/exit doesn't allow for
introspection by a debugger, compiler, or any other lisp tool, unless
the binding mechanism of the lisp itself could be traversed. For example,
If I have a form
(let ((foo 'a))
(let ((bar 'b))
(break ...)
...))
Here, %venv% would presumably be bound twice, once at entry to each of
the let forms. But there is no way to actually describe those contours,
apart from knowing how in the debugger to find %venv%'s previous binding,
and the previous binding before that, and so on. Furthermore, if macrolets,
function calls, or other binders of %fenv% were added to the mix, it would
be fairly complex for a debugger to find out which of these bindings
occurred in what order. Finally, consider sibling contours:
(let ((foo 'a))
(let ((bar 'b))
(break ...))
(let ((bas 'c))
(break ...)
...))
Here we have a dynamic-extent problem; how, in the second break, do we
describe the binding for 'bar? (Answer: we don't - it is gone). We
would like at a higher level to be able to describe these three let
forms in relationship to each other.
You can get the Trial version of Allegro CL and play around with
environments (if you are doing beta testing for 7.0 there is even
documentation that you can read about the interface). I won't go
too deep into this, only deep enough to answer your questions, but
an understanding of some of our goals is necessary to know what you
are seeing.
1. We want to provide as close to O(1) as possible. This implies hash
tables and no chained access at the implementational level.
2. We want to allow for introspection after code is walked, the
requirements of which are different between compilatio and execution,
as we've been discussing.
3. We wanted to stay as close as possible to the CLtL2 specification,
pp 207-214, so that explanatory documentation could start with that
definition as a springboard. If you haven't read these pages in CLtL2,
take the opportunity now to do that, and my further explanations will
make more sense.
4. ... There are others, but I think these will suffice for now.
> What is the structure of the data being indexed?
When you get your hands on an environment object, it has been already
been built up in structure for you (CL doesn't specify how to create
these, only how to pass them around, altough we do provide
sys::make-augmentable-environment that you can use to create your own).
These augmentable-environments are lightweight structures that always
point directly to a heavyweight structure called a base (short for
augmentable-environment-base). This base has several slots in it which
represent the accumulation of all lexical environments that are currently
saved, which might include all bindings for all namespaces ever done in an
execution or a compilation (in reality, this is not true, but it is
theoretically possible).
Each entry in the base includes the "index" of the environment contour
which created it. The environment object itself that is returned from
sys:augment-environment has the same "index" as the associations that were
added during that call to augment-environment.
Note: sys:augment-environment is different from CLtL2 in that it accepts
a :reuse keyword argument. This argument defaults to false, but can be
set to true in order to add multiple bindings at the same contour level
as a previous call to augment-environment. The descriptions in CLtL2
imply that augment-environment always returns a new environment. With
the :reuse argument, environment information for a particular contour
need not be saved up and applied all at once in a single call to
sys:augment-environment.
Only
> one number was added in the interpreted environment while two were added in
> the compiled environment (and one changed, if the lists are aligned right
> justified). What is the difference between the interpreted and compiled
> structures being indexed?
You are correct in noticing that the index lists are right justified. In a
lisp implementation this is obviously the most efficient representation.
Index-list structure is shared, and very little consing is necessary to
increment indeces in either depth or width.
So how is an index "incremented"?
It is only done with augment-environment, but if :reuse is true, it is not
done. If an environment is at a leaf on a tree of environments, then
augment-environment will push a 1 onto the index. If a sister environment
is created, then the first item in the environment's index is incremented
and pushed onto the cdr of the index. In that way, one gets a tree of
environments. Here is a sample run of some null augmentations (just to
demonstrate the index creation technique and the relationship between
contours). It's not a very useful series, because the actual
environment-base has nothing in it (because we didn't put anyting into it):
CL-USER(4): (setq e1 (sys::make-augmentable-environment))
#<Augmentable EVALUATION environment NIL @ #x10b0a732>
CL-USER(5): (setq e2 (sys:augment-environment e1))
#<Augmentable EVALUATION environment (1) @ #x10b0ac72>
CL-USER(6): (setq e3 (sys:augment-environment e2))
#<Augmentable EVALUATION environment (1 1) @ #x10b0af7a>
CL-USER(7): (setq e4 (sys:augment-environment e2))
#<Augmentable EVALUATION environment (2 1) @ #x10b0b252>
CL-USER(8): (setq e5 (sys:augment-environment e4))
#<Augmentable EVALUATION environment (1 2 1) @ #x10b0b52a>
CL-USER(9): (setq e6 (sys:augment-environment e3))
#<Augmentable EVALUATION environment (1 1 1) @ #x10b0b802>
CL-USER(10):
Note in the hierarchy that e2 is the only child of e1, and e3 and e4 are
siblings and children of e2. Note also that e4 was created after e3
time-wise. We might see such an association of these environments in
the three LET statements, above, where e2 represented the let which
binds foo, e3 reprexsents the let which binds bar, and e4 represents
the let which binds bas. I included a couple of first-cousins in the
mix as well, for clarity of the relationships...
Note that all of e1 - e6 have the same base.
> I've seen references to indexing into fields within lexical contours using
> <depth><position> pairs but I can't make that fit with what I'm seeing
> above.
The problem with depth/position pairs is that manufaturing a new index is
more heavyweight, and knowing whether an object is "in" an environment
takes more artihmetic calculation. Also, depth/position pairs only show
you one aspect of the location of your contour - it doesn't tell the
story of prcisely where in the whole this environment is located. Thus,
you would indeed need to chain environments to establish the rest of the
information needed. As it is, this index style supplies three degrees of
information: depth (i.e. the length of the list), position (i.e. the
integer at the car of the list) and path (i.e. represeinting recursive
positions down to the head of the tree).
> Also, I have seen references to lexical contours as a graphical
> representation of a chain of environment objects, with rectangles drawn
> within rectangles representing each new scope/environment object. (I am
> supposing the graphical representation reminds one of contours on a
> geographic contour map.) I get the feeling that understanding is off base
> given your statement about "the shape of the contours to which
> these lightweight environment objects refer".
Note that these indeces give all of the information necessary to identify
all relationships between environments with the same base. With chained
environments, it is indeed necessary to draw some pictures in order to
understand the relationshps of two environment objects that don't point
one to the other (for example, you would only know that e5 and e6 are
cousins in a chained implemetation by following their chains back until
you found a common ancestor, in this case, e2. Then you could draw the
graphical representation between them. But with our indexing style, the
index _is_ he graphical representation.
As for why the compiled code forms more contours than the interpreted code,
there are a couple of reasons:
1. The compiler starts with a longer chain itself - some setup is
required to take care of any possible break-off environments
2. The example is larger - note that the interpreted version was
just a let form, and the compiled version was a function definition.
3. The compiler might need to create more finely grained contours,
depending on what it needs to do.
4. The interpreter can clean up after itself, which allows index
components to start over.
> Do you mind defining contours,
> and the shapes of contours, for me?
I think I've mostly done this, by description. Essentially a new contour
is entered whenever a call to augement-environment has reuse argument of
nil. The shape of that contour depends on whether progeny or siblings
are created.
> Thanks for your help,
I hope it _did_ help :-)
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Duane Rettig <·····@franz.com> writes:
> [ I think I figuerd out the problem with my post ... Sorry for the
> resend, but the first was incomplete... Duane]
What this was all about, was that apparently nntp doesn't like to see
a % as the first character on a line. I had tried to post directly
(thinking perhaps that gnus was at fault when I had tried to send four
or five times) but when it got caught soon after a line that started
with %venv%, I realized that there was some kind of protocol intervening.
I fixed the line, and what you saw apologizing for the resend was really
the only message that got through. So here I am once more, apologizing
in advance if any of the garbage messages get through later.
Protocols... Sheesh.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Duane Rettig <·····@franz.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > [ I think I figuerd out the problem with my post ... Sorry for the
> > resend, but the first was incomplete... Duane]
>
> What this was all about, was that apparently nntp doesn't like to see
> a % as the first character on a line. I had tried to post directly
> (thinking perhaps that gnus was at fault when I had tried to send four
> or five times) but when it got caught soon after a line that started
> with %venv%, I realized that there was some kind of protocol intervening.
> I fixed the line, and what you saw apologizing for the resend was really
> the only message that got through. So here I am once more, apologizing
> in advance if any of the garbage messages get through later.
>
> Protocols... Sheesh.
Hmm. That's very weird. Let's see. Here are (I hope) two lines
of text ...
% This line starts with a "%" sign, but the next doesn't.
* This line doesn't start with a "%" sign, but the previous one does.
--
Gareth McCaughan
.sig under construc
I wrote:
[Duane Rettig:]
> > What this was all about, was that apparently nntp doesn't like to see
> > a % as the first character on a line. I had tried to post directly
> > (thinking perhaps that gnus was at fault when I had tried to send four
> > or five times) but when it got caught soon after a line that started
> > with %venv%, I realized that there was some kind of protocol intervening.
> > I fixed the line, and what you saw apologizing for the resend was really
> > the only message that got through. So here I am once more, apologizing
> > in advance if any of the garbage messages get through later.
> >
> > Protocols... Sheesh.
>
> Hmm. That's very weird. Let's see. Here are (I hope) two lines
> of text ...
>
> % This line starts with a "%" sign, but the next doesn't.
> * This line doesn't start with a "%" sign, but the previous one does.
Well, from where I'm sitting that appeared to work. :-) Duane,
did it reach you?
--
Gareth McCaughan
.sig under construc
Gareth McCaughan <················@pobox.com> writes:
> I wrote:
>
> [Duane Rettig:]
> > > What this was all about, was that apparently nntp doesn't like to see
> > > a % as the first character on a line. I had tried to post directly
> > > (thinking perhaps that gnus was at fault when I had tried to send four
> > > or five times) but when it got caught soon after a line that started
> > > with %venv%, I realized that there was some kind of protocol intervening.
> > > I fixed the line, and what you saw apologizing for the resend was really
> > > the only message that got through. So here I am once more, apologizing
> > > in advance if any of the garbage messages get through later.
> > >
> > > Protocols... Sheesh.
> >
> > Hmm. That's very weird. Let's see. Here are (I hope) two lines
> > of text ...
> >
> > % This line starts with a "%" sign, but the next doesn't.
> > * This line doesn't start with a "%" sign, but the previous one does.
>
> Well, from where I'm sitting that appeared to work. :-) Duane,
> did it reach you?
Yes. I must have a buggy (or "featureful") nntp...
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Hi Duane,
Yes, it helped enormously. You took a great deal of time and trouble to
write all of that, thank you. It was clear, I was interested in the topic,
and I enjoyed reading about it. It is interesting that CLtL2 8.5
Environments pp.207-214 was deleted by ANSI, so what you're doing is kind of
"back to the future". 8.5 states that environments returned by
augment-environment are for syntactic analysis only (code walkers) and are
not to be passed to *evalhook*, which also was deleted by ANSI.* (As per
http://home.comcast.net/~bc19191/cltl2-ansi.htm).
Even though environments returned by augment-environment have restrictions,
it seems that (enclose lambda-expression &optional env) can be used to
produce a usable function. The restrictions on lambda-expression are
interesting. It can't reference variable or function bindings lexically
visible in env, go do 'go' or 'return-from' to lexically visible tags or
blocks. The go and return-from restrictions seem clear to me but I'm
confused by the variable and function-bindings restrictions. If I can't use
the environment for those bindings, why use the environment at all?
Does this mean that environments returned by augment-environment are really
useful only for syntactic analysis, or can they be used to implement
behavior similar to that in Erann's implementation of locales?
Thanks again for a great read.
Jeff
Jeff
* Lispworks for Windows 4.3.7 seems to have at least augment-environment,
meaning possibly the rest as well, and does have *evalhook* and evalhook.
"Duane Rettig" <·····@franz.com> wrote in message
··················@franz.com...
...
> A complete explanation of this whole thing would probably require
> me to write up a tutorial for the implementation of our
> environments-access module, which I probably will eventually do
> at some ILC conference, especially when we prepare to actually
> release the module into opensource. It is quite a clever system,
> invented by Steve Haflich,...
...
> > Thanks for your help,
>
> I hope it _did_ help :-)
>
> --
> Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
> 555 12th St., Suite 1450 http://www.555citycenter.com/
> Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
"Jeff Caldwell" <·········@yaSPhooAM.com> writes:
> Hi Duane,
>
> Yes, it helped enormously. You took a great deal of time and trouble to
> write all of that, thank you. It was clear, I was interested in the topic,
> and I enjoyed reading about it. It is interesting that CLtL2 8.5
> Environments pp.207-214 was deleted by ANSI, so what you're doing is kind of
> "back to the future". 8.5 states that environments returned by
> augment-environment are for syntactic analysis only (code walkers) and are
> not to be passed to *evalhook*, which also was deleted by ANSI.* (As per
> http://home.comcast.net/~bc19191/cltl2-ansi.htm).
I've tried to ignore restrictions made in CLtL2, since some of them, when taken
as a whole, might have been a large part of the reason why environments-access
was removed before the ANS was finalized. Besides, as you observe, some of
the restrictions are due to clashes with features that have been removed
anyway in and of themselves, and so the restrictions are moot.
> Even though environments returned by augment-environment have restrictions,
> it seems that (enclose lambda-expression &optional env) can be used to
> produce a usable function. The restrictions on lambda-expression are
> interesting. It can't reference variable or function bindings lexically
> visible in env, go do 'go' or 'return-from' to lexically visible tags or
> blocks. The go and return-from restrictions seem clear to me but I'm
> confused by the variable and function-bindings restrictions. If I can't use
> the environment for those bindings, why use the environment at all?
Well, yes. I haven't yet seen a use for enclose, and haven't implemented
it (I'm open to enlightenment from anyone who can demonstrate a good use).
> Does this mean that environments returned by augment-environment are really
> useful only for syntactic analysis, or can they be used to implement
> behavior similar to that in Erann's implementation of locales?
The latter. One of the most subtle (but perhaps one of the most important)
differences in our environments-access implementation is the presence of
"values". The CLtL2 environments have carefully defined the presence of
names in appropriate namespaces, and have also defined associations
between those names and declarations, most of which I was able to use
with little change. But the great missing feature was the ability to
attach a "value" to the name, where "value" could mean anything that
gave semantics to the name at the point in the environment it appears.
In an evaluation environment, this could include an actual value, when
taken in the variable namespace, or it could mean a definition in the
function namespace. In a compilation environment, a value could be
a semantic compiler node. Of course, one could argue that some sort
of declaration could be created for such "values", but one of the
requirements for these values is that they be mutable, and in general
environments must be thought of as immutable. So with few exceptions,
every name has a value locative (a cons cell which contains the value and
which _is_ the only thing mutable in the environment). This establishes
a name/value binding mechanism that can be used in any way desired,
under the umbrella of the environments package. Because it was such
a radical departure from CLtL2, I tested it first by reimplementing our
interpreter using environments. That proved that it could be done, _and_
that it could be fairly fast. And it opens up a lot of doors to uses
beyond mere syntax.
> Thanks again for a great read.
You're welcome.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
In article <·············@franz.com>, Duane Rettig <·····@franz.com>
wrote:
> Well, yes. I haven't yet seen a use for enclose, and haven't implemented
> it (I'm open to enlightenment from anyone who can demonstrate a good use).
One possibility is if one is writing some code in SICP-OO style, e.g.:
(let (...) (list (lambda ...) (lambda ...)))
or some such thing, then one could add "methods" incrementally, and
without having to reset the state variables. I'm not sure how "useful"
that is given that we have CLOS, but it's a possibility. I can see it
serving pedagogical purposes if nothing else.
E.
In article <····················@sea-read.news.verio.net>,
"Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
> Hi Duane,
>
> Yes, it helped enormously. You took a great deal of time and trouble to
> write all of that, thank you. It was clear, I was interested in the topic,
> and I enjoyed reading about it.
Hear, hear. One comment:
> CLtL2 8.5 Environments pp.207-214
My copy of CLTL2 is on-line and doesn't have page numbers, so having the
section number reference is more useful.
E.
On Fri, 09 Jul 2004 11:14:58 -0700, Erann Gat <·········@flownet.com> wrote:
> In article <····················@sea-read.news.verio.net>,
> "Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
>
>> CLtL2 8.5 Environments pp.207-214
>
> My copy of CLTL2 is on-line and doesn't have page numbers, so having
> the section number reference is more useful.
Hmm, I thought that "8.5" was pretty clear...
--
"Lisp doesn't look any deader than usual to me."
(David Thornley, reply to a question older than most languages)
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
In article <··············@bird.agharta.de>,
Edi Weitz <········@agharta.de> wrote:
> On Fri, 09 Jul 2004 11:14:58 -0700, Erann Gat <·········@flownet.com> wrote:
>
> > In article <····················@sea-read.news.verio.net>,
> > "Jeff Caldwell" <·········@yaSPhooAM.com> wrote:
> >
> >> CLtL2 8.5 Environments pp.207-214
> >
> > My copy of CLTL2 is on-line and doesn't have page numbers, so having
> > the section number reference is more useful.
>
> Hmm, I thought that "8.5" was pretty clear...
It was, but that was Jeff Caldwell's addition. Duane's original
citation had page numbers only.
E.
Erann Gat <·········@flownet.com> writes:
> In article <··············@javamonkey.com>,
> Peter Seibel <·····@javamonkey.com> wrote:
>
>> Erann Gat <·········@flownet.com> writes:
>>
>> > In article <··············@javamonkey.com>,
>> > Peter Seibel <·····@javamonkey.com> wrote:
>> >> Though I think I still might argue that there could be some use
>> >> for a version of MACROLET that passes E0 (in this example) as
>> >> the &environment rather than E2 and E3.
>> >
>> > You have that. E0 is what you get by default for all code in the
>> > body of FOO.
>>
>> Not quite because if we call MACROEXPAND-1 without passing an
>> environment object we get the global lexical environment.
>
> Yes quite. You forget that there is more than one way to macroexpand
> a macro. In fact, there are three ways, each of which gives you a
> different lexical environment:
>
> (macroexpand '(foo)) ; Uses the global lexical environment
> (macroexpand '(foo) env) ; Uses env
> (foo) ; Uses the lexical environment at the point where the call appears
Yes but none of those three is what I want. Consider:
;; E0: global lexical environment
(macrolet ((foo () ...))
;; E1: E0 + binding for FOO
(macrolet ((foo (&environment env) ...))
;; E2: E1 + new (shadowing) binding for FOO
(macrolet ((bar () ...))
;; E3: E2 + binding for BAR
(foo))))
In the body of the FOO macro defined in the second MACROLET we have,
as I understand it:
(macroexpand '(foo)) ; Uses E0
(macroexpand '(foo) env) ; Uses E3
(foo) ; Uses E2
But there's no way to expand '(foo) in E1 which is what I want.
On the other hand I may be confused about this because I thoroughly
expected this to signal an error because BAR is not bound in the
global lexical environment.
(macrolet ((foo () (macroexpand-1 '(bar))))
(macrolet ((bar () ''lexical-bar))
(foo)))
However in Allegro, SBCL, and CLISP it evaluates to LEXICAL-BAR. This
seems to me to contravene this passage from the MACROEXPAND-1
dictionary entry:
If only form is supplied as an argument, then the environment is
effectively null, and only global macro definitions as established
by defmacro are considered.
Maybe someone can explain to me why a macroexpansion that uses "only
global macro definitions established by defmacro" can see the local
definiton of BAR. At any rate, this signals an error as I'd expect:
(macrolet ((foo () (bar)))
(macrolet ((bar () ''lexical-bar))
(foo)))
And this expands to LEXICAL-BAR, this time as I'd expect:
(macrolet ((foo (&environment env) (macroexpand-1 '(bar) env)))
(macrolet ((bar () ''lexical-bar))
(foo)))
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
"Peter Seibel" <·····@javamonkey.com> wrote in message
···················@javamonkey.com...
...
> Yes but none of those three is what I want. Consider:
>
> ;; E0: global lexical environment
> (macrolet ((foo () ...))
> ;; E1: E0 + binding for FOO
> (macrolet ((foo (&environment env) ...))
> ;; E2: E1 + new (shadowing) binding for FOO
> (macrolet ((bar () ...))
> ;; E3: E2 + binding for BAR
> (foo))))
>
> In the body of the FOO macro defined in the second MACROLET we have,
> as I understand it:
>
> (macroexpand '(foo)) ; Uses E0
> (macroexpand '(foo) env) ; Uses E3
> (foo) ; Uses E2
>
> But there's no way to expand '(foo) in E1 which is what I want.
>
> On the other hand I may be confused about this because I thoroughly
> expected this to signal an error because BAR is not bound in the
> global lexical environment.
>
> (macrolet ((foo () (macroexpand-1 '(bar))))
> (macrolet ((bar () ''lexical-bar))
> (foo)))
...
In foo, (macroexpand-1 '(bar)) => (bar) because the environment is nil and
(bar) is not a macro, so:
(macrolet ((foo () (macroexpand-1 '(bar))))
(macrolet ((bar () ''lexical-bar))
(foo)))
=>
(macrolet ((foo () (macroexpand-1 '(bar))))
(macrolet ((bar () ''lexical-bar))
(bar)))
=>
(macrolet ((foo () (macroexpand-1 '(bar))))
(macrolet ((bar () ''lexical-bar))
'lexical-bar))
=>
LEXICAL-BAR
;;;;;;;;;;;;;;;;;;;;;;
> ;; E0: global lexical environment
> (macrolet ((foo () ...))
> ;; E1: E0 + binding for FOO
> (macrolet ((foo (&environment env) ...))
> ;; E2: E1 + new (shadowing) binding for FOO
> (macrolet ((bar () ...))
> ;; E3: E2 + binding for BAR
> (foo))))
>
> In the body of the FOO macro defined in the second MACROLET we have,
> as I understand it:
>
> (macroexpand '(foo)) ; Uses E0
> (macroexpand '(foo) env) ; Uses E3
> (foo) ; Uses E2
>
> But there's no way to expand '(foo) in E1 which is what I want.
If "to expand '(foo) in E1" is what you want, what is wrong with:
(macrolet ((foo () ...))
;; E1: E0 + binding for FOO
(foo) ;;
<---------------
(macrolet ((foo (&environment env) ...))
;; E2: E1 + new (shadowing) binding for FOO
(macrolet ((bar () ...))
;; E3: E2 + binding for BAR
(foo))))
Isn't it true that, excepting side effects, there cannot be a reason for
expanding E1's foo later than before the second macrolet because once E1's
foo is shadowed in E2, it is impossible for anything to alter E1 foo's
expansion?
Jeff
"Jeff Caldwell" <·········@yaSPhooAM.com> writes:
> "Peter Seibel" <·····@javamonkey.com> wrote in message
> ···················@javamonkey.com...
> ...
>> Yes but none of those three is what I want. Consider:
>>
>> ;; E0: global lexical environment
>> (macrolet ((foo () ...))
>> ;; E1: E0 + binding for FOO
>> (macrolet ((foo (&environment env) ...))
>> ;; E2: E1 + new (shadowing) binding for FOO
>> (macrolet ((bar () ...))
>> ;; E3: E2 + binding for BAR
>> (foo))))
>>
>> In the body of the FOO macro defined in the second MACROLET we have,
>> as I understand it:
>>
>> (macroexpand '(foo)) ; Uses E0
>> (macroexpand '(foo) env) ; Uses E3
>> (foo) ; Uses E2
>>
>> But there's no way to expand '(foo) in E1 which is what I want.
>>
>> On the other hand I may be confused about this because I thoroughly
>> expected this to signal an error because BAR is not bound in the
>> global lexical environment.
>>
>> (macrolet ((foo () (macroexpand-1 '(bar))))
>> (macrolet ((bar () ''lexical-bar))
>> (foo)))
> ...
>
> In foo, (macroexpand-1 '(bar)) => (bar) because the environment is nil and
> (bar) is not a macro, so:
Ah, that makes sense. That's sort of amusing.
> ;;;;;;;;;;;;;;;;;;;;;;
>
>> ;; E0: global lexical environment
>> (macrolet ((foo () ...))
>> ;; E1: E0 + binding for FOO
>> (macrolet ((foo (&environment env) ...))
>> ;; E2: E1 + new (shadowing) binding for FOO
>> (macrolet ((bar () ...))
>> ;; E3: E2 + binding for BAR
>> (foo))))
>>
>> In the body of the FOO macro defined in the second MACROLET we have,
>> as I understand it:
>>
>> (macroexpand '(foo)) ; Uses E0
>> (macroexpand '(foo) env) ; Uses E3
>> (foo) ; Uses E2
>>
>> But there's no way to expand '(foo) in E1 which is what I want.
>
> If "to expand '(foo) in E1" is what you want, what is wrong with:
>
> (macrolet ((foo () ...))
> ;; E1: E0 + binding for FOO
> (foo) ;;
> <---------------
> (macrolet ((foo (&environment env) ...))
> ;; E2: E1 + new (shadowing) binding for FOO
> (macrolet ((bar () ...))
> ;; E3: E2 + binding for BAR
> (foo))))
>
> Isn't it true that, excepting side effects, there cannot be a reason for
> expanding E1's foo later than before the second macrolet because once E1's
> foo is shadowed in E2, it is impossible for anything to alter E1 foo's
> expansion?
Well, I'm trying to draw a distinction between the body of the macros
defined by a given MACROLET and the body of the MACROLET itself. This
seems to me analogous--though Erann claims I'm wrong about that--to
the difference between FLET and LABELs. I.e. in FLET the definitions
created by the FLET are not in scope within the bodies of the
functions being defined while in LABELS they are.
My point is that within the definition of the second FOO I want to be
able to use the enclosing FOO macro. That's all. Though "want" may be
too strong a term. Really I'm just saying that it doesn't seem crazy
to have a MACROLET like construct that is more like FLET than LABELS.
Unless I'm missing something, at the time the inner MACROLET is
compiled the outer MACROLET's binding for FOO is around somewhere and
there's no *in principle* reason why I shouldn't be able to get at any
more than there's some deep principle why it's wrong that this:
(flet ((foo () 'outer))
(flet ((foo () (foo)))
(foo)))
returns OUTER rather than recursing forever. And, as I just tried to
demonstrate over in another thread, there's some benefits to having
FLET work the way it does. I'm arguing--albiet in a somewhat handwavy
way--that the same benefits might acruue to a FLET-like version of
MACROLET. I understand the reasons that MACROLET doesn't work that
way. I even agree that if you had to pick one semantics for MACROLET
the current one is probably more useful than the one I'm proposing.
I'm just saying that no one has been able to explain to me some deep
reason why the semantics I'm proposing couldn't be supported. Which is
not to say that there isn't such a reason or even that someone hasn't
explained it already and that I've just been too dense to understand
it when they did.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
"Peter Seibel" <·····@javamonkey.com> wrote in message
···················@javamonkey.com...
> "Jeff Caldwell" <·········@yaSPhooAM.com> writes:
>
> > "Peter Seibel" <·····@javamonkey.com> wrote in message
...
> >
> >> ;; E0: global lexical environment
> >> (macrolet ((foo () ...))
> >> ;; E1: E0 + binding for FOO
> >> (macrolet ((foo (&environment env) ...))
> >> ;; E2: E1 + new (shadowing) binding for FOO
> >> (macrolet ((bar () ...))
> >> ;; E3: E2 + binding for BAR
> >> (foo))))
> >>
> >> In the body of the FOO macro defined in the second MACROLET we have,
> >> as I understand it:
> >>
> >> (macroexpand '(foo)) ; Uses E0
> >> (macroexpand '(foo) env) ; Uses E3
> >> (foo) ; Uses E2
> >>
> >> But there's no way to expand '(foo) in E1 which is what I want.
> >
> > If "to expand '(foo) in E1" is what you want, what is wrong with:
> >
> > (macrolet ((foo () ...))
> > ;; E1: E0 + binding for FOO
> > (foo) ;;
> > <---------------
> > (macrolet ((foo (&environment env) ...))
> > ;; E2: E1 + new (shadowing) binding for FOO
> > (macrolet ((bar () ...))
> > ;; E3: E2 + binding for BAR
> > (foo))))
> >
> > Isn't it true that, excepting side effects, there cannot be a reason for
> > expanding E1's foo later than before the second macrolet because once
E1's
> > foo is shadowed in E2, it is impossible for anything to alter E1 foo's
> > expansion?
>
> Well, I'm trying to draw a distinction between the body of the macros
> defined by a given MACROLET and the body of the MACROLET itself.
I'm going to be picky about terminology.
CL-USER 3 > (macrolet ((f1 () ''f1)
(f2 () ''f2))
(list (f1) (f2)))
(F1 F2)
f1 has a body and f2 has a body, so it's "the bodies of the macros"
instead of "the body of the macros". The syntax for MACROLET is:
macrolet ((name lambda-list [[local-declaration* | local-documentation]]
local-form*)*) declaration* form*
Strictly speaking, the macrolet itself does not have a "body" in the sense
of
&body. The CLHS goes on to say:
"The body forms (but not the lambda list) of each ... macro created by
macrolet
are enclosed in an implicit block..."
Each macro within the macrolet has a (set of) body forms.
> This
> seems to me analogous--though Erann claims I'm wrong about that--to
> the difference between FLET and LABELs. I.e. in FLET the definitions
> created by the FLET
These definitions are functions in the sense of a lambda form...
> are not in scope within the bodies of the
> functions being defined while in LABELS they are.
>
...
> My point is that within the definition of the second FOO I want to be
> able to use the enclosing FOO macro.
The definition of the second FOO is: (foo (&environment env) ...). You
said "definition" but I think below you are using "definition" to mean
"scope".
To use the first FOO within the definition of the second foo, I simply refer
to your example below (reproduced here, with comments, for clarity):
> (flet ((foo () 'outer)) ;; definition of first foo is (foo ()
'outer)
> (flet ((foo () (foo))) ;; definition of second foo is (foo ()
(foo))
> (foo))) ;; expands second foo
That does indeed return OUTER, so in fact you already _are_ doing
what you said you want to do (repeating your earlier statement):
> My point is that within the definition of the second FOO I want to be
> able to use the enclosing FOO macro.
Within the definition of the second FOO you are, in fact, using the
enclosing FOO macro. That's why I say you were using "definition"
as if you meant "scope".
> That's all. Though "want" may be
> too strong a term. Really I'm just saying that it doesn't seem crazy
> to have a MACROLET like construct that is more like FLET than LABELS.
The above shows that MACROLET already is more like FLET than LABELS.
In your example, the second FOO, (foo () (foo)), is like FLET because
it refers to the outer FOO. If the MACROLET for the second FOO were
like LABELS instead of FLET, then (foo () (foo)) would be recursive. It
isn't, so (foo () (foo)) is like FLET, not LABELS.
> Unless I'm missing something, at the time the inner MACROLET is
> compiled the outer MACROLET's binding for FOO is around somewhere and
> there's no *in principle* reason why I shouldn't be able to get at any
> more than there's some deep principle why it's wrong that this:
>
> (flet ((foo () 'outer))
> (flet ((foo () (foo)))
> (foo)))
>
> returns OUTER rather than recursing forever. And, as I just tried to
> demonstrate over in another thread, there's some benefits to having
> FLET work the way it does.
And MACROLET works like FLET...
> I'm arguing--albiet in a somewhat handwavy
> way--that the same benefits might acruue to a FLET-like version of
> MACROLET.
MACROLET already is like FLET. Maybe a MACROLET
like LABELS would be useful?
> I understand the reasons that MACROLET doesn't work that
> way.
I'm becoming more convinced that you're really saying something
about the _scope_ of a MACROLET, instead of "within the definition"
of a MACROLET, although I haven't homed in yet on exactly what
you're saying about that scope.
Jeff
> I even agree that if you had to pick one semantics for MACROLET
> the current one is probably more useful than the one I'm proposing.
> I'm just saying that no one has been able to explain to me some deep
> reason why the semantics I'm proposing couldn't be supported. Which is
> not to say that there isn't such a reason or even that someone hasn't
> explained it already and that I've just been too dense to understand
> it when they did.
>
> -Peter
>
> --
> Peter Seibel ·····@javamonkey.com
>
> Lisp is the red pill. -- John Fraser, comp.lang.lisp
"Jeff Caldwell" <·········@yaSPhooAM.com> wrote in message
·························@sea-read.news.verio.net...
....
Sigh -- I apologize for, in my previous post, not replacing ...
> To use the first FOO within the definition of the second foo, I simply
refer
> to your example below (reproduced here, with comments, for clarity):
>
> > (flet ((foo () 'outer)) ;; definition of first foo is (foo ()
> 'outer)
> > (flet ((foo () (foo))) ;; definition of second foo is (foo ()
> (foo))
> > (foo))) ;; expands second foo
>
> That does indeed return OUTER, so in fact you already _are_ doing
> what you said you want to do (repeating your earlier statement):
...
with
CL-USER 7 > (macrolet ((f1 () '''outer))
(macrolet ((f1 () (f1)))
(f1)))
OUTER
You were showing FLET, and I should have converted your example
to MACROLET, but did not. I'm hoping that the actual MACROLET
example above doesn't change the arguments I made in the prior post.
The MACROLET example above works like your FLET example.
As I reviewed that post, I also thought I didn't put in enough, or any,
"It seems to me that", or "Correct me if I'm wrong, but ...". I'm just
trying to understand it, too, so please point out what I'm missing.
Jeff
"Jeff Caldwell" <·········@yaSPhooAM.com> writes:
> "Jeff Caldwell" <·········@yaSPhooAM.com> wrote in message
> ·························@sea-read.news.verio.net...
> ....
>
> Sigh -- I apologize for, in my previous post, not replacing ...
>
>> To use the first FOO within the definition of the second foo, I
>> simply refer to your example below (reproduced here, with comments,
>> for clarity):
>>
>> > (flet ((foo () 'outer)) ;; definition of first foo is (foo ()
>> 'outer)
>> > (flet ((foo () (foo))) ;; definition of second foo is (foo ()
>> (foo))
>> > (foo))) ;; expands second foo
>>
>> That does indeed return OUTER, so in fact you already _are_ doing
>> what you said you want to do (repeating your earlier statement):
> ...
>
> with
>
> CL-USER 7 > (macrolet ((f1 () '''outer))
> (macrolet ((f1 () (f1)))
> (f1)))
> OUTER
>
> You were showing FLET, and I should have converted your example
> to MACROLET, but did not. I'm hoping that the actual MACROLET
> example above doesn't change the arguments I made in the prior post.
> The MACROLET example above works like your FLET example.
Well, sort of. What's happening here is that the definition of the
inner F1 can indeed "call" the outer F1. The problem with that is what
that's really doing is using the code that the outer F1 expands into
to in turn compute the expansion of the inner F1. But what I want
is--in certain situations--to use the outer F1 to compute the
expansion to be produced by the inner F1. You can see the problem by
comparing these two expresions:
(macrolet ((f1 (x) `(list ,x ,x)))
(f1 10)) ==> (10 10)
vs.
(macrolet ((f1 (x) `(list ,x ,x)))
(macrolet ((f1 (x) (f1 x)))
(f1 10))) ==> ERROR, "Funcall of 10 which is a non-function."
This is, I imagine, the same reason you had to add an extra quote
before the symbol returned by the outer F1 in your example.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
"Peter Seibel" <·····@javamonkey.com> wrote in message
···················@javamonkey.com...
> "Jeff Caldwell" <·········@yaSPhooAM.com> writes:
>
...
> > CL-USER 7 > (macrolet ((f1 () '''outer))
> > (macrolet ((f1 () (f1)))
> > (f1)))
> > OUTER
> >
...
> > The MACROLET example above works like your FLET example.
>
> Well, sort of. What's happening here is that the definition of the
> inner F1 can indeed "call" the outer F1. The problem with that is what
> that's really doing is using the code that the outer F1 expands into
> to in turn compute the expansion of the inner F1. But what I want
> is--in certain situations--to use the outer F1 to compute the
> expansion to be produced by the inner F1. You can see the problem by
> comparing these two expresions:
>
> (macrolet ((f1 (x) `(list ,x ,x)))
> (f1 10)) ==> (10 10)
>
> vs.
>
> (macrolet ((f1 (x) `(list ,x ,x)))
> (macrolet ((f1 (x) (f1 x)))
> (f1 10))) ==> ERROR, "Funcall of 10 which is a non-function."
...
CL-USER 560 (macrolet ((f1 (x) `(list ,x ,x)))
(macrolet ((f1 (x) `',(f1 x)))
(f1 10)))
=> (10 10)
It is impossible to have the 10 passed to the outer F1. It
is not an "environment" issue, it's just the difference
between functions and macros.
I used Steele's backquote program from CLtL2
Appendix C, with debugging output added. Steele's
program uses $ for backquote and % for comma.
<backquote$start>
<comma%start>X<comma%end><comma%start>X<comma%end>
(LIST (COMMA X) (COMMA X))
<backquote$end>
<backquote$start>
<comma%start>(F1 X)<comma%end>
(QUOTE (COMMA (F1 X)))
<backquote$end>
CL-USER 577 > (macrolet ((f1 (x) (print 'outer) $(list %x %x)))
(macrolet ((f1 (x) (print 'inner) $'%(f1 x)))
(print 'inside-inner)
(f1 10)))
(APPEND (LIST (QUOTE LIST)) (LIST X) (LIST X)
(QUOTE NIL)) ;; expansion of outer F1
(APPEND (LIST (QUOTE QUOTE)) (LIST (F1 X))
(QUOTE NIL)) ;; expansion of inner F1
OUTER ;; expanded code for outer F1 executes
INSIDE-INNER ;; expanded code for inner F1 executes
INNER ;; body of inner F1 executes
(10 10)
The expanded code from the outer F1 has computed a result
and never will be called again before any code in the scope
of the inner F1 begins to execute, including any
expanded code from the inner F1 itself. It is impossible to pass
the 10 to the outer F1 because the environment containing
the binding of X to10 is not constructed until after the expanded
outer F1 code has executed and completed.
Surely more than having access to the environments hidden
by shadowing, which you mentioned in a previous post,
would have to change in order for the 10 to be passed
to F1. The entire order of evaluation would have to change.
I guess I don't know why you want MACROLET to
work like FLET when FLET already works that way :)
Jeff
"Jeff Caldwell" <·········@yaSPhooAM.com> writes:
> "Peter Seibel" <·····@javamonkey.com> wrote in message
> ···················@javamonkey.com...
>> "Jeff Caldwell" <·········@yaSPhooAM.com> writes:
>>
> ...
>> > CL-USER 7 > (macrolet ((f1 () '''outer))
>> > (macrolet ((f1 () (f1)))
>> > (f1)))
>> > OUTER
>> >
> ...
>> > The MACROLET example above works like your FLET example.
>>
>> Well, sort of. What's happening here is that the definition of the
>> inner F1 can indeed "call" the outer F1. The problem with that is what
>> that's really doing is using the code that the outer F1 expands into
>> to in turn compute the expansion of the inner F1. But what I want
>> is--in certain situations--to use the outer F1 to compute the
>> expansion to be produced by the inner F1. You can see the problem by
>> comparing these two expresions:
>>
>> (macrolet ((f1 (x) `(list ,x ,x)))
>> (f1 10)) ==> (10 10)
>>
>> vs.
>>
>> (macrolet ((f1 (x) `(list ,x ,x)))
>> (macrolet ((f1 (x) (f1 x)))
>> (f1 10))) ==> ERROR, "Funcall of 10 which is a non-function."
> ...
>
> CL-USER 560 (macrolet ((f1 (x) `(list ,x ,x)))
> (macrolet ((f1 (x) `',(f1 x)))
> (f1 10)))
> => (10 10)
>
> It is impossible to have the 10 passed to the outer F1. It is not an
> "environment" issue, it's just the difference between functions and
> macros.
Okay, you'll perhaps be happy to know that I am now fully convinced
that my analogy with FLET *is* flawed--certainly it doesn't seem to be
helping anyone (including me) understand what I'm trying to say. Let
my try another approach:
Suppose I wrote this:
(macrolet ((foo (x) `(list :something-else ',x)))
(macrolet ((bar (&whole whole x &environment env)
(if (numberp x)
`(list :number ,x)
(macroexpand-1 (cons 'foo (cdr whole)) env))))
(bar "string")))
The BAR macro will either compute its expansion directly or use the
FOO macro to compute it via a call to MACROEXPAND-1. (Obviously it
would be silly to write code like this by hand--the assumption is that
these nested MACROLETs are themselves generated by some other macro.)
In this case everything works fine because FOO and BAR have different
names. But this almost identical expression will recurse forever:
(macrolet ((foo (x) `(list :something-else ',x)))
(macrolet ((foo (&whole whole x &environment env)
(if (numberp x)
`(list :number ,x)
(macroexpand-1 (cons 'foo (cdr whole)) env))))
(foo "string")))
because the &environment parameter is bound to an environment in which
FOO is bound to the inner MACROLET's macro, shadowing the outer
definition. But at compile time there *is* another environment around
somewhere in which FOO is bound to the outer MACROLET's macro. If
there was a way to get a hold of that environment or to get at the
shadowed mappings in the environment some variant of this latter
expression could be made to work just the way the former expression
did.
> Surely more than having access to the environments hidden by
> shadowing, which you mentioned in a previous post, would have to
> change in order for the 10 to be passed to F1. The entire order of
> evaluation would have to change.
Well, calling MACROEXPAND does affect the order somewhat which I'm
fine with. I just can't get at the shadowed definition which does seem
to me to be a question of not being able to get at the right parts of
the compile-time environment.
> I guess I don't know why you want MACROLET to work like FLET when
> FLET already works that way :)
Same reason I'm glad DEFMACRO exists even though we already have
DEFUN; because I want to do stuff at compile-time, not runtime.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
"Peter Seibel" <·····@javamonkey.com> wrote in message
···················@javamonkey.com...
...
>
> Okay, you'll perhaps be happy to know that I am now fully convinced
> that my analogy with FLET *is* flawed--certainly it doesn't seem to be
> helping anyone (including me) understand what I'm trying to say. Let
> my try another approach:
>
> Suppose I wrote this:
>
> (macrolet ((foo (x) `(list :something-else ',x)))
> (macrolet ((bar (&whole whole x &environment env)
> (if (numberp x)
> `(list :number ,x)
> (macroexpand-1 (cons 'foo (cdr whole)) env))))
> (bar "string")))
In that code, FOO executes in BAR's environment. Any
environment present at the point of FOO's definition plays
no role in FOO's expansion. Whatever that environment
might be, it is completely replaced by BAR's environment
in the MACROEXPAND-1 call.
>
> The BAR macro will either compute its expansion directly or use the
> FOO macro to compute it via a call to MACROEXPAND-1. (Obviously it
> would be silly to write code like this by hand--the assumption is that
> these nested MACROLETs are themselves generated by some other macro.)
> In this case everything works fine because FOO and BAR have different
> names. But this almost identical expression will recurse forever:
>
> (macrolet ((foo (x) `(list :something-else ',x)))
> (macrolet ((foo (&whole whole x &environment env)
> (if (numberp x)
> `(list :number ,x)
> (macroexpand-1 (cons 'foo (cdr whole)) env))))
> (foo "string")))
>
> because the &environment parameter is bound to an environment in which
> FOO is bound to the inner MACROLET's macro, shadowing the outer
> definition. But at compile time there *is* another environment around
> somewhere in which FOO is bound to the outer MACROLET's macro.
There may not be an environment for the outer FOO at all. A smart compiler
could see that FOO is never called with a default environment and so not
bother to construct one. Also, the FLET, LABELS, and MACROLET
page in the CLHS specifies that "the consequences are undefined if the
local macro definitions reference any local variable or function bindings
that are visible in (the) lexical environment." I interpret the paragraph
in which that text appears to mean that the macro can expand to code that
references local variable and/or function bindings but those references
cannot be made until after the expansion is complete. Is that a correct
interpretation?
If so, does that leave room for anything at all in the environment other
than other MACROLET definitions, parameters to the macro, and
unshadowed top-level bindings? CLISP and Lispworks have different
behavior which, as I read the CLHS, seems conforming in both instances,
(undefined monkeys flying through noses). I don't know if the situation
is ugly and will be corrected by, for example, Duane's new environments,
or whether the situation is beautiful and I don't see the whole picture yet.
Lispworks (LWW 4.3.7)
CL-USER 734 > (defun foo (x) ''top-level-foo)
FOO
CL-USER 735 > (flet ((foo (x) ''flet-foo)))
(macrolet ((foo (x) (foo x)))
(foo 10)))
Error: Argument to apply/funcall is not a function: #<unbound>.
A compile-region on line 735 gives:
Compilation aborted due to error in (TOP-LEVEL-FORM 1):
Undefined function #:|internal-flet-name-for-FOO4198| called
with arguments (10).
That's consistent with the CLHS as I read it. The MACROLET referenced
a function binding in the lexical environment. Even though a top-level
foo existed, the lexical foo shadowed it so far as the MACROLET
was concerned, and that's conforming.
CLISP
[54]> (defun foo (x) ''top-level-foo)
FOO
[55]> (flet ((foo (x) ''flet-foo))
(macrolet ((foo (x) (foo x)))
(foo 10)))
TOP-LEVEL-FOO
CLISP did not shadow the top-level foo even though a different foo
existed in the lexical environment. Again, as I read it, this is conforming
because the result of a reference by a MACROLET to a variable
or function binding in the lexical environment is undefined.
You talk about an environment that exists for the outer MACROLET.
What could possibly be in that environment except for other
MACROLET definitions and "unshadowed" top-level bindings?
I don't know if a) I understand the CLHS and the behavior of the two
Lisps, one of the Lisps is exhibiting non-conforming behavior, or
something else is going on.
I think if a macro's expansion ensures that all of its new lexicals are
gensymed, then the situation above will never occur, and so this
isn't a problem. Presumably, the programmer has control of the
lexical environment and knows to avoid the undefined behavior.
The key is that one must pass as parameters everything the macro
is supposed to use, other than other MACROLETs and top-level
forms. (Perhaps something else is OK too? Just not lexical variable
or function bindings.) The environment in effect in the body of the
definition of the macro has the restricted environment. Once the
macro has been expanded, the expansion is then free to make full
use of the lexical environment.
I keep waiting for the "click" to happen in my head so that I understand
exactly why MACROLETs work this way. I have a better grasp
than I did of how they behave but not why. Perhaps that's what you're
saying, too, that they could have different useful behavior, and perhaps
Duane's project will enable that different behavior.
> If
> there was a way to get a hold of that environment or to get at the
> shadowed mappings in the environment some variant of this latter
> expression could be made to work just the way the former expression
> did.
>
...
> Well, calling MACROEXPAND does affect the order somewhat which I'm
> fine with. I just can't get at the shadowed definition which does seem
> to me to be a question of not being able to get at the right parts of
> the compile-time environment.
As I said, I don't think there is a shadowed environment that is useful to
you.
(Could be, in a different Lisp, but is not, in CL).
>
> > I guess I don't know why you want MACROLET to work like FLET when
> > FLET already works that way :)
>
> Same reason I'm glad DEFMACRO exists even though we already have
> DEFUN; because I want to do stuff at compile-time, not runtime.
FLET can be used during compile-time:
CL-USER 759 > (defun foo (x)
(flet ((foo (x) (list 'foo-1 x)))
(flet ((foo (x) (list 'foo-2 (foo x))))
(flet ((foo (x) (list 'foo-3 (foo x))))
(foo x)))))
FOO
CL-USER 760 > (macrolet ((bar (x) `',(foo x)))
(macrolet ((bar (x) `',(bar x)))
(bar x)))
(FOO-3 (FOO-2 (FOO-1 X)))
Jeff
Of course, my example below would have been more interesting as:
CL-USER 780 > (macrolet ((bar (x) `(foo ,x)))
(macrolet ((bar (x) (list 'quote (bar x))))
(bar 10)))
(FOO-3 (FOO-2 (FOO-1 10)))
or more simply using FLET's nesting to avoid the nested MACROLETs
completely:
CL-USER 781 > (macrolet ((bar (x) `(foo ,x)))
(bar 10))
(FOO-3 (FOO-2 (FOO-1 10)))
"Jeff Caldwell" <·········@yaSPhooAM.com> wrote in message
·························@sea-read.news.verio.net...
>
> "Peter Seibel" <·····@javamonkey.com> wrote in message
> ···················@javamonkey.com...
...
> > > I guess I don't know why you want MACROLET to work like FLET when
> > > FLET already works that way :)
> >
> > Same reason I'm glad DEFMACRO exists even though we already have
> > DEFUN; because I want to do stuff at compile-time, not runtime.
>
> FLET can be used during compile-time:
>
> CL-USER 759 > (defun foo (x)
> (flet ((foo (x) (list 'foo-1 x)))
> (flet ((foo (x) (list 'foo-2 (foo x))))
> (flet ((foo (x) (list 'foo-3 (foo
x))))
> (foo x)))))
> FOO
>
> CL-USER 760 > (macrolet ((bar (x) `',(foo x)))
> (macrolet ((bar (x) `',(bar x)))
> (bar x)))
> (FOO-3 (FOO-2 (FOO-1 X)))
>
> Jeff
"Jeff Caldwell" <·········@yaSPhooAM.com> writes:
> "Peter Seibel" <·····@javamonkey.com> wrote in message
> ···················@javamonkey.com...
> ...
>>
>> Okay, you'll perhaps be happy to know that I am now fully convinced
>> that my analogy with FLET *is* flawed--certainly it doesn't seem to be
>> helping anyone (including me) understand what I'm trying to say. Let
>> my try another approach:
>>
>> Suppose I wrote this:
>>
>> (macrolet ((foo (x) `(list :something-else ',x)))
>> (macrolet ((bar (&whole whole x &environment env)
>> (if (numberp x)
>> `(list :number ,x)
>> (macroexpand-1 (cons 'foo (cdr whole)) env))))
>> (bar "string")))
>
> In that code, FOO executes in BAR's environment. Any environment
> present at the point of FOO's definition plays no role in FOO's
> expansion. Whatever that environment might be, it is completely
> replaced by BAR's environment in the MACROEXPAND-1 call.
Well, more precisely, FOO executes in the lexical environment of the
call to BAR that occurs within the inner MACROLET. That environment
happens to contain two macro bindings, one for FOO and one for BAR.
>> The BAR macro will either compute its expansion directly or use the
>> FOO macro to compute it via a call to MACROEXPAND-1. (Obviously it
>> would be silly to write code like this by hand--the assumption is that
>> these nested MACROLETs are themselves generated by some other macro.)
>> In this case everything works fine because FOO and BAR have different
>> names. But this almost identical expression will recurse forever:
>>
>> (macrolet ((foo (x) `(list :something-else ',x)))
>> (macrolet ((foo (&whole whole x &environment env)
>> (if (numberp x)
>> `(list :number ,x)
>> (macroexpand-1 (cons 'foo (cdr whole)) env))))
>> (foo "string")))
>>
>> because the &environment parameter is bound to an environment in which
>> FOO is bound to the inner MACROLET's macro, shadowing the outer
>> definition. But at compile time there *is* another environment around
>> somewhere in which FOO is bound to the outer MACROLET's macro.
>
> There may not be an environment for the outer FOO at all. A smart compiler
> could see that FOO is never called with a default environment and so not
> bother to construct one.
Well, obviously in a Lisp where there was a way to get at the outer
FOO binding, a compiler couldn't make that assumption.
> Also, the FLET, LABELS, and MACROLET page in the CLHS specifies that
> "the consequences are undefined if the local macro definitions
> reference any local variable or function bindings that are visible
> in (the) lexical environment."
Right. But I'm not talking about acessing local *variable* or
*function* bindings--I'm talking about other macro bindings. I assume
the reason accessing variable and function bindings is undefined
behavior is because it doesn't make any sense--they can't be used
until runtime while macros operate at compile time.
> I interpret the paragraph in which that text appears to mean that
> the macro can expand to code that references local variable and/or
> function bindings but those references cannot be made until after
> the expansion is complete. Is that a correct interpretation?
I believe so.
[snip]
> You talk about an environment that exists for the outer MACROLET.
> What could possibly be in that environment except for other MACROLET
> definitions and "unshadowed" top-level bindings?
Nothing. I'm only interested in bindings created by other MACROLETs
and global macros.
> Perhaps that's what you're saying, too, that they could have
> different useful behavior, and perhaps Duane's project will enable
> that different behavior.
Yes. That is what I'm saying.
>> If there was a way to get a hold of that environment or to get at
>> the shadowed mappings in the environment some variant of this
>> latter expression could be made to work just the way the former
>> expression did.
>>
>
> ...
>
>> Well, calling MACROEXPAND does affect the order somewhat which I'm
>> fine with. I just can't get at the shadowed definition which does
>> seem to me to be a question of not being able to get at the right
>> parts of the compile-time environment.
>
> As I said, I don't think there is a shadowed environment that is
> useful to you. (Could be, in a different Lisp, but is not, in CL).
Well, conceptually there *is* a shadowed environment. That the
compiler may be able to avoid creating a data structure because it can
prove that it can't affect the results of the compilation is a simple
matter of optimization. Though it seems to me a rather strange
optimization to make--since it's purely a compiletime data structure
anyway, there's not a lot to be gained by doing the work to figure out
that certain bindings are never used just to avoid some compile-time
memory allocation or whatever you'd be saving. Unused FLET bindings,
on the other hand, might reasonably be removed via dead-code pruning
to remove the size of the compiled code.
>> > I guess I don't know why you want MACROLET to work like FLET when
>> > FLET already works that way :)
>>
>> Same reason I'm glad DEFMACRO exists even though we already have
>> DEFUN; because I want to do stuff at compile-time, not runtime.
>
> FLET can be used during compile-time:
Only in the sense that *all* of Common Lisp can be used at compile
time--that's the whole point of macros!
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
"Jeff Caldwell" <·········@yaSPhooAM.com> writes:
> If so, does that leave room for anything at all in the environment other
> than other MACROLET definitions, parameters to the macro, and
> unshadowed top-level bindings?
The spec suggests declarations and SYMBOL-MACROLET definitions
are also part of the environment and may be used.
Is the following code valid?
;; Imagine an EVAL-WHEN if necessary.
(setf (symbol-value 'foo) 42)
(locally
(declare (special foo))
(macrolet ((bar ()
;; (declare (special foo))
foo))
(bar)))
It seems to me that the SPECIAL declaration should affect the
local macro BAR, which should then expand to 42. However, SBCL
0.8.11 warns that the variable FOO is undefined. Uncommenting
the second declaration avoids the warning.
Does this fall in "the consequences are undefined" territory?
The macro references a binding that is visible in the lexical
environment, due to the declaration, but the binding itself is
not in the lexical environment.
Peter Seibel <·····@javamonkey.com> writes:
> (defmacro foo () ''global-foo)
>
> (macrolet ((foo () ''macrolet-foo))
> (macrolet ((bar () (macroexpand '(foo))))
> (bar))) ==> GLOBAL-FOO
I don't think that code will necessarily work when a file is
being compiled. DEFMACRO saves the definition in the compilation
environment, which may be distinct from the evaluation
environment that MACROEXPAND uses by default.