Ok, I rewrote my code to create code, or execute it, depending on the
switch *execute-mode*. For that I changed one function to a macro, but
with a problem:
(defmacro emit-action (exp)
`(if *execute-mode*
,exp
(progn (collect-emit-buf) (push ',exp *actions*))))
; this expands to code that either executes exp or pushes its quotation
to an action list.
; observe especially the ' quote, which causes whatever exp is to be
inserted verbatim
(defun emit-stuff (val)
(emit-action (do-stuff-with val)))
Now the problem is that I don't want VAL to be pushed, i.e. I want
something more like
(emit-action `(do-stuff-with ,val)) but then the emit-action macro
won't execute that code, but expand to (if *bla* quoted-do-stuff-exp
...), so it will only *return* that expression, instead of executing it.
Again I'm in a situation where I need to eval stuff :(
I'd like to be able to write the macro as:
`...(push `,exp *actions*)...
with a second quasiquote, and the call like (emit-action (do-stuff-with
,val)), but this doesn't work because the ` and , seem to be only read
statically, i.e. the macro's second ` can't cause the , to unquote its
argument.
Any ideas?
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann <···········@web.de> writes:
> Ok, I rewrote my code to create code, or execute it, depending on the
> switch *execute-mode*. For that I changed one function to a macro,
> but with a problem:
> (defmacro emit-action (exp)
> `(if *execute-mode*
> ,exp
> (progn (collect-emit-buf) (push ',exp *actions*))))
> ; this expands to code that either executes exp or pushes its
> quotation to an action list.
> ; observe especially the ' quote, which causes whatever exp is to be
> inserted verbatim
>
> (defun emit-stuff (val)
> (emit-action (do-stuff-with val)))
Don't use quote and eval, use lambda and funcall.
--
/|_ .-----------------------.
,' .\ / | Free Mumia Abu-Jamal! |
,--' _,' | Abolish the racist |
/ / | death penalty! |
( -. | `-----------------------'
| ) |
(`-. '--.)
`. )----'
Ulrich Hobelmann wrote:
> Now the problem is that I don't want VAL to be pushed, i.e. I want
> something more like
> (emit-action `(do-stuff-with ,val)) but then the emit-action macro
> won't execute that code, but expand to (if *bla* quoted-do-stuff-exp
> ...), so it will only *return* that expression, instead of executing it.
> Again I'm in a situation where I need to eval stuff :(
What EMIT-ACTION will do is push the expanded backquote object onto
*ACTIONS*. And so when later you want to evaluate that action, that
action will want to evaluate the VAL variable, in the wrong context.
I.e. what you have here is exactly the same as:
(emit-action (list 'do-stuff-with val))
but the LIST is not evaluated, since this is a macro. So what ends up
happening is that this expands to the code:
(if *execute-mode*
(list 'do-stuff-with val)
(progn (collect-emit-buf) (push '(list 'do-stuff-with val)
*actions*))
In excute mode, the (LIST 'DO-STUFF-WITH VAL) doesn't do the right
thing now. Although it can correctly resolve the value of VAL from the
surrounding lexical environment, all it does is compute the expression
and throw it away.
In delayed mode, the expression (LIST 'DO-STUFF-WITH VAL) is evaled,
but it can't resolve the correct value of VAL.
> Any ideas?
This isn't a macro problem, but a problem of wanting to evaluate the
same expression in two different contexts, yet allow it to resolve
variables in the same way. That is done using closures, rather rhan
backquote substitution tricks.
Yes, the two branches of the IF do need a different solution, but that
solution can be based on exactly the same piece of source code that
comes in as a macro argument.
You want this code to come out of the macro:
(if *execute-mode*
(do-stuff-with val)
(progn
(collect-emit-buf)
(push (lambda () (do-stuff-with val)) *actions*)))
>From this pattern, it should be obvious how to write the macro which
build this out of (EMIT-ACTION (DO-STUFF-WITH-VAL)).
So *ACTIONS* isn't a list containing fragments of source code. It's a
list of first-class funtions (which are compiled if the places that
generate them are compiled!)
You call these first class functions using FUNCALL, rather than EVAL.
Kaz Kylheku wrote:
> but the LIST is not evaluated, since this is a macro. So what ends up
> happening is that this expands to the code:
>
> (if *execute-mode*
> (list 'do-stuff-with val)
> (progn (collect-emit-buf) (push '(list 'do-stuff-with val)
> *actions*))
>
> In excute mode, the (LIST 'DO-STUFF-WITH VAL) doesn't do the right
> thing now. Although it can correctly resolve the value of VAL from the
> surrounding lexical environment, all it does is compute the expression
> and throw it away.
In execute mode it creates a list, instead of evaluating the expression,
yes.
> In delayed mode, the expression (LIST 'DO-STUFF-WITH VAL) is evaled,
> but it can't resolve the correct value of VAL.
No, in delayed mode it pushes a list of the symbol 'do-stuff-with, along
with the *value* of val, at least until I modified the code :)
Later, the macro simply fetches all those Lisp expressions and lets the
top-level-macro expand to all that code. Then the code can be compiled
into a function.
> You want this code to come out of the macro:
>
> (if *execute-mode*
> (do-stuff-with val)
> (progn
> (collect-emit-buf)
> (push (lambda () (do-stuff-with val)) *actions*)))
>
>>From this pattern, it should be obvious how to write the macro which
> build this out of (EMIT-ACTION (DO-STUFF-WITH-VAL)).
>
> So *ACTIONS* isn't a list containing fragments of source code. It's a
> list of first-class funtions (which are compiled if the places that
> generate them are compiled!)
But then what would the macro expand to? Right now it outputs a list of
all the collected statements to compile, and execute at runtime. But a
macro can't return *functions*, can it?
> You call these first class functions using FUNCALL, rather than EVAL.
Yes... still thinking...
The problem is to reconcile the old macro code with the new mode of
simply running the code.
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann wrote:
> Kaz Kylheku wrote:
> > In delayed mode, the expression (LIST 'DO-STUFF-WITH VAL) is evaled,
> > but it can't resolve the correct value of VAL.
>
> No, in delayed mode it pushes a list of the symbol 'do-stuff-with, along
> with the *value* of val, at least until I modified the code :)
>
> Later, the macro simply fetches all those Lisp expressions and lets the
^^^^^
> top-level-macro expand to all that code. Then the code can be compiled
> into a function.
Ah, you are trying to do some macro-time code motion. He he, well that
is possible, but the problem is that if you move code from one
environment to the other, if that code contains environmental
references, it won't work properly at the site that it is moved to.
You can't fix the problem by using backquote to interpolate those
values, because you are working in macro time; run time isn't happening
yet.
That is to say, you want something like
(defun foo (var)
(delayed-execute (+ var var))
(defun do-them ()
(expand-delayed-executions))
I mean, that doesn't even make sense. You have to pin down the exact
semantics of what this does and what is allowed.
What if FOO is never called, but DO-THEM is called? What happens with
(+ var var), given that since FOO was never called, there never existed
a binding for VAR?
What if FOO is called three times, and then DO-THEM is called? Three
bindings for VAR will have existed; which one is supposed to be used?
I think you have to start by writing down some example programs written
in your macro language and identify what they do, how the symbolic
references resolve, and explain any corner cases.
The solutions based on closures and funcall assume that
(DELAYED-EXECUTE ...) is dynamic. It does not gather up source code at
macro-expansion time. Rather, after all macros have been expanded and
the software is running, each time the program evaluates
(DELAYED-EXECUTE ...) (or, more precisely, the macro-expansion that
used to be a DELAYED-EXECUTE form) it pushes some function on a list.
Periodically, something has to go through that list, call the functions
(and probably clear them out, too).
Kaz Kylheku wrote:
>> Later, the macro simply fetches all those Lisp expressions and lets the
> ^^^^^
>
>> top-level-macro expand to all that code. Then the code can be compiled
>> into a function.
>
> Ah, you are trying to do some macro-time code motion. He he, well that
> is possible, but the problem is that if you move code from one
> environment to the other, if that code contains environmental
> references, it won't work properly at the site that it is moved to.
No, all free variables occurring inside the macro code (i.e. all
variables that my functions get passed) are part of the user's macro
parameters, i.e. they are expected to exist where the user invokes the
macro.
> You can't fix the problem by using backquote to interpolate those
> values, because you are working in macro time; run time isn't happening
> yet.
>
> That is to say, you want something like
>
> (defun foo (var)
> (delayed-execute (+ var var))
>
> (defun do-them ()
> (expand-delayed-executions))
>
> I mean, that doesn't even make sense. You have to pin down the exact
> semantics of what this does and what is allowed.
No, I only want (emit-action `(some-func ,some-var)) (where some-var is
user-supplied, i.e. is bound wherever the user uses my macro) to expand to:
(if *execute-mode* (some-func some-var)
(push `(some-func ,some-var) *actions*)
so that it either executes immediately, or - in compilation mode - puts
the stuff into a list that is later returned by the macro, so that the
macro-caller will get that code directly.
> What if FOO is never called, but DO-THEM is called? What happens with
> (+ var var), given that since FOO was never called, there never existed
> a binding for VAR?
>
> What if FOO is called three times, and then DO-THEM is called? Three
> bindings for VAR will have existed; which one is supposed to be used?
I can't quite follow you. My main macro only calls some functions to
create code, save it in a list, and then returns that list as a PROGN
for the user to compile. I want to extend my set of functions to also
have a *function* instead of a macro, that will not append that code to
a list to return from a macro, but execute those actions at runtime,
dynamically.
> I think you have to start by writing down some example programs written
> in your macro language and identify what they do, how the symbolic
> references resolve, and explain any corner cases.
So far it's just that (defun foo (x) (the-macro some-values ... x)) will
assemble a list of actions (that may use X) and compile them directly,
with fixed values.
Now I want to be able to say - somewhere - (the-function x y z) where
all of x y z are variable at runtime. THE-FUNCTION sets *execute-mode*
to T and then calls the same stuff as the macro would; only this time
the macro expansion (above) would not push the code to a list, to later
return them as a macro, but it would just execute the code.
> The solutions based on closures and funcall assume that
> (DELAYED-EXECUTE ...) is dynamic. It does not gather up source code at
> macro-expansion time. Rather, after all macros have been expanded and
> the software is running, each time the program evaluates
> (DELAYED-EXECUTE ...) (or, more precisely, the macro-expansion that
> used to be a DELAYED-EXECUTE form) it pushes some function on a list.
> Periodically, something has to go through that list, call the functions
> (and probably clear them out, too).
Well, I could just write an interpreter, but changing maybe five lines
to get the same result is too tempting to me ;)
--
Suffering from Gates-induced brain leakage...
"Ulrich Hobelmann" <···········@web.de> wrote in message
···················@individual.net...
>
> No, I only want (emit-action `(some-func ,some-var)) (where some-var is
> user-supplied, i.e. is bound wherever the user uses my macro) to expand
> to:
> (if *execute-mode* (some-func some-var)
> (push `(some-func ,some-var) *actions*)
> so that it either executes immediately, or - in compilation mode - puts
> the stuff into a list that is later returned by the macro, so that the
> macro-caller will get that code directly.
You would be better to provide some more concrete examples, but I can't help
but wonder if you really want #. ?
(my-macro (a (+ b c)))
vs
(defvar *foo* '(+ b c))
(my-macro (a #.foo))
--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
Coby Beck wrote:
> "Ulrich Hobelmann" <···········@web.de> wrote in message
> ···················@individual.net...
>> No, I only want (emit-action `(some-func ,some-var)) (where some-var is
>> user-supplied, i.e. is bound wherever the user uses my macro) to expand
>> to:
>> (if *execute-mode* (some-func some-var)
>> (push `(some-func ,some-var) *actions*)
>> so that it either executes immediately, or - in compilation mode - puts
>> the stuff into a list that is later returned by the macro, so that the
>> macro-caller will get that code directly.
>
> You would be better to provide some more concrete examples, but I can't help
> but wonder if you really want #. ?
>
> (my-macro (a (+ b c)))
> vs
>
> (defvar *foo* '(+ b c))
>
> (my-macro (a #.foo))
Argh, I can't really find it in the CLHS, nor on Google. Is that the
shortcut for directly evaluating something, at read time? In that case
I'm not sure if it would work, as the macro's argument is supposed to
contain free variables of the surrounding function.
--
Suffering from Gates-induced brain leakage...
"Ulrich Hobelmann" <···········@web.de> wrote in message
···················@individual.net...
> Coby Beck wrote:
>> You would be better to provide some more concrete examples, but I can't
>> help but wonder if you really want #. ?
>>
>> (my-macro (a (+ b c)))
>> vs
>>
>> (defvar *foo* '(+ b c))
>>
>> (my-macro (a #.foo))
>
> Argh, I can't really find it in the CLHS, nor on Google. Is that the
> shortcut for directly evaluating something, at read time? In that case
> I'm not sure if it would work, as the macro's argument is supposed to
> contain free variables of the surrounding function.
Yes, that's right. And if free variable means lexical then, yeah, it
wouldn't work.
--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
Ulrich Hobelmann <···········@web.de> writes:
> Coby Beck wrote:
>> I can't help but wonder if you really want #. ?
> Argh, I can't really find it in the CLHS, nor on Google.
http://clhs.lisp.se/Body/02_dhf.htm
Ulrich Hobelmann wrote:
>Now the problem is that I don't want VAL to be pushed, i.e. I want
>something more like
>(emit-action `(do-stuff-with ,val)) but then the emit-action macro
>won't execute that code, but expand to (if *bla* quoted-do-stuff-exp
>...), so it will only *return* that expression, instead of executing it.
> Again I'm in a situation where I need to eval stuff :(
Hey Ulrich,
Will something like this work?
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defun ev-cdr (tree)
"returns a copy of TREE, replacing symbols in the cdr of
TREE or any of it's elements with their corresponding symbol-values"
(cons (car tree)
(map 'list #'(lambda (x) (cond ((symbolp x) (symbol-value x))
((listp x) (ev-cdr x))
(t x)))
(cdr tree))))
(defvar *emit-execute* t)
(defmacro emit-action (expr)
`(if *emit-execute*
,expr
(ev-cdr ',expr)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
CL-USER> (setf x 5)
5
CL-USER> (emit-action (+ 4 x))
9
CL-USER > (setf *emit-execute* nil)
NIL
CL-USER > (emit-action (+ 4 x))
(+ 4 5)
CL-USER >
Nick
········@gmail.com wrote:
> Ulrich Hobelmann wrote:
> >Now the problem is that I don't want VAL to be pushed, i.e. I want
> >something more like
> >(emit-action `(do-stuff-with ,val)) but then the emit-action macro
> >won't execute that code, but expand to (if *bla* quoted-do-stuff-exp
> >...), so it will only *return* that expression, instead of executing it.
> > Again I'm in a situation where I need to eval stuff :(
>
> Hey Ulrich,
>
> Will something like this work?
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> (defun ev-cdr (tree)
> "returns a copy of TREE, replacing symbols in the cdr of
> TREE or any of it's elements with their corresponding symbol-values"
This is almost certainly headed down the wrong path.
> (cons (car tree)
> (map 'list #'(lambda (x) (cond ((symbolp x) (symbol-value x))
> ((listp x) (ev-cdr x))
> (t x)))
> (cdr tree))))
Note that X is a variable that holds a symbol, then (SYMBOL-VALUE X)
computes the same thing as (EVAL X).
In different words, EVAL evaluates a free variable as if by calling
SYMBOL-VALUE on it.
So what you are writing here is a flavor of EVAL: a function that hunts
down all the symbols in a nested tree and substitutes them for their
dynamic value, ignoring lexical bindings.
> CL-USER> (setf x 5)
In Common Lisp, creating a variable by assigning to it is nonportable,
and undefined according to the standard.
I wrote:
>> "returns a copy of TREE, replacing symbols in the cdr of
>> TREE or any of it's elements with their corresponding symbol-values"
um... maybe it should actually be something like
(defun ev-cdr (tree)
(cond ((symbolp tree) (symbol-value tree))
((not (listp tree)) tree)
((macro-function (car tree)) tree)
(t (cons (car tree)
(map 'list #'ev-cdr (cdr tree))))))
I realized as soon as I left the house bad things would happen if it
got fed a DOLIST or... an atom or something ;-) still, though, now the
problem of instances of VAL in the macro... maybe best would be to not
to use MACRO-FUNCTION but rather wrap SYMBOL-VALUE in a HANDLER-CASE
that spits out the symbol if there is an error?
Kaz wrote:
>This is almost certainly headed down the wrong path.
Really? Why? Did you try it?
Kaz wrote:
>Note that X is a variable that holds a symbol, then (SYMBOL-VALUE X)
>computes the same thing as (EVAL X).
"If all that is required for some application is to obtain the current
dynamic value of a given symbol, the function symbol-value may be more
efficient than eval." -cltl2 20.1
Kaz wrote:
>So what you are writing here is a flavor of EVAL: a function that hunts
>down all the symbols in a nested tree and substitutes them for their
>dynamic value, ignoring lexical bindings.
>
>> CL-USER> (setf x 5)
>
>In Common Lisp, creating a variable by assigning to it is nonportable,
>and undefined according to the standard.
sweet, thanks for the lesson, dude
········@gmail.com wrote:
> Ulrich Hobelmann wrote:
>> Now the problem is that I don't want VAL to be pushed, i.e. I want
>> something more like
>> (emit-action `(do-stuff-with ,val)) but then the emit-action macro
>> won't execute that code, but expand to (if *bla* quoted-do-stuff-exp
>> ...), so it will only *return* that expression, instead of executing it.
>> Again I'm in a situation where I need to eval stuff :(
>
> Hey Ulrich,
>
> Will something like this work?
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> (defun ev-cdr (tree)
> "returns a copy of TREE, replacing symbols in the cdr of
> TREE or any of it's elements with their corresponding symbol-values"
> (cons (car tree)
> (map 'list #'(lambda (x) (cond ((symbolp x) (symbol-value x))
> ((listp x) (ev-cdr x))
> (t x)))
> (cdr tree))))
>
> (defvar *emit-execute* t)
>
> (defmacro emit-action (expr)
> `(if *emit-execute*
> ,expr
> (ev-cdr ',expr)))
So it would look up all symbol-values inside the macro. I'm a bit
confused right now; I put this into my macro EMIT-ACTION now:
`(if *execute-mode*
,exp
(progn ... (push (ev-cdr ',exp) *actions*)))
with the caller using
(emit-action (stuff val)) where val is a parameter there. But then the
call to EV-CDR only receives 'exp as parameter, complaining that the
symbol isn't a list :(
Right now I wonder why, since ,exp is unquoted...
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> CL-USER> (setf x 5)
> 5
> CL-USER> (emit-action (+ 4 x))
> 9
> CL-USER > (setf *emit-execute* nil)
> NIL
> CL-USER > (emit-action (+ 4 x))
> (+ 4 5)
> CL-USER >
Hm, looks good. I'll have to investigate it.
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann wrote:
> Ok, I rewrote my code to create code, or execute it, depending on the
> switch *execute-mode*. For that I changed one function to a macro, but
> with a problem:
> (defmacro emit-action (exp)
> `(if *execute-mode*
> ,exp
> (progn (collect-emit-buf) (push ',exp *actions*))))
> ; this expands to code that either executes exp or pushes its quotation
> to an action list.
> ; observe especially the ' quote, which causes whatever exp is to be
> inserted verbatim
>
> (defun emit-stuff (val)
> (emit-action (do-stuff-with val)))
>
> Now the problem is that I don't want VAL to be pushed, i.e. I want
> something more like
> (emit-action `(do-stuff-with ,val)) but then the emit-action macro
> won't execute that code, but expand to (if *bla* quoted-do-stuff-exp
> ...), so it will only *return* that expression, instead of executing it.
> Again I'm in a situation where I need to eval stuff :(
>
> I'd like to be able to write the macro as:
> `...(push `,exp *actions*)...
> with a second quasiquote, and the call like (emit-action (do-stuff-with
> ,val)), but this doesn't work because the ` and , seem to be only read
> statically, i.e. the macro's second ` can't cause the , to unquote its
> argument.
>
> Any ideas?
>
All I can think of is to construct the code programmatically instead of
using backquote. Backquotes and commas are a convenience that the Lisp
system (reader? Interpreter/Compiler?) expands into list-building code
for you!
Ulrich Hobelmann wrote:
> Ok, I rewrote my code to create code, or execute it, depending on the
> switch *execute-mode*. For that I changed one function to a macro, but
> with a problem:
> (defmacro emit-action (exp)
> `(if *execute-mode*
> ,exp
> (progn (collect-emit-buf) (push ',exp *actions*))))
changed this to
,(something exp)
(progn ... (push ,exp ...))...
with the call like (emit-action `(some-func ,some-val)), so I can at
least insert it into the list as before.
Now the problem is reduced to having the function SOMETHING transform
the quasiquotation exp into *just* the code for exp, so the macro can
return it, so that at the compile-site, it will just execute it, instead
of creating a stupid list.
I wrote the following:
CL-USER> (defmacro foo (exp)
(print exp)
exp)
FOO
CL-USER> (let ((x 'bar))
(foo `(func 5 ,x)))
(LIST* 'FUNC (LIST* 5 (LIST X)))
(FUNC 5 BAR)
; output ends
So it seems like the quasiquotation expands to some funny term with LIST*.
Is there any standard function that will transform that LIST* expression
into the expression (as the Lisp writer prints it at the end)?
Otherwise, I'm gonna write that, and the problem will be solved. :)
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann wrote:
> Ulrich Hobelmann wrote:
> > Ok, I rewrote my code to create code, or execute it, depending on the
> > switch *execute-mode*. For that I changed one function to a macro, but
> > with a problem:
> > (defmacro emit-action (exp)
> > `(if *execute-mode*
> > ,exp
> > (progn (collect-emit-buf) (push ',exp *actions*))))
>
> changed this to
> ,(something exp)
> (progn ... (push ,exp ...))...
>
> with the call like (emit-action `(some-func ,some-val)), so I can at
> least insert it into the list as before.
So you stepped on a bump in the carpet to flatten it, and it worked,
but another carpet bump showed up elsewhere in the room!
Well, when the carpet is 11 feet wide, and the room is 10 feet wide,
that's what happens.
> Now the problem is reduced to having the function SOMETHING transform
> the quasiquotation exp into *just* the code for exp, so the macro can
> return it, so that at the compile-site, it will just execute it, instead
> of creating a stupid list.
That function SOMETHING is called EVAL! :)
> So it seems like the quasiquotation expands to some funny term with LIST*.
Yes, backquote is a shorthand notation for list construction code. To
actually make the object that the backquote represents, you have to
evaluate it. Doing so runs the list construction code. So the SOMETHING
function you are looking for is EVAL, which will take the backquote
object that was passed into the macro and convert it to the code that
it represents.
But now you run into the EVAL problem: EVAL doesn't have the lexical
environment to be able to resolve the variable reference within that
list construction code.
You can't solve the problem this way at all. The problem is now that
the caller who uses the EMIT-ACTION macro is constructing code at
run-time. So that code cannot possibly be incoporated into that
caller's body. The caller gives EMIT-ACTION a backquote, which refers
to lexical variables in the caller.
You can't embed into the body of a function the values of lexical
variables whose bindings are established when that function is invoked.
That function has to exist in order to be invoked, and by the time a
function exists, all macro processing had been done already.
You don't actually /need/ to embed the value of VAL into the source
code.You just need to generate code which contains references to VAL.
Do not try to resolve VAL at macro-expansion time, it doesn't exist at
macro-expansion time!
Kaz Kylheku wrote:
>> Now the problem is reduced to having the function SOMETHING transform
>> the quasiquotation exp into *just* the code for exp, so the macro can
>> return it, so that at the compile-site, it will just execute it, instead
>> of creating a stupid list.
>
> That function SOMETHING is called EVAL! :)
No, when I call for instance
(let ((x 'bar)) (eval '(list* 'func (list* 5 (list x)))))
it complains that x isn't known. I.e. EVAL seems to ignore the lexical
environment. Unfortunately :(
>> So it seems like the quasiquotation expands to some funny term with LIST*.
>
> Yes, backquote is a shorthand notation for list construction code. To
> actually make the object that the backquote represents, you have to
> evaluate it. Doing so runs the list construction code. So the SOMETHING
> function you are looking for is EVAL, which will take the backquote
> object that was passed into the macro and convert it to the code that
> it represents.
Using EVAL doesn't work in the code, either.
> But now you run into the EVAL problem: EVAL doesn't have the lexical
> environment to be able to resolve the variable reference within that
> list construction code.
Yes. Strange that it doesn't. Lisp is quite well designed overall, but
EVAL is the exception ;)
> You can't solve the problem this way at all. The problem is now that
> the caller who uses the EMIT-ACTION macro is constructing code at
> run-time. So that code cannot possibly be incoporated into that
> caller's body. The caller gives EMIT-ACTION a backquote, which refers
> to lexical variables in the caller.
>
> You can't embed into the body of a function the values of lexical
> variables whose bindings are established when that function is invoked.
> That function has to exist in order to be invoked, and by the time a
> function exists, all macro processing had been done already.
>
> You don't actually /need/ to embed the value of VAL into the source
> code.You just need to generate code which contains references to VAL.
> Do not try to resolve VAL at macro-expansion time, it doesn't exist at
> macro-expansion time!
There are two cases: compilation, and interpretation. In compilation
the macro may expand to actual Lisp variables, because these are
expected to be in the lexical Lisp environment, so that it will compile
and run.
In interpretation the macro expansion should only contain the verbatim
code, which refers to the variable that exists at the macro-call site;
in this case VAL is a function parameter. So that should work too.
I only can't find a way to have one syntactic form expand to the code
that will expand to the if-clause that either executes the verbatim
code, or else pushes its quotation onto a list.
Hm, I'll see what ideas tomorrow will bring.
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann wrote:
> Kaz Kylheku wrote:
> >> Now the problem is reduced to having the function SOMETHING transform
> >> the quasiquotation exp into *just* the code for exp, so the macro can
> >> return it, so that at the compile-site, it will just execute it, instead
> >> of creating a stupid list.
> >
> > That function SOMETHING is called EVAL! :)
>
> No, when I call for instance
> (let ((x 'bar)) (eval '(list* 'func (list* 5 (list x)))))
>
> it complains that x isn't known. I.e. EVAL seems to ignore the lexical
> environment. Unfortunately :(
Okay, so the SOMETHING you are looking for is an EVAL which does not
ignore the lexical environment. Unfortunately, that doesn't exist.
When I say "that doesn't exist", I'm being deliberately ambiguous.
You can take that demonstrative pronoun to refer to "the lexical
environment" or to "SOMETHING". Because neither of them exist. :)
At macro-expansion time, the lexical environment doesn't exist. There
is no variable X.
If you have this;
(let ((x 'bar)) (some-macro (list 'func x)))
If SOME-MACRO applies EVAL to the form (LIST 'FUNC X), it will complain
that X isn't known. But you see, at macro-expansion time, not only is
not X known, it doesn't exist at all.
Consider if the above is put into a function
(defun foo ()
(let ((x 'bar)) (some-macro (list 'func x))))
The macro is expanded when the DEFUN is processed. The DEFUN can be
processed when you type it to the listener. Or when COMPILE-FILE
processes a source file that contains DEFUN. In either case, the
function is not being called, it is being scanned to produce the
definition, or even being compiled. Since the function is not being
called, the semantic actions of the LET never happen, and the variable
X doesn't exist at the time the macro is called. So even if you had a
magic EVAL that could see lexical variables, that function would not
work in these situations. You would have no environment that could be
passed to it.
In this case, such an eval could work:
(let ((x 'bar)) (eval '(list* 'func (list* 5 (list x)))))
you could stick an extra environment argument on this eval:
(let ((x 'bar)) (eval '(list* 'func (list* 5 (list x)))
(extensions::current-environment)))
This fictitous (EXTENSIONS::CURRENT-ENVIRONMENT) operator would return
an object representing the lexical environment, which would be passed
into EVAL and used to resolve the reference to X.
In the macro, though, you would have no way of getting such an object,
since the lexical environment doesn't exist.
> Using EVAL doesn't work in the code, either.
That's right. It's the function you are looking for, /and/ it won't
work.
:)
Kaz Kylheku wrote:
> If you have this;
>
> (let ((x 'bar)) (some-macro (list 'func x)))
>
> If SOME-MACRO applies EVAL to the form (LIST 'FUNC X), it will complain
> that X isn't known. But you see, at macro-expansion time, not only is
> not X known, it doesn't exist at all.
Of course it does: in the user's code, the LET. The macro can expand to
any code that uses X, and X will be 'bar.
> Consider if the above is put into a function
>
> (defun foo ()
> (let ((x 'bar)) (some-macro (list 'func x))))
Here X would simply be replaced by BAR by the compiler. If FOO had X as
a parameter, instead of the LET, X would be dynamic at runtime,
depending on the call.
> The macro is expanded when the DEFUN is processed. The DEFUN can be
> processed when you type it to the listener. Or when COMPILE-FILE
> processes a source file that contains DEFUN. In either case, the
> function is not being called, it is being scanned to produce the
> definition, or even being compiled. Since the function is not being
> called, the semantic actions of the LET never happen, and the variable
> X doesn't exist at the time the macro is called. So even if you had a
But the macro expansion occurs inside X's environment, so the compiler
won't see any X that's unknown. Remember, in macro-mode *execute-mode*
is NIL, so X isn't used at compile-time to execute, but it's merely
returned from the macro as code that will be compiled and run later.
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann wrote:
> Kaz Kylheku wrote:
> > If you have this;
> >
> > (let ((x 'bar)) (some-macro (list 'func x)))
> >
> > If SOME-MACRO applies EVAL to the form (LIST 'FUNC X), it will complain
> > that X isn't known. But you see, at macro-expansion time, not only is
> > not X known, it doesn't exist at all.
>
> Of course it does: in the user's code, the LET. The macro can expand to
> any code that uses X, and X will be 'bar.
Well, of course the syntax (LET ((X ...)) ...) exists, and the symbol X
has been interned. But that's not what I mean when I say X doesn't
exist. I mean only that the lexical variable binding X doesn't exist.
> > Consider if the above is put into a function
> >
> > (defun foo ()
> > (let ((x 'bar)) (some-macro (list 'func x))))
>
> Here X would simply be replaced by BAR by the compiler.
Only as an optimization, after seeing that X receives this constant
object as a value, and is never assigned any other value.
That is not relevant; consider X to be bound to something that isn't
known at compile time, like the result of calling some function.
> If FOO had X as a parameter, instead of the LET, X would be dynamic at runtime,
> depending on the call.
Okay, that's another way. Function parameters are just lexical
variables. As a parameter, the newly created binding for X could
receive some dynamic value from the argument expression. Inside the
LET, it could receive a dynamic value from an initializing expression.
> > The macro is expanded when the DEFUN is processed. The DEFUN can be
> > processed when you type it to the listener. Or when COMPILE-FILE
> > processes a source file that contains DEFUN. In either case, the
> > function is not being called, it is being scanned to produce the
> > definition, or even being compiled. Since the function is not being
> > called, the semantic actions of the LET never happen, and the variable
> > X doesn't exist at the time the macro is called. So even if you had a
>
> But the macro expansion occurs inside X's environment, so the compiler
You have to be careful when you say "macro expansion" in Lisp
discussions, because it's ambiguous: it refers to the process of
expansion as well as to its result.
Surely enough, if the macro's result, a form which evaluates X, is
substituted into the LET which sets up the binding for X, the reference
will later be correct.
The macro expansion process does not occur in an environment where X is
known, however.
> won't see any X that's unknown.
> Remember, in macro-mode *execute-mode*
> is NIL, so X isn't used at compile-time to execute, but it's merely
> returned from the macro as code that will be compiled and run later.
Yes, but it will be incorporated into some remote part of the program,
by means of some other macro which retrieves the elements of
*ACTION-LIST*, and produces them as an expansion. The result of that
expansion won't be placed in an where X is known.
It's just a cut and paste job: moving snippets of code around using
macros.
Here is how you could potentially do it. Okay, so the form to be moved
around can contain variable references:
In *EXECUTE-MODE*, the variable references simply resolve to the
surrounding lexical variables.
If *EXECUTE-MODE* is off, you can specify alternate bindings that these
actions will use to resolve the variables. Your EMIT-ACTION will take
two arguments, like this:
(defun foo (x)
(emit-action ((x 42)) (func 5 x)))
If actions are being deferred, then the macro will compute this form:
(let ((x 42)) (func 5 x))
and that is what will be pushed into *ACTION-LIST*. The form contains
no free variables, and so can be subject to EVAL.
I really don't see how else you can do it. You can't transport (FOO 5
X) into some other location of the program without arranging for the X
reference to be somehow satisfied.
(defmacro emit-action (deferred-bindings form)
(if *execute-mode*
form
(progn
...
(push `(let ,deferred-bindings ,form) *action-list*))))
Kaz Kylheku wrote:
>>> Consider if the above is put into a function
>>>
>>> (defun foo ()
>>> (let ((x 'bar)) (some-macro (list 'func x))))
>> Here X would simply be replaced by BAR by the compiler.
>
> Only as an optimization, after seeing that X receives this constant
> object as a value, and is never assigned any other value.
>
> That is not relevant; consider X to be bound to something that isn't
> known at compile time, like the result of calling some function.
True.
>>> The macro is expanded when the DEFUN is processed. The DEFUN can be
>>> processed when you type it to the listener. Or when COMPILE-FILE
>>> processes a source file that contains DEFUN. In either case, the
>>> function is not being called, it is being scanned to produce the
>>> definition, or even being compiled. Since the function is not being
>>> called, the semantic actions of the LET never happen, and the variable
>>> X doesn't exist at the time the macro is called. So even if you had a
>> But the macro expansion occurs inside X's environment, so the compiler
>
> You have to be careful when you say "macro expansion" in Lisp
> discussions, because it's ambiguous: it refers to the process of
> expansion as well as to its result.
Oops, you're right. Mea culpa.
> Surely enough, if the macro's result, a form which evaluates X, is
> substituted into the LET which sets up the binding for X, the reference
> will later be correct.
>
> The macro expansion process does not occur in an environment where X is
> known, however.
Exactly, but that's not a problem, because the macro doesn't need the
value of X. It's only juggling statements and symbols.
Only the function, that sets *execute-mode* to T, will cause the code to
be executed, and in that context the X is supposed to refer to the
function's parameter.
>> won't see any X that's unknown.
>
>> Remember, in macro-mode *execute-mode*
>> is NIL, so X isn't used at compile-time to execute, but it's merely
>> returned from the macro as code that will be compiled and run later.
>
> Yes, but it will be incorporated into some remote part of the program,
> by means of some other macro which retrieves the elements of
> *ACTION-LIST*, and produces them as an expansion. The result of that
> expansion won't be placed in an where X is known.
No, it's the same macro. The macro calls a function that recurses
around its argument, puts some stuff into a global variable, and then
returns that list as a PROGN, basically. It's all in one invocation.
At the beginning of the macro, it also initializes *actions* to NIL, so
it's always fresh and empty, and deleted in the end.
> It's just a cut and paste job: moving snippets of code around using
> macros.
>
> Here is how you could potentially do it. Okay, so the form to be moved
> around can contain variable references:
>
> In *EXECUTE-MODE*, the variable references simply resolve to the
> surrounding lexical variables.
Yes, the function parameter at runtime.
> If *EXECUTE-MODE* is off, you can specify alternate bindings that these
> actions will use to resolve the variables. Your EMIT-ACTION will take
> two arguments, like this:
>
> (defun foo (x)
> (emit-action ((x 42)) (func 5 x)))
>
> If actions are being deferred, then the macro will compute this form:
>
> (let ((x 42)) (func 5 x))
>
> and that is what will be pushed into *ACTION-LIST*. The form contains
> no free variables, and so can be subject to EVAL.
>
> I really don't see how else you can do it. You can't transport (FOO 5
> X) into some other location of the program without arranging for the X
> reference to be somehow satisfied.
>
> (defmacro emit-action (deferred-bindings form)
> (if *execute-mode*
> form
> (progn
> ...
> (push `(let ,deferred-bindings ,form) *action-list*))))
Hm, interesting idea. I'll think about it tomorrow, but maybe I'll just
hand-create a list instead of using `, so I can reconstruct the
expression in the macro. That looks easier to me, just one function
needed to write.
--
Suffering from Gates-induced brain leakage...
Kaz Kylheku wrote:
> Here is how you could potentially do it. Okay, so the form to be moved
> around can contain variable references:
>
> In *EXECUTE-MODE*, the variable references simply resolve to the
> surrounding lexical variables.
>
> If *EXECUTE-MODE* is off, you can specify alternate bindings that these
> actions will use to resolve the variables. Your EMIT-ACTION will take
> two arguments, like this:
>
> (defun foo (x)
> (emit-action ((x 42)) (func 5 x)))
>
> If actions are being deferred, then the macro will compute this form:
>
> (let ((x 42)) (func 5 x))
>
> and that is what will be pushed into *ACTION-LIST*. The form contains
> no free variables, and so can be subject to EVAL.
>
> I really don't see how else you can do it. You can't transport (FOO 5
> X) into some other location of the program without arranging for the X
> reference to be somehow satisfied.
>
> (defmacro emit-action (deferred-bindings form)
> (if *execute-mode*
> form
> (progn
> ...
> (push `(let ,deferred-bindings ,form) *action-list*))))
Hm, I did something similar to this, and it in fact seems to work.
Well, I don't have anything like a test suite, so it'll probably break
sometime later. Well, I'll see, but it'll take a while, since for the
next week I'll have more pressing stuff to do. :(
The solution is thus:
What I absolutely need is a quoted form of the code, but it has to refer
to actual variable names, so I use quasiquote at the macro's call site:
(emit-action `(some-func ,val)) (not quite there yet, but see below).
But the macro has to expand to something that will execute that same
code, if in *execute-mode*. So, because the macro isn't called with a
Lisp expression, but with a quasiquoted Lisp-exp, I have to remove the
quotes -> EVAL!
Now to circumvent the stupid problem that EVAL doesn't *get* its lexical
environment, I introduce a LET, but in the opposite place as your code
above.
Now the macro looks like this:
(defmacro emit-action (var-bindings exp)
`(if *execute-mode*
(eval (list 'let ,var-bindings ,exp))
(progn (collect-emit-buf) (push ,exp *actions*))))
I use the explicit LIST because right now I can't afford getting more
confused by another set of ` and , (especially no double ,,). ;)
The downside is that every call site has to explicitly pass whatever
unquoted arguments it uses, so a call now looks like
(defun some-fun (varname)
(emit-action `((varname ,varname)) `(some-function ,varname)))
Well, that's it.
In another post I asked if there are valid reasons for not separating
the reading and quasiquote-expansion phase (as in Scheme), so the reader
would only translate Lisp syntax into QUASIQUOTE or UNQUOTE expressions,
and most of all, if there are valid reasons for expanding macros after
quasiquote-expansion, because otherwise macros could use cooler quoting
combinations which would be resolved after all macro-expansion (except
the macros inside the quasiquote-expansion, which would have to be done
again later. I hope anybody knows about those design decisions.
Anyway, I hope I'll never have to deal with intricate knots of
quasiquote, macros, and - most of all - EVAL, again. :)
Well, chapter closed for now, off Lisp for a while, and back to more
boring stuff...
Many many thanks to you Kaz, to Nick, Eli, and everybody else!
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann wrote:
> Now to circumvent the stupid problem that EVAL doesn't *get* its lexical
> environment, I introduce a LET, but in the opposite place as your code
> above.
>
> Now the macro looks like this:
> (defmacro emit-action (var-bindings exp)
> `(if *execute-mode*
> (eval (list 'let ,var-bindings ,exp))
> (progn (collect-emit-buf) (push ,exp *actions*))))
Strange. This assures that in *EXECUTE-MODE*, the expression sees only
the bindings specified in VAR-BINDINGS, and not the surrounding ones.
> I use the explicit LIST because right now I can't afford getting more
> confused by another set of ` and , (especially no double ,,). ;)
Easy:
`(if *execute-mode*
(eval `(let ,',var-bindings ,',exp))
(progn (collect-emit-buf) (push ,exp *actions*))))
You put in two commas, so the rightmost comma goes with the outer
backquote. So the outer backquote puts in VAR-BINDINGS, and that then
reduces to ,'(...) for the outer backquote where the , and ' "cancel"
each other. (If you leave out the quote and just have two commas you
get double evaluation; one from each comma. Here, we just want to put
the object into the template and then have it "survive" the outer
expansion verbatim).
Even the famed Ron Rivest had trouble with nested backquote, according
to a MIT Lisp mailing list archive dating back to 1980. He missed the
,', trick. :)
> The downside is that every call site has to explicitly pass whatever
> unquoted arguments it uses, so a call now looks like
> (defun some-fun (varname)
> (emit-action `((varname ,varname)) `(some-function ,varname)))
>
> Well, that's it.
>
> In another post I asked if there are valid reasons for not separating
> the reading and quasiquote-expansion phase (as in Scheme), so the reader
The valid reason is that unless you specify something, it doesn't get
done. That is to say, that separation is an /extra/ requirement for
implementors. When people put together programming language standards
for ANSI or ISO, particularly in the first draft, they tend to avoid
inventing features, but rather codify what is already out there.
If you are making a standard and there are, say, eight language vendors
already, and none of them implement some feature the same way, you
don't barge in and require them all to implement it one way.
> would only translate Lisp syntax into QUASIQUOTE or UNQUOTE expressions,
> and most of all, if there are valid reasons for expanding macros after
> quasiquote-expansion, because otherwise macros could use cooler quoting
> combinations which would be resolved after all macro-expansion (except
If you have an S-expression based target syntax for the backquote
language, you can write code that writes backquotes, which can be
useful.
There are things you can do even if you don't have this. For instance,
you might think that in order to splice in a variable number of
"commas" you might need such a feature. But in fact, the "cross
producting" behavior of nested ,@ splicing unquotes will do that. E.g.
(let ((expr-list '((+ 1 1) (+ 2 2) (* 3 3))))
``(,,@expr-list))
This produces a piece of code which, when evaled will compute the list
(2 4 9). So it's just like if you wrote
(let ((expr-list '((+ 1 1) (+ 2 2) (* 3 3))))
``(,,(first expr-list) ,,(second expr-list) ,,(third expr-list)))
except that here, we manually produced three elements, hard-coding for
a list of length 3.
The inner backquote splices a variable number of elements into the
outer backquote, and the outer backquotes comma magically "distributes"
into those spliced elements.
On 9379 day of my life Ulrich Hobelmann wrote:
>> Consider if the above is put into a function
>> (defun foo ()
>> (let ((x 'bar)) (some-macro (list 'func x))))
>
> Here X would simply be replaced by BAR by the compiler.
It wouldn't. Symbol X may have special meaning in SOME-MACRO. Of
course X may be replaced by in result of macroexpansion of SOME-MACRO
if compiler is sure that X is used as a variable. But SOME-MACRO will
never get (LIST 'FUNC 'BAR) or (LIST 'FUNC BAR) as input.
--
Ivan Boldyrev
Perl is a language where 2 x 2 is not equal to 4.
Ivan Boldyrev wrote:
> On 9379 day of my life Ulrich Hobelmann wrote:
>>> Consider if the above is put into a function
>>> (defun foo ()
>>> (let ((x 'bar)) (some-macro (list 'func x))))
>> Here X would simply be replaced by BAR by the compiler.
>
> It wouldn't. Symbol X may have special meaning in SOME-MACRO. Of
> course X may be replaced by in result of macroexpansion of SOME-MACRO
> if compiler is sure that X is used as a variable. But SOME-MACRO will
> never get (LIST 'FUNC 'BAR) or (LIST 'FUNC BAR) as input.
Right. I'm assuming that X is a variable that will not be used by the
macro, so that after expansion it too will be free and thus refer to 'bar.
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann <···········@web.de> writes:
> So it seems like the quasiquotation expands to some funny term with LIST*.
>
> Is there any standard function that will transform that LIST*
> expression into the expression (as the Lisp writer prints it at the
> end)? Otherwise, I'm gonna write that, and the problem will be solved.
> :)
The representation of quasiquotation is not standarized, only its
effect when executed. Different Lisp implementations use different
representations.
--
__("< Marcin Kowalczyk
\__/ ······@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Marcin 'Qrczak' Kowalczyk wrote:
> Ulrich Hobelmann <···········@web.de> writes:
>
>> So it seems like the quasiquotation expands to some funny term with LIST*.
>>
>> Is there any standard function that will transform that LIST*
>> expression into the expression (as the Lisp writer prints it at the
>> end)? Otherwise, I'm gonna write that, and the problem will be solved.
>> :)
>
> The representation of quasiquotation is not standarized, only its
> effect when executed. Different Lisp implementations use different
> representations.
Hm, then I suppose I should construct my expressions manually. They are
all rather simple anyway.
--
Suffering from Gates-induced brain leakage...
Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote:
+---------------
| Ulrich Hobelmann <···········@web.de> writes:
| > So it seems like the quasiquotation expands to some funny term with LIST*.
|
| The representation of quasiquotation is not standarized, only its
| effect when executed. Different Lisp implementations use different
| representations.
+---------------
Yup. This is one place where Scheme got it right, IMHO, by at least
standardizing the name of the quasiquotation macro and the associated
internal syntax markers -- QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING --
that the reader transforms `,,@ into:
http://schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-7.html#%_sec_4.2.6
Also, by making QUASIQUOTE an explicit operator ("syntax", but it
could have been a macro in CL), it allows the embedding of "implicit
quasiquotation" in the forms of *other* wrapper macros, such as are
used by the Scheme Shell, for example. [I have written elsewhere
(several times) on why the lack of such standarization makes it
practically impossible to write a "CL Shell" with Scsh-style syntax.]
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock wrote:
> Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote:
> +---------------
> | Ulrich Hobelmann <···········@web.de> writes:
> | > So it seems like the quasiquotation expands to some funny term with LIST*.
> |
> | The representation of quasiquotation is not standarized, only its
> | effect when executed. Different Lisp implementations use different
> | representations.
> +---------------
>
> Yup. This is one place where Scheme got it right, IMHO, by at least
> standardizing the name of the quasiquotation macro and the associated
> internal syntax markers -- QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING --
> that the reader transforms `,,@ into:
>
> http://schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-7.html#%_sec_4.2.6
>
> Also, by making QUASIQUOTE an explicit operator ("syntax", but it
> could have been a macro in CL), it allows the embedding of "implicit
> quasiquotation" in the forms of *other* wrapper macros, such as are
> used by the Scheme Shell, for example. [I have written elsewhere
> (several times) on why the lack of such standarization makes it
> practically impossible to write a "CL Shell" with Scsh-style syntax.]
>
>
> -Rob
>
> -----
> Rob Warnock <····@rpw3.org>
> 627 26th Avenue <URL:http://rpw3.org/>
> San Mateo, CA 94403 (650)572-2607
>
Why couldn't you do similar things in CL? All a quasiquote does is
stand in for explicit list construction!
Eli Gottlieb <···········@gmail.com> wrote:
+---------------
| Rob Warnock wrote:
| > Yup. This is one place where Scheme got it right, IMHO, by at least
| > standardizing the name of the quasiquotation macro and the associated
| > internal syntax markers -- QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING --
| > http://schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-7.html#%_sec_4.2.6
| > Also, by making QUASIQUOTE an explicit operator ("syntax", but it
| > could have been a macro in CL), it allows the embedding of "implicit
| > quasiquotation" in the forms of *other* wrapper macros, such as are
| > used by the Scheme Shell, for example. [I have written elsewhere
| > (several times) on why the lack of such standarization makes it
| > practically impossible to write a "CL Shell" with Scsh-style syntax.]
|
| Why couldn't you do similar things in CL? All a quasiquote does is
| stand in for explicit list construction!
+---------------
In CL perhaps [though even there the situation is a little more
complicated than you imply -- it is a stand-in for construction
of a *code* sequence that, when evaluated, will perform an explicit
list construction], but in Scheme the separation between the reader
and the evaluator is complete: *All* the reader does is change the
`,,@ markers to QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING forms
(respectively); it is the evaluator which "executes" the resulting
QUASIQUOTE forms. And in particular, even though R5RS warns that:
Unpredictable behavior can result if any of the symbols
QUASIQUOTE, UNQUOTE, or UNQUOTE-SPLICING appear in positions
within a <QQ template> otherwise than as described above.
all Scheme systems I have access to allow the reading of "isolated"
or "improperly-nested" UNQUOTE and UNQUOTE-SPLICING subforms, e.g.:
> (quote (a ,b ,@(c d) e))
==> (a (unquote b) (unquote-splicing (c d)) e)
>
This means that *other* macros besides QUASIQUOTE can use those
forms [albeit by doing a tree walk and interpreting the embedded
UNQUOTE and UNQUOTE-SPLICING subforms], which is what happens in
Scsh, allowing one to write things like this:
(let ((foo "filename")) ; RUN is a Scsh macro that does what
(run (ls -l ,foo))) ; might be called "implicit quasiquoting".
(run (cc ,file ,@flags)) ; Compile FILE with FLAGS.
Unfortunately those are illegal to a standard CL reader, so in a
(hypothetical) "CLsh" the above would have to look like this:
(let ((foo "filename"))
(run `(ls -l ,foo))) ; Note explicit backquote
(run `(cc ,file ,@flags)) ; (ditto)
This is why I said it's "practically impossible to write a CL Shell
with Scsh-style syntax" [unless you manually add explicit backquotes
at various points].
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Alexander Schmolck <··········@gmail.com> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > This is why I said it's "practically impossible to write a CL Shell
| > with Scsh-style syntax" [unless you manually add explicit backquotes
| > at various points].
|
| I haven't really thought much about it or tested it, but why doesn't
| this simple approach below work for that purpose...
+---------------
Yeah, it sort of works, for simple cases, though I haven't tested
the corner cases [since I don't have a QUASIQUOTE expander yet to
test it with (see below)]:
> (defun run-func (form)
(format t "RUN-FUNC saw this form: ~s~%" form))
RUN-FUNC
> (defmacro run (run-form)
(list 'run-func (list 'quote run-form)))
RUN
> (run (ls -l foo))
RUN-FUNC saw this form: (LS -L FOO)
NIL
>
Note that for a real "CLsh", you'd probably want to run with
readtable-case :INVERT to avoid upcasing the Unix commands. ;-} ;-}
Unfortunately, having usurped backquote/comma/comma-at from READ,
one is now stuck needing to do one's own full backquote expansion,
which includes all the gore of handling lexical environments as well.
That is, you get this:
> (let ((file "some.file")
(flags '(-lm -lbsd)))
(run (cc ,file ,@flags)))
; In: LET ((FILE "some.file") (FLAGS '#))
; (LET (# #)
; (RUN #))
; Note: Variable FILE defined but never used.
;
; Note: Variable FLAGS defined but never used.
RUN-FUNC saw this form: (CC (UNQUOTE FILE) (UNQUOTE-SPLICING FLAGS))
NIL
>
when you really wanted this: ;-} ;-}
RUN-FUNC saw this form: (CC "some.file" -LM -LBSD)
And worse than that, even. Notice that RUN-FUNC got the variable
*names*, not their values, and that the bindings are *lexical*,
not global, so RUN-FUNC isn't going to be able to figure out what
to replace FILE & FLAGS with. We're going to have to extend the
whole infrastructure with something like this:
> (defun run-func (form env)
(format t "RUN-FUNC saw this form: ~s~%In this env: ~s~%" form env))
RUN-FUNC
> (defmacro run (run-form &environment env)
(list 'run-func (list 'quote run-form) (list 'quote env)))
RUN
> (run (ls -l foo))
RUN-FUNC saw this form: (LS -L FOO)
In this env: NIL
NIL
> (let ((file "some.file")
(flags '(-lm -lbsd)))
(run (cc ,file ,@flags)))
; In: LET ((FILE "some.file") (FLAGS '#))
; (LET (# #)
; (RUN #))
; Note: Variable FILE defined but never used.
;
; Note: Variable FLAGS defined but never used.
RUN-FUNC saw this form: (CC (UNQUOTE FILE) (UNQUOTE-SPLICING FLAGS))
In this env: #S(C::LEXENV
:FUNCTIONS NIL
:VARIABLES ((FLAGS
. #<C::LAMBDA-VAR #x58930165 NAME= FLAGS>)
(FILE . #<C::LAMBDA-VAR #x5893010D NAME= FILE>))
:BLOCKS NIL
:TAGS NIL
:TYPE-RESTRICTIONS NIL
:LAMBDA #<LAMBDA #x5893021D
NAME= NIL
TYPE= #<KERNEL::BUILT-IN-CLASS
FUNCTION (read-only) {280576F5}>
WHERE-FROM= :DEFINED
VARS= (FILE FLAGS)>
:CLEANUP NIL
:COOKIE #S(C::COOKIE
:SPEED 1
:SPACE 1
:SAFETY 1
:CSPEED 1
:BREVITY 1
:DEBUG 2)
:INTERFACE-COOKIE #S(C::COOKIE
:SPEED NIL
:SPACE NIL
:SAFETY NIL
:CSPEED NIL
:BREVITY NIL
:DEBUG NIL)
:OPTIONS NIL
:DYNAMIC-EXTENT NIL)
NIL
>
Ooooh! Look at that loverly idiosyncratic environment! How is my
QUASIQUOTE macro expander going to extract the values of FILE & FLAGS
from *that* in a portable way??!?!?
[Cue Duane Rettig, waiting in the wings for an encore presentation
of his Environments Access module... ;-} ;-} ]
Anyway, like I said before (too many posts ago), doing a "CLsh" that
used "pure" Scsh syntax would be quite hard. It might be better to
just give up on trying to achieve the "pure" syntax and accept that
we would type the extra backquote character in all of the run-process
convenience forms. Then it "just works", right out of the box:
> (defun run (form)
(format t "RUN saw this form: ~s~%" form))
> (let ((file "some.file")
(flags '(-lm -lbsd)))
(run `(cc ,file ,@flags)))
RUN saw this form: (CC "some.file" -LM -LBSD)
NIL
>
+---------------
| (you don't need backquote to accept additional commas, right?)?
+---------------
What do you mean by "additional commas"? Nesting? Of course!
It needs to work *just* like normal CL backquote, otherwise
your "shell" syntax won't be "CL".
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
In article <······················@speakeasy.net>,
Rob Warnock <····@rpw3.org> wrote:
> In this env: #S(C::LEXENV
> :FUNCTIONS NIL
> :VARIABLES ((FLAGS
> . #<C::LAMBDA-VAR #x58930165 NAME= FLAGS>)
> (FILE . #<C::LAMBDA-VAR #x5893010D NAME= FILE>))
> :BLOCKS NIL
[...]
> :DYNAMIC-EXTENT NIL)
> NIL
> >
>
> Ooooh! Look at that loverly idiosyncratic environment! How is my
> QUASIQUOTE macro expander going to extract the values of FILE & FLAGS
> from *that* in a portable way??!?!?
>
> [Cue Duane Rettig, waiting in the wings for an encore presentation
> of his Environments Access module... ;-} ;-} ]
I don't think you need to. You're outputting a macroexpander. It will
get evaluated in the lexical context of the let, so you just need to
reference the variables:
> > (let ((file "some.file")
> (flags '(-lm -lbsd)))
> (run (cc ,file ,@flags)))
> RUN-FUNC saw this form: (CC (UNQUOTE FILE) (UNQUOTE-SPLICING FLAGS))
If you had a QUASIQUOTE expander that'd just be putting a QUASIQUOTE
around the command form.
(defun process-qq (args)
`(append ,@(loop for arg in args
collect (cond ((listp arg)
(case (car arg)
(unquote `(list ,(second arg)))
(unquote-splicing (second arg))
(otherwise `(list ,(process-qq arg)))))
(t `(list (quote ,arg)))))))
(defmacro quasiquote (args)
(process-qq args))
That one's extremely pessimal and almost certainly broken in the general
case, but hey, it's a five-minute hack:
(defun run-func (form)
(format t "RUN-FUNC saw this form: ~s~%" form))
(defmacro run (run-form)
(list 'run-func (list 'quasiquote run-form)))
SSBR> (let ((file "some.file")
(flags '(-lm -lbsd)))
(run (cc ,file ,@flags)))
RUN-FUNC saw this form: (CC "some.file" -LM -LBSD)
SSBR> (macroexpand-1 '(let ((file "some.file")
(flags '(-lm -lbsd)))
(run (cc ,file ,@flags))))
(LET ((FILE "some.file") (FLAGS '(-LM -LBSD)))
(RUN (CC (UNQUOTE FILE) (UNQUOTE-SPLICING FLAGS))))
NIL
SSBR> (macroexpand-1 '(run (cc ,file ,@flags)))
(RUN-FUNC (QUASIQUOTE (CC (UNQUOTE FILE) (UNQUOTE-SPLICING FLAGS))))
T
SSBR> (macroexpand-1 '(QUASIQUOTE (CC (UNQUOTE FILE)
(UNQUOTE-SPLICING FLAGS))))
(APPEND (LIST 'CC) (LIST FILE) FLAGS)
T
No environment access needed!
-bcd
--
*** Brian Downing <bdowning at lavos dot net>
Brian Downing <·············@lavos.net> wrote:
+---------------
| Rob Warnock <····@rpw3.org> wrote:
| > Ooooh! Look at that loverly idiosyncratic environment! How is my
| > QUASIQUOTE macro expander going to extract the values of FILE & FLAGS
| > from *that* in a portable way??!?!?
| >
| > [Cue Duane Rettig, waiting in the wings for an encore presentation
| > of his Environments Access module... ;-} ;-} ]
|
| I don't think you need to. You're outputting a macroexpander. It will
| get evaluated in the lexical context of the let, so you just need to
| reference the variables....
...
| If you had a QUASIQUOTE expander that'd just be putting a QUASIQUOTE
| around the command form.
...
| (defmacro quasiquote (args)
| (process-qq args))
+---------------
Good point. You do need to completely macroexpand the effective
or implicit (RUN macro generated) QUASIQUOTE form & subforms before
emitting any function calls, but that said, it *is* possible.[1]
Good to know, thanks!
"CLsh", anyone...? ;-}
-Rob
[1] Well, one still has to usurp the built-in readmacros for `/,/,@
with something like Alexander Schmolck's code [and make sure
that doesn't break something else!], but at least the simple
cases can be made to work.
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
In article <······················@speakeasy.net>,
Rob Warnock <····@rpw3.org> wrote:
> Good point. You do need to completely macroexpand the effective
> or implicit (RUN macro generated) QUASIQUOTE form & subforms before
> emitting any function calls, but that said, it *is* possible.[1]
I'm not sure what you mean here. As long as you don't try to pass your
created QUASIQUOTE form to another macro that messes it up, the normal
course of macroexpansion should take care of everything (as it will of
course happen before any lexical variable references are evaluated)...
-bcd
--
*** Brian Downing <bdowning at lavos dot net>
Brian Downing <·············@lavos.net> wrote:
+---------------
| Rob Warnock <····@rpw3.org> wrote:
| > Good point. You do need to completely macroexpand the effective
| > or implicit (RUN macro generated) QUASIQUOTE form & subforms before
| > emitting any function calls, but that said, it *is* possible.[1]
|
| I'm not sure what you mean here. As long as you don't try to pass your
| created QUASIQUOTE form to another macro that messes it up, the normal
| course of macroexpansion should take care of everything (as it will of
| course happen before any lexical variable references are evaluated)...
+---------------
Sorry if I was confusing. All I meant was that before you can hand
the processing of the form off to some normal function (DEFUN-defined),
the chain of macroexpansion had to continue up to at least the point
where CL code had been generated that *used* [rather than just named]
all of the lexical variables involved in any of the unquotes.
Which what you just said, except that the desired end result *is* to
eventually pass the created QUASIQUOTE form off to an external function
for further processing [in the CLsh case, running a shell command].
It's just that the *values* of any lexical variables need to have
been made visible to CL code that will be executed before (or during)
that hand-off. E.g., this:
(let ((file "some.file")
(ccflags '(-O))
(ldflags '(-lm -lbsd)))
(run (cc ,@ccflags ,file ,@ldflags)))
should probably expand to something like this [assuming no
optimization of QUASIQUOTE whatsoever!!], where RUN-PROCESS
is a normal function:
(let ((file "some.file")
(ccflags '(-O))
(ldflags '(-lm -lbsd)))
(run-process (cons 'cc
(append ccflags
(cons file
(append ldflags nil))))))
That's all.
Are we violently agreeing now? ;-}
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
In article <································@speakeasy.net>,
Rob Warnock <····@rpw3.org> wrote:
> Which what you just said, except that the desired end result *is* to
> eventually pass the created QUASIQUOTE form off to an external function
> for further processing [in the CLsh case, running a shell command].
> It's just that the *values* of any lexical variables need to have
> been made visible to CL code that will be executed before (or during)
> that hand-off. E.g., this:
>
> should probably expand to something like this [assuming no
> optimization of QUASIQUOTE whatsoever!!], where RUN-PROCESS
> is a normal function:
>
> That's all.
>
> Are we violently agreeing now? ;-}
Yeah, something like that. :)
I just think you'd have to be intentionally perverse to /not/ have the
above happen.
-bcd
--
*** Brian Downing <bdowning at lavos dot net>
····@rpw3.org (Rob Warnock) writes:
> Note that for a real "CLsh", you'd probably want to run with
> readtable-case :INVERT to avoid upcasing the Unix commands. ;-} ;-}
Oh, I'd want that anyway (given the current absence of sane case handling in
most CLs. Well at least it has warded off CamelCase).
> Unfortunately, having usurped backquote/comma/comma-at from READ,
> one is now stuck needing to do one's own full backquote expansion,
> which includes all the gore of handling lexical environments as well.
> That is, you get this:
I can't really follow you (this might be my fault though). I think the worst
case scenario would be that you grab one of several freely available backquote
readers and trivially modify it not to barf on supercilious unquotes. I don't
think you need to handle environments in any way at all -- you just need to to
macrolet the UNQUOTE appropriately in your RUN macro.
> Anyway, like I said before (too many posts ago), doing a "CLsh" that
> used "pure" Scsh syntax would be quite hard.
I'm not sure why scsh's syntax would be that desirable anyway, see below.
> It might be better to just give up on trying to achieve the "pure" syntax
> and accept that we would type the extra backquote character in all of the
> run-process convenience forms. Then it "just works", right out of the box:
>
> > (defun run (form)
> (format t "RUN saw this form: ~s~%" form))
> > (let ((file "some.file")
> (flags '(-lm -lbsd)))
> (run `(cc ,file ,@flags)))
> RUN saw this form: (CC "some.file" -LM -LBSD)
> NIL
> >
>
> +---------------
> | (you don't need backquote to accept additional commas, right?)?
> +---------------
>
> What do you mean by "additional commas"? Nesting? Of course!
> It needs to work *just* like normal CL backquote, otherwise
> your "shell" syntax won't be "CL".
What I meant is: is `,,foo supposed to work (i.e. return an unquote form)? If
so, why? Personally I compleltely fail to see the point of
(run (ls ,@flags ,dir))
as opposed to
(run `(ls ,@flags ,dir))
It saves you between -1 and 1 keystrokes (-1 if you bind ` to insert `(_),
where _ is cursor position, which would seem a very reasonable thing to do
(the negligible cost is needing to type C-q `,foo to write `,foo)), but
greatly complicates reading nested backquote expressions.
You no longer can predict what some unquote will do at all unless you know
*all* the implicitly backquoting forms around it and even then it will be much
harder to parse visually; you can't just do a quick C-r ` or C-s ` to navigate
nesting levels either and on top of that you loose early error recognition --
not a good tradeoff in my book. So at the very least for writing macros I'd
want to use a non-backquoting variant of run.
Now I *could* see the point of
,run ls ,@flags ,dir
or even
, ls ,@flags ,dir
in, e.g. the slime-repl because that *is* considerable less hassle to type. In
fact I'd rather like to see something like this (i'd prefer "sys" to "run",
though) -- having the ability to quickly execute and also capture the output
of shell commands would be *really* convenient to my mind (it certainly is in
ipython, where you've got 3 options: just run the command; run and capture
output as string; run and capture output as list of lines; in an ideal world,
of course, you would want to capture not strings but *typed* output, i.e. a
list of pathnames in the above example -- either by implementing pluggable
parsers for common shell commands or just providing the most important shell
functionality (ls, cp, kill etc.) directly in lisp, with a nicer lispy
interface).
Of course this functionality also ought to be available as a library, so that
it can easiliy be transferred to real CL code. As for the *library* syntax, I
think there are two options: messing with the readtable to allow using the
same syntax or the repl just automatically printing a non-shorthand notation
rewrite on issuing such commands (that can then by copied and pasted into real
code).
I'd prefer the second approach.
'as
Alexander Schmolck <··········@gmail.com> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > Note that for a real "CLsh", you'd probably want to run with
| > readtable-case :INVERT to avoid upcasing the Unix commands. ;-} ;-}
|
| Oh, I'd want that anyway (given the current absence of sane case
| handling in most CLs. Well at least it has warded off CamelCase).
+---------------
Well, except, readtable-case :INVERT *preserves* CamelCase, you know... ;-}
+---------------
| > Unfortunately, having usurped backquote/comma/comma-at from READ,
| > one is now stuck needing to do one's own full backquote expansion,
| > which includes all the gore of handling lexical environments as well.
|
| I can't really follow you...
+---------------
Brian Downing later showed that I was incorrect about that point.
Please excuse the confusion.
+---------------
| > Anyway, like I said before (too many posts ago), doing a "CLsh" that
| > used "pure" Scsh syntax would be quite hard.
|
| I'm not sure why scsh's syntax would be that desirable anyway, see below.
+---------------
Because my original question was *NOT* what a "better" syntax would be,
but whether CL *could* duplicate Scsh's exact macro syntax [at least the
basics, where the Scheme/Common Lisp differences don't show up much].
+---------------
| > +---------------
| > | (you don't need backquote to accept additional commas, right?)?
| > +---------------
| >
| > What do you mean by "additional commas"? Nesting? Of course!
| > It needs to work *just* like normal CL backquote, otherwise
| > your "shell" syntax won't be "CL".
|
| What I meant is: is `,,foo supposed to work (i.e. return an unquote form)?
| If so, why?
+---------------
As to "why", I'd say that it has to only if Scsh's syntax needs it,
so CLsh can copy it (see below).
As to *what* it's supposed to return, that would be the same as what
Scheme's READ returns, namely (QUASIQUOTE (UNQUOTE (UNQUOTE FOO))),
which if evaluated *by itself* might produce an error, but inside
a macro which provides "implicit backquoting" [see below] would
produce the value of FOO, as you'd expect.
+---------------
| > It might be better to just give up on trying to achieve the "pure"
| > syntax and accept that we would type the extra backquote character
| > in all of the run-process convenience forms. Then it "just works",
| > right out of the box:
| > > (defun run (form)
| > (format t "RUN saw this form: ~s~%" form))
| > > (let ((file "some.file")
| > (flags '(-lm -lbsd)))
| > (run `(cc ,file ,@flags)))
| > RUN saw this form: (CC "some.file" -LM -LBSD)
| > NIL
| > >
| ...
| Personally I compleltely fail to see the point of
| (run (ls ,@flags ,dir))
| as opposed to
| (run `(ls ,@flags ,dir))
+---------------
Yes, well, in the occasional hacks I've taken at doing a "CLsh" from
time to time, I *have* used the latter form. But as I said earlier,
I was just curious whether the former, which is what Scsh accepts
[what it *requires*, actually!], would be possible in CL. I now
believe it would be possible, provided that one replaces the built-in
backquote processing with your own readmacros.
Or said another way: This is not an issue of deciding from scratch
what would be a "good" syntax for a Lisp-based shell; it's the case
that there already *is* a "Scheme Shell" a.k.a. "Scsh", based on
Scheme-48, that has a long history of use and publication and idiom
[see <http://www.scsh.net/> for more details]. If one were to try
to create a "CLsh" and wished to benefit from the experience of the
Scsh community, it would behoove one to attempt to match the "process
notation for running programs and setting up pipelines and redirections"
as closely as possible. And the "implicit backquoting" within process
macros that allows the (run (ls ,@flags ,dir)) form is an explicit part
of that notation, see <http://www.scsh.net/docu/html/man-Z-H-3.html>.
I hope that answers the "why" part...
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock wrote:
> In CL perhaps [though even there the situation is a little more
> complicated than you imply -- it is a stand-in for construction
> of a *code* sequence that, when evaluated, will perform an explicit
> list construction], but in Scheme the separation between the reader
> and the evaluator is complete: *All* the reader does is change the
> `,,@ markers to QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING forms
> (respectively); it is the evaluator which "executes" the resulting
> QUASIQUOTE forms. And in particular, even though R5RS warns that:
I'm wondering: could one create reader-macros to do the conversion as in
Scheme, and implement QUASIQUOTE as a macro? I don't have the time or
interest to do so right now, but in theory...
Since macros are (if I'm correct) expanded from outermost to innermost,
that would even work, and macros could accept code with UNQUOTEs in it,
or manipulate structures like that.
Finally, the QUASIQUOTE macro would expand its arguments accordingly
(while looking for the , and ,@).
--
Suffering from Gates-induced brain leakage...
Ulrich Hobelmann <···········@web.de> wrote:
+---------------
| Rob Warnock wrote:
| > In CL perhaps [though even there the situation is a little more
| > complicated than you imply -- it is a stand-in for construction
| > of a *code* sequence that, when evaluated, will perform an explicit
| > list construction], but in Scheme the separation between the reader
| > and the evaluator is complete: *All* the reader does is change the
| > `,,@ markers to QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING forms
| > (respectively); it is the evaluator which "executes" the resulting
| > QUASIQUOTE forms. And in particular, even though R5RS warns that:
|
| I'm wondering: could one create reader-macros to do the conversion as in
| Scheme, and implement QUASIQUOTE as a macro? I don't have the time or
| interest to do so right now, but in theory...
+---------------
In theory, yes. In fact, one could probably "simply" port the
QUASIQUOTE macro from some Scheme implementation into Common Lisp.
But it might be tricky to preserve all of the original semantics
of the native backquote implementation, especially any performance
optimization tricks they might have done. That is, one of the
*advantages* of conflating the roles of READ & EVAL in backquote
handling in CL is that special cases can be parsed "up front",
when the expression is first read, and not have to wait for when
QUASIQUOTE is expanded later when the optimization might not be
so obvious or available. E.g., when READ sees ",@" it's preseumably
in the middle of reading a list, and it can stop right there and
recursively read the *rest* of the list [pre-empting the list-reading
that's already going on] and convert both the ",@" arg and the tail
of the list after it into a call to some kind of "append", e.g.,
as in CMUCL:
cmu> (let ((*print-pretty* nil))
(princ '`(a ,@(list 'b) c d e f)))
(LISP::BACKQ-CONS (QUOTE A)
(LISP::BACKQ-APPEND (LIST (QUOTE B)) (QUOTE (C D E F))))
`(A ,@(LIST 'B) C D E F)
cmu>
where LISP::BACKQ-CONS & LISP::BACKQ-APPEND are just alternate names
for CONS & APPEND to help the pretty-printer:
cmu> (LISP::BACKQ-CONS 1 2)
(1 . 2)
cmu> (LISP::BACKQ-APPEND '(1 2 3) '(4 5 6))
(1 2 3 4 5 6)
cmu>
Whereas a Scheme-style reader would *have* to return this:
(quasiquote a (unquote-splicing (list 'b) c d e f))
and then the QUASIQUOTE macro would have to re-parse the list into
equivalent executable code at macro-expansion time [which for an
interpreter might be on every use].
+---------------
| Since macros are (if I'm correct) expanded from outermost to innermost,
| that would even work, and macros could accept code with UNQUOTEs in it,
| or manipulate structures like that.
+---------------
Yes, but... It's pretty hairy to get right. This following URL
contains a discussion of some of the subtleties:
http://www.codecomments.com/message296922.html
BTW, I stumbled across the following naive implementation of
QUASIQUOTE as a syntax-rules macro in the Scheme FAQ. I'm not sure
whether it has any of the bugs mentioned in the previous reference:
http://community.schemewiki.org/?scheme-faq-language
...
(define-syntax quasiquote
(syntax-rules (unquote unquote-splicing quasiquote)
((_ (unquote form))
form)
((_ ((unquote-splicing form) . rest))
(append form (quasiquote rest)))
((_ (quasiquote form) . depth)
(list 'quasiquote (quasiquote form #f . depth)))
((_ (unquote form) x . depth)
(list 'unquote (quasiquote form . depth)))
((_ (unquote-splicing form) x . depth)
(list 'unquote-splicing (quasiquote form . depth)))
((_ (car . cdr) . depth)
(cons (quasiquote car . depth) (quasiquote cdr . depth)))
((_ #(elt ...) . depth)
(list->vector (quasiquote (elt ...) . depth)))
((_ atom . depth)
'atom)))
The expansion is not "optimal" - it does not exploit Scheme's
abilitity to represent constant lists and vectors as literals.
It is possible to write a macro that does that, but it is rather
long and hence not included here.
+---------------
| Finally, the QUASIQUOTE macro would expand its arguments accordingly
| (while looking for the , and ,@).
+---------------
You meant "while looking for any UNQUOTEs, UNQUOTE-SPLICINGs, or
additional QUASIQUOTEs", I hope, since all of the ` , ,@ characters
would be long gone before the QUASIQUOTE macro runs.
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock wrote:
> Yes, but... It's pretty hairy to get right. This following URL
> contains a discussion of some of the subtleties:
>
> http://www.codecomments.com/message296922.html
Hm, ok.
> +---------------
> | Finally, the QUASIQUOTE macro would expand its arguments accordingly
> | (while looking for the , and ,@).
> +---------------
>
> You meant "while looking for any UNQUOTEs, UNQUOTE-SPLICINGs, or
> additional QUASIQUOTEs", I hope, since all of the ` , ,@ characters
> would be long gone before the QUASIQUOTE macro runs.
Oh, I forgot about the nested quasiquotes, yes.
As for the , and ,@ : I just didn't want to spell them all out ;)
--
Suffering from Gates-induced brain leakage...
Rob Warnock wrote:
> But it might be tricky to preserve all of the original semantics
> of the native backquote implementation, especially any performance
> optimization tricks they might have done. That is, one of the
> *advantages* of conflating the roles of READ & EVAL in backquote
> handling in CL is that special cases can be parsed "up front",
> when the expression is first read, and not have to wait for when
> QUASIQUOTE is expanded later when the optimization might not be
> so obvious or available.
I strongly disagree with this. I can't think of backquote optimization
opportunities that are exclusively available at read time. The
following is certainly no example of it:
> E.g., when READ sees ",@" it's preseumably
> in the middle of reading a list, and it can stop right there and
A ,@ is not necessarily in a list. It can be preceded by a row of other
unquotes, e.g
,,@,@
> recursively read the *rest* of the list [pre-empting the list-reading
Ah but if you recursively read something, then, in a very real sense,
that something is no longer at read time! The recursive read for that
subexpression is already done. You have reduced it to a structure which
you take advantage of in the higher level read function. This isn't
surprising to us, but to a classical, Lisp-ignorant computer scientist,
that would be a big deal: it's tantamount to the lexical analyzer
looking at parse trees to do transformations. :)
> that's already going on] and convert both the ",@" arg and the tail
> of the list after it into a call to some kind of "append", e.g.,
This isn't really optimization, but a "kludgy sort of farting around".
:)
> as in CMUCL:
>
> cmu> (let ((*print-pretty* nil))
> (princ '`(a ,@(list 'b) c d e f)))
> (LISP::BACKQ-CONS (QUOTE A)
> (LISP::BACKQ-APPEND (LIST (QUOTE B)) (QUOTE (C D E F))))
> `(A ,@(LIST 'B) C D E F)
> cmu>
I can hardly believe that this is the final result of the backquote
processing.What we are seeing here is surely some strange intermediate
representation for the backquote.
The above backquote can be reduced all the way to:
'(A B C D E F)
Where will that be done by CMUCL? If BACKQ-CONS and BACKQ-APPEND are
just functions, will the above code be optimized somewhere so that it
folds to the literal '(A B C D E F)?
The CLISP backquote system does it right in the backquote macro:
(macroexpand-1 '`(a ,@(list 'b) c d e f))
--> '(A B C D E F)
That's right, an actual list literal pops out of the macro itself.
The backquote form looks like this. (Disabling *PRINT-PRETTY* won't
reveal the operators, but we can print the REST of the compound form):
(rest '`(a ,@(list 'b) c d e f))
--> ((A (SYSTEM::SPLICE (LIST 'B)) C D E F))
The CLISP printer will note that these unquotes and splices are
dangling outside of any backquote and not print them specially.
The operator in the first position is SYSTEM::BACKQUOTE, so the whole
thing is:
(SYSTEM::BACKQUOTE (A (SYSTEM::SPLICE (LIST 'B)) C D E F))
This is a much better intermediate representation than some kludgy
conses and appends! It takes less work to spin back into pretty-printed
form. The pretty printer just has to locally recognize these symbols.
(Though a bit of communication through special variables tells it not
to expand unquotes that are outside of a backquote). When the printer
sees a (SYSTEM::SPLICE X ...) it just has to validate the number of
arguments and then spit it out a ,@ and recurse into printing X.
Oh, by the way, CLISP's backquote optimizer can be turned off for
debugging:
(setf system::*backquote-optimize* nil)
(macroexpand-1 '`(a ,@(list 'b) c d e f))
--> (APPEND (LIST 'A) (LIST 'B) (LIST 'C) (LIST 'D) (LIST 'E) (LIST
'F))
Note that this coincides with what the HyperSpec calls for in its
abstract semantic description of what backquote does.
The optimizer uses this as the starting point, and analyzes for cases
that it can reduce. By doing it this way, we can separately render our
two concerns into code: getting the backquote right according to the
spec, and correct optimization.
In this case, it notes that there is an APPEND, all of whose elements
are LIST forms. This can be transformed by taking out all of the
arguments of the LIST forms and making one big LIST form:
(LIST 'A 'B 'C 'D 'E 'F)
Now, we can note that all of the arguments of the LIST form are
constant forms. This means that we can turn the whole thing into a
quoted list, in which the constant forms are reduced to their values:
'(A B C D E F)
> where LISP::BACKQ-CONS & LISP::BACKQ-APPEND are just alternate names
> for CONS & APPEND to help the pretty-printer:
One would hope that they also help the optimizer to generate better
code too! Because this BACKQ-CONS does not have to have the same
semantics as ordinary CONS. The ordinary CONS produces a cell which the
program is allowed to mutate. The cell coming out of the BACKQ-CONS can
be assumed to be immutable.
So when the CMUCL compiler encounters, for instance:
(BACKQ-CONS 'A (BACKQ-CONS 'B NIL))
it should be able to reduce that to
'(A B)
A compiler macro for BACK-CONS could do this, in fact. Whereas that
optimization would be invalid for the ordinary CONS. Even if a cell is
treated as immutable, and the optimizer can prove that, there is the
question that the program may break if two calls to CONS produce
objects that are EQ.
Ulrich Hobelmann wrote:
> I'm wondering: could one create reader-macros to do the conversion as in
> Scheme, and implement QUASIQUOTE as a macro? I don't have the time or
> interest to do so right now, but in theory...
>
> Since macros are (if I'm correct) expanded from outermost to innermost,
> that would even work, and macros could accept code with UNQUOTEs in it,
> or manipulate structures like that.
>
> Finally, the QUASIQUOTE macro would expand its arguments accordingly
> (while looking for the , and ,@).
Steele does something very similar to this in CLtL2
<http://tinyurl.com/94oyh>. As others have mentioned elsewhere in this
thread, one can not use the same syntax, but Steele just implements the
same functionality with an alternate syntax, using "$" for quasiquote
and "%" for comma.
From: Ulrich Hobelmann
Subject: Quotation und macro eval order (was: Re: Dynamic unquote ( , )?)
Date:
Message-ID: <44ujf0F41u0pU1@individual.net>
Rob Warnock wrote:
> Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote:
> +---------------
> | Ulrich Hobelmann <···········@web.de> writes:
> | > So it seems like the quasiquotation expands to some funny term with LIST*.
> |
> | The representation of quasiquotation is not standarized, only its
> | effect when executed. Different Lisp implementations use different
> | representations.
> +---------------
>
> Yup. This is one place where Scheme got it right, IMHO, by at least
> standardizing the name of the quasiquotation macro and the associated
> internal syntax markers -- QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING --
> that the reader transforms `,,@ into:
I agree. Do you know in what order Scheme applies quasiquote expansion
and macro expansion?
Are there reasons for applying macros last, instead of before quasiquote
expansion?
In my case, if the reader, as in Scheme, transformed the punctuation
merely into Lisp expressions, I could access those inside my macro, and
the problem would be solved.
In particular, I could use a lone , in the macro's argument, but inside
the macro it would be find a matching ` . That's what I in fact meant
with the "dynamic unquote" heading. Lisp merely complains, because the
reader tries to expand the quasiquote and comma first.
--
Suffering from Gates-induced brain leakage...
From: Rob Warnock
Subject: Re: Quotation und macro eval order (was: Re: Dynamic unquote ( , )?)
Date:
Message-ID: <28udnZmi4_y7snbeRVn-iQ@speakeasy.net>
Ulrich Hobelmann <···········@web.de> wrote:
+---------------
| Rob Warnock wrote:
| > Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote:
| > +---------------
| > | The representation of quasiquotation is not standarized, only its
| > | effect when executed. Different Lisp implementations use different
| > | representations.
| > +---------------
| >
| > Yup. This is one place where Scheme got it right, IMHO, by at least
| > standardizing the name of the quasiquotation macro and the associated
| > internal syntax markers -- QUASIQUOTE, UNQUOTE, and UNQUOTE-SPLICING --
| > that the reader transforms `,,@ into:
|
| I agree. Do you know in what order Scheme applies quasiquote expansion
| and macro expansion?
+---------------
As I said in my parallel reply to Eli, basically *all* a Scheme
reader does is change the ` , ,@ readmacros to QUASIQUOTE, UNQUOTE,
and UNQUOTE-SPLICING forms:
mz> (quote `(a ,b ,@(c d) e))
==> (quasiquote (a (unquote b) (unquote-splicing (c d)) e))
mz>
It is up to the evaluator which "executes" a QUASIQUOTE form
[effectively a macro call] to create the final list structure:
mz> (let ((b 'foo) (c iota) (d 4))
(quasiquote (a (unquote b) (unquote-splicing (c d)) e)))
==> (a foo 0 1 2 3 e)
mz>
+---------------
| Are there reasons for applying macros last, instead of before
| quasiquote expansion?
+---------------
QUASIQUOTE *is* just a macro [for all intents & purposes, albeit
R5RS calls it "syntax"], so it's evaluated (as above) in the
ordinary course of events just like any other macro.
+---------------
| In my case, if the reader, as in Scheme, transformed the punctuation
| merely into Lisp expressions, I could access those inside my macro,
| and the problem would be solved.
+---------------
Exactly. But in CL, unlike Scheme, "naked" comma and comma-at are
illegal, so you can't.
+---------------
| In particular, I could use a lone , in the macro's argument, but inside
| the macro it would be find a matching ` . That's what I in fact meant
| with the "dynamic unquote" heading.
+---------------
This is exactly the property that Scsh relies upon with its
"process-form" macros, e.g.:
(let ((foo "filename")) ; RUN is a Scsh macro that does what
(run (ls -l ,foo))) ; might be called "implicit quasiquoting".
(run (cc ,file ,@flags)) ; Compile FILE with FLAGS.
I have called it "implicit quasiquoting" because the RUN macro behaves
"as if" it were a QUASIQUOTE (plus some other stuff). As you note, the
RUN macro must traverse its arguments "the same as" QUASIQUOTE [or perhaps
build up an explicit QUASIQUOTE form internally and call EVAL].
In CL the backquote must be explicit, so in "CLsh" you'd have to
write it like this:
(let ((foo "filename"))
(run `(ls -l ,foo))) ; Note explicit backquote
(run `(cc ,file ,@flags)) ; (ditto)
+---------------
| Lisp merely complains, because the reader tries to expand the
| quasiquote and comma first.
+---------------
To say it another way, in CL the backquote/comma/comma-at readmacros
are defined only in terms of the *final* value after both reading and
evaluation; the intermediate form that the reader returns that EVAL sees
is never explicitly defined in the standard, and may (and *does*!) vary
wildly across implementations (or even *within* an implementation,
depending on the exact input presented to it). Thus the user cannot
depend upon the details of that intermediate form, and the reader can
legally refuse to handle any input that would result in an intermediate
form that the evaluator wouldn't accept -- even if a *user* macro that
consumed that intermediate form might find it quite acceptable!
Conversely, in Scheme the mapping from the reader input readmacros
to intermediate s-expr forms is explicitly defined in R5RS, each in
isolation from the others:
`form <==> (QUASIQUOTE form)
,form <==> (UNQUOTE form)
,@form <==> (UNQUOTE-SPLICING form)
The practical result is that you *can* define macros in Scheme which
"implicitly quasiquote" their arguments, and you *can't* do that in CL
[at least, not in the ANSI standard or any implementation I've tested].
Pity, really...
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock wrote:
> As I said in my parallel reply to Eli, basically *all* a Scheme
> reader does is change the ` , ,@ readmacros to QUASIQUOTE, UNQUOTE,
> and UNQUOTE-SPLICING forms:
>
> mz> (quote `(a ,b ,@(c d) e))
> ==> (quasiquote (a (unquote b) (unquote-splicing (c d)) e))
> mz>
I know that much.
> QUASIQUOTE *is* just a macro [for all intents & purposes, albeit
> R5RS calls it "syntax"], so it's evaluated (as above) in the
> ordinary course of events just like any other macro.
Ok. That answers my other post...
Remains to create short syntax for it, or to replace part of the Lisp
reader with reader-macros.
> The practical result is that you *can* define macros in Scheme which
> "implicitly quasiquote" their arguments, and you *can't* do that in CL
> [at least, not in the ANSI standard or any implementation I've tested].
>
> Pity, really...
Yes.
--
Suffering from Gates-induced brain leakage...
From: Joerg Hoehle
Subject: Re: Quotation und macro eval order (was: Re: Dynamic unquote ( , )?)
Date:
Message-ID: <ulkwfntnj.fsf@users.sourceforge.net>
····@rpw3.org (Rob Warnock) writes:
> The practical result is that you *can* define macros in Scheme which
> "implicitly quasiquote" their arguments, and you *can't* do that in CL
> [at least, not in the ANSI standard or any implementation I've tested].
Well, thanks to Kaz Kylheku's efforts a few years ago, the backquote
implementation of CLISP was changed. The internals are now documented
and useable, like in Scheme.
E.g. I wrote a compiler-macro optimizer which recognizes backquoted
forms. One has to be careful, though, because unquote can occur in
*any* subform.
Note also that I started a thread, a few years ago, where KM Pitman
responded that it's a bad idea to standardize those names, because
IIRC of the confusion that would arise: "now is this an explicit use
of QUASIQUOTE by the reader or is this symbol accidentally here?"
E.g. the topic (I said (quote uh)?) vs. (I said 'uh ?)
IMHO, a solution path would be to standardizse using a different
package than COMMON-LISP, not in USE-PACKAGE, so conflicts would not
be accidental with user code.
Regards,
Jorg Hohle
Telekom/T-Systems Technology Center
From: Rob Warnock
Subject: Re: Quotation und macro eval order (was: Re: Dynamic unquote ( , )?)
Date:
Message-ID: <RtednaqZZrO-KGzeRVn-og@speakeasy.net>
Joerg Hoehle <······@users.sourceforge.net> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > The practical result is that you *can* define macros in Scheme which
| > "implicitly quasiquote" their arguments, and you *can't* do that in CL
| > [at least, not in the ANSI standard or any implementation I've tested].
|
| Well, thanks to Kaz Kylheku's efforts a few years ago, the backquote
| implementation of CLISP was changed. The internals are now documented
| and useable, like in Scheme.
+---------------
Neat!
+---------------
| IMHO, a solution path would be to standardizse using a different
| package than COMMON-LISP, not in USE-PACKAGE, so conflicts would not
| be accidental with user code.
+---------------
I think that would be fine, as long that other package were also
standardized along with the names. And as far as I'm concerned,
SYSTEM::BACKQUOTE, SYSTEM::UNQUOTE, and SYSTEM::UNQUOTE-SPLICING
would be fine.
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607