Hi all,
I'm becoming familiar with CMUCL's code walker. The documentation is
essentially the source walk.lisp. Eric Marsden's post from last month has
been helpful in understanding how the code is accessed:
http://groups.google.com/groups?selm=wzi1y79tc75.fsf%40melbourne.laas.fr
I hope someone can help me overcome this issue of macro expansion. As a
test I'm merely replacing the symbol a with b in
(loop for x from 1 to 10 do (write a)):
(walker:walk-form
'(loop for x from 1 to 10 do (write a))
nil
(lambda (subform context env)
(declare (ignore context env))
(if (eq subform 'a) 'b subform)))
It results in this expansion:
(let ((x 1))
(declare (type real x))
(block nil
(tagbody
ansi-loop::next-loop
(write b)
(ansi-loop::loop-really-desetq x (1+ x))
(when (> x '10) (go ansi-loop::end-loop))
(go ansi-loop::next-loop)
ansi-loop::end-loop)))
Because the loop macro has been expanded the resulting code is unmaintainable.
walk-form-expand-macros-p is nil by default.
Since the code walker isn't supposed to expand macros when
walk-form-expand-macros-p is nil this appears to be a bug. Certainly the
code walker doesn't understand the loop template. But since loop is a macro it
shouldn't matter: it should leave the parent form alone when replacing the
symbol a with b.
Please let me know if I have misunderstood the interface and there is a
simple way around this. It doesn't appear that adding a loop template to
the code walker should be necessary--but if it is the code should make a
great foundation.
Thanks,
Adam
On Mon, 21 Oct 2002 15:03:46 +1300, "Adam Warner" <······@consulting.net.nz> said:
[...]
AW> Since the code walker isn't supposed to expand macros when
AW> walk-form-expand-macros-p is nil this appears to be a bug. Certainly the
AW> code walker doesn't understand the loop template. But since loop is a macro it
AW> shouldn't matter: it should leave the parent form alone when replacing the
AW> symbol a with b.
For a form that is not a function call, you seem to deny the code
walker both special knowledge about the form and permission to
macroexpand the form. What might it do, then, to find out which
exactly are the places where A is used as a variable?
---Vassil.
--
Non-googlable is googlable.
Hi Vassil Nikolov,
> For a form that is not a function call, you seem to deny the code walker
> both special knowledge about the form and permission to macroexpand the
> form. What might it do, then, to find out which exactly are the places
> where A is used as a variable?
If the walker expands all macros even when walk-form-expand-macros-p is
nil then the output it produces is of no use for later incorporation into
a program.
Try running this code. I have just renamed loop as custom-loop:
(walker:walk-form
'(custom-loop for x from 1 to 10 do (write a))
nil
(lambda (subform context env)
(declare (ignore context env))
(if (eq subform 'a) 'b subform)))
This is the output:
(custom-loop for x from 1 to 10 do (write b))
Can you see how the above output might be more useful in some
circumstances than:
(let ((x 1))
(declare (type real x))
(block nil
(tagbody
ansi-loop::next-loop
(write b)
(ansi-loop::loop-really-desetq x (1+ x))
(when (> x '10) (go ansi-loop::end-loop))
(go ansi-loop::next-loop)
ansi-loop::end-loop)))
Regards,
Adam
"Adam Warner" <······@consulting.net.nz> wrote in message
···································@consulting.net.nz...
> Hi Vassil Nikolov,
>
> > For a form that is not a function call, you seem to deny the code walker
> > both special knowledge about the form and permission to macroexpand the
> > form. What might it do, then, to find out which exactly are the places
> > where A is used as a variable?
>
> If the walker expands all macros even when walk-form-expand-macros-p is
> nil then the output it produces is of no use for later incorporation into
> a program.
But the same is true if it blindly calls the function with :eval context on all
subforms. It wouldn't be a code walker anymore.
It looks like you need something specific so will have to implement that
yourself.
--
Martin Simmons, Xanalys Software Tools
······@xanalys.com
rot13 to reply
On Tue, 22 Oct 2002 19:45:30 +1300, "Adam Warner" <······@consulting.net.nz> said:
AW> If the walker expands all macros even when walk-form-expand-macros-p is
AW> nil then the output it produces is of no use for later incorporation into
AW> a program.
If a code walker claims to produce output that is useful for
incorporation into a human-maintained program, that's a pretty big
claim...
AW> Try running this code. I have just renamed loop as custom-loop:
(walker:walk-form
'(custom-loop for x from 1 to 10 do (write a))
nil
(lambda (subform context env)
(declare (ignore context env))
(if (eq subform 'a) 'b subform)))
AW> This is the output:
(custom-loop for x from 1 to 10 do (write b))
One assumption here is that the variable being substituted will
never be named by any of the symbols FOR, FROM, etc. and I am not
sure this is a good assumption to make. By the way, what happens
if the form to be walked contains (QUOTE A)? Or if it contains a
nested form that introduces a local binding for A? Or it contains
a CHECK-TYPE-like macro where A is a type?
In other words, the above example using LOOP is one very special
case. If the cases that you are going to deal with are all that
special, maybe they are special enough so that something simpler
might suffice, maybe even something almost as simple as
SUBST---after all,
(subst 'a 'b '(custom-loop for x from 1 to 10 do (write a)))
produces the same result. (I am not seriously suggesting to use
SUBST, I just want to make the point that more realistic examples
need to be considered, or that maybe you don't need `the heavy
artillery.')
AW> Can you see how the above output might be more useful in some
AW> circumstances than:
(let ((x 1))
(declare (type real x))
(block nil
(tagbody
ansi-loop::next-loop
(write b)
(ansi-loop::loop-really-desetq x (1+ x))
(when (> x '10) (go ansi-loop::end-loop))
(go ansi-loop::next-loop)
ansi-loop::end-loop)))
Well, in any case a correct form is more useful than any nicely
looking but incorrect form (especially if it is hard to notice that
the form is incorrect).
---Vassil.
--
Non-googlable is googlable.
Hi Vassil Nikolov,
> For a form that is not a function call, you seem to deny the code
> walker both special knowledge about the form and permission to
> macroexpand the form. What might it do, then, to find out which
> exactly are the places where A is used as a variable?
By the way, the specific answer to your question is to treat an unknown
and unexpanded macro as if the list is a non-special form (i.e. treat
the first element like a function call and the remaining elements as
arguments). While not ideal in the case of the loop macro it would still
work in most cases where variable replacements such as for, from, in or do
were avoided (plus the replacement procedure could be made more
intelligent than the rudimentary one below):
* (walker:walk-form
'(custom-loop for x from 1 to 10 do (write x))
nil
(lambda (subform context env)
(declare (ignore context env))
(if (eq subform 'x) 'new-var subform)))
(custom-loop for new-var from 1 to 10 do (write new-var))
I have no wish to deny the code walker any special knowledge. I have the
reality that the code walker does not (appear to) understand loop. If the
objective is to transform code for later modification then the inline
expansion of unknown/custom macros renders the code unmaintainable.
If the code walker was just being used for temporary transformation of
code--where the original would continue to be used--then the behaviour of
expanding out all macros would be ideal as it provides for more robust
transformation of the code.
Regards,
Adam
On Tue, 22 Oct 2002 21:05:07 +1300, "Adam Warner" <······@consulting.net.nz> said:
AW> By the way, the specific answer to your question is to treat an unknown
AW> and unexpanded macro as if the list is a non-special form (i.e. treat
AW> the first element like a function call and the remaining elements as
AW> arguments).
That sounds like asking for trouble...
AW> While not ideal in the case of the loop macro it would still
AW> work in most cases where variable replacements such as for, from, in or do
AW> were avoided
Again, I don't think this is realistic enough, and it is not the
only problem.
AW> (plus the replacement procedure could be made more
AW> intelligent than the rudimentary one below):
* (walker:walk-form
'(custom-loop for x from 1 to 10 do (write x))
nil
(lambda (subform context env)
(declare (ignore context env))
(if (eq subform 'x) 'new-var subform)))
(custom-loop for new-var from 1 to 10 do (write new-var))
It will have to be *much* more intelligent: how about doing the same
substitution while walking
(custom-loop
for x from 1 to 10 do
(custom-loop for new-var from 11 to 20 do
(format t "~%~D ~D" x new-var)))
---Vassil.
--
Non-googlable is googlable.
Hi Vassil Nikolov,
> It will have to be *much* more intelligent: how about doing the same
> substitution while walking
>
> (custom-loop
> for x from 1 to 10 do
> (custom-loop for new-var from 11 to 20 do
> (format t "~%~D ~D" x new-var)))
While I understand your general point thanks Vassil it is just as easy to
substitute for x as before:
* (walker:walk-form
'(custom-loop
for x from 1 to 10 do
(custom-loop for new-var from 11 to 20 do
(format t "~%~D ~D" x new-var)))
nil
(lambda (subform context env)
(declare (ignore context env))
(if (eq subform 'x) 'different-x subform)))
(custom-loop for different-x from 1 to 10 do
(custom-loop for new-var from 11 to 20 do
(format t "~%~D ~D" different-x new-var)))
This works fine because the problem you specified conforms with treating
the unknown and unexpanded "macro" (custom-loop) as a non-special form.
You may have been trying to make a point that x was being replaced with a
chosen variable that also happens to exist (new-var). I'm only concerned
with reliably replacing x with another arbitrary symbol/form. If x was to
become new-var and new-var was to become something else then the first
step would be to transform new-var to a different symbol and then
transform x to new-var.
Regards,
Adam
On Wed, 23 Oct 2002 19:04:15 +1300, "Adam Warner" <······@consulting.net.nz> said:
[...]
AW> You may have been trying to make a point that x was being replaced with a
AW> chosen variable that also happens to exist (new-var).
My point is that in the general case, when one does such
substitutions, one does not know if the same name exists or not
(when arbitrary forms are to be handled).
And there are many other situations, too. Another example would be
encountering a LET-like macro, e.g. substituting A with B in the
form (CUSTOM-LET (A) (FOO A)). The naive substitution produces
(CUSTOM-LET (A) (FOO B)) which does not seem to be what is wanted.
Besides, if the form is to be treated as a function call, I don't
know how the second element of (CUSTOM-LET ((A 'BAR)) (FOO B))
would be treated.
AW> I'm only concerned with reliably replacing x with another
AW> arbitrary symbol/form.
If `reliably' means you know in advance there would be no conflicts
or any other problems, then something simpler than a code walker
would suffice. Otherwise, if arbitrary forms are to be handled
reliably, this becomes a fairly difficult task. Consider that even
Lisp implementations themselves often fail to provide the original
programmer-written source during debugging, if that source involves
macros, especially ones not defined by the language itself (that
might be implemented as special forms).
---Vassil.
--
Non-googlable is googlable.