From: Doug Philips
Subject: macro question...
Date: 
Message-ID: <3EA29C30.3050507@mac.com>
I think I know the answer to this (hmmm, that petard has a darn short 
fuse!), but figured asking for confirmation/clarification couldn't hurt...

Macros... tres cool way to run code at compile time to alter the code 
that the compiler is going to see.

The context a macro runs in is not the context of the code that it 
returns will be compiled/eval'ed in.

What if I want to return code from a macro and simultaneously be able to 
do some bookkeepping that will let me find that code, in its context, 
later, maybe putting it into a hash or a list... Is that just something 
that screams "You don't really have continuations!" or is it more subtle 
than that?

Macros called from the toplevel are really different from macros called 
from elsewhere. They can return code that will be invoked "right away" 
and so some bookkeepping can be done as a side effect of the 
eval/compile that is happening "at the same time". But that doesn't work 
when the macro is invoked in some other context (say a defun).

-D'gou

If a little knowledge is dangerous, how worse is 90% knowledge? ;-)

From: Kenny Tilton
Subject: Re: macro question...
Date: 
Message-ID: <3EA2BF20.1020705@nyc.rr.com>
Doug Philips wrote:
> Macros called from the toplevel are really different from macros called 
> from elsewhere. They can return code that will be invoked "right away" 

But that is not about macros, that is about being at the top-level: if I 
place (print "cells rule!") at the top-level it runs right away, too.

> and so some bookkeepping can be done as a side effect of the 
> eval/compile that is happening "at the same time". But that doesn't work 
> when the macro is invoked in some other context (say a defun).

You can still do book-keeping at runtime, just include the bookkeeping 
in the expansion (aka "returned code"):

(defmacro c? (&body body)
   `(make-c-dependent
     :code ',body ;; <== this be the bookkeeping
     :rule (lambda (c &aux (self (c-model c)))
               (declare (ignorable self c))
               ,@body)))

Above I grab the code (as a tree of symbols) and stuff it (for possible 
later debugging inspection of source) into a slot in the c-dependent 
struct, as well as returning it as the code of the rule for the compiler 
to munch on.

So there are two ways to go [untested]:

(defmacro def-this (&body cell-code)
     (record-source cell-code) ;; pass in, at this point, the symbol tree
     `(progn ,@cell-code))

or:

(defmacro def-this (&body cell-code)
     `(progn
        (record-source ',cell-code) ;; quote so compiler sees symbols
        ,@cell-code))

I won't bring up eval-when unless you do. :)

-- 

  kenny tilton
  clinisys, inc
  http://www.tilton-technology.com/
  ---------------------------------------------------------------
"Everything is a cell." -- Alan Kay
From: Tim Bradshaw
Subject: Re: macro question...
Date: 
Message-ID: <ey3el3xnsbu.fsf@cley.com>
* Doug Philips wrote:

> What if I want to return code from a macro and simultaneously be able
> to do some bookkeepping that will let me find that code, in its
> context, later, maybe putting it into a hash or a list... Is that just
> something that screams "You don't really have continuations!" or is it
> more subtle than that?

Just store stuff in a hash table or wherever at macroexpansion time,
or if you want the information to be stored only at runtime arrange
that the expansion stores the information.

--tim
From: Peter Seibel
Subject: Re: macro question...
Date: 
Message-ID: <m3fzod85te.fsf@javamonkey.com>
Tim Bradshaw <···@cley.com> writes:

> * Doug Philips wrote:
> 
> > What if I want to return code from a macro and simultaneously be able
> > to do some bookkeepping that will let me find that code, in its
> > context, later, maybe putting it into a hash or a list... Is that just
> > something that screams "You don't really have continuations!" or is it
> > more subtle than that?
> 
> Just store stuff in a hash table or wherever at macroexpansion time,
> or if you want the information to be stored only at runtime arrange
> that the expansion stores the information.

You'll probably want to bang your head against section 3.2.3.1 of the
CLHS for a while too in order to get straight on how to do somethings
at compile time, and others at runtime, or both.

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

  The intellectual level needed   for  system design is  in  general
  grossly  underestimated. I am  convinced  more than ever that this
  type of work is very difficult and that every effort to do it with
  other than the best people is doomed to either failure or moderate
  success at enormous expense. --Edsger Dijkstra
From: Doug Philips
Subject: Re: macro question...
Date: 
Message-ID: <3EA3148D.1030108@mac.com>
I see that I wasn't as clear as I thought I meant to be!

What I should have asked was, how do I get this:


(defun blah ()
    ; boring stuff here...
    (invokeMacro "tag" (fun1 arg1 arg2) (fun2 arg3 arg4 arg 5 ) ;...
    )
    ; boring stuff here...
)

What I want to have 'invokeMacro' do is both:
      a) return its arguments (except for the 1st one).... as if it 
never existed, like a ghost...

AND

      b) stash a closure of the code that is its arguments (except for 
the 1st one) off somewhere in a list or hash table that I can get to later.

If I whap the args into a lambda and eval or compile or coerce it, I 
lose the lexical scoping, so the arguments can't refer to anything 
local. :-(

Its looking like I'll have to leave behind a 'thunk' that'll register 
the closure at run time (but then I don't know it exists :-( until much 
later (if ever))... Plus I need a gensym to capture the closure and 
refer to it, as well as to invoke it...

I can't help thinking there is an easier way...

Thanks for all the responses so far, I'm amazed for a holiday weekend!

-D'gou
From: Peter Seibel
Subject: Re: macro question...
Date: 
Message-ID: <m3u1cs7pde.fsf@javamonkey.com>
Doug Philips <····@mac.com> writes:

> I see that I wasn't as clear as I thought I meant to be!
> 
> What I should have asked was, how do I get this:
> 
> 
> (defun blah ()
>     ; boring stuff here...
>     (invokeMacro "tag" (fun1 arg1 arg2) (fun2 arg3 arg4 arg 5 ) ;...
>     )
>     ; boring stuff here...
> )
> 
> What I want to have 'invokeMacro' do is both:
>       a) return its arguments (except for the 1st one).... as if it
>       never existed, like a ghost...

> AND
> 
>       b) stash a closure of the code that is its arguments (except for
>       the 1st one) off somewhere in a list or hash table that I can
>       get to later.
> 
> If I whap the args into a lambda and eval or compile or coerce it, I
> lose the lexical scoping, so the arguments can't refer to anything
> local. :-(

If you want to refer to runtime state you have to wait until runtime
to make the closure. In that case all you need to do is something like
this:

    (defvar *captured-code* (make-hash-table :test #'equal))

    (defmacro invoke-macro (tag &body body)
      `(progn
        (setf (gethash ,tag *captured-code*) #'(lambda () ,@body))
        ,@body))

    (defun get-capturede-code (tag)
      (gethash tag *captured-code*))

If you want to use this macro in the file where it is declared you'll
need to wrap the DEFVAR of *captured-code* in an EVAL-WHEN:

    (eval-when (:compile-toplevel :load-toplevel :execute)
      (defvar *captured-code* (make-hash-table :test #'equal)))


> Its looking like I'll have to leave behind a 'thunk' that'll register
> the closure at run time (but then I don't know it exists :-( until
> much later (if ever))... Plus I need a gensym to capture the closure
> and refer to it, as well as to invoke it...

I'm not clear on what exactly you're trying to do. Worse yet--I have a
small suspicion that you aren't either. The scheme I showed above will
replace the closure under a given tag everytime the code that was
generated from invoke-macro is run. Which seems to not be what you
want. But you do want a closure that refers to runtime state so you
don't have much choice. Maybe if you pop up a level and tell us what
you are really trying to do (i.e. why do you think you need a macro
that behaves this way) and we might be able to help you out with your
actual problem.

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

  The intellectual level needed   for  system design is  in  general
  grossly  underestimated. I am  convinced  more than ever that this
  type of work is very difficult and that every effort to do it with
  other than the best people is doomed to either failure or moderate
  success at enormous expense. --Edsger Dijkstra
From: Doug Philips
Subject: Re: macro question...
Date: 
Message-ID: <3EA35A84.10304@mac.com>
Peter Seibel indited:
> If you want to refer to runtime state you have to wait until runtime
> to make the closure. In that case all you need to do is something like
> this:

...thanks for the example (elided)...

> I'm not clear on what exactly you're trying to do. Worse yet--I have a
> small suspicion that you aren't either. The scheme I showed above will
> replace the closure under a given tag everytime the code that was
> generated from invoke-macro is run. Which seems to not be what you
> want. But you do want a closure that refers to runtime state so you
> don't have much choice. Maybe if you pop up a level and tell us what
> you are really trying to do (i.e. why do you think you need a macro
> that behaves this way) and we might be able to help you out with your
> actual problem.

I started with code for doing tracing and debugging. But then as I tried 
to simplify it, I realized that what I was trying to do was:
    (other stuff
      ...
      ... (eval `(lambda () ,@body))
      ...

but that wasn't doing "the right thing" either. ;-)

I guess it just "hit me" that while a macro can run at compile or eval 
time, its not really aware of what's going on around it. The code you 
showed is going to evaluate a lambda every time through, but what will 
be different is not the code in the lambda, but the environment around it.

What I'm trying to do is be able to add debugging code that I can invoke 
at will later (not really like a handler or a restart though). But 
without a concrete example (I don't have one yet), its hard to 
express... By keeping track of all the "debugging points" it would be 
easy to map/loop over all of them and a get snap shot of the entire 
system (or as much has been instrumented). Probably not "The Lisp Way", 
but much faster than using the debugger to move around (and besides, 
there is no way from the debugger to look back in time to a stack that 
isn't there anymore, whereas the closures do capture that)...

hmmmmm.....

-D'gou
From: Rob Warnock
Subject: Re: macro question...
Date: 
Message-ID: <RLadnUlyAfrfuzijXTWc-g@speakeasy.net>
Peter Seibel  <·····@javamonkey.com> wrote:
+---------------
| Doug Philips <····@mac.com> writes:
| > What I want to have 'invokeMacro' do is both:
| >       a) return its arguments (except for the 1st one)....
| > AND   b) stash a closure of the code that is its arguments...
...
|     (defmacro invoke-macro (tag &body body)
|       `(progn
|         (setf (gethash ,tag *captured-code*) #'(lambda () ,@body))
|         ,@body))
+---------------

If the bodies can be very large, rather than duplicating the code
one might prefer to do something like this:

	(defmacro invoke-macro (tag &body body)
	  (let ((thunk (gensym)))
	    `(flet ((,thunk () ,@body))
	       (setf (gethash ,tag *captured-code*) #',thunk)
	       (,thunk))))

But in any case, be aware that each time control passes through
(the expansion of) INVOKE-MACRO you're going to overwrite the
save-away code with a new version, so if you do this inside of
a loop you will only save the closure over the last iteration
of the body.


-Rob

-----
Rob Warnock, PP-ASEL-IA		<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607