From: Daniel Katz
Subject: Macrology and automated invocation of defclass?
Date: 
Message-ID: <m21wf671vc.fsf@Daniel-Katzs-Computer.local.i-did-not-set--mail-host-address--so-tickle-me>
Hi -

I'm trying to automate the creation of a bunch of classes and am
getting myself a bit confused.  To begin, I tried to write a function
which called defclass appropriately (i.e. with an appropriate name and
docstring) as follows:

(defun generate-instruction-class (name symbol)
  "Create a SYMBOL class corresponding to the NAME string.  This
should be called like (generate-instruction-class \"foo\" 'foo)."
  (let ((docstring (concatenate 'string "For standalone instruction: " name)))
    (defclass symbol (vm-instruction)
      ()
      (:documentation docstring))))

(where vm-instruction is a previously defined class) but the
compiler (SBCL 1.07) complained

; in: DEFUN GENERATE-INSTRUCTION-CLASS => DEFCLASS SYMBOL
;     DEFCLASS
; 
; caught ERROR:
;   (in macroexpansion of (DEFCLASS SYMBOL # ...))
;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
;   DOCSTRING is not a legal :documentation value

which I guess makes sense upon reading the hyperspec (which says that
the :documentation class option takes a string argument while not
saying anything about taking symbols or forms which evaluate to a
string).  So I then switched to a macro of the form

(defmacro generate-instruction-class (name symbol)
  `(defclass ,symbol (vm-instruction)
     ()
     (:documentation ,(concatenate 'string
				   "For standalone instruction: " name))))

which the compiler was happy with and which seemed perfectly fine
under inspection via macroexpand-1:

> (macroexpand-1 '(generate-instruction-class "foo" foo))
(DEFCLASS FOO (VM-INSTRUCTION) NIL
          (:DOCUMENTATION "For standalone instruction: foo"))
T


So now my first question: am I correct that generate-instruction-class
will need to be a macro in order to use constructed docstrings as I'm
trying to do here?  Or did I miss a better way to do this as a
function?




Anyway, once I had a working generate-instruction-class macro that I
could call, I ran into the same problem all over again.  I had a list
of string/symbol pairs I wanted to run it on:

(defparameter *my-classes*
  '(("fooname" foo)
    ("barname" bar)
    ("bazname" baz)))

and tried to approach the problem of automating the creation of these
classes via generate-instruction-class by using a function (note the
name pluralization to distinguishe from the previous macro):

(defun generate-instruction-classes ()
  (dolist (pair *my-classes* 'done)
    (generate-instruction-class (car pair) (cadr pair))))

which gives the compiler fits in the obvious way:

; in: DEFUN GENERATE-INSTRUCTION-CLASSES
;     GENERATE-INSTRUCTION-CLASS
; 
; caught ERROR:
;   (in macroexpansion of (GENERATE-INSTRUCTION-CLASS # #))
;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
;   The value CAR is not of type CHARACTER.

which again makes sense since generate-instruction-classes is
expecting the first argument to be a string for use in the docstring
rather than a form which will happen to return a string.  The second
argument will be similarly compromised, I think.  So instead I ended
up with another macro:

(defmacro generate-instruction-classes ()
  `(progn
     ,@(loop for pair in *my-classes*
	     collecting `(generate-instruction-class ,(car pair)
						     ,(cadr pair)))))

which expands to produce

> (macroexpand-1 '(generate-instruction-classes))
(PROGN
  (GENERATE-INSTRUCTION-CLASS "fooname" FOO)
  (GENERATE-INSTRUCTION-CLASS "barname" BAR)
  (GENERATE-INSTRUCTION-CLASS "bazname" BAZ))
T

and which does the job I'm trying to do, but which leads me to the
second question (in parallel to the first) of: was there was a better
way to do this without having to define generate-instruction-classes
as a macro?


Any insight or commentary would be appreciated.

Thanks!

Dan

P.S.  For the record, I don't really have any stylistic objections to
macros -- I just prefer functions if I can get away with them since I
find the function evaluation logic easier to remember when I'm trying
to write code...

From: Barry Margolin
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <barmar-EA4942.01183318072007@comcast.dca.giganews.com>
In article 
<··············@Daniel-Katzs-Computer.local.i-did-not-set--mail-host-add
ress--so-tickle-me>,
 Daniel Katz <······@yahoo.com> wrote:

> Hi -
> 
> I'm trying to automate the creation of a bunch of classes and am
> getting myself a bit confused.  To begin, I tried to write a function
> which called defclass appropriately (i.e. with an appropriate name and
> docstring) as follows:
> 
> (defun generate-instruction-class (name symbol)
>   "Create a SYMBOL class corresponding to the NAME string.  This
> should be called like (generate-instruction-class \"foo\" 'foo)."
>   (let ((docstring (concatenate 'string "For standalone instruction: " name)))
>     (defclass symbol (vm-instruction)
>       ()
>       (:documentation docstring))))
> 
> (where vm-instruction is a previously defined class) but the
> compiler (SBCL 1.07) complained
> 
> ; in: DEFUN GENERATE-INSTRUCTION-CLASS => DEFCLASS SYMBOL
> ;     DEFCLASS
> ; 
> ; caught ERROR:
> ;   (in macroexpansion of (DEFCLASS SYMBOL # ...))
> ;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
> ;   DOCSTRING is not a legal :documentation value
> 
> which I guess makes sense upon reading the hyperspec (which says that
> the :documentation class option takes a string argument while not
> saying anything about taking symbols or forms which evaluate to a
> string).  So I then switched to a macro of the form
> 
> (defmacro generate-instruction-class (name symbol)
>   `(defclass ,symbol (vm-instruction)
>      ()
>      (:documentation ,(concatenate 'string
> 				   "For standalone instruction: " name))))
> 
> which the compiler was happy with and which seemed perfectly fine
> under inspection via macroexpand-1:
> 
> > (macroexpand-1 '(generate-instruction-class "foo" foo))
> (DEFCLASS FOO (VM-INSTRUCTION) NIL
>           (:DOCUMENTATION "For standalone instruction: foo"))
> T
> 
> 
> So now my first question: am I correct that generate-instruction-class
> will need to be a macro in order to use constructed docstrings as I'm
> trying to do here?  Or did I miss a better way to do this as a
> function?

You could do it in a function either by calling EVAL or using the 
Meta-Object Protocol.  Here's the EVAL method:

(defun generate-instruction-class (name symbol)
  (eval `(defclass ,symbol (vm-instruction)
           ()
           (:documentation ,(concatenate 'string
              "For standalone instruction: " name))))

> Anyway, once I had a working generate-instruction-class macro that I
> could call, I ran into the same problem all over again.  I had a list
> of string/symbol pairs I wanted to run it on:
> 
> (defparameter *my-classes*
>   '(("fooname" foo)
>     ("barname" bar)
>     ("bazname" baz)))
> 
> and tried to approach the problem of automating the creation of these
> classes via generate-instruction-class by using a function (note the
> name pluralization to distinguishe from the previous macro):

The function solutions operate at run time rather than compile time, so 
they can accept computed data.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
From: Daniel Katz
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <m2tzrwftk1.fsf@Daniel-Katzs-Computer.local.i-did-not-set--mail-host-address--so-tickle-me>
>  Daniel Katz <······@yahoo.com> wrote:
>>
>> So I then switched to a macro of the form
>>
>> (defmacro generate-instruction-class (name symbol)
>>   `(defclass ,symbol (vm-instruction)
>>      ()
>>      (:documentation ,(concatenate 'string
>> 				   "For standalone instruction: " name))))
>>
>> which the compiler was happy with and which seemed perfectly fine
>> under inspection via macroexpand-1:
>>
>> > (macroexpand-1 '(generate-instruction-class "foo" foo))
>> (DEFCLASS FOO (VM-INSTRUCTION) NIL
>>           (:DOCUMENTATION "For standalone instruction: foo"))
>> T
>>
>>
>> So now my first question: am I correct that generate-instruction-class
>> will need to be a macro in order to use constructed docstrings as I'm
>> trying to do here?  Or did I miss a better way to do this as a
>> function?

And Barry Margolin replied:
>
> You could do it in a function either by calling EVAL or using the
> Meta-Object Protocol.  Here's the EVAL method:
>
> (defun generate-instruction-class (name symbol)
>   (eval `(defclass ,symbol (vm-instruction)
>            ()
>            (:documentation ,(concatenate 'string
>               "For standalone instruction: " name))))


Mmmm... I thought about this but then rejected it -- really for no
better reason than a kneejerk bias against calling "eval" directly.


This general problem (that calling a macro prevents you from using
forms as arguments and thus seems to force you to write the caller as
yet another macro) must come up from time to time.  Is this the
general idiom (i.e. use eval to break the macro-calling chain and get
you back to functions)?


Dan
From: Kjetil S. Matheussen
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <Pine.LNX.4.64.0707221634310.9781@ttleush>
On Sun, 22 Jul 2007, Daniel Katz wrote:
>>
>> You could do it in a function either by calling EVAL or using the
>> Meta-Object Protocol.  Here's the EVAL method:
>>
>> (defun generate-instruction-class (name symbol)
>>   (eval `(defclass ,symbol (vm-instruction)
>>            ()
>>            (:documentation ,(concatenate 'string
>>               "For standalone instruction: " name))))
>
>
> Mmmm... I thought about this but then rejected it -- really for no
> better reason than a kneejerk bias against calling "eval" directly.
>

Unless the generate-instruction-class function requires high performance, 
theres nothing wrong using eval in this situation. Its clean, does exactly 
what its supposed to do, and its easy to read. Therefore its probably also 
the right thing to do, I would guess.
From: Pascal Costanza
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <5gh9qoF3fka3aU1@mid.individual.net>
Daniel Katz wrote:

> This general problem (that calling a macro prevents you from using
> forms as arguments and thus seems to force you to write the caller as
> yet another macro) must come up from time to time.  Is this the
> general idiom (i.e. use eval to break the macro-calling chain and get
> you back to functions)?

I don't know the context of this discussion, but just about this 
specific question:

No, that's not the general idiom. A good macro is designed in such a way 
that there is an underlying functional abstraction to which the macro 
expands which is also part of the official API. This allows you to use 
the functional abstraction in your own macro expansions (or even as part 
of your regular code).

Unfortunately, CLOS as it is specified in ANSI Common Lisp doesn't 
provide the functional abstractions (or only a few of them) for the 
high-level macros. In that case, using eval is ok. (You could use the 
API provided by the CLOS MOP, but then your code is less portable.)


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Daniel Trstenjak
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <20070722161641.GC7205@linux.ver>
On Sun, Jul 22, 2007 at 05:01:43PM +0200, Pascal Costanza wrote:
> No, that's not the general idiom. A good macro is designed in such a way that there is an underlying
> functional abstraction to which the macro expands which is also part of the official API. This allows you to
> use the functional abstraction in your own macro expansions (or even as part of your regular code).

I can see the advantage of the functional abstraction when the
implementation of the macro has to be changed, no need for
recompling the code which uses the macro, are there other
ones?


Regards,
Daniel
From: Daniel Trstenjak
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <20070722162727.GA7600@linux.ver>
I wrongly replied directly to Pascal Costanza, so here the answer I got from him:
  

>On 22 Jul 2007, at 18:13, Daniel Trstenjak wrote:
>I can see the advantage of the functional abstraction when the
>implementation of the macro has to be changed, no need for
>recompling the code which uses the macro, are there other
>ones?

Yes. The most important one is that functional abstractions are easier to debug - you still see the function
invocations on the call stack, and debuggers allow you to inspect the call stack. A macro, on the other, is
expanded away at compile time, so when you debug, you will probably not be able to see anymore where some
specific code originated from.

So it's a good idea that macros are only thin layers on otherwise functional abstractions.

Another advantage of functions is that you can pass them around as first-class values. This is not possible
for macros (or doesn't make a lot of sense).


Cheers,
Pascal
From: Rob Warnock
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <EYSdnchxK9RvnTnbnZ2dnUVZ_uCinZ2d@speakeasy.net>
Daniel Trstenjak  <················@online.de> wrote:
+---------------
| Pascal Costanza wrote:
| > No, that's not the general idiom. A good macro is designed in such a
| > way that there is an underlying functional abstraction to which the
| > macro expands which is also part of the official API. This allows you
| > to use the functional abstraction in your own macro expansions
| > (or even as part of your regular code).
| 
| I can see the advantage of the functional abstraction when the
| implementation of the macro has to be changed, no need for
| recompling the code which uses the macro...
+---------------

Actually, this is *NOT* true!! You still need to recompile the code
to get the effect of any changes in the functional abstraction behind
the macro, since the macro (and hence the function behind it) is only
called during the macro-expansion phase of compilation, and is *not*
called at runtime! See the following for the official verbiage:

    http://www.alu.org/HyperSpec/Body/sec_3-2-2-2.html
    3.2.2.2 Minimal Compilation

Also note that while the macro function is forbidden to modify its
argument form in any way [and thus so-called "displacing" macroexpansion
is forbidden]:

    http://www.alu.org/HyperSpec/Body/sec_3-1-2-1-2-2.html
    3.1.2.1.2.2 Macro Forms

    http://www.alu.org/HyperSpec/Issues/iss301-writeup.html
    Issue SELF-MODIFYING-CODE Writeup

many implementations (CMUCL, CLISP, most others) get around that for
performance reasons by actually doing at least Minimal Compilation
of even "interpreted" code [REPL & LOADed source], so even there
changing the function behind a macro will require re-evaluating any
defining forms [DEFUN, DEFVAR, etc.] whose values contained a call
of that macro. Example:

    > (defmacro foo () 12)

    FOO
    > (defun bar () (foo))

    BAR
    > (compiled-function-p 'bar)

    NIL      ; Note: This would be T in SBCL
    > (bar)

    12
    > (defmacro foo () 17)

    FOO
    > (bar)

    12	     ; But 17 is also legal (albeit an inefficient implementation)
    > 

+---------------
| ... are there other ones?
+---------------

Yes. [Pascal answered this in his parallel reply.]


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Barry Margolin
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <barmar-1350C0.21423330072007@newsgroups.comcast.net>
In article <································@speakeasy.net>,
 ····@rpw3.org (Rob Warnock) wrote:

> Daniel Trstenjak  <················@online.de> wrote:
> +---------------
> | Pascal Costanza wrote:
> | > No, that's not the general idiom. A good macro is designed in such a
> | > way that there is an underlying functional abstraction to which the
> | > macro expands which is also part of the official API. This allows you
> | > to use the functional abstraction in your own macro expansions
> | > (or even as part of your regular code).
> | 
> | I can see the advantage of the functional abstraction when the
> | implementation of the macro has to be changed, no need for
> | recompling the code which uses the macro...
> +---------------
> 
> Actually, this is *NOT* true!! You still need to recompile the code
> to get the effect of any changes in the functional abstraction behind
> the macro, since the macro (and hence the function behind it) is only
> called during the macro-expansion phase of compilation, and is *not*
> called at runtime! See the following for the official verbiage:

It depends on how the macro is written.  It's possible to design some 
macros so that they simply expand into a call to a function, and that 
function can be redefined at run time.

I remember that many macros in the Symbolics table and presentation 
system were written something like this:

(defmacro with-table ((stream &rest options) &body body)
  `(with-table-internal
     #'(lambda (,stream) ,@body)
     ,@options))

The macro is so trivial that it virtually never needs to be redefined.  
All the real work takes place in the function WITH-TABLE-INTERNAL, which 
can be redefined on the fly.

Almost any macro could conceivably be rewritten this way, although some 
cases would require a good optimizer, e.g.:

(defmacro if (test then &optional else)
  `(if-internal #'(lambda () ,test)
                #'(lambda () ,then)
                #'(lambda () ,else)))
(declaim (inline test-internal))
(defun if-internal (test-fun then-fun else-fun)
  (cond ((funcall test-fun) (funcall then-fun))
        (t (funcall else-fun))))

You wouldn't want to use this macro unless the compiler detected that 
the anonymous functions are temporary and the FUNCALLs could be elided.  
Of course, if the call to IF-INTERNAL is expanded inline, you then have 
the same redefinition problem as macros.

And doing a macro like LOOP this way would effectively mean that all the 
syntax parsing would be done at run time, which you probably wouldn't 
want.  So not all macros are amenable to this style.  It's really only 
useful for cases like Symbolics did, where the syntax is simple and the 
code it invokes has high run-time complexity, so that the function call 
overhead is negligible.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
From: Rob Warnock
Subject: Re: Macrology and automated invocation of defclass?
Date: 
Message-ID: <HY2dnRafmrljYTPbnZ2dnUVZ_sKqnZ2d@speakeasy.net>
Barry Margolin  <······@alum.mit.edu> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) wrote:
| > Actually, this is *NOT* true!! You still need to recompile the code
| > to get the effect of any changes in the functional abstraction behind
| > the macro, since the macro (and hence the function behind it) is only
| > called during the macro-expansion phase of compilation, and is *not*
| > called at runtime! See the following for the official verbiage: ...
| 
| It depends on how the macro is written.  It's possible to design some 
| macros so that they simply expand into a call to a function, and that 
| function can be redefined at run time.
| 
| I remember that many macros in the Symbolics table and presentation 
| system were written something like this:
| 
| (defmacro with-table ((stream &rest options) &body body)
|   `(with-table-internal
|      #'(lambda (,stream) ,@body)
|      ,@options))
| 
| The macro is so trivial that it virtually never needs to be redefined.  
| All the real work takes place in the function WITH-TABLE-INTERNAL, which 
| can be redefined on the fly.
+---------------

Sure, many "WITH"-style macros are of this style, and I've written
plenty of those myself. One set in particular was for wrapping a
"container" HTML web page around some locally-specified content.
E.g., a call to:

    (with-normal-results-page (title)
      ...[some HTOUT or CL-WHO code]...)

expanded into:

    (normal-results-page (http-request-stream *http-request*)
			 (http-request-self *http-request*)
			 title
			 (lambda (s)
			   (with-html-output (s s t)
			     ...[some HTOUT or CL-WHO code]...)))

Clearly one could change the layout of the header/footer/sidebars of
such pages by redefining the NORMAL-RESULTS-PAGE *without* recompling
ot reloading any code containing uses of WITH-NORMAL-RESULTS-PAGE.
So my comment [to which you replied] was overly general. Mea culpa.

But I was mainly trying to warn about those macros that do source
code *rewriting*, for which the "underlying function" is the rewriting
algorithm itself, and which are usually of the form:

    (defmacro my-macro (macro-args...)
      (my-expansion-function macro-args...))

or possibly with some very simple rearranging of the MACRO-ARGS before
calling the expansion function.

Those macros do indeed require that any binding forms calling them
be recompiled [or, for those interpreters that always do at least
ANSI CL "minimal compilation", reloaded] when the expansion function
is redefined.


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607