From: Andreas Thiele
Subject: MOP/Macroexpansion
Date: 
Message-ID: <d7cb1h$8f9$04$1@news.t-online.com>
Hi,

the following code snippet works in my application if def-class is used at
top-level:

(defmacro def-methods (name)
  `(progn ,@(class-methods name)))

(defmacro def-class (name parents &body body)
  `(progn
     (defclass ,name ,(if parents parents '(db-root))
       ,@(transform-source name body)
       (:metaclass db-class))
     (def-methods ,name)))

class-methods computes a list of defmethod forms and needs to call
find-class on the created class. Thus defclass needs to be executed before
def-methods is expanded. This does not work, if I use def-class within a
function (not at top-level). In this case - I think - everything gets
expanded and thus def-methods gets expanded before defclass is executed.
class-method will fail on calling find-class of course.

At the moment I am more astonished that the macro works at top-level.

Can anybody give a hint?
Any suggestions how I can solve the problem?

Andreas

From: Pascal Costanza
Subject: Re: MOP/Macroexpansion
Date: 
Message-ID: <3fu0nrF9l9acU1@individual.net>
Andreas Thiele wrote:
> Hi,
> 
> the following code snippet works in my application if def-class is used at
> top-level:
> 
> (defmacro def-methods (name)
>   `(progn ,@(class-methods name)))
> 
> (defmacro def-class (name parents &body body)
>   `(progn
>      (defclass ,name ,(if parents parents '(db-root))
>        ,@(transform-source name body)
>        (:metaclass db-class))
>      (def-methods ,name)))
> 
> class-methods computes a list of defmethod forms and needs to call
> find-class on the created class. Thus defclass needs to be executed before
> def-methods is expanded. This does not work, if I use def-class within a
> function (not at top-level). In this case - I think - everything gets
> expanded and thus def-methods gets expanded before defclass is executed.
> class-method will fail on calling find-class of course.
> 
> At the moment I am more astonished that the macro works at top-level.

ANSI Common Lisp is very specific about what should happen at the top 
level in this case. From the Hyperspec: "If a defclass form appears as a 
top level form, the compiler must make the class name be recognized as a 
valid type name in subsequent declarations (as for deftype) and be 
recognized as a valid class name for defmethod parameter specializers 
and for use as the :metaclass option of a subsequent defclass. The 
compiler must make the class definition available to be returned by 
find-class when its environment argument is a value received as the 
environment parameter of a macro."

Note that PROGN doesn't invalidate the "top-level-ness" of enclosed forms.

For non-top-level defclass forms, this is not specified because a) the 
defclass form may not be executed at all and b) the initforms for slots 
may close over the lexical environment of the defclass form which is 
only available at runtime.

The idea would be to not rely on the defmethod macro to create your 
methods at runtime, but to programmatically create those methods. Then 
you are able to refer to the run-time-generated class directly. A 
portable way to do this is by using EVAL:

(let ((my-class (defclass ...)))
   (eval `(defmethod my-method ((object ,my-class) ...)
            ...)))

Note that ANSI Common Lisp explicitly allows you to use class objects as 
specializers instead of class names, so this is indeed portable.

In some implementations, the use of EVAL may lead to inefficiencies 
because the evaluated form is not compiled but interpreted, including 
the method body of the generated method! (!!!)

You can circumvent this by doing the following:

(let ((my-class (defclass ...)))
   (funcall
     (compile nil
       `(lambda ()
          (defmethod my-method ((object ,my-class) ...)
            ...)))))

This indeed also compiles the generated method body.

The clean but only semi-portable way is to use the idiom for 
programmatically creating methods suggested by AMOP:

(let ((my-class (defclass ...)))
   (multiple-value-bind
       (method-lambda method-args)
       (make-method-lambda
         gf (class-prototype (generic-function-method-class gf))
         '(lambda (...) ...) nil)
     (let ((method (apply #'make-instance
                     (generic-function-method-class gf)
                     :qualifiers ()
                     :lambda-list '(...)
                     :specializers (list my-class ...)
                     :function (compile nil method-lambda)
                     method-args)))
       (add-method gf method))))

This only works in CLOS implementations that provide a correct 
implementation of make-method-lambda. Of the ones that I have seen, only 
CMUCL and SBCL provide correct versions, and the one in LispWorks is 
nearly correct (modulo argument list and generated parameter passing 
convention for the method function). Allegro, clisp, MCL and OpenMCL 
don't provide make-method-lambda.

Note that none of these approaches allow you to close over the lexical 
environment of the class- and method-generating code. If you delay 
generating methods to runtime, you don't have a chance in this regard 
because ANSI Common Lisp doesn't require the lexical environments to be 
available at runtime. This is different when you directly say (defmethod 
...) instead of one the solutions I have proposed above because the 
macroexpansion of defmethod can call make-method-lambda at 
macroexpansion time and thus close over the (then-accessible) lexical 
environment.

(I am not aware of any Common Lisp that gives you first-class access to 
lexical environments at runtime. OpenLisp does - see 
http://www.eligis.com/ - as apparently do some Scheme implementations. 
Whether you can take advantage of this in some CLOS-style object system 
+ a working MOP for it is left as an exercise to the reader. ;)

Another workaround for your problem could be to define the respective 
class globally. The non-global defclass forms would then just redefine 
the global class which may be sufficient for your purposes.

I needed to be able to create methods programmatically in my code as 
well, therefore I have added a utility function ENSURE-METHOD to my 
Closer to MOP compatibility library. It's not part of the public version 
yet, but will be in the next release.


One more thing: You take care of superclass defaulting in your def-class 
macro. However, the "MOP-way" to do this is in methods specialized on 
initialize-instance and reinitialize-instance, as follows:

(defmethod initialize-instance :around
   ((class my-standard-class)
    &rest initargs
    &key direct-superclasses
    &allow-other-keys)
   (declare (dynamic-extent initargs))
   (if (loop for super in direct-superclasses
             thereis (subtypep super 'my-standard-object))
      (call-next-method)
      (apply #'call-next-method
             class
             :direct-superclasses
             (append direct-superclasses
                     (list (find-class 'my-standard-object)))
             initargs)))

(defmethod reinitialize-instance :around
   ((class my-standard-class)
    &rest initargs
    &key (direct-superclasses () direct-superclasses-p)
    &allow-other-keys)
   (declare (dynamic-extent initargs))
   (if (or (not direct-superclasses-p)
           (loop for super in direct-superclasses
                 thereis (subtypep super 'my-standard-object)))
      (call-next-method)
      (apply #'call-next-method
             ... ; as in initialize-instance
                  )))

This is more robust than to do it in a macro because now you are sure 
that your superclass is added even when someone programmatically 
manipulates your classes.

That idiom doesn't work in some CLOS implementations though, because 
they do the defaulting of standard-object too early (at macroexpansion 
time instead of class object initialization time). Fortunately, Closer 
to MOP fixes this so that this idiom becomes indeed portable across most 
CLOS MOPs.

See http://common-lisp.net/project/closer/ for more details on Closer to 
MOP.


Pascal

-- 
2nd European Lisp and Scheme Workshop
July 26 - Glasgow, Scotland - co-located with ECOOP 2005
http://lisp-ecoop05.bknr.net/
From: Pascal Costanza
Subject: Re: MOP/Macroexpansion
Date: 
Message-ID: <3fu1c8F9l9acU2@individual.net>
Pascal Costanza wrote:

> Note that none of these approaches allow you to close over the lexical 
> environment of the class- and method-generating code. If you delay 
> generating methods to runtime, you don't have a chance in this regard 
> because ANSI Common Lisp doesn't require the lexical environments to be 
> available at runtime. This is different when you directly say (defmethod 
> ...) instead of one the solutions I have proposed above because the 
> macroexpansion of defmethod can call make-method-lambda at 
> macroexpansion time and thus close over the (then-accessible) lexical 
> environment.

Of course, your own macro could call make-method-lambda at 
macroexpansion-time as well.

(Furthermore, Closer to MOP also fixes the different arguments for 
make-method-lambda in LispWorks, so in this way you could write code 
that is portable across CMUCL, SBCL and LispWorks.)



Pascal

-- 
2nd European Lisp and Scheme Workshop
July 26 - Glasgow, Scotland - co-located with ECOOP 2005
http://lisp-ecoop05.bknr.net/
From: Coby Beck
Subject: Re: MOP/Macroexpansion
Date: 
Message-ID: <a1nme.23033$on1.18449@clgrps13>
"Andreas Thiele" <······@nospam.com> wrote in message 
····················@news.t-online.com...
> Hi,
>
> the following code snippet works in my application if def-class is used at
> top-level:
>
> (defmacro def-methods (name)
>  `(progn ,@(class-methods name)))
>
> (defmacro def-class (name parents &body body)
>  `(progn
>     (defclass ,name ,(if parents parents '(db-root))
>       ,@(transform-source name body)
>       (:metaclass db-class))
>     (def-methods ,name)))
>
> class-methods computes a list of defmethod forms and needs to call
> find-class on the created class. Thus defclass needs to be executed before
> def-methods is expanded. This does not work, if I use def-class within a

Make def-methods a function instead, thus it will run after the execution of 
the defclass form.

-- 
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
From: Andreas Thiele
Subject: Re: MOP/Macroexpansion
Date: 
Message-ID: <d7gadt$v6a$03$1@news.t-online.com>
Thanks for your answers - both have been important hints for me.

I think Cobis suggestion will lead to some eval or 'compiling by hand', like
Pascal pointed out.  I tracked down macroexpansion and have been able to
shift the eval/compile to some other place in my application where it would
occur only seldomly. I wanted to use macroexpansion instead of eval or
compile to get my code compiled in one step.

Fortunately things totally changed and it turned out that I only need to
call my def-class (a modification of defclass) from toplevel.

Although I dynamically generate classes and methods they are only dynamic
during developement. The delivered application will not do any dynamic
defclass or defmethod besides loading my definition file.

So at the moment, my def-class macro works at least in LispWorks like it was
before and I cannot focus on portability yet.

Still a little astonishing for me, that def-class will not be fully expanded
if called at toplevel - and thus is working.


Thanks

Andreas