From: Andreas Willig
Subject: Question about a macro-defining macro
Date: 
Message-ID: <874r4gcl3c.fsf@prospero.hpi.uni-potsdam.de>
Hi there,

i am stuck on the following: for my private fun i want to write a
matrix package (yeah, i know there is matlisp out there).

My matrices are defined as:

(defclass matrix ()
  ((rows     :accessor  rows    :initarg :rows     :initform    1)
   (columns  :accessor  columns :initarg :columns  :initform    1)
   (field    :accessor  field   :initarg :field    :initform    nil)))

and field is typically a two-dimensional array (initialized in various
make-xxx-matrix functions)

Given some matrix m i want to be able to refer to its elements (say
row i and column j) in a fashion like (x i j). It seems that macrolet
is a good way to do this since it allows to be used with setf. I want
to use this to write things like:
   (with-matrix-elements (m x)
     (setf (x 0 0) 1)
     (setf (x 1 1) (x 0 0)))
which is a (weird) way to construct the 2x2 identity matrix. Such a
macro would allow me to express the matrix addition as:

   (with-matrix-elements (mres z)
      (with-matrix-elements (m1 x)
         (with-matrix-elements (m2 y)
            (dotimes (i rows)
               (dotimes (j cols)
                  (setf (z i j) (+ (x i j) (y i j))))))))


My problem is: i could not figure out how to write the
with-matrix-elements macro. My first version was:

(defmacro with-matrix-elements ((m var) &body body)
  (assert (symbolp var) (var) "var should be a symbol")
  (with-basic-gensyms (g-m)
    `(let* ((,g-m ,m)
            (ref ,(list 'field g-m)))
      (macrolet ((,var (r c)
                   `(aref ref ,r ,c)))
        ,@body)
      nil)))

which, however, obviously has the problem of variable capture (ref)
which bites me when i nest with-matrix-elements constructs. My second
bet was:

(defmacro with-matrix-elements-new ((m var) &body body)
  (assert (symbolp var) (var) "var should be a symbol")
  (with-basic-gensyms (g-m g-field)
    `(let* ((,g-field (field ,m)))
      (macrolet ((,var (r c)
                   `(aref ,g-field ,r ,c)))
        ,@body)
      nil)))

but with only a single comma before the reference to g-field i get the
message that variable g-field has no value (during runtime) and with
two commas i get that the variable  #:G2415 has no value.

Anyone who can illuminate me? I have meditated over chapter 16 of
Grahams "On Lisp", but could not figure out how to solve this.


Andreas


-- 
Andreas Willig                   |    'Als Schnitzel sieht man bekloppt aus.'
Hasso-Plattner-Institute         |           
University of Potsdam            |                  (Alte Mastschweinweisheit)
·······@hpi.uni-potsdam.de       |     
                                 |    

From: Tim Bradshaw
Subject: Re: Question about a macro-defining macro
Date: 
Message-ID: <ey3d6j4tchu.fsf@cley.com>
* Andreas Willig wrote:

> (defmacro with-matrix-elements-new ((m var) &body body)
>   (assert (symbolp var) (var) "var should be a symbol")
>   (with-basic-gensyms (g-m g-field)
>     `(let* ((,g-field (field ,m)))
>       (macrolet ((,var (r c)
>                    `(aref ,g-field ,r ,c)))
>         ,@body)
>       nil)))

> but with only a single comma before the reference to g-field i get the
> message that variable g-field has no value (during runtime) and with
> two commas i get that the variable  #:G2415 has no value.

> Anyone who can illuminate me? I have meditated over chapter 16 of
> Grahams "On Lisp", but could not figure out how to solve this.


The standard idiom is ,', for this kind of thing.  It's quite hard to
persuade yourself that this is correct, and huge discussions have
happened on cll caused by people thinking that backquote was
inadequately specified & generally misunderstanding the issue.  My
technique is to have some idioms (as above), and if they fail, try
things till they work, and to believe that backquote is adequately
specified but it's too hard for me to intuit.

--tim
From: Andreas Willig
Subject: Re: Question about a macro-defining macro
Date: 
Message-ID: <87znm8b0x3.fsf@prospero.hpi.uni-potsdam.de>
Hi,

indeed,

Tim Bradshaw <···@cley.com> writes:

> * Andreas Willig wrote:
> 
> > (defmacro with-matrix-elements-new ((m var) &body body)
> >   (assert (symbolp var) (var) "var should be a symbol")
> >   (with-basic-gensyms (g-m g-field)
> >     `(let* ((,g-field (field ,m)))
> >       (macrolet ((,var (r c)
> >                    `(aref ,g-field ,r ,c)))
> >         ,@body)
> >       nil)))
> 
> > but with only a single comma before the reference to g-field i get the
> > message that variable g-field has no value (during runtime) and with
> > two commas i get that the variable  #:G2415 has no value.
[...] 
> 
> The standard idiom is ,', for this kind of thing.  It's quite hard
> to

this does the trick. Thanks.

> persuade yourself that this is correct, and huge discussions have
> happened on cll caused by people thinking that backquote was
> inadequately specified & generally misunderstanding the issue.  My

Well, this is really hard to grok ... i should spend a lazy afternoon
and a good cup of coffee on this ....

> technique is to have some idioms (as above), and if they fail, try
> things till they work, and to believe that backquote is adequately
> specified but it's too hard for me to intuit.
> 
> --tim

Andreas

-- 
Andreas Willig                   |    'Als Schnitzel sieht man bekloppt aus.'
Hasso-Plattner-Institute         |           
University of Potsdam            |                  (Alte Mastschweinweisheit)
·······@hpi.uni-potsdam.de       |     
                                 |    
From: Kaz Kylheku
Subject: Re: Question about a macro-defining macro
Date: 
Message-ID: <cf333042.0304301415.2dd148e1@posting.google.com>
Andreas Willig <·······@hpi.uni-potsdam.de> wrote in message news:<··············@prospero.hpi.uni-potsdam.de>...
 but with only a single comma before the reference to g-field i get
the
> message that variable g-field has no value (during runtime) and with
> two commas i get that the variable  #:G2415 has no value.
> 
> Anyone who can illuminate me? I have meditated over chapter 16 of
> Grahams "On Lisp", but could not figure out how to solve this.

I have started working on an advanced tutorial on backquotes that
should turn anyone into a seasoned expert.

Nested backquotes with multiple commas are simple to understand, once
you form the right mental model. However, you have to keep in mind
that a convenient mental model that explains how the backquote works
does not necessarily correspond to how the backquote is actually
implemented.

One way to understand it like this:

1. when a backquote form is evaluated, then all of the commas which
belong to that form evaluate their adjoint expressions at that time
and these are inserted into the list.

2. Any inner backquotes and commas enclosed in them remain for a
further evaluation round.

But in reality, this isn't necessarily how it happens; it may be that
an entire backquote form and all inner backquotes are expanded into
list-constructing code in one step. To make matters more confusing,
it's possible for the expander to produce more than one representation
of the backquote: an expanded version containing the code, and another
version for pretty-printing, so that when you print the object, it
appears as nice nested backquote rather than the corresponding hairy
code.

The consequence of rule 2 is that if you have two commas in a row,
then two evaluations happen (one for each round of evaluation of the
backquote form). For instance if you have ``(,,x),  the first
evaluation *conceptually* produces the non-nested backquote object
`(,<value-of-form 'x>). The leftmost comma goes with the inner
backquote and so it stays. Now if you evaluate that again, you finally
get the equivalent of (list <value-of-form <value-of-form 'x>>). (I
don't want to say (eval 'x); the <value-of-form 'x> notation indicates
evaluation, but not necessarily with EVAL). The ``natural'' evaluation
of a backquote can see lexical variables, whereas EVAL does not).

If you have N nesting levels of backquote (with N being 1 for a single
backquote) then the first evaluation round will evaluate the
*rightmost* commas in those expressions that have N commas---in other
words, those expressions that have the maximum number of commas
possible, matching the number of levels. Given `````,,,,,(+ 2 2) the
first evaluation will ``cancel'' the outermost backquote with the
rightmost comma, causing the (+ 2 2) expression to be reduced to 4,
and four backquotes and commas to remain, as if the form ````,,,,4
were produced.

But if there are fewer commas than backquotes, then none of those
commas belong to the outer backquote, and so they are not touched by
evaluation. No cancelation takes place. Thus the evaluation of
`````,,,,(+ 2 2) conceptually just strips one backquote level away,
leaving ````,,,,(+ 2 2). The (+ 2 2) is not reduced. But it will be if
another evaluation is applied, because now there are four commas
against four backquotes, so cancelation can take place.

Suppose you have this: ``,,(list '+ 2 2) . The first evaluation will
conceptually reduce to `,(+ 2 2). And then the next one finally to 4.
Two commas, two evaluations. Now suppose that you did not want this to
happen; you want the list (+ 2 2) to ``survive'' through the second
evaluation round. What do you do? You apply the trick Tim Bradshaw
showed you; you insert a quote to cancel the second evaluation:
``,',(list '+ 2 2).  So the first evaluation reduces to this: `,'(+ 2
2) or equivalently `,(quote (+ 2 2)). And the next evaluation will
then cancel the backquote and comma, evaluate the (quote ...) and
leave you with (+ 2 2). This is exactly what is often needed in
macro-defining-macros; you want some form that is input into the macro
to survive into the second-level macro without undergoing an
additional evaluation.
From: BK
Subject: Re: Question about a macro-defining macro
Date: 
Message-ID: <39d9c156.0305020156.18d2cfb2@posting.google.com>
···@ashi.footprints.net (Kaz Kylheku) wrote ...

> I have started working on an advanced tutorial on backquotes

[snip]

> don't want to say (eval 'x); the <value-of-form 'x> notation indicates
> evaluation, but not necessarily with EVAL). The ``natural'' evaluation
> of a backquote can see lexical variables, whereas EVAL does not).

It may be helpful in this context to emphasise to the audience of yor
tutorial that backquotes are expanded by the reader and not by the
evaluator or compiler. This might help to avoid confusion beteen what
you paraphrased "evaluation, but not necessarily with EVAL" and
"``natural'' evaluation of a backquote".

rgds
bk