From: jmckitrick
Subject: Why is this macro misbehaving?
Date: 
Message-ID: <1161890653.658584.11430@h48g2000cwc.googlegroups.com>
I'd like to take a string like this: "1/2/3"
and bind it like this:

(let ((item "1") (arg1 "2") (arg2 "3"))
  body)

Here's the macro:

(defmacro with-url ((&rest vars) seq &body body)
  (let ((subseqs (split-sequence #\/ seq :remove-empty-subseqs t)))
    `(let ,(loop
	      for var in vars
	      for subseq in subseqs collect
		`(,var ,subseq))
       ,@body)))

However, using the macro declares all 'args' after 'item' to be
undefined:

(with-url (item arg1 arg2) (request-unhandled-part request)
  (format t "arg1: ~A~%" arg1))

+  Warnings (2)
 |-- undefined variable: ARG1
 `-- This variable is undefined:
       ARG1

Why would 'item' work but not the subsequent 'arg' values?

From: ·············@gmail.com
Subject: Re: Why is this macro misbehaving?
Date: 
Message-ID: <1161895190.299212.290290@i42g2000cwa.googlegroups.com>
jmckitrick wrote:
> Why would 'item' work but not the subsequent 'arg' values?

I'm no expert in Common Lisp, but I think you misunderstood how macros
work.
Remember that macros get expanded at compile time (or, if you run
interpreted code, before your code is actually interpreted). In your
example, you call your macro binding seq to the form
(request-unhandled-part request), not to the result of evaluating it.
So the call (split-sequence #\/ seq :remove-empty-subseqs t) becomes,
after macroexpansion, equivalent to (split-sequence #\/
'(request-unhandled-part request) :remove-empty-subseqs t) .
Since '(request-unhandled-part request) is a list, it is a sequence as
well, so no errors occur when the call to split-sequence is evaluated,
but of course the result is not what you'd expect. I don't know
precisely how split-sequence works, but I guess the result of the call
will be '((request-unhandled-part request)), that is, a list (or maybe
a vector) of only one element, the entire sequence (which didn't
contain any #\/ element). That's why only the first variable is bound
to something (i.e. the entire sequence) while the others are unbound.
I'd write that macro as

(defmacro with-url ((&rest vars) seq &body body)
  (labels ((create-bindings (vars subseqs)
            (unless (null vars)
	      (nconc `((,(car vars) ,(car subseqs)))
		     (create-bindings (cdr vars) (cdr subseqs))))))
    (let ((subseqs (gensym)))
      `(let ((,subseqs (split-sequence #\/ ,seq :remove-empty-subseqs
t)))
	 (let ,(loop
		 for var in vars
		 for i from 0
		 collect `(,var (nth ,i ,subseqs)))
	   ,@body)))))

but again, I'm not an expert so there are probably better ways to do
that.
From: Zach Beane
Subject: Re: Why is this macro misbehaving?
Date: 
Message-ID: <m3k62mhn6g.fsf@unnamed.xach.com>
"jmckitrick" <···········@yahoo.com> writes:

> I'd like to take a string like this: "1/2/3"
> and bind it like this:
> 
> (let ((item "1") (arg1 "2") (arg2 "3"))
>   body)
> 
> Here's the macro:
> 
> (defmacro with-url ((&rest vars) seq &body body)
>   (let ((subseqs (split-sequence #\/ seq :remove-empty-subseqs t)))
>     `(let ,(loop
> 	      for var in vars
> 	      for subseq in subseqs collect
> 		`(,var ,subseq))
>        ,@body)))

The splitting must be done at runtime, not macroexpansion time.
 
> (with-url (item arg1 arg2) (request-unhandled-part request)
>   (format t "arg1: ~A~%" arg1))

In this call, your macro gets the list (REQUEST-UNHANDLED-PART
REQUEST) as the SEQ argument. Trying to use split-sequence on it will
not work.

You need to move the splitting work into runtime; that's when the
value (rather than the form that produces the value) is available.

At that point, though, there's very little to be gained over simply
using DESTRUCTURING-BIND.

Zach
From: jmckitrick
Subject: Re: Why is this macro misbehaving?
Date: 
Message-ID: <1161897138.098513.157370@f16g2000cwb.googlegroups.com>
On Oct 26, 3:49 pm, Zach Beane <····@xach.com> wrote:
> You need to move the splitting work into runtime; that's when the
> value (rather than the form that produces the value) is available.

I guess I haven't learned the subtlety yet of when that happens.  I
thought any unknown values simply delayed binding until run-time.
There's a similar macro in araneida (with-url-params (params)
(request-url request) body) that works the way I had hoped, but it is
actually a function, and a complicated one at that, on closer
inspection.

> At that point, though, there's very little to be gained over simply
> using DESTRUCTURING-BIND.

Yes, I see your point.  Thanks.
From: Kaz Kylheku
Subject: Re: Why is this macro misbehaving?
Date: 
Message-ID: <1161968679.521791.118110@h48g2000cwc.googlegroups.com>
jmckitrick wrote:
> On Oct 26, 3:49 pm, Zach Beane <····@xach.com> wrote:
> > You need to move the splitting work into runtime; that's when the
> > value (rather than the form that produces the value) is available.
>
> I guess I haven't learned the subtlety yet of when that happens.

Quite simply

1. Write an example of the desired target code by hand first. Get it
working right.
2. Identify the variable and collapsible pieces of syntax in that
target
3. Write the macro which generates that target code.

You can start by just making it a macro with no arguments that
evaluates a backquoted template containing the entire example, with no
unquotes. Then identify parts that can be replaced, using unquotes, by
material computed by the macro, keeping in mind that you are operating
purely on syntax.
From: Sidney Markowitz
Subject: Re: Why is this macro misbehaving?
Date: 
Message-ID: <4541347b$0$88659$742ec2ed@news.sonic.net>
jmckitrick wrote, On 27/10/06 10:12 AM:
> On Oct 26, 3:49 pm, Zach Beane <····@xach.com> wrote:
>> You need to move the splitting work into runtime; that's when the
>> value (rather than the form that produces the value) is available.
> 
> I guess I haven't learned the subtlety yet of when that happens.  I
> thought any unknown values simply delayed binding until run-time.
> There's a similar macro in araneida (with-url-params (params)
> (request-url request) body) that works the way I had hoped, but it is
> actually a function, and a complicated one at that, on closer
> inspection.

You can think of a macro as a way to define a different syntax for
something that you can code as a function in Lisp but that would look
uglier, require too much typing, or otherwise look undesirable.

So start by writing out an example of the syntax you want to be able to
use and what it would look like in Lisp without a macro:

(with-url (item arg1 arg2) (request-unhandled-part request)
  (format t "arg1: ~A~%" arg1))

should evaluate as if you had entered

(let ((seq (request-unhandled-part request)))
  (destructuring-bind (item arg1 arg2)
     (split-sequence #\/ seq :remove-empty-subseqs t)
   (format t "arg1: ~A~%" arg1)))

Now lets write a function that transforms an expression that looks like
the former into the latter:

(defun %with-url-function (arglist seqexpr body)
   (let ((seqvar (gensym)))
    `(let ((,seqvar ,seqexpr))
       (destructuring-bind ,arglist
          (split-sequence #\/ ,seqvar :remove-empty-subseqs t)
         ,@body))))

I wrote this to use a gensym instead of seq to make sure there are no
name collisions. The binding of the gensym ensures that the sequence
expresson is evaluated only once at runtime in case we end up changing
the macro to use it more than the one time in split-sequence.

Note that backquote and comma syntax is not something that you only use
in defmacro definitions. It actually is a shorthand syntax that is
useful when you are writing something that outputs s-expresions, and
macro definitions happen to be a common use for it. You could write
(list 'let (list (list seqvar seqexpr)) (list 'destructurin-bind ...
but why bother when we have backquote syntax to make it easier?

Now verify that this function does transform the macro syntax into the
correct Lisp syntax and debug it as a function if it doesn't. *** Note
*** I have not typed this into a Lisp listener. When I say verify this,
I mean it ... Code that has not been eve compiled probably will not
compile and run properly. Fixing my typos is left as an exercise for the
reader.

(%with-url-function
   '(item arg1 arg2)
   '(request-unhandled-part request)
   '((format t "arg1: ~A~%" arg1)))


Once you see it works, define the macro itself:

(defmacro with-url (arglist seqexpr &body body)
  (%with-url-function arglist seqexpr body))


When people define macros they usually do not explicitly define a
function and then have the defmacro do nothing but call it. That is a
stylistic choice. If the macro gets at all complicated I prefer to
separate out the function, which is called at compile time, and make the
actual defmacro simpler. In this case, once you have the definition of
the helper function %with-url-function working, you could move it into
the defmacro itself to get

(defmacro with-url (arglist seqexpr &body body)
   (let ((seqvar (gensym)))
    `(let ((,seqvar ,seqexpr))
       (destructuring-bind ,arglist
          (split-sequence #\/ ,seqvar :remove-empty-subseqs t)
         ,@body))))


Again, I have deliberately not typed this into a Lisp listener as I'm
just explaining the broad concepts and I think that fixing any mistakes
would help in understanding.

-- 
    Sidney Markowitz
    http://www.sidney.com
From: jmckitrick
Subject: Re: Why is this macro misbehaving?
Date: 
Message-ID: <1161968620.292843.52980@k70g2000cwa.googlegroups.com>
Thanks for taking the time to explain things, Sidney.  I fear, though,
that Zach is right in this case, because the (request-unhandled-part
request) cannot be evaluated until execution time.  But I've saved your
post for future reference.  I especially need to remind myself to use
gensym more often.
From: Sidney Markowitz
Subject: Re: Why is this macro misbehaving?
Date: 
Message-ID: <45424556$0$88682$742ec2ed@news.sonic.net>
jmckitrick wrote, On 28/10/06 6:03 AM:
> that Zach is right in this case, because the (request-unhandled-part
> request) cannot be evaluated until execution time

If you follow what I've written, that is taken into account in my
example. In fact that is the point of that approach to dealing with
macros: The macro is used to translate one syntax into another, and you
know before you start what the generated Lisp code is going to look
like, so whatever needs to be evaluated at runtime is evaluated at runtime.

See Kaz's post to this thread which summarizes in three lines what I was
trying to say in more detail.

-- 
    Sidney Markowitz
    http://www.sidney.com