Hi,
I have a few quick questions about macros. I've been reading On Lisp, but it
takes time for it all to sink in, and for now I'm stuck with a macro
problem.
I simply want to take a list of arguments, each a 2-item list, and generate
code like this:
(cffi:defcallback mbox-add-field :pointer ((self :pointer) (args :pointer))
(with-parsetuple-arg 'string buf
(with-parsetuple-arg 'string val
(cffi:foreign-funcall "PyArg_ParseTuple" :pointer args :string
"ss" :pointer buf :pointer val)
(mbox-add-field (pygetarg buf) (pygetarg val))
)
)
(py_buildvalue "s" ""))
I've handled pygetarg, which looks like this:
(defmacro pygetarg(variable)
`(cffi:foreign-string-to-lisp(cffi:mem-ref ,variable :string)))
I can handle things that only expand once, but I don't know how to expand on
a list!
The macro has this input: '(('string buf) ('string val))
The macro should first generate a with-parsetuple-arg for each input
argument, wrapping the body twice. Then, for each input argument, it should
generate a list of arguments to cffi:foreign-funcall, and finally it should
create bindings so that the value of the variables given (buf and val) is
preferably the value of (pygetarg buf) and (pygetarg val).
with-parsetuple-arg is like this:
(defmacro with-parsetuple-arg(type var &body body)
`(cffi:with-foreign-pointer (,var 255) ,@body))
It's not particularly useful or important, I was just trying to simplify
things.
I did try to write make-parsetuple-args though - it works, but it will make
some seasoned Lispers laugh.
;Takes args like this: '(('string buf) ('string val)) etc
;Could be tidied up a bit more to accept lists or single items
(defmacro make-parsetuple-args(&rest vars)
`(eval(let ((retval (reverse(list 'cffi:foreign-funcall
"PyArg_ParseTuple" :pointer args :string))))
;First build the arg string
(let ((arg-string ""))
(dolist (arg ,@vars)
(when (eq (first arg) 'string)
;Should use macro string-append
(setf arg-string (str-format "~as" arg-string))))
(push arg-string retval))
;Now add the variables
(dolist (arg ,@vars)
(push :pointer retval)
(push (second arg) retval))
(nreverse retval))))
PyArg_ParseTuple takes the following arguments: An args foreign address
pointer (supplied by Python), a multi-digit string indicating the args
required ('s' for string, 'l' for long), and then a foreign address pointer
to store the variables, which can be used afterwards until freed up by
CFFI's with-foreign-pointer. I want to simplify it so people (such as
myself) can easily write Python-to-Lisp callbacks.
I couldn't figure out how to generate the code without running it, so I
created a list of the code required and stuck an 'eval' there. There, I
said it.
This is about where my brain turned to jelly. I couldn't figure out how to
generate macro code recursively. I've just read something about it in On
Lisp (I'm on page 144), but it isn't very useful.
(defmacro make-parsetuple-with(&rest vars)
`(let ((retval))
(dolist (arg ,@vars)
(push ''with-parsetuple-arg retval))
(push (reverse(make-parsetuple-with (first arg) (second arg))) retval)
retval))
It doesn't work! It complains that arg isn't bound.
So I thought I'd drop by comp.lang.lisp and see if anyone could help.
I'm working on python-on-lisp, which is a new project to make it possible
(and easy, and hopefully efficient) to use Python's standardised libraries
in Lisp (of whatever variety). I haven't announced it here yet, because
it's not finished, but I am forced to ask a little macro guidance from the
Lisp masters hereabouts. ;-)
Thanks,
Jeremy.
From: Thomas A. Russ
Subject: Re: Help needed with macros: Brain turning to jelly
Date:
Message-ID: <ymi4q36g4q6.fsf@sevak.isi.edu>
Sorry I don't have time to go through this more thoroughly and analyze
exactly what you are trying to do, but the following struck me:
Jeremy Smith <············@decompiler.org> writes:
> ;Takes args like this: '(('string buf) ('string val)) etc
> ;Could be tidied up a bit more to accept lists or single items
> (defmacro make-parsetuple-args(&rest vars)
> `(eval(let ((retval (reverse(list 'cffi:foreign-funcall
> "PyArg_ParseTuple" :pointer args :string))))
> ;First build the arg string
> (let ((arg-string ""))
> (dolist (arg ,@vars)
> (when (eq (first arg) 'string)
> ;Should use macro string-append
> (setf arg-string (str-format "~as" arg-string))))
> (push arg-string retval))
> ;Now add the variables
> (dolist (arg ,@vars)
> (push :pointer retval)
> (push (second arg) retval))
> (nreverse retval))))
One of the important points about writing macros is to realize that the
first item inside the macro form does NOT need to be the backquote. If
you have things that you want to have evaluated at macroexpansion time,
you put them just into the body of the macro form.
What is happening in the code above is that you are deferring the
computations until runtime (when EVAL gets invoked) rather than doing
them at macro-expansion time.
For a simple example, let's consider a simple LET form that initializes
each of its variables that don't have an initial value to zero (instead
of nil):
(defmacro integer-let (var-list &body body)
(let ((new-var-list nil))
(dolist (var-spec var-list)
(cond ((atom var-spec) ; Just the symbol
(push `(,var-spec 0) new-var-list))
((rest var-spec) ; Initial value given
(push var-spec new-var-list))
(t ; e.g., (x), no value.
(push `(,(first var-spec) 0) new-var-list))))
`(let ,(reverse new-var-list)
,@body)))
In this example, the new-var-list is computed at macro-expansion time,
and the result of that computation is substituted into the code returned
by the macro.
(macroexpand-1 '(integer-let (a b (c 3) (d))
(+ a b c d)))
==>
(LET ((A 0)
(B 0)
(C 3)
(D 0))
(+ A B C D))
Now what might make this a bit easier for some of the rest of us to
follow is to give an example of a macro call and the code you would like
to have it produce.
--
Thomas A. Russ, USC/Information Sciences Institute
Thomas A. Russ wrote:
> One of the important points about writing macros is to realize that the
> first item inside the macro form does NOT need to be the backquote. If
> you have things that you want to have evaluated at macroexpansion time,
> you put them just into the body of the macro form.
> What is happening in the code above is that you are deferring the
> computations until runtime (when EVAL gets invoked) rather than doing
> them at macro-expansion time.
Thanks for pointing this out, I don't think On Lisp or the chapter in PCL
was clear enough on this point, as every example started with a backquote.
It's a tough choice when to try and write something without knowing every
last detail, as I'm a quarter the way through On Lisp, but I'll be another
few weeks finishing it and I would like to get this project rolling long
before then.
I think I did okay, but Pascal and your postings helped me understand the
bits I was missing!
> Now what might make this a bit easier for some of the rest of us to
> follow is to give an example of a macro call and the code you would like
> to have it produce.
Pascal's post pretty much shows what I was trying to do, which was make a
body and wrap it several times in multiple with- blocks. The simplest
solution was to do it with a function, as I did, but I didn't know you
could put commas next to function calls to include their value, ,@ to
include a list, and I didn't know you could build up a list in the way that
Pascal did.
Thanks for the help, anyway, I'm more savvy about macros now.
All the best,
Jeremy.
From: Peter Seibel
Subject: Re: Help needed with macros: Brain turning to jelly
Date:
Message-ID: <m2hd7563os.fsf@gigamonkeys.com>
Jeremy Smith <············@decompiler.org> writes:
> Thomas A. Russ wrote:
>
>> One of the important points about writing macros is to realize that the
>> first item inside the macro form does NOT need to be the backquote. If
>> you have things that you want to have evaluated at macroexpansion time,
>> you put them just into the body of the macro form.
>
>> What is happening in the code above is that you are deferring the
>> computations until runtime (when EVAL gets invoked) rather than doing
>> them at macro-expansion time.
>
> Thanks for pointing this out, I don't think On Lisp or the chapter in PCL
> was clear enough on this point, as every example started with a backquote.
Hey, that's not true. The very *first* example of a macro in the book,
in Chapter 3 not only doesn't start with a backquote, it doesn't even
*use* a backquote:
(defmacro backwards (expr) (reverse expr))
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
Peter Seibel wrote:
> Jeremy Smith <············@decompiler.org> writes:
>
>> Thanks for pointing this out, I don't think On Lisp or the chapter in PCL
>> was clear enough on this point, as every example started with a
>> backquote.
>
> Hey, that's not true. The very *first* example of a macro in the book,
> in Chapter 3 not only doesn't start with a backquote, it doesn't even
> *use* a backquote:
>
> (defmacro backwards (expr) (reverse expr))
>
> -Peter
>
Sorry, my mistake. :-( I suppose I just didn't get the significance of the
backquote, and thus I remember every macro as having one.
But Chapter 8 (the "Macros: defining your own" chapter) is great (you packed
a lot of ideas in there), I just suppose I should have re-read it a few
times. I'll read it again tonight.
Cheers,
Jeremy.
Just a quick note, but this is the error I get from running the recursive
macro (in Clisp):
[11]> (py::make-parsetuple-with(list (list 'string 'arg) (list 'string
'val)))
*** - EVAL: variable PYTHONONLISP::ARG has no value
Jeremy.
Jeremy Smith <············@decompiler.org> writes:
> [...]
> The macro has this input: '(('string buf) ('string val))
Most probably NOT.
QUOTE is used to avoid evaluation of the following form (to take it as
data), and is needed when we want to pass that form as unevaluated
data (literal) to a FUNCTION, because functions receive their
arguments evaluated.
Macros don't receive their arguments evaluated, so you don't need to
QUOTE their argument, usually. If you want to quote the arguments of
a macro, it's a hint you are wanting a function.
'(('string buf) ('string val)) --> (((quote string) buf) ((quote string) val))
(remember, --> means "evaluates to")
I don't understand what these symbols QUOTE have to do in your data
structure. What are you trying to describe with this data? I gather
that you are describing a list of parameters, each with a name and a
type. So just write it:
( ; a list
; of parameters; we'll write each parameter as
( ; a list
; containing the name of the parameter:
buf
; and the type of the parameter:
string)
( ; a list
; containing the name of the parameter:
val
; and the type of the parameter:
string))
((buf string) (val string))
Why would you need to insert symbols named QUOTE in this data structure?
> I simply want to take a list of arguments, each a 2-item list, and generate
> code like this:
>
> (cffi:defcallback mbox-add-field :pointer ((self :pointer) (args :pointer))
> (with-parsetuple-arg 'string buf
> (with-parsetuple-arg 'string val
> (cffi:foreign-funcall "PyArg_ParseTuple" :pointer args :string
> "ss" :pointer buf :pointer val)
> (mbox-add-field (pygetarg buf) (pygetarg val))
> )
> )
> (py_buildvalue "s" ""))
You should indent properly your code!!!
(cffi:defcallback mbox-add-field :pointer ((self :pointer) (args :pointer))
(with-parsetuple-arg 'string buf
(with-parsetuple-arg 'string val
(cffi:foreign-funcall "PyArg_ParseTuple"
:pointer args
:string "ss"
:pointer buf
:pointer val)
(mbox-add-field (pygetarg buf) (pygetarg val))))
(py_buildvalue "s" ""))
Now we can see clearly that we have to data structures:
((buf string) (val string))
and:
(cffi:defcallback mbox-add-field :pointer ((self :pointer) (args :pointer))
(with-parsetuple-arg (quote string) buf
(with-parsetuple-arg (quote string) val
(cffi:foreign-funcall "PyArg_ParseTuple"
:pointer args
:string "ss"
:pointer buf
:pointer val)
(mbox-add-field (pygetarg buf) (pygetarg val))))
(py_buildvalue "s" ""))
and assuming that all occurences of the symbols found in the first
tree are to be substituted in the places where they occur in the
second list, we can describe the transformation with the following
patterns:
((<name> <type>)...)
==>
(cffi:defcallback mbox-add-field :pointer ((self :pointer) (args :pointer))
(with-parsetuple-arg (quote <type>) <name>
...
(cffi:foreign-funcall "PyArg_ParseTuple"
:pointer args
:string "ss"
:pointer <name>
...)
(mbox-add-field (pygetarg <name>) ...))
(py_buildvalue "s" ""))
Now, that the problem is specified, can you write a function to do
this transformation between these two data structures?
(defun gen-callback (input)
(loop
:with body = `(progn
(cffi:foreign-funcall "PyArg_ParseTuple"
:pointer args
:string "ss"
,@(loop :for (<name> <ype>) :in input
:nconc `(:pointer ,<name>)))
(mbox-add-field ,@(loop :for (<name> <type>) :in input
:collect `(pygetarg ,<name>))))
:for (<name> <type>) :in (reverse input)
:do (setf body `(with-parsetyple-arg (quote ,<type>) ,<name> ,body))
:finally (return body)))
[90]> (gen-callback '((buf string) (val integer)))
(WITH-PARSETYPLE-ARG 'STRING BUF
(WITH-PARSETYPLE-ARG 'INTEGER VAL
(PROGN
(CFFI:FOREIGN-FUNCALL "PyArg_ParseTuple" :POINTER ARGS :STRING "ss" :POINTER
BUF :POINTER VAL)
(MBOX-ADD-FIELD (PYGETARG BUF) (PYGETARG VAL)))))
Now, if you want to avoid the ' in the gen-callback call, or if you
want to have the data generated be compiled, you can call gen-callback
from a macro:
(defmacro callback (input) (gen-callback input))
[92]> (macroexpand-1 '(callback ((buf string) (val integer))))
(WITH-PARSETYPLE-ARG 'STRING BUF
(WITH-PARSETYPLE-ARG 'INTEGER VAL
(PROGN
(CFFI:FOREIGN-FUNCALL "PyArg_ParseTuple" :POINTER ARGS :STRING "ss" :POINTER
BUF :POINTER VAL)
(MBOX-ADD-FIELD (PYGETARG BUF) (PYGETARG VAL))))) ;
T
> I've handled pygetarg, which looks like this:
>
> (defmacro pygetarg(variable)
> `(cffi:foreign-string-to-lisp(cffi:mem-ref ,variable :string)))
What has this to do with the wanted macros? Can't you concentrate on
the job at hand?
> I couldn't figure out how to generate the code without running it, so I
> created a list of the code required and stuck an 'eval' there. There, I
> said it.
So you can't figure how to avoid EVAL, therefore you add one more EVAL.
Logical.
Use MACROEXPAND-1 to expand a macro!
(But first, write a function to do the job!)
--
__Pascal Bourguignon__ http://www.informatimago.com/
The rule for today:
Touch my tail, I shred your hand.
New rule tomorrow.
Pascal Bourguignon wrote:
> Jeremy Smith <············@decompiler.org> writes:
>> [...]
>> The macro has this input: '(('string buf) ('string val))
>
> Most probably NOT.
>
> QUOTE is used to avoid evaluation of the following form (to take it as
> data), and is needed when we want to pass that form as unevaluated
> data (literal) to a FUNCTION, because functions receive their
> arguments evaluated.
Okay, I understand this now. I had a vague idea of it, but sometimes it
takes me a few goes to understand a concept.
> Macros don't receive their arguments evaluated, so you don't need to
> QUOTE their argument, usually. If you want to quote the arguments of
> a macro, it's a hint you are wanting a function.
>
> '(('string buf) ('string val)) --> (((quote string) buf) ((quote string)
> val))
>
> (remember, --> means "evaluates to")
>
> I don't understand what these symbols QUOTE have to do in your data
> structure. What are you trying to describe with this data? I gather
> that you are describing a list of parameters, each with a name and a
> type. So just write it:
That's right.
> Why would you need to insert symbols named QUOTE in this data structure?
I intended the quote to make string into 'string and thus a symbol. Now I
realise that this is only needed to pass unevaluated data such as code.
> Now, that the problem is specified, can you write a function to do
> this transformation between these two data structures?
I tried to do it with a function, but I was missing a few key concepts which
are clear in your code.
> Now, if you want to avoid the ' in the gen-callback call, or if you
> want to have the data generated be compiled, you can call gen-callback
> from a macro:
>
> (defmacro callback (input) (gen-callback input))
That's handy.
>> I've handled pygetarg, which looks like this:
>>
>> (defmacro pygetarg(variable)
>> `(cffi:foreign-string-to-lisp(cffi:mem-ref ,variable :string)))
>
> What has this to do with the wanted macros? Can't you concentrate on
> the job at hand?
I did this before I started work on the main macro, I was just doing it
piece by piece.
>> I couldn't figure out how to generate the code without running it, so I
>> created a list of the code required and stuck an 'eval' there. There, I
>> said it.
>
> So you can't figure how to avoid EVAL, therefore you add one more EVAL.
> Logical.
>
> Use MACROEXPAND-1 to expand a macro!
> (But first, write a function to do the job!)
I'll give it a go! :-)
Anyway, your code works very well and is educational, because I already know
what it does. I've learnt a lot from it.
I've inserted conditionals to only insert the PyArg calls if the return
value or args are NIL. I've let people specify a block of code to be
executed instead of calling a Lisp function to do it, but that's
experimental.
Now there is just a one-line argument, defpycallback, which specifies a
function name, a return type, and a list of arguments. The user still needs
to make a separate Lisp function, but I can integrate both by wrapping the
defun inside defpycallback. This is great, because it makes it easier for
me (and others) to write Lisp-to-Python modules.
Thanks for the help, it's really clarified my understanding of writing
macros. I'll credit you in the next release if you like, and now I can
continue this on my own. :-)
Jeremy.
Jeremy Smith <············@decompiler.org> writes:
> Pascal Bourguignon wrote:
>>> I've handled pygetarg, which looks like this:
>>>
>>> (defmacro pygetarg(variable)
>>> `(cffi:foreign-string-to-lisp(cffi:mem-ref ,variable :string)))
>>
>> What has this to do with the wanted macros? Can't you concentrate on
>> the job at hand?
>
> I did this before I started work on the main macro, I was just doing it
> piece by piece.
What I meant is that you don't need do consider all the macros or
functions at once, you don't need to macroexpand all of them
recursively to make it work. The compiler will process the macros
recursively when needed for you.
So you can concentrate on each macro one at a time.
--
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
__Pascal Bourguignon__ http://www.informatimago.com/