From: Joe Marshall
Subject: Macros for seriously interested people
Date: 
Message-ID: <1179681955.315029.87840@z28g2000prd.googlegroups.com>
Here are a couple of links that demonstrate good uses of macros in
Common Lisp.  These examples are not short, nor are they particularly
macro-laden.  They show how a couple of judiciously placed macros can
make useful changes to the language.

Pragmatic Parsing in Common Lisp
Henry Baker
http://home.pipeline.com/~hbaker1/Prag-Parse.html

SERIES package for Common Lisp
Richard Waters
http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html

@techreport{ siskind91screamer,
    author = "Jeffrey Mark Siskind and David Allen McAllester",
    title = "Screamer: {A} Portable Efficient Implementation of
Nondeterministic Common Lisp",
    number = "IRCS-93-03",
    address = "Philadelphia, PA",
    year = "1991",
    url = "citeseer.ist.psu.edu/siskind93screamer.html" }

Funny, but I seem to have argued this question before with Alex
Martelli.
http://mail.python.org/pipermail/python-list/2003-October/229805.html

I gave two examples of interesting macro expansion.  One from series:

  (defun example ()
    (let ((elements '(3 -1 4 -1 5 -9 2 -6 5 -3)))
      (/ (reduce #'max (map 'list #'abs elements))
         (reduce #'+ elements))))

This works, but it is inefficient because MAP and the second REDUCE
both traverse the list, and the intermediate list generated by MAP is
immediately discarded (consumed) by the first REDUCE.

The series version appears quite similar:

  (defun series-example ()
    (let ((elements (scan '(3 -1 4 -1 5 -9 2 -6 5 -3))))
      (/ (collect-max (#m abs elements))
         (collect-sum elements))))

but if you macroexpand the body, you get this:

  (COMMON-LISP:LET* (ELEMENTS (#:LISTPTR-702 '(3 -1 4 -1 5 -9 2 -6 5
-3)) #:ITEMS-710 (#:NUMBER-707 NIL) (#:SUM-714 0))
    (DECLARE (TYPE LIST #:LISTPTR-702) (TYPE NUMBER #:SUM-714))
    (TAGBODY
     #:LL-717 (IF (ENDP #:LISTPTR-702) (GO SERIES::END))
              (SETQ ELEMENTS (CAR #:LISTPTR-702))
              (SETQ #:LISTPTR-702 (CDR #:LISTPTR-702))
              (SETQ #:ITEMS-710 ((LAMBDA (#:V-708) (ABS #:V-708))
ELEMENTS))
              (IF (OR (NULL #:NUMBER-707) (< #:NUMBER-707
#:ITEMS-710)) (SETQ #:NUMBER-707 #:ITEMS-710))
              (SETQ #:SUM-714 (+ #:SUM-714 ELEMENTS))
              (GO #:LL-717)
      SERIES::END)
    (IF (NULL #:NUMBER-707) (SETQ #:NUMBER-707 NIL))
    (/ #:NUMBER-707 #:SUM-714))

This code avoids traversing the series more than once and does not
create an intermediate series just to discard it.  It does this by
analyzing the code and determining that the computation may procede in
`lock step'.

This example is from Screamer:
(and seems relevant to a different thread on this list!)

(defun pythagorean-triples (n)
 (all-values
  (let ((a (an-integer-between 1 n))
        (b (an-integer-between 1 n))
        (c (an-integer-between 1 n)))
   (unless (= (+ (* a a) (* b b)) (* c c)) (fail))
   (list a b c))))

The body of this function expands into:

(LET ((VALUES 'NIL) (SCREAMER::LAST-VALUE-CONS NIL))
  (LET ((SCREAMER::TRAIL-POINTER (FILL-POINTER SCREAMER::*TRAIL*)))
    (SCREAMER::CHOICE-POINT-INTERNAL
      (PROGN
           (LET ((#:DUMMY-29301 N))
             (LET ((#:CONTINUATION-29303
                    #'(LAMBDA (&OPTIONAL #:DUMMY-29283 &REST
#:OTHER-29284)
                        (DECLARE (SCREAMER::MAGIC) (IGNORE
#:OTHER-29284))
                        (PROGN
                          (LET ((#:DUMMY-29296 N))
                            (LET ((#:CONTINUATION-29298
                                   #'(LAMBDA (&OPTIONAL #:DUMMY-29285
&REST #:OTHER-29286)
                                       (DECLARE (SCREAMER::MAGIC)
(IGNORE #:OTHER-29286))
                                       (PROGN
                                         (LET ((#:DUMMY-29291 N))
                                           (LET ((#:CONTINUATION-29293
                                                  #'(LAMBDA (&OPTIONAL
#:DUMMY-29287 &REST #:OTHER-29288)
                                                      (DECLARE
(SCREAMER::MAGIC) (IGNORE #:OTHER-29288))
                                                      (LET ((C
#:DUMMY-29287)
                                                            (B
#:DUMMY-29285)
                                                            (A
#:DUMMY-29283))
                                                        (PROGN
                                                          (IF (NULL (=
(+ (* A A) (* B B)) (* C C)))
                                                              (PROGN
(FAIL)))
                                                          (LET
((#:DUMMY-29281 (LIST A B C)))
                                                            (LET
((SCREAMER::VALUE #:DUMMY-29281))
                                                              (PROGN
 
(GLOBAL
 
(COND ((NULL VALUES)
 
(SETF
 
SCREAMER::LAST-VALUE-CONS
 
(LIST SCREAMER::VALUE))
 
(SETF
 
VALUES
 
SCREAMER::LAST-VALUE-CONS))
 
(T
 
(SETF
 
(REST SCREAMER::LAST-VALUE-CONS)
 
(LIST SCREAMER::VALUE))
 
(SETF
 
SCREAMER::LAST-VALUE-CONS
 
(REST
 
SCREAMER::LAST-VALUE-CONS)))))
 
(FAIL)))))))))
                                             (DECLARE (DYNAMIC-EXTENT
#:CONTINUATION-29293))
                                             (SCREAMER::AN-INTEGER-
BETWEEN-NONDETERMINISTIC #:CONTINUATION-29293
 
1
 
#:DUMMY-29291)))))))
                              (DECLARE (DYNAMIC-EXTENT
#:CONTINUATION-29298))
                              (SCREAMER::AN-INTEGER-BETWEEN-
NONDETERMINISTIC #:CONTINUATION-29298
 
1
 
#:DUMMY-29296)))))))
               (DECLARE (DYNAMIC-EXTENT #:CONTINUATION-29303))
               (SCREAMER::AN-INTEGER-BETWEEN-NONDETERMINISTIC
#:CONTINUATION-29303 1 #:DUMMY-29301))))))
  VALUES)

From: Rainer Joswig
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <joswig-4B4467.19360620052007@news-europe.giganews.com>
In article <·······················@z28g2000prd.googlegroups.com>,
 Joe Marshall <··········@gmail.com> wrote:

sometimes the fun reading postings is proportional to the line width...

could you repost it without those added line breaks? Please.

> Here are a couple of links that demonstrate good uses of macros in
> Common Lisp.  These examples are not short, nor are they particularly
> macro-laden.  They show how a couple of judiciously placed macros can
> make useful changes to the language.
> 
> Pragmatic Parsing in Common Lisp
> Henry Baker
> http://home.pipeline.com/~hbaker1/Prag-Parse.html
> 
> SERIES package for Common Lisp
> Richard Waters
> http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html
> 
> @techreport{ siskind91screamer,
>     author = "Jeffrey Mark Siskind and David Allen McAllester",
>     title = "Screamer: {A} Portable Efficient Implementation of
> Nondeterministic Common Lisp",
>     number = "IRCS-93-03",
>     address = "Philadelphia, PA",
>     year = "1991",
>     url = "citeseer.ist.psu.edu/siskind93screamer.html" }
> 
> Funny, but I seem to have argued this question before with Alex
> Martelli.
> http://mail.python.org/pipermail/python-list/2003-October/229805.html
> 
> I gave two examples of interesting macro expansion.  One from series:
> 
>   (defun example ()
>     (let ((elements '(3 -1 4 -1 5 -9 2 -6 5 -3)))
>       (/ (reduce #'max (map 'list #'abs elements))
>          (reduce #'+ elements))))
> 
> This works, but it is inefficient because MAP and the second REDUCE
> both traverse the list, and the intermediate list generated by MAP is
> immediately discarded (consumed) by the first REDUCE.
> 
> The series version appears quite similar:
> 
>   (defun series-example ()
>     (let ((elements (scan '(3 -1 4 -1 5 -9 2 -6 5 -3))))
>       (/ (collect-max (#m abs elements))
>          (collect-sum elements))))
> 
> but if you macroexpand the body, you get this:
> 
>   (COMMON-LISP:LET* (ELEMENTS (#:LISTPTR-702 '(3 -1 4 -1 5 -9 2 -6 5
> -3)) #:ITEMS-710 (#:NUMBER-707 NIL) (#:SUM-714 0))
>     (DECLARE (TYPE LIST #:LISTPTR-702) (TYPE NUMBER #:SUM-714))
>     (TAGBODY
>      #:LL-717 (IF (ENDP #:LISTPTR-702) (GO SERIES::END))
>               (SETQ ELEMENTS (CAR #:LISTPTR-702))
>               (SETQ #:LISTPTR-702 (CDR #:LISTPTR-702))
>               (SETQ #:ITEMS-710 ((LAMBDA (#:V-708) (ABS #:V-708))
> ELEMENTS))
>               (IF (OR (NULL #:NUMBER-707) (< #:NUMBER-707
> #:ITEMS-710)) (SETQ #:NUMBER-707 #:ITEMS-710))
>               (SETQ #:SUM-714 (+ #:SUM-714 ELEMENTS))
>               (GO #:LL-717)
>       SERIES::END)
>     (IF (NULL #:NUMBER-707) (SETQ #:NUMBER-707 NIL))
>     (/ #:NUMBER-707 #:SUM-714))
> 
> This code avoids traversing the series more than once and does not
> create an intermediate series just to discard it.  It does this by
> analyzing the code and determining that the computation may procede in
> `lock step'.
> 
> This example is from Screamer:
> (and seems relevant to a different thread on this list!)
> 
> (defun pythagorean-triples (n)
>  (all-values
>   (let ((a (an-integer-between 1 n))
>         (b (an-integer-between 1 n))
>         (c (an-integer-between 1 n)))
>    (unless (= (+ (* a a) (* b b)) (* c c)) (fail))
>    (list a b c))))
> 
> The body of this function expands into:
> 
> (LET ((VALUES 'NIL) (SCREAMER::LAST-VALUE-CONS NIL))
>   (LET ((SCREAMER::TRAIL-POINTER (FILL-POINTER SCREAMER::*TRAIL*)))
>     (SCREAMER::CHOICE-POINT-INTERNAL
>       (PROGN
>            (LET ((#:DUMMY-29301 N))
>              (LET ((#:CONTINUATION-29303
>                     #'(LAMBDA (&OPTIONAL #:DUMMY-29283 &REST
> #:OTHER-29284)
>                         (DECLARE (SCREAMER::MAGIC) (IGNORE
> #:OTHER-29284))
>                         (PROGN
>                           (LET ((#:DUMMY-29296 N))
>                             (LET ((#:CONTINUATION-29298
>                                    #'(LAMBDA (&OPTIONAL #:DUMMY-29285
> &REST #:OTHER-29286)
>                                        (DECLARE (SCREAMER::MAGIC)
> (IGNORE #:OTHER-29286))
>                                        (PROGN
>                                          (LET ((#:DUMMY-29291 N))
>                                            (LET ((#:CONTINUATION-29293
>                                                   #'(LAMBDA (&OPTIONAL
> #:DUMMY-29287 &REST #:OTHER-29288)
>                                                       (DECLARE
> (SCREAMER::MAGIC) (IGNORE #:OTHER-29288))
>                                                       (LET ((C
> #:DUMMY-29287)
>                                                             (B
> #:DUMMY-29285)
>                                                             (A
> #:DUMMY-29283))
>                                                         (PROGN
>                                                           (IF (NULL (=
> (+ (* A A) (* B B)) (* C C)))
>                                                               (PROGN
> (FAIL)))
>                                                           (LET
> ((#:DUMMY-29281 (LIST A B C)))
>                                                             (LET
> ((SCREAMER::VALUE #:DUMMY-29281))
>                                                               (PROGN
>  
> (GLOBAL
>  
> (COND ((NULL VALUES)
>  
> (SETF
>  
> SCREAMER::LAST-VALUE-CONS
>  
> (LIST SCREAMER::VALUE))
>  
> (SETF
>  
> VALUES
>  
> SCREAMER::LAST-VALUE-CONS))
>  
> (T
>  
> (SETF
>  
> (REST SCREAMER::LAST-VALUE-CONS)
>  
> (LIST SCREAMER::VALUE))
>  
> (SETF
>  
> SCREAMER::LAST-VALUE-CONS
>  
> (REST
>  
> SCREAMER::LAST-VALUE-CONS)))))
>  
> (FAIL)))))))))
>                                              (DECLARE (DYNAMIC-EXTENT
> #:CONTINUATION-29293))
>                                              (SCREAMER::AN-INTEGER-
> BETWEEN-NONDETERMINISTIC #:CONTINUATION-29293
>  
> 1
>  
> #:DUMMY-29291)))))))
>                               (DECLARE (DYNAMIC-EXTENT
> #:CONTINUATION-29298))
>                               (SCREAMER::AN-INTEGER-BETWEEN-
> NONDETERMINISTIC #:CONTINUATION-29298
>  
> 1
>  
> #:DUMMY-29296)))))))
>                (DECLARE (DYNAMIC-EXTENT #:CONTINUATION-29303))
>                (SCREAMER::AN-INTEGER-BETWEEN-NONDETERMINISTIC
> #:CONTINUATION-29303 1 #:DUMMY-29301))))))
>   VALUES)

-- 
http://lispm.dyndns.org
From: Joe Marshall
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <1179690016.396975.110720@a26g2000pre.googlegroups.com>
On May 20, 10:36 am, Rainer Joswig <······@lisp.de> wrote:
> In article <·······················@z28g2000prd.googlegroups.com>,
>  Joe Marshall <··········@gmail.com> wrote:
>
> sometimes the fun reading postings is proportional to the line width...
>
> could you repost it without those added line breaks? Please.

Stupid Google news posting grumble grumble.  I don't think I can
disable the extra line breaks.  I copied a good chunk of my response
from
http://mail.python.org/pipermail/python-list/2003-October/229805.html

so you can get a lot of the text there with the correct line breaks.
From: Xah Lee
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <1179806297.841391.264260@r3g2000prh.googlegroups.com>
Joe, y'know?, in Mathematica, just about every line of code is like a
lisp's macro, except that it's more powerful?

This analogy is helpful:

in imperative lang, their people are often proud and wonder at the
power of their eval() function. For example, Perl, PHP, Python all
have eval(). I explain, in pratical ways, that lisp's inherent symbol
system,
basically makes the entire language a “eval” system. (see What is
Expressiveness in a Computer Language
http://xahlee.org/perl-python/what_is_expresiveness.html )

Sometimes few months ago, i was studying lisp macros, and realized a
similar analogy. One often hear Lisper wonder and marvel about lisp's
macro system, which gives the lisp touting right as a meta-programing
language.  But in Mathematica, in practical sense, almost every line
of code functions as lisp's macro; the entire language is just one
giant macro system! It is not a wonder, that one line of Mathematica
is like 10 to 100 lines of Common Lisp. (not considering the
mathematical function in mathematica, like derivative, integral,
differential equation, transcendental (math) functions... etc. If we
consider these, then one line of mathematica is pratically like one
thousand lines of Common Lisp, Allah!)

  Xah
  ···@xahlee.org
∑ http://xahlee.org/
From: Joe Marshall
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <1179806748.468147.323230@u36g2000prd.googlegroups.com>
On May 21, 8:58 pm, Xah Lee <····@xahlee.org> wrote:
> Joe, y'know?, in Mathematica, just about every line of code is like a
> lisp's macro, except that it's more powerful?

I'm afraid that people like Harrop have made me jaded.  Show me
something.  Make my jaw drop.
From: Jon Harrop
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <4655cd94$0$8739$ed2619ec@ptn-nntp-reader02.plus.net>
Joe Marshall wrote:
> ... Show me something ...

Typesetting macros convert mathematical notation into scene graphs that can
be rendered:

  http://www.ffconsultancy.com/tmp/mathematica.png

-- 
Dr Jon D Harrop, Flying Frog Consultancy
The F#.NET Journal
http://www.ffconsultancy.com/products/fsharp_journal/?usenet
From: Raymond Wiker
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <m2irainm01.fsf@RawMBP.local>
Jon Harrop <···@ffconsultancy.com> writes:

> Joe Marshall wrote:
>> ... Show me something ...
>
> Typesetting macros convert mathematical notation into scene graphs that can
> be rendered:
>
>   http://www.ffconsultancy.com/tmp/mathematica.png

	Bah.

        http://www.fractalconcept.com/ex.pdf

   
From: Jon Harrop
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <46561878$0$8745$ed2619ec@ptn-nntp-reader02.plus.net>
Raymond Wiker wrote:
> Bah.
> 
>         http://www.fractalconcept.com/ex.pdf

Nice. :-)

-- 
Dr Jon D Harrop, Flying Frog Consultancy
The F#.NET Journal
http://www.ffconsultancy.com/products/fsharp_journal/?usenet
From: Abdulaziz Ghuloum
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <1179808677.683886.139480@q66g2000hsg.googlegroups.com>
On May 21, 11:58 pm, Xah Lee <····@xahlee.org> wrote:

> one line of mathematica is pratically like one thousand lines of Common Lisp

Your statement reminds me of "Master Foo and the Ten Thousand
Lines"[1] except that Master Foo knows what he's talking about.

Aziz,,,

[1] http://catb.org/~esr/writings/unix-koans/ten-thousand.html
From: fireblade
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <1179750894.570076.282700@b40g2000prd.googlegroups.com>
On May 20, 7:25 pm, Joe Marshall <··········@gmail.com> wrote:
> Here are a couple of links that demonstrate good uses of macros in
> Common Lisp.  These examples are not short, nor are they particularly
> macro-laden.  They show how a couple of judiciously placed macros can
> make useful changes to the language.
>
> Pragmatic Parsing in Common Lisp
> Henry Bakerhttp://home.pipeline.com/~hbaker1/Prag-Parse.html
>
> SERIES package for Common Lisp
> Richard Watershttp://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html
>

Just out of curiosity, I downloaded series package after reading cltl2
but I didn't had a chance to play with it. What's your experiences?

thanks
bobi
From: Joe Marshall
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <1179770323.532386.130210@y2g2000prf.googlegroups.com>
On May 21, 5:34 am, fireblade <·················@gmail.com> wrote:
> On May 20, 7:25 pm, Joe Marshall <··········@gmail.com> wrote:
>
> > Here are a couple of links that demonstrate good uses of macros in
> > Common Lisp.  These examples are not short, nor are they particularly
> > macro-laden.  They show how a couple of judiciously placed macros can
> > make useful changes to the language.
>
> > Pragmatic Parsing in Common Lisp
> > Henry Bakerhttp://home.pipeline.com/~hbaker1/Prag-Parse.html
>
> > SERIES package for Common Lisp
> > Richard Watershttp://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html
>
> Just out of curiosity, I downloaded series package after reading cltl2
> but I didn't had a chance to play with it. What's your experiences?

It's a mixed bag.  For the things it does well, it does a *very* nice
job.  For some of the other things, it can be persuaded.  For some
things it is simply the wrong tool.  I was using it for a lot of file
processing and it was great to write things like this:

(defun utf-8->ucs-2 (utf8string)
  "Convert a ustring to a lw:text-string."
  (collect 'string
           (#m code-char
               (ucs-4->ucs-2
                (utf-8->ucs-4
                 (scan 'simple-vector-8b
                       (code-unit-vector utf8string)))))))

But since series will only optimize those computations that can go in
lock-step, it was, shall we say, *interesting* to write utf-8->ucs-4
where we were consuming between 1 and 6 bytes per output.
From: Tim X
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <87fy5pe1c9.fsf@lion.rapttech.com.au>
Joe Marshall <··········@gmail.com> writes:

> Subject: Macros for seriously interested people

Thanks Joe, there is some nice 'meat' in that post!

Tim
-- 
tcross (at) rapttech dot com dot au
From: Rob St. Amant
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <f2uldo$h0e$1@blackhelicopter.databasix.com>
Tim X <····@nospam.dev.null> writes:

> Joe Marshall <··········@gmail.com> writes:
>
>> Subject: Macros for seriously interested people
>
> Thanks Joe, there is some nice 'meat' in that post!

I find Screamer in particular a really impressive piece of work.
From: Matthew Swank
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <pan.2007.05.22.20.09.48.793739@c.net>
On Tue, 22 May 2007 07:50:40 -0400, Rob St. Amant wrote:

> Tim X <····@nospam.dev.null> writes:
> 
>> Joe Marshall <··········@gmail.com> writes:
>>
>>> Subject: Macros for seriously interested people
>>
>> Thanks Joe, there is some nice 'meat' in that post!
> 
> I find Screamer in particular a really impressive piece of work.

Screamer is an impressive grab bag of techniques to embed an extension
into lisp (not that I have a deep understanding of the code).  However,
it's not seamless.  My frustrations in writing my own extensions come
from lacking the nuance to both hide the machinery, and stay in the
language at the same time (especially where flow control is involved).

Matt 

-- 
"You do not really understand something unless you
 can explain it to your grandmother." - Albert Einstein.
From: Tim X
Subject: Re: Macros for seriously interested people
Date: 
Message-ID: <871wh8e057.fsf@lion.rapttech.com.au>
·······@ncsu.edu (Rob St. Amant) writes:

> Tim X <····@nospam.dev.null> writes:
>
>> Joe Marshall <··········@gmail.com> writes:
>>
>>> Subject: Macros for seriously interested people
>>
>> Thanks Joe, there is some nice 'meat' in that post!
>
> I find Screamer in particular a really impressive piece of work.

I'd noticed the Debian screamer package and had put it on my "Things to check
out" list. No, after Joe's post and looking at some of those links, I've just
started checking it out. I does look pretty impressive to me, but then again,
I'm still at the stage of being impressed on pretty much a weekly basis by
the way things are done in CL.

Tim
-- 
tcross (at) rapttech dot com dot au