From: Tim Bradshaw
Subject: What you can do with macros in Lisp
Date:
Message-ID: <ey3u16e4rkq.fsf@cley.com>
Someone mentioned that there's been the usual futile argumentation
between Lisp people and you-don't-need-macros-at-all and/or
you-can-do-it-all-with-HOFs people.
Here's something I'd like to see non-macro people do.
Imagine you have a language with a CASE expression. You're writing
(or generating: for instance in the output of some kind of FSM system)
code that has a huge number of large, dense, integer CASE expressions,
which can obviously be optimised into jump-tables. But your compiler
won't do this, so your code is spending lots of time testing integers
repeatedly.
So: implement NCASE, which looks exactly like your language's CASE
expression, but, if all the keys are numeric and if they pass some
user-definable test - for instance that there are enough keys and they
are dense enough in their range, will expand to a jump-table.
Otherwise it should just expand to the ordinary CASE expression.
Can you do this with macros in Lisp? Yes: in fact I did this ages
ago, with results I've now put up at
http://www.tfeb.org/lisp/toys.html#NCASE.
Can you do it any other way (in partiucular: in any way which isn't
equivalent to writing a macro system...)
--tim
I've randomly sampled posts from the various interminable threads on
this subject from time to time, and one issue seems never to get
discussed. Python and LISP represent two very different development
methodologies. In my world of practical programming, Python is almost
always adjunct to development in C, C++, Java, and hopefully (in the
future) something like Pyrex. In other words, there is pure Python,
then there is the other stuff you do behind the scenes to make Python
go fast, or (to refer to your example) to do something really nasty &
ugly that might be better handled by say C++ templates or the like.
OTOH, it seems like LISP took a tangent whereby there is a need (or
perhaps a percieved need) to do *everything* in LISP. This mandate
creates the need for a plethora of features that, in the Pythonic way
of doing things, should (and are) left outside the scope of the
language proper.
And anyway, if I needed to generate a lot of ugly lookup code, my
first pass would be to write a Python program to generate it. Is that
a macro system?
-dbm
Tim Bradshaw <···@cley.com> wrote in message news:<···············@cley.com>...
> Someone mentioned that there's been the usual futile argumentation
> between Lisp people and you-don't-need-macros-at-all and/or
> you-can-do-it-all-with-HOFs people.
>
> Here's something I'd like to see non-macro people do.
>
> Imagine you have a language with a CASE expression. You're writing
> (or generating: for instance in the output of some kind of FSM system)
> code that has a huge number of large, dense, integer CASE expressions,
> which can obviously be optimised into jump-tables. But your compiler
> won't do this, so your code is spending lots of time testing integers
> repeatedly.
>
> So: implement NCASE, which looks exactly like your language's CASE
> expression, but, if all the keys are numeric and if they pass some
> user-definable test - for instance that there are enough keys and they
> are dense enough in their range, will expand to a jump-table.
> Otherwise it should just expand to the ordinary CASE expression.
>
> Can you do this with macros in Lisp? Yes: in fact I did this ages
> ago, with results I've now put up at
> http://www.tfeb.org/lisp/toys.html#NCASE.
>
> Can you do it any other way (in partiucular: in any way which isn't
> equivalent to writing a macro system...)
>
> --tim
From: ·············@comcast.net
Subject: Re: What you can do with macros in Lisp
Date:
Message-ID: <k77a2hx8.fsf@comcast.net>
·········@yahoo.com (dan) writes:
> I've randomly sampled posts from the various interminable threads on
> this subject from time to time, and one issue seems never to get
> discussed. Python and LISP represent two very different development
> methodologies. In my world of practical programming, Python is
> almost always adjunct to development in C, C++, Java, and hopefully
> (in the future) something like Pyrex. In other words, there is pure
> Python, then there is the other stuff you do behind the scenes to
> make Python go fast, or (to refer to your example) to do something
> really nasty & ugly that might be better handled by say C++
> templates or the like.
Exactly the problem. Sometimes you need to step up and program at
the meta-level. If your language can't do that, you step outside
the language.
> OTOH, it seems like LISP took a tangent whereby there is a need (or
> perhaps a percieved need) to do *everything* in LISP.
Or perhaps `no obstacle'.
> And anyway, if I needed to generate a lot of ugly lookup code, my
> first pass would be to write a Python program to generate it. Is that
> a macro system?
Sort of. If the source code were not very Pythonesque, I'd be more
tempted to call it a compiler.
dan wrote:
> And anyway, if I needed to generate a lot of ugly lookup code, my
> first pass would be to write a Python program to generate it. Is that
> a macro system?
Yes, sort of. Typically, macros are embedded in the language that they
generate code for so that you don't need an explicit separate program
generation stage.
Pascal
From: Tim Bradshaw
Subject: Re: What you can do with macros in Lisp
Date:
Message-ID: <ey38ynqvxr1.fsf@cley.com>
* dan wrote:
> And anyway, if I needed to generate a lot of ugly lookup code, my
> first pass would be to write a Python program to generate it. Is that
> a macro system?
Almost. A macro system in the (Common) Lisp sense is one where the
language you are generating is the same as the one as the one in which
you write the macros, and the source text which you are using to
generate is also the same language.
A system where you take a source language and transform it into some
target language, using a possibly third language to do the
transformation (although this is very commonly the same as the source
language) is really a compiler, at least to Lisp people.
Of course, a macro system has to bottom-out somewhere - eventually the
macros have to expand to something which isn't yet more macros.
But the point of my article was that a macro system is in fact quite
close to a compiler, and it can do some of the sort of things that
compilers can do - namely transform comprehensible user code to
incomprehensible `machine language' (which happens to be written in
the same language, although often not in a form that any human would
wish to write).
I would, incidentally, much like to see C++ templates doing this kind
of transformation of CASE.
--tim
·········@yahoo.com (dan) wrote:
> OTOH, it seems like LISP took a tangent whereby there is a
> need (or perhaps a percieved need) to do *everything* in
> LISP.
This idea would be better expressed by saying
OTOH, LISP aims at the commercially attractive prospect of
achieving substantial cost savings by standardising on a
single language.
Look at the rest of industry. It keeps asking questions such
as
Do we need both 50Hz and 60Hz electrical machinery?
Could we save money by standardising?
Why do we have both battery and propane forklifts in our
warehouse? Could we save money by standardising?
Why are we ever installing single electric sockets? Why
don't we standarise on just double sockets?
Why do we have different engines for 7-series and
Rolls-Royce motorcar. Wouldn't we save a fortune by
using a single engine.
Why does our fleet use both Pratt and Whitney, and Rolls
Royce areo-engines?
Seeking cost savings through standarisation is conventional
wisdom across the whole of industry. Treating Lisp as an
exception in which this concern is "tangential" mis-frames
the debate.
Alan Crowe
dan wrote:
> I've randomly sampled posts from the various interminable threads on
> this subject from time to time, and one issue seems never to get
> discussed. Python and LISP represent two very different development
> methodologies. In my world of practical programming, Python is almost
> always adjunct to development in C, C++, Java, and hopefully (in the
> future) something like Pyrex. In other words, there is pure Python,
> then there is the other stuff you do behind the scenes to make Python
> go fast, or (to refer to your example) to do something really nasty &
> ugly that might be better handled by say C++ templates or the like.
>
> OTOH, it seems like LISP took a tangent whereby there is a need (or
> perhaps a percieved need) to do *everything* in LISP. This mandate
> creates the need for a plethora of features that, in the Pythonic way
> of doing things, should (and are) left outside the scope of the
> language proper.
Since I don't know much about how Python is used, I can't comment on the
Pythonic Way. But the answer to your question about Lisp is, Yes and
No. Lisp is a general-purpose programming language, so, yes, you should
be able to do lots of stuff with it. On the other hand, if you want to
connect to a program written in another language, why not?
Lisp goes beyond There's More than One Way to Do It, and beyond There's
One Right Way to Do it, all the way to Anyway You Want to Do It, You Can
Do It That Way in Lisp. You know what they say, anarchy isn't perfect,
but it's better than no government at all.
>
> And anyway, if I needed to generate a lot of ugly lookup code, my
> first pass would be to write a Python program to generate it. Is that
> a macro system?
I endorse Pascal Costanza's reply to this question, with the following
addendum: there's no separate code-generation phase, but there's also no
separate code-compilation-and-use phase. You generate the ugly code and
use it immediately. Program-generated code is mixed in with the stuff
the programmer wrote.
--
-- Drew McDermott
Yale Computer Science Department
Drew McDermott wrote:
> Lisp goes beyond There's More than One Way to Do It, and beyond There's
> One Right Way to Do it, all the way to Anyway You Want to Do It, You Can
> Do It That Way in Lisp.
I'd vote for making the latter the official slogan for Lisp! ;)
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
Pascal Costanza wrote:
> Drew McDermott wrote:
>
>> Lisp goes beyond There's More than One Way to Do It, and beyond
>> There's One Right Way to Do it, all the way to Anyway You Want to Do
>> It, You Can Do It That Way in Lisp.
>
>
> I'd vote for making the latter the official slogan for Lisp! ;)
Yes, and then we can replace "I Love NY" with "See you at the Port
Authority Men's Room".
The language is verstile, not promiscuous. It has multiple modalities so
you can find The Right One for the problem at hand. There is exactly one
way to skin that cat in that condition with these available knives.[1]
kenny
[1] No animals were harmed in this rant.
--
http://tilton-technology.com
What?! You are a newbie and you haven't answered my:
http://alu.cliki.net/The%20Road%20to%20Lisp%20Survey
Tim Bradshaw wrote:
> Someone mentioned that there's been the usual futile argumentation
> between Lisp people and you-don't-need-macros-at-all and/or
> you-can-do-it-all-with-HOFs people.
The ancient context for this thread was a prior incarnation arising
amongst Pythonistas discussing on clp whether or not Python should have
macros in version 42, due out in fifteen years.
So I am adding clp in this otherwise unmodified version of your original.
Me, I finally realized two things: (a) Graham explains it better:
http://www.paulgraham.com/onlisp.html
..and two, hey, one of Python's most important design principle is
visually clean code. I think I use macros more for that than anything else.
kenny
>
> Here's something I'd like to see non-macro people do.
>
> Imagine you have a language with a CASE expression. You're writing
> (or generating: for instance in the output of some kind of FSM system)
> code that has a huge number of large, dense, integer CASE expressions,
> which can obviously be optimised into jump-tables. But your compiler
> won't do this, so your code is spending lots of time testing integers
> repeatedly.
>
> So: implement NCASE, which looks exactly like your language's CASE
> expression, but, if all the keys are numeric and if they pass some
> user-definable test - for instance that there are enough keys and they
> are dense enough in their range, will expand to a jump-table.
> Otherwise it should just expand to the ordinary CASE expression.
>
> Can you do this with macros in Lisp? Yes: in fact I did this ages
> ago, with results I've now put up at
> http://www.tfeb.org/lisp/toys.html#NCASE.
>
> Can you do it any other way (in partiucular: in any way which isn't
> equivalent to writing a macro system...)
>
> --tim
>
>
--
http://tilton-technology.com
What?! You are a newbie and you haven't answered my:
http://alu.cliki.net/The%20Road%20to%20Lisp%20Survey
Tim Bradshaw <···@cley.com> writes:
> Someone mentioned that there's been the usual futile argumentation
> between Lisp people and you-don't-need-macros-at-all and/or
> you-can-do-it-all-with-HOFs people.
>
> Here's something I'd like to see non-macro people do.
>
> Imagine you have a language with a CASE expression. You're writing
> (or generating: for instance in the output of some kind of FSM system)
> code that has a huge number of large, dense, integer CASE expressions,
> which can obviously be optimised into jump-tables. But your compiler
> won't do this, so your code is spending lots of time testing integers
> repeatedly.
>
> So: implement NCASE, which looks exactly like your language's CASE
> expression, but, if all the keys are numeric and if they pass some
> user-definable test - for instance that there are enough keys and they
> are dense enough in their range, will expand to a jump-table.
> Otherwise it should just expand to the ordinary CASE expression.
>
> Can you do this with macros in Lisp? Yes: in fact I did this ages
> ago, with results I've now put up at
> http://www.tfeb.org/lisp/toys.html#NCASE.
>
> Can you do it any other way (in partiucular: in any way which isn't
> equivalent to writing a macro system...)
>
> --tim
Well, as explained in Paul Graham's "On Lisp" in his extended coverage
of LISP macros, if you don't mind the typing (and most importantly the
associated obfuscation by redundant details), you can always replace
macro calls by function calls encapsulating the bodies in closure
lambdas (or even, merely quoting them and let the called function make
a function of these lists).
The most difficult stuff is the manipulation of variables (lexical
scope) and declarations in general. But since LISP is highly
(totally?) dynamic, you can always generate a declaration at run-time
rather than compilation time. Well, perhaps doing compilation-time
stuff at run-time is the equivalent of writing a macro system...
http://lib1.store.vip.sc5.yahoo.com/lib/paulgraham/onlisp.pdf
(defmacro with-var (varname value &body body)
`(let ((,varname ,value)) ,@body))
(with-var x (* 6 7)
(format t "x =~D~%" x)
(format t "2x =~D~%" (* 2 x)))
x =42
2x =84
NIL
(defun f-with-var-1 (varname value body)
(set varname value) ;; wrong, we're setting a global variable
(funcall body))
(f-with-var-1 'x (* 6 7)
(lambda () (format t "x =~D~%" x)
(format t "2x =~D~%" (* 2 x))))
x =42
2x =84
NIL
(format t "x= ~D~%" x)
x= 42
NIL
(makunbound 'x)
;;X
(defun f-with-var-2 (varname value body)
;; cheating or not?
(eval (list `(lambda (,varname) (progn ,@body)) value)))
(f-with-var-2 'x '(* 6 7)
'((format t "x =~D~%" x)
(format t "2x =~D~%" (* 2 x))))
x =42
2x =84
NIL
Of course, if you add the constraint of not having to change the
source and quote everything, it's trivially true that you cannot do
what you're asking without a macro system.
I note that when I program in languages other than LISP, my sources
are covered by comments containing emacs lisp code, followed by some
automatically generated code...
--
__Pascal_Bourguignon__
http://www.informatimago.com/
Do not adjust your mind, there is a fault in reality.
Pascal Bourguignon wrote:
> Well, as explained in Paul Graham's "On Lisp" in his extended coverage
> of LISP macros, if you don't mind the typing (and most importantly the
> associated obfuscation by redundant details),...
Right. I want to start this almost 500 message thread over again and say
to the Pythonistas, "I thought you valued visually cleaner code?".
kenny
--
http://tilton-technology.com
What?! You are a newbie and you haven't answered my:
http://alu.cliki.net/The%20Road%20to%20Lisp%20Survey
On Sun, 12 Oct 2003 14:49:24 GMT,
Kenny Tilton <·······@nyc.rr.com> wrote:
> Right. I want to start this almost 500 message thread over again and say
> to the Pythonistas, "I thought you valued visually cleaner code?".
Yes, but we also value people who know when the horse is dead.
--amk
From: Tim Bradshaw
Subject: Re: What you can do with macros in Lisp
Date:
Message-ID: <ey3llrq4ijs.fsf@cley.com>
* Pascal Bourguignon wrote:
> Well, as explained in Paul Graham's "On Lisp" in his extended coverage
> of LISP macros, if you don't mind the typing (and most importantly the
> associated obfuscation by redundant details), you can always replace
> macro calls by function calls encapsulating the bodies in closure
> lambdas (or even, merely quoting them and let the called function make
> a function of these lists).
And this generates a jump-table at compile-time? How does it do that?
--tim
Pascal Bourguignon <····@thalassa.informatimago.com> wrote in message news:<··············@thalassa.informatimago.com>...
> Tim Bradshaw <···@cley.com> writes:
> > Can you do this with macros in Lisp? Yes: in fact I did this ages
> > ago, with results I've now put up at
> > http://www.tfeb.org/lisp/toys.html#NCASE.
> >
> > Can you do it any other way (in partiucular: in any way which isn't
> > equivalent to writing a macro system...)
> >
> > --tim
>
> Well, as explained in Paul Graham's "On Lisp" in his extended coverage
> of LISP macros, if you don't mind the typing (and most importantly the
> associated obfuscation by redundant details), you can always replace
> macro calls by function calls encapsulating the bodies in closure
> lambdas (or even, merely quoting them and let the called function make
> a function of these lists).
It doesn't work this simply. If you let the called function make
functions out of lists, then you break the connection to the lexical
environment. The function behaves like EVAL or COMPILE.
You'd need a language in which everything is already a function that
carries the lexical environment, and in which the evaluation of
everything can be delayed to the latest possible time and controlled.
That still doesn't give you the complete power of macros, because a
macro can not only control the timing or repetition of the evaluation
of a piece of the program, it can control every aspect of the
relationship between its syntax and semantics.
It's true that higher order functions can adequately support some of
the use cases served by macros, but not all. Some macro uses require
that the macro have access to the original source code, not to a
cooked and canned function-object! And yet, those same uses still want
access to the lexical environment.
Consider a macro which understands some language that is quite
different from ordinary Lisp, but whose utterances seamlessly
interoperate with the surrounding Lisp: so that for example instances
of that language make references to lexical variables defined in the
surrounding Lisp and the like.
> The most difficult stuff is the manipulation of variables (lexical
> scope) and declarations in general. But since LISP is highly
> (totally?) dynamic, you can always generate a declaration at run-time
> rather than compilation time.
Not if it is to be bound to the lexical environment that was processed
at compilation time!
> Well, perhaps doing compilation-time
> stuff at run-time is the equivalent of writing a macro system...
There is no limit to what you can achieve at run time: your run time
can implement an interpreter or compiler for a programming language.
> (defmacro with-var (varname value &body body)
> `(let ((,varname ,value)) ,@body))
>
> (with-var x (* 6 7)
> (format t "x =~D~%" x)
> (format t "2x =~D~%" (* 2 x)))
>
> x =42
> 2x =84
> NIL
(let ((y 43))
(with-var x (* 6 7)
(format t "x = ~D~%" x)
(format t "y = ~D~%" y)))
;; output
;; 42
;; 43
> (defun f-with-var-2 (varname value body)
> ;; cheating or not?
> (eval (list `(lambda (,varname) (progn ,@body)) value)))
>
>
> (f-with-var-2 'x '(* 6 7)
> '((format t "x =~D~%" x)
> (format t "2x =~D~%" (* 2 x))))
(let ((y 43))
(f-with-var-2 'x '(* 7)
'((format t "x = ~D~%" x)
(format t "y = ~D~%" y))))
;; error: inner EVAL refers to nonexistent dynamic binding of Y.
Your run time compilation is disconnected from the lexical environment
which invokes it.
> x =42
> 2x =84
> NIL
>
>
>
> Of course, if you add the constraint of not having to change the
> source and quote everything, it's trivially true that you cannot do
> what you're asking without a macro system.
There is more to it than the quote, as you can see. The macro system
does more than just provide you with a way to wrap F-WITH-VAR-2 to
eliminate quoting! If you do that, it's still broken:
(defmacro with-var (varname value body)
`(f-with-var-2 ',varname ',value ',body))
(let ((y 43))
(with-var x (* 6 7)
(format t "x = ~D~%" x)
(format t "y = ~D~%" y))) ;; error!
One of my favorite macro packages, and one I use almost every day,
is Waters' COVER package.
http://www-2.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/testing/cover/0.html
Less than 300 (nonblank noncomment) lines of lisp implements a code coverage tool
for the language, embedded in the language rather than off in some compiler or preprocessor.
How hard would it be to implement this in Python?
Paul
···@ashi.footprints.net (Kaz Kylheku) writes:
> It doesn't work this simply. If you let the called function make
> functions out of lists, then you break the connection to the lexical
> environment. The function behaves like EVAL or COMPILE.
>
> [...]
>
> (let ((y 43))
> (f-with-var-2 'x '(* 7)
> '((format t "x = ~D~%" x)
> (format t "y = ~D~%" y))))
>
> ;; error: inner EVAL refers to nonexistent dynamic binding of Y.
>
> Your run time compilation is disconnected from the lexical environment
> which invokes it.
Indeed. I still keep some emacs lisp neurons :-)
--
__Pascal_Bourguignon__
http://www.informatimago.com/
Do not adjust your mind, there is a fault in reality.