On 2005-11-26 21:23:22 +0000, ········@gmail.com said:
> kinda silly but hey, why not? http://www.lisperati.com/no_macros.html
> :P
Kind of cute, but on the other hand ...
"We, in Lisp land, have backquote!"
- no confusion with the word "macro"
- kind of weird/magical feeling (most people, apart from
froggies, don't even know where the key fits on the keyboard)
- certainly less pompous than "SPEL" once the acronym is
explained
- the use of "backquote" to refer to "macros" mixes the
means and the ends, just like Lisp mixes code and data
when using macros ...
Nice try, though.
--
JFB ()
I like the concept of calling them "backquotes"...
It does capture the concept pretty well... only problem is, that that
word already has a meaning in Lisp... maybe if you call it a "backquote
command" and then use "backquote" for short when there is no ambiguity
in what is being said...
-Conrad Barski
www.lisperati.com
······@gmail.com writes:
> I like the concept of calling them "backquotes"...
Please don't.
> It does capture the concept pretty well... only problem is, that
> that word already has a meaning in Lisp... maybe if you call it a
> "backquote command" and then use "backquote" for short when there is
> no ambiguity in what is being said...
It seems to me that one of the biggest stumbling blocks to a proper
understanding of macros is not getting that macros are *just*
functions that manipulate s-expressions to produce s-expressions that
represent Lisp code. Because so many macros, especially the simple
macros used in tutorials, can be written with just a backquote
template, new Lispers get the impression that the "magic" of macros
comes from backquote. To use terminology that encourages this
confusion seems like a really bad idea to me.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
In reality, there is little to no difference between Lisp macros and
the macros in C++. Both do exactly the same thing (at the end): token
substitution. The difference with Lisp lies in how the text is
substituted.
In C/C++, compiling a program is [basically] broken up into:
1. Parsing/tokenising
2. Pre-compiling
3. Lexical analysis
4. Compiling
5. Linking
Now, the C/C++ compiler works per file, and code generation doesn't
actually happen until the end (there may be intermediate code
generation, and optimization is in there as well, but essentially this
is the order). Since the pre-compiler doesn't actually have information
about the generated code, C/C++ macros can *only* do simple token
substitution.
In Lisp, however, (I'm gonna botch this and I;m sure I'll get corrected
by those who know more) there are distinct phases per READ operation:
parsing/tokenising (read), macro-expansion, then compiling. This means
that in subsequent read operations, Lisp can execute previously
generated code. So, while Lisp is technically only doing token
substitution as well, it can execute code that generates the desired
tokens.
Jeff M.
"Jeff M." <·······@gmail.com> writes:
> In Lisp, however, (I'm gonna botch this and I;m sure I'll get corrected
> by those who know more) there are distinct phases per READ operation:
> parsing/tokenising (read), macro-expansion, then compiling. This means
> that in subsequent read operations, Lisp can execute previously
> generated code. So, while Lisp is technically only doing token
> substitution as well,
No, Lisp is doing tree manipulation, of which token substitution is
only a small (and uninteresting) part.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
On Sun, 27 Nov 2005 01:30:06 +0100, Jeff M. <·······@gmail.com> wrote:
> In reality, there is little to no difference between Lisp macros and
> the macros in C++. Both do exactly the same thing (at the end): token
> substitution. The difference with Lisp lies in how the text is
> substituted.
>
Lisp has access to the entire language when generating code.
This is a lot more powerfull than token substitution.
For instance to embedd sql in a c++ program you need
a seperate c++ preprosessor which translates a "exec sql <>"
to C function calls.
In Lisp of course the same can be done with macroes.
I dare you to write the loop macro using #define :)
Another difference is that once a macro is defined it can be used
anywhere not just in the souce file. Care must be taken though
so that files that use the macro are also recompiled when the
definition changes. Hence the need for defsystem/ADSF.
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
I don't know how you got what you implied from what I posted. To my
knowledge it is just token substitution at read-time. The only
difference is in how the tokens are generated. In C/C++ and other
languages with a typical pre-compiler, the tokens are predefined. In
Lisp, since it has acces to the language at read-time, it can generate
tokens to use. We stated exactly the same thing.
Jeff M.
Jeff M. wrote:
> I don't know how you got what you implied from what I posted. To my
> knowledge it is just token substitution at read-time. The only
> difference is in how the tokens are generated. In C/C++ and other
> languages with a typical pre-compiler, the tokens are predefined. In
> Lisp, since it has acces to the language at read-time, it can generate
> tokens to use. We stated exactly the same thing.
"The only difference is..." -- while technically correct, in this
difference lie worlds! While in C/C++ you are limited to replace
expressions by patttern matching with others, you have the full
Lisp language at hand to calculate the substitution, and what is
even more powerfull, you have state. That means, while doing the
macroexpansion, you can save any interesting information in your
Lisp image as well as read out any variable that may contain
relevant information. So the expansion of one macro may result
in quite different code over time.
Peter
--
Ltk, the easy lisp gui http://www.peter-herth.de/ltk/
"Jeff M." <·······@gmail.com> wrote in message
····························@f14g2000cwb.googlegroups.com...
>I don't know how you got what you implied from what I posted. To my
> knowledge it is just token substitution at read-time. The only
> difference is in how the tokens are generated. In C/C++ and other
> languages with a typical pre-compiler, the tokens are predefined. In
> Lisp, since it has acces to the language at read-time, it can generate
> tokens to use. We stated exactly the same thing.
Hi Jeff,
I think one of the biggest misunderstandings we are having here is just what
you mean by "substitution". Clearly most people are taking that in the
C/C++ sense of rearranging a set of tokens in another pattern. You seem to
be using it in a way that includes generating entirely new source code from
that passed to the macro. That's fine, then, in that sense yes it is a just
a fancier and more versatile substitution of one set of forms for another.
But you are are too casual about the fact that a Lisp macro has the use of
the full language, plus your own code, at its disposal. This means not just
amazing code transformations, it means side effects. Your macro can define
functions, interact with a user, print to *standard-output*, store and
retrieve data, you name it. This is nothing to sneeze at.
--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
"Jeff M." <·······@gmail.com> writes:
> there is little to no difference between Lisp macros and the macros
> in C++. Both do exactly the same thing (at the end): token substitution.
Perhaps you are using a definition of token substitution that is
different from the one in use in comp.lang.lisp. I believe it's clear
that in
#define foo(x, y, z) bar(z, y) + baz(x)
the tokens x, y, and z in the token sequence making up the macro
template are substituted with the corresponding expressions in the
macro call.
If your experience with Lisp macros is limited to things like
(defmacro foo (x y z)
`(+ (bar ,z ,y) (baz ,x)))
it would seem like this is just like C preprocessor macros. However,
in general, Lisp macros can do arbitrary transformations on their
inputs, and the output doesn't have to come from a template. Actually,
Lisp macros are (or can be) compilers: they translate from an input
language to an output language. Token substituion can't do that.
Lars Brinkhoff wrote:
> "Jeff M." <·······@gmail.com> writes:
>
>>there is little to no difference between Lisp macros and the macros
>>in C++. Both do exactly the same thing (at the end): token substitution.
>
>
> Perhaps you are using a definition of token substitution that is
> different from the one in use in comp.lang.lisp. I believe it's clear
> that in
>
> #define foo(x, y, z) bar(z, y) + baz(x)
>
> the tokens x, y, and z in the token sequence making up the macro
> template are substituted with the corresponding expressions in the
> macro call.
>
> If your experience with Lisp macros is limited to things like
>
> (defmacro foo (x y z)
> `(+ (bar ,z ,y) (baz ,x)))
>
> it would seem like this is just like C preprocessor macros. However,
> in general, Lisp macros can do arbitrary transformations on their
> inputs, and the output doesn't have to come from a template. Actually,
> Lisp macros are (or can be) compilers: they translate from an input
> language to an output language. Token substituion can't do that.
In addition to arbitrary transformations on their inputs, Lisp macros
can also do arbitrary side effects that don't show up (directly) in
their outputs (or what one typically thinks of as their outputs). The
whole genre of "with-" macros (with-open-file being probably the best
known) is based on setting up an environment/context/whatever in which
the arbitrary-transformation-of-inputs code will run. The ability to do
whatever you darn please in a macro _in addition to_ playing obviously
viisble transformations on your inputs is the thing that I find so
remarkable (and sometimes such a royal pain).
paul
Paul Wallich <··@panix.com> writes:
> In addition to arbitrary transformations on their inputs, Lisp macros
> can also do arbitrary side effects that don't show up (directly) in
> their outputs (or what one typically thinks of as their outputs).
This is actually a very important point that is often missed (or
forgotten) even by experienced Lisp users in these sorts of
discussions. Certainly things like type inferencing or optimizer code
walkers and such would not be doable without this.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
"Jeff M." <·······@gmail.com> writes:
> In reality, there is little to no difference between Lisp macros and
> the macros in C++. Both do exactly the same thing (at the end): token
> substitution. The difference with Lisp lies in how the text is
... Remaining utter nonsense snipped ...
This is so wrong and otherwise misleading that it is difficult to know
how to even start a correction.
"Token" substitution has little or nothing to do with Lisp macros.
For one thing, Lisp macros do not operate on _text_ (they don't even
operate on "tokens" for any reasonable typical compiler oriented
definition of the term), they operate on various _data structures_,
whether of your own devise, predefined in core CL, or some other 3rd
party definitions.
Further Lisp macros are _functions_, not some sort of "templates" +
"rules of substitution" operated upon by some external processor (ala'
the C/C++ preprocessor). By being functions (just like any other
function, except they are called at a special time), they have access
to and use of any and all existing functions visible to them - this
includes the entire CL base language, any 3rd party libs, any libs you
may have created yourself, any other in house libs, any operating
system libs, etc.
Here's a couple of things to think about given these facts:
i) it would not be _technically_ misguided to write a C++ compiler
_as_ a macro (plus various ancillary helper functions). Indeed, it
would be far easier and simpler than trying to do it in C/C++ (due to
the fact that writing compilers is so much easier in general in CL
and, as we see, the entire language is available to the C++-compiler
macro). Try doing that with just C++ macrs and "token substitution".
ii) It is perfectly feasible (and under some circumstances, perhaps
even reasonable), for a Lisp macro to call out over a socket to get
needed information in order to complete its work, or to interact via a
web client to obtain such information or query a database, or perhaps
all of these together, etc. Try doing this with C++ macros and "token
substitution".
Anyone with even the ghost of a clue, knows that if you are going to
make an analogy/comparison here, the proper comparison is with C++
template "programming". But even that is but a ghost of a ghost of
what Lisp macros provide.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
Thanks for one of the most arrogant replies ever. I don't write Lisp
compilers, so I don't know the intricate details of under-the-hood
operations of them, hence why I typed:
> I'm gonna botch this and I'm sure I'll get corrected
> by those who know more
However, your comments are extremely degrading:
> Remaining utter nonsense snipped
...
> Anyone with even the ghost of a clue
Etc. I know what Lisp macros are capable of. And I didn't say that C++
macros can do what Lisp macros can. It was never even implied. However,
some of the comments you made were new things that I did get from your
post. Thank you for those.
I would like to ask a question however. Given this statement,
> "Token" substitution has little or nothing to do with Lisp macros.
> For one thing, Lisp macros do not operate on _text_, ...
It has to at some point. The compiler has to parse the text, tokenise
it, then (possibly) evaluate, and then compile if needed. The macro is
expanded in there somewhere, my understanding is that it happens
somewhere after tokenising, but before eval (call it a macro phase, but
I'm sure there is a standard name for it). During the expansion, one
set of tokens (or data) is potentially replaced with another set, which
is then passed onto the next phase.
If what I said is correct, technically both languages do data/token
substitution. The difference lies in HOW they can do it. If what I
stated is incorrect, can someone correct me in a constructive manner?
Anyways, I shall bow out. Lata, folks.
Jeff M.
"Jeff M." <·······@gmail.com> writes:
It might help if you actually included who you were replying to up
front. That's just common courtesy.
> Thanks for one of the most arrogant replies ever.
Really? I wouldn't have thought so, but YMMV. For example, your post
was very arrogant, in that it stated incorrect things and misleading
information as though they/it were "obviously" true and said them in
such a way as to indicate that you knew you were absolutely right in
your general observations (if not the details). Turns out you were
wrong in all of that. It appears that someone pointing out that you
were wrong in this ticks you off. Such is life.
> I don't write Lisp compilers, so I don't know the intricate details
> of under-the-hood operations of them, hence why I typed:
>
> > I'm gonna botch this and I'm sure I'll get corrected
> > by those who know more
Sure, but that is totally irrelevant. You don't need to write Lisp
compilers or even know anything about them to understand that Lisp
macros are not simple "token substitution" templates ala' C/C++
macros. You just have to have studied and understood them for what
they are via the abundantly available resources.
> However, your comments are extremely degrading:
>
> > Remaining utter nonsense snipped
> ...
> > Anyone with even the ghost of a clue
I'm sorry if that hurt you.
> Etc. I know what Lisp macros are capable of.
But here you state again that you know all about them, yet you made
rather egregious errors and passed them off as fact. How do you
square this in your own mind?
> And I didn't say that C++ macros can do what Lisp macros can. It was
> never even implied.
Actually, you said they were basically them same:
> In reality, there is little to no difference between Lisp macros and
> the macros in C++. Both do exactly the same thing (at the end): token
> substitution.
Now, how can any reasonable reading of those two comments from you not
conclude that you are contradicting yourself?
> However, some of the comments you made were new things that I did
> get from your post. Thank you for those.
>
> I would like to ask a question however. Given this statement,
>
> > "Token" substitution has little or nothing to do with Lisp macros.
> > For one thing, Lisp macros do not operate on _text_, ...
>
> It has to at some point.
The Lisp reader operates on text. The text is _long_ gone before
macros are invoked. Of course, reader-macros operate on text, but
that is again at a _very_ different level. Effectively reader-macros
allow for customization of the _lexer_.
> The compiler has to parse the text, tokenise it, then (possibly)
> evaluate, and then compile if needed. The macro is expanded in there
> somewhere
Actually, the compiler _may_ not even be involved (note that macros
are run at _macro expansion_ time, which is not the same as _compile_
time. Anyway, by the time macros are expanded the current working
artifact is an annotated syntax tree - a data structure recursively
composed of other data structures.
> my understanding is that it happens somewhere after tokenising, but
> before eval
It's after "parsing", but note this doesn't imply that compilation is
involved. For example:
(macroexpand '(defun foo (a) :foo-to-you))
==> ...bunch of stuff...
Nothing is compiled or evaluated here.
> (call it a macro phase, but I'm sure there is a standard
> name for it).
Macro expansion time is the term.
> During the expansion, one set of tokens (or data) is potentially
> replaced with another set, which is then passed onto the next phase.
Well, "tokens" are not involved. The input data is s-expressions.
Whether it is replaced depends on what the macro does (for example, it
could just return exactly what it was passed after noting various
things about it into the environment). There may not even be a "next
phase", as in the above example.
> If what I said is correct,
But it's not.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
My reply was written in haste. If it came across poorly, my appologies.
When I stated:
> Etc. I know what Lisp macros are capable of.
I'm not implying that I know everything about them. I know what my car
is capable of, and I use it every day, all the time. I don't know [in
detail] how the engine works. These are two different levels of
knowledge.
Your follow-up explained some more things to me. Thank you. My
understanding is increasing. :-)
However, you did the following as an example:
> (macroexpand '(defun foo (a) :foo-to-you))
> Nothing is compiled or evaluated here.
How is that the case? The REPL had to read it, and had to evaluate it
in order to display some output. True, nothing was compiled, but that
may only be because your implementation decided not to. Couldn't
another implementation decide to compile before executing it?
I think the big misunderstanding came from my use of the work "tokens".
In my opinion, I could have said "data" or "code".
Regardless, thanks for the reply.
Jeff M.
>> (macroexpand '(defun foo (a) :foo-to-you))
>> Nothing is compiled or evaluated here.
>
> How is that the case? The REPL had to read it, and had to evaluate it
> in order to display some output. True, nothing was compiled, but that
> may only be because your implementation decided not to. Couldn't
> another implementation decide to compile before executing it?
macroexpand expands the macro. It does not execute it.
maxroexpand, and the more usefull macroexpand-1 are tools
for debugging macroes. I would recomend trying a macroexpand-1
on the first expression in your code in which you intend to
use your macro. That is BEFORE making the function that
uses the macro test it first in the REPL and verify that the output is
what you intended. Particularly if you are inexperienced with
macroes this can save you a lot of frustration.
> I think the big misunderstanding came from my use of the work "tokens".
> In my opinion, I could have said "data" or "code".
Language is about comminication. If your definition of token
is different from everybody elses you are misread.
Better to change your definition to fit the accepted interpretation
(symbol output from a lexer) than make your own.
For the record macro : sexpr --> sexpr
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
"Jeff M." <·······@gmail.com> writes:
> Your follow-up explained some more things to me. Thank you. My
> understanding is increasing. :-)
>
> However, you did the following as an example:
>
> > (macroexpand '(defun foo (a) :foo-to-you))
> > Nothing is compiled or evaluated here.
>
> How is that the case? The REPL had to read it, and had to evaluate it
Well, strictly speaking, there was something evaluated - the function
form macroexpand with the quoted list as argument. Nothing about the
_macro form_ DEFUN was evaluated or compiled (or executed for that
matter), which was the salient point for the discussion. Also, I am
using the terms "evaluate", "compile", "form", et al. as formally
defined in CLHS - not as popular venacular. Which I believe makes the
most sense given the context. Also, the typical venacular meanings
are pretty fuzzy.
> in order to display some output. True, nothing was compiled, but that
> may only be because your implementation decided not to.
Actually, not quite. First the macroexpand form could have been
compiled, but the macro form could not have.
> Couldn't another implementation decide to compile before executing
> it?
The function form macroexpand, yes, the macro form no. Also the macro
form was not evaluated and thus not executed and this would be true
independent of implementation.
> I think the big misunderstanding came from my use of the work "tokens".
> In my opinion, I could have said "data" or "code".
There are two problems with this:
i) By substituting "data" for "tokens", you can sort of make your
original claim, but it then becomes empty. Why? Because in that case
you can claim that pretty much every program/function/application does
"exactly the same thing" as C/C++ macros. That is all you are saying
in this case is that they all do some sort of input to output
processing. Not particularly enlightening or of much semantic
content.
ii) It discounts the all important parts of _what_ and _how_ the
transformations are done. However that is the key bit as it is this
aspect which when added to something like i) above turns it from a
completely vacuous statement into something meaningful.
Here are a couple more things to note:
The key thing to understanding macros is to realize they are
functions. Closely followed by understanding that they are called at
a special time and how that fits into the overall processing times.
Many times here you will see the claim that macros are functions which
transform input s-expressions to output s-expressions. Strictly
speaking this is not accurate on a couple counts:
1) "s-expression" is not formally defined by the standard - at least I
don't see it in the glossary, where such things should be found,
nor does it appear in the master index. CLTL2 has it indexed under
the FORMAT ~S directive, which seems to fit well with the
traditional meaning: the _readable_ _print_ representation of some
object. Of course, some objects do _not_ have such a
representation.
2) Macros operate on forms, and forms can be any object, not only the
typically thought of case of _compound form_, which is usually what
many people have in mind by "s-expression" in these discussions.
They can also return any object.
For example,
(defclass fooclass ()
((s1 :initarg :s1)))
(defmacro foo (x)
(make-instance 'fooclass :s1 x))
Note that this macro does not produce any "code" (and certainly not
something that could be an s-expression, since in the current state of
affairs, an instance of fooclass does not have a readable print
representation). This macro is defined to return an instance of type
FOOCLASS as its result _at macroexpansion time_.
It's instructive to inspect some results of a few cases involving (or
looking like they involve) this macro:
(inspect `(foo "hi"))
==>
A proper list @ #x773e6c21 with 2 elements
0-> The symbol FOO
1-> A simple-string (2) "hi"
;; Just a quoted list here
(inspect (foo "hi"))
==>
FOOCLASS @ #x7735408a = #<FOOCLASS @ #x7735408a>
0 Class --------> #<STANDARD-CLASS FOOCLASS>
1 S1 -----------> A simple-string (2) "hi"
;; An instance of FOOCLASS
(inspect (foo (make-instance 'fooclass)))
==>
FOOCLASS @ #x77355712 = #<FOOCLASS @ #x77355712>
0 Class --------> #<STANDARD-CLASS FOOCLASS>
1 S1 -----------> (MAKE-INSTANCE ...), a proper list with 2 elements
;; An instance of FOOCLASS. Think about why S1 here _isn't_ an
;; instance of FOOCLASS.
(inspect `(foo ,(make-instance 'fooclass)))
==>
A proper list @ #x773c9839 with 2 elements
0-> The symbol FOO
1-> #<FOOCLASS @ #x7736b0ea>
;; Quoted list, but with the second element evaluated due to the comma
(inspect (macroexpand `(foo ,(make-instance 'fooclass))))
==>
FOOCLASS @ #x7736e602 = #<FOOCLASS @ #x7736e602>
0 Class --------> #<STANDARD-CLASS FOOCLASS>
1 S1 -----------> #<FOOCLASS @ #x7736e5da>
;; An instance of FOOCLASS and now S1 is also an instance of FOOCLASS.
We can force this sort of effect in more "normal" looking calls of foo
with any argument by making a slight change:
(defmacro foo (x)
(make-instance 'fooclass :s1 (eval x)))
(inspect (foo (make-instance 'fooclass)))
==>
FOOCLASS @ #x77377852 = #<FOOCLASS @ #x77377852>
0 Class --------> #<STANDARD-CLASS FOOCLASS>
1 S1 -----------> #<FOOCLASS @ #x7737782a>
While these sort of macros are not nearly as widely useful as the more
typical "code transformers", they are not useless, as they permit the
constuction of "constants" of arbitrary complexity, that may require
_huge_ amounts of processing to compute, which may be saved as part of
a function definition in a fasl output - just like "ordinary"
constants. For example, in some file foo-stuff.cl:
...
(defun some-initializer (...args...)
(let ((the-special-foo (foo ...))
...)
...))
...
In build system somewhere:
;;; NOTE: this will take a looooonnnnnnnnggg time to compile because
;;; FOO is computing the SPECIAL foo here!!
;;;
(compile-file "foo-stuff.cl")
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
> If what I said is correct, technically both languages do data/token
> substitution. The difference lies in HOW they can do it.
That HOW is a significant HOW.
You use the word "Token" where you mean "Data".
And by "substitution", you don't mean simple substitution as in C macro
but any "Transformation and calculation".
So when you said "Token substitution", Don't you actually mean "Data
transformation" ?
Don't you think there is a big difference between two words?
Most of what every application do is "transforming data".
A program that calculates any mathematical formula is just doing token
substitution; it substitutes a math expression with the result of
calculation.
You feed data to web server, and it returns a web page. In a sense,
Apache substitute URL for an HTML page.
An SQL server just substitute SQL query with the result on query.
All of the above is valid by your definition of "substitution"; You
don't care how complex it has to go through before the output is
derived, just that an input is used to derived the output.
With all implication above, I could have said that C++ macro is like
SQL Server or Apache, just a token substitution system with some minor
different :-)
Pisin Bootvong <··········@gmail.com> wrote:
+---------------
| You feed data to web server, and it returns a web page. In a sense,
| Apache substitute URL for an HTML page.
|
| An SQL server just substitute SQL query with the result on query.
|
| All of the above is valid by your definition of "substitution"; You
| don't care how complex it has to go through before the output is
| derived, just that an input is used to derived the output.
|
| With all implication above, I could have said that C++ macro is like
| SQL Server or Apache, just a token substitution system with some minor
| different :-)
+---------------
Except that -- unlike C/C++ macros -- Lisp macros can easily, during
the process of expansion, do such things as query HTTP web servers and/or
SQL databases and incorporate the results into the macro expansion!!
Try *that* with other languages' macros... ;-} ;-}
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
"Jeff M." <·······@gmail.com> writes:
> During the expansion, one set of tokens (or data) is potentially
> replaced with another set, which is then passed onto the next phase.
>
> If what I said is correct, technically both languages do data/token
> substitution. The difference lies in HOW they can do it. If what I
> stated is incorrect, can someone correct me in a constructive manner?
To most people "tokens" is not synonymous with "data" and that's why
everyone is jumping up and down on you. In both Lisp and compiler
terminology "token" usually refers to the thing that comes out of the
lexer; a sequence of tokens is then fed into a parser which builds
some sort of tree representation, which is then compiled. The key
characteristic of Lisp macros is that they operate on trees, not
tokens.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
Am 27.11.2005 1:30 Uhr schrieb "Jeff M." unter <·······@gmail.com> in
························@g49g2000cwa.googlegroups.com:
> In reality, there is little to no difference between Lisp macros and
> the macros in C++. Both do exactly the same thing (at the end): token
> substitution. The difference with Lisp lies in how the text is
> substituted.
>
> In C/C++, compiling a program is [basically] broken up into:
>
> 1. Parsing/tokenising
> 2. Pre-compiling
> 3. Lexical analysis
> 4. Compiling
> 5. Linking
>
> Now, the C/C++ compiler works per file, and code generation doesn't
> actually happen until the end (there may be intermediate code
> generation, and optimization is in there as well, but essentially this
> is the order). Since the pre-compiler doesn't actually have information
> about the generated code, C/C++ macros can *only* do simple token
> substitution.
>
> In Lisp, however, (I'm gonna botch this and I;m sure I'll get corrected
> by those who know more) there are distinct phases per READ operation:
> parsing/tokenising (read), macro-expansion, then compiling. This means
> that in subsequent read operations, Lisp can execute previously
> generated code. So, while Lisp is technically only doing token
> substitution as well, it can execute code that generates the desired
> tokens.
>
> Jeff M.
>
Just a few notes.
READ does not do macro expansion or compiling.
READ reads expressions from a stream. READ has a mechanism for so called
'Read Macros'. But those are different from 'Macros'. READ returns
Lisp datastructures.
For example COMPILE, COMPILE-FILE, STEP and EVAL are doing macro expansions.
Plus functions like MACROEXPAND and MACROEXPAND-1.
Conceptually a Macro is a function that takes a Lisp expression and
returns a Lisp expression. You can use these transformations for
your own purpose, but usually they are used to transform Lisp code.
The macro expansion works on Lisp data structures and returns
Lisp datastructure. Since the transformation is written in Lisp,
you can do almost arbitrary transformations (best when they
do not run forever or allocate infinite amount of data ;-) ).
So (procedural) Lisp macros are not providing 'just' token substitution, but
arbitrary code transformations. This is a relatively 'simple' but
powerful mechanism since transformations are written in Lisp and are
transforming Lisp data to Lisp data. So you can reuse a lot of machinery.
Also keep in mind that not always all generated code is available during
compilation.
For example if you have a file example.lisp:
;---
(defun transform (foo)
(list 'print foo))
(defmacro bar (a)
(transform a))
(defun oops ()
(bar 'a))
;---
If you try to do (compile-file "example.lisp") you will get a compilation
error. The code for the transformation function TRANSFORM has been
generated, but is not available to the compiler. This is where you
need to a) load the code before or b) make the function TRANSFORM
available to the compilation environment using EVAL-WHEN.