Hi,
I've been learning lisp recently, with a background in various imperative
and functional (ML and Haskell) languages.
Macros in lisp are certainly powerful, but many of the ways to use them
that I think of I could equally well use higher order functions. Can
someone suggest the kinds of things macros are better for rather than high
order functions? I guess this is part of my functional language
indoctrination:-) Literature pointers also welcom.
tia,
niall
> I've been learning lisp recently, with a background in various imperative
> and functional (ML and Haskell) languages.
>
> Macros in lisp are certainly powerful, but many of the ways to use them
> that I think of I could equally well use higher order functions. Can
> someone suggest the kinds of things macros are better for rather than high
> order functions? I guess this is part of my functional language
> indoctrination:-) Literature pointers also welcom.
One of the things I think macros are for is to define your own "compiler"
for a language construct. Macros allow one to "compile" a form into a CL
representation. Just as many language compilers (Gwydion Dylan for example)
will compile to C first and then assembler. A simple expression can be be
expanded into multiple defuns, defvars and generated documention (for
example).
Wade
> One of the things I think macros are for is to define your own "compiler"
> for a language construct. Macros allow one to "compile" a form into a CL
> representation. Just as many language compilers (Gwydion Dylan for example)
> will compile to C first and then assembler. A simple expression can be be
> expanded into multiple defuns, defvars and generated documention (for
> example).
Actually defining these embedded languages were exactly the reason I was
looking at macros versus functions. In Haskell there are many domain
specific embedded languages for reactive animations, music and more. but
in Haskell they are defined using types, monads and functions.
While learning Lisp, I wanted to see if using macros could give me
a better way to create these little languages, and it seems that
with strict argument passing and side effects they are probably the
only way to go for me.
niall
"Niall Dalton" <·······@ics.uci.edu> wrote
> Actually defining these embedded languages were exactly the reason I was
> looking at macros versus functions. In Haskell there are many domain
> specific embedded languages for reactive animations, music and more. but
> in Haskell they are defined using types, monads and functions.
> While learning Lisp, I wanted to see if using macros could give me
> a better way to create these little languages, and it seems that
> with strict argument passing and side effects they are probably the
> only way to go for me.
Here is a well known example of a macro that generate HTML writing code
(html:html
(:html :head
(:body ((:div :style "width:650")
((:img :src "logo.jpg")) :br
((:p :style "font-family:verdana;font-size:14") title)
((:p :style "font-family:verdana;font-size:12") text)
((:a :href "andt.html")"back to the screenshots list") :br
:br
((:img :format (:src "~a.jpg" file)))))))
Expands to
(progn
(write-string "<HTML><HEAD></HEAD><BODY><DIV STYLE=\"width:650\"><IMG
SRC=\"logo.jpg\"><BR><P STYLE=\"font-family:verdana;font-size:14\">"
html:*html-stream*)
(princ title html:*html-stream*)
(write-string "</P><P STYLE=\"font-family:verdana;font-size:12\">"
html:*html-stream*)
(princ text html:*html-stream*)
(write-string "</P><A HREF=\"andt.html\">back to the screenshots
list</A><BR><BR><IMG SRC=\"" html:*html-stream*)
(format html:*html-stream* "~a.jpg" file)
(write-string "\"></DIV></BODY></HTML>" html:*html-stream*))
It's obviously much easier to work with the first form but the expanded code
is rather efficient. I don't see how you could do this with higher order
functions.
Marc
"Marc Battyani" <·············@fractalconcept.com> writes:
>
> "Niall Dalton" <·······@ics.uci.edu> wrote
>
> > Actually defining these embedded languages were exactly the reason I was
> > looking at macros versus functions. In Haskell there are many domain
> > specific embedded languages for reactive animations, music and more. but
> > in Haskell they are defined using types, monads and functions.
> > While learning Lisp, I wanted to see if using macros could give me
> > a better way to create these little languages, and it seems that
> > with strict argument passing and side effects they are probably the
> > only way to go for me.
>
> Here is a well known example of a macro that generate HTML writing code
>
> (html:html
> (:html :head
> (:body ((:div :style "width:650")
> ((:img :src "logo.jpg")) :br
> ((:p :style "font-family:verdana;font-size:14") title)
> ((:p :style "font-family:verdana;font-size:12") text)
> ((:a :href "andt.html")"back to the screenshots list") :br
> :br
> ((:img :format (:src "~a.jpg" file)))))))
>
> Expands to
>
> (progn
> (write-string "<HTML><HEAD></HEAD><BODY><DIV STYLE=\"width:650\"><IMG
> SRC=\"logo.jpg\"><BR><P STYLE=\"font-family:verdana;font-size:14\">"
> html:*html-stream*)
> (princ title html:*html-stream*)
> (write-string "</P><P STYLE=\"font-family:verdana;font-size:12\">"
> html:*html-stream*)
> (princ text html:*html-stream*)
> (write-string "</P><A HREF=\"andt.html\">back to the screenshots
> list</A><BR><BR><IMG SRC=\"" html:*html-stream*)
> (format html:*html-stream* "~a.jpg" file)
> (write-string "\"></DIV></BODY></HTML>" html:*html-stream*))
>
> It's obviously much easier to work with the first form but the expanded code
> is rather efficient. I don't see how you could do this with higher order
> functions.
I hate it when people use mathy terms like "higher order functions" or
"types, monads, and functions" in arguments about how to write code.
I'm not a formalist, I'm a seat-of-the-pants kind of intuitivist, so I
get bogged down in this kind of terminology. I want to know what
specific functions and coding patterns are used. Without that, I'm
not sure who to agree with, or whether there is even a disagreement.
Am I confused or are MAPCAR, FUNCALL, etc. not higher order functions.
What would it mean to say either that you wouldn't use them in Lisp or
that you only have them in other languages.
Yes, other languages have strong static typing, but all that's doing is
telling you what programs will not be permitted to run. It isn't telling
you what the programs will do when they do run.
My personal library for HTML which I'm developing for hypermeta.com
(which I'm using at my web site but have not released) does just this
cited optimization above as a macro, but it employs classes, methods
and typep, and mapcar, funcall, apply, etc. The library first conjures
structure at compile time and then uses object-oriented techniques to
determine how not to have to do the same thing at runtime, reconstructing
a partially evaluated form.
I'm not math-literate enough to know what a monad is, so I guess I'm
missing out there, but I didn't find myself stopping at any point and
saying "gee, there's this thing I want to do and I wish I had a more
powerful language that could help me over the hump", so I conclude
that it's either an incidental syntactic issue related to a particular
coding style. (I tried a web search, but that found me only either
non-CS/math references or Haskell doc references that assumed I
already knew.)
Anyway, my point is that I agree with Marc that I'll be a little
surprised if Haskell makes this particular macro easy to write, but
at the same time for all I can tell, Haskell's got a lot of Lispy notions
in it, it just makes the notation painful to Lispers. So if some Haskell
guy says "oh, we can do that, all you have to do is <insert something
that looks awful and painful to lispers but probably feels natural to
haskell folks>" I won't be that surprised. Further, it seems to me very
unlikely that Marc has phrased his question right since what else do
lispers use but higher order functions? We just do it in a more
engineering way, calling them when we actually need them, just not
religiously as part of everything we do in order to create some mystic
style.
And, on the other hand, I think Niall's apparent point that somehow Lisp
is impoverished for not having any of these. Lisp does not come with all
possible notations for describing things as the base language, yes. But
I daresay Lisp is more capable of morphing its syntax and capabilities into
something suitable to people with different expressional needs than other
languages. A programmer with enough desire can make a .lisp file look like
(and have underlying support for) pretty much any language; I cite CGOL
as an example of it having been done. I doubt Haskell is as accomodating.
Kent M Pitman <······@world.std.com> writes:
> I hate it when people use mathy terms like "higher order functions" or
> "types, monads, and functions" in arguments about how to write code.
> I'm not a formalist, I'm a seat-of-the-pants kind of intuitivist, so I
> get bogged down in this kind of terminology. I want to know what
> specific functions and coding patterns are used. Without that, I'm
> not sure who to agree with, or whether there is even a disagreement.
I believe I can quote Gerard Berry (www.esterel.org) without him
getting upset. I heard him say something along the lines of "with
enough effort, you can `formalize' anything". I.e. you can write down
a "mathematical-looking" formula for anything. :) Of course, taking a
look at his Esterel papers, you get the feel why he is entitled to
such an opinion. :)
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
Marco Antoniotti <·······@cs.nyu.edu> writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > I hate it when people use mathy terms like "higher order functions" or
> > "types, monads, and functions" in arguments about how to write code.
> > I'm not a formalist, I'm a seat-of-the-pants kind of intuitivist, so I
> > get bogged down in this kind of terminology. I want to know what
> > specific functions and coding patterns are used. Without that, I'm
> > not sure who to agree with, or whether there is even a disagreement.
>
> I believe I can quote Gerard Berry (www.esterel.org) without him
> getting upset. I heard him say something along the lines of "with
> enough effort, you can `formalize' anything". I.e. you can write down
> a "mathematical-looking" formula for anything. :) Of course, taking a
> look at his Esterel papers, you get the feel why he is entitled to
> such an opinion. :)
I should qualify this. The quote was in the context of a discussion
about what is the "right" level of "mathematical abstraction". The
point being that getting carried away with heavy mathematical notation
and notions, while potentially being very precise, does not always buy
you much.
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
Kent M Pitman <······@world.std.com> writes:
> "Marc Battyani" <·············@fractalconcept.com> writes:
>
> >
> > "Niall Dalton" <·······@ics.uci.edu> wrote
> >
> > > Actually defining these embedded languages were exactly the reason I was
> > > looking at macros versus functions. In Haskell there are many domain
> > > specific embedded languages for reactive animations, music and more. but
> > > in Haskell they are defined using types, monads and functions.
> > > While learning Lisp, I wanted to see if using macros could give me
> > > a better way to create these little languages, and it seems that
> > > with strict argument passing and side effects they are probably the
> > > only way to go for me.
> >
> > Here is a well known example of a macro that generate HTML writing code
> >
> > (html:html
> > (:html :head
> > (:body ((:div :style "width:650")
> > ((:img :src "logo.jpg")) :br
> > ((:p :style "font-family:verdana;font-size:14") title)
> > ((:p :style "font-family:verdana;font-size:12") text)
> > ((:a :href "andt.html")"back to the screenshots list") :br
> > :br
> > ((:img :format (:src "~a.jpg" file)))))))
> >
> > Expands to
> >
> > (progn
> > (write-string "<HTML><HEAD></HEAD><BODY><DIV STYLE=\"width:650\"><IMG
> > SRC=\"logo.jpg\"><BR><P STYLE=\"font-family:verdana;font-size:14\">"
> > html:*html-stream*)
> > (princ title html:*html-stream*)
> > (write-string "</P><P STYLE=\"font-family:verdana;font-size:12\">"
> > html:*html-stream*)
> > (princ text html:*html-stream*)
> > (write-string "</P><A HREF=\"andt.html\">back to the screenshots
> > list</A><BR><BR><IMG SRC=\"" html:*html-stream*)
> > (format html:*html-stream* "~a.jpg" file)
> > (write-string "\"></DIV></BODY></HTML>" html:*html-stream*))
> >
> > It's obviously much easier to work with the first form but the expanded code
> > is rather efficient. I don't see how you could do this with higher order
> > functions.
>
> I hate it when people use mathy terms like "higher order functions" or
> "types, monads, and functions" in arguments about how to write code.
> I'm not a formalist, I'm a seat-of-the-pants kind of intuitivist, so I
> get bogged down in this kind of terminology. I want to know what
> specific functions and coding patterns are used. Without that, I'm
> not sure who to agree with, or whether there is even a disagreement.
>
> Am I confused or are MAPCAR, FUNCALL, etc. not higher order functions.
> What would it mean to say either that you wouldn't use them in Lisp or
> that you only have them in other languages.
I agree with you somewhat, but I think it can be quite useful to talk
about higher-order functions, just usually not when talking about
Lisp. In lisp, it's like being a fish in the middle of the ocean and
talking about water -- it's hard to imagine doing anything without it.
Even our most trivial macros probably use higher-order functions
during expansion and/or in the expanded code. It's hard to sneeze in
lisp without writing one.
On the other hand, some languages, like SmallTalk and Perl, use
higher-order functions (or messages in the case of ST) to do a lot of
the syntactic sugar things that macros can be used for in CL. You can
do the same in Lisp, and often do (or I do), then wrap it in a macro
to make it nicer. In these other languages, you'll see someone
writing some brain-damaged, hand-expanded-all-over-the-place code, and
suggest that they abstract that away in a higher-order function. In
lisp, you might suggest a macro, or just say "write a function to do
that", but if you told a Perl hacker to "write a function to do that",
you might get more of the same mess. "Use a higher-order function"
will probably get better results. And if they hadn't ever thought
about functions manipulating other functions, or had considered it
black magic, you get to tell them about it.
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
On Tue, 16 Oct 2001, Kent M Pitman wrote:
> I hate it when people use mathy terms like "higher order functions" or
> "types, monads, and functions" in arguments about how to write code.
> I'm not a formalist, I'm a seat-of-the-pants kind of intuitivist, so I
> get bogged down in this kind of terminology. I want to know what
> specific functions and coding patterns are used. Without that, I'm
> not sure who to agree with, or whether there is even a disagreement.
Ya, and I hate it when you Lisp guys talk about special forms and special
variables. I mean many languages have variables, what so special about
lisp. I wish people would shut up and show me the code ;-)
Yes this is tongue in cheek!
My point is that in some languages the "mathy terms" are common, one need
not know the maths but can know what coding patterns people are talking
about when they use them. This is especially true of monads in Haskell.
> Am I confused or are MAPCAR, FUNCALL, etc. not higher order functions.
> What would it mean to say either that you wouldn't use them in Lisp or
> that you only have them in other languages.
The crux of my question is not "can lisp do this" or "haskell rules" or
anything like that. Simply that many of my initial thoughts on macros
could also be formulated as functions, and I wanted to appreciate the
real power and elegance of macros and the situations where one would
prefer them over functions.
> Yes, other languages have strong static typing, but all that's doing is
> telling you what programs will not be permitted to run. It isn't telling
> you what the programs will do when they do run.
But, if I understand correctly, I can also statically type Lisp by
declaring the types, which the compiler will then check and use during
compilation. If that's true, then this is much closer to how I would like
to program. Initially not declare types, do some exploratory bottom-up
design and coding. People tend to tackle tasks opportunistically, and
I've noted this myself while programming. Dynamically typed languages seem
to support this much better than statically types ones. Allowing me to
declare types if it turns out some part needs to be better optimized by
the compiler, or just for my peace of mind appeals as well - and fits
perfectly with the way I tend to refine code as I develop it.
I was not arguing that static typing is the way to go, in fact I did not
think I was arguing in either direction. I simply was saying the
background I was coming from and what I was trying to think about doing
in case I was coming at it completely assways.
> I'm not math-literate enough to know what a monad is, so I guess I'm
> missing out there, but I didn't find myself stopping at any point and
> saying "gee, there's this thing I want to do and I wish I had a more
> powerful language that could help me over the hump", so I conclude
> that it's either an incidental syntactic issue related to a particular
> coding style. (I tried a web search, but that found me only either
> non-CS/math references or Haskell doc references that assumed I
> already knew.)
I'm not math-literate enough either to go into gory details on what
a monad is when it comes to the real they were introduced (as I recall
Moggi found that they were especially useful to structure denotational
semantics of languages). In Haskell, monads were found to be useful as
a tool in structuring programs, pulling out the common patterns and
allowing people to write general functions which operate on code
in these patterns.
Rather than attempt a fumbling explanation myself, I'd suggest:
http://www.dcs.gla.ac.uk/~nww/Monad.html
which shows monads from the programmer's rather than the math-freak's
point of view.
I'm quite certain that monads are easily used in Lisp, and who knows,
may even be an interesting little pattern with some uses for lisp
programmers.
> in it, it just makes the notation painful to Lispers. So if some Haskell
> guy says "oh, we can do that, all you have to do is <insert something
> that looks awful and painful to lispers but probably feels natural to
> haskell folks>" I won't be that surprised.
I wouldn't be surprised either, but that's really not the point.
The point is, in lisp, which is the better or more natural way to do it
(or preferably both).
> We just do it in a more engineering way, calling them when we
> actually need them, just not religiously as part of everything we do
> in order to create some mystic style.
Ah, that's what the parentheses in lisp are for, right?:-)
I'm a pretty promiscious programmer, I'll use anyting that gets the job
done.
> And, on the other hand, I think Niall's apparent point that somehow Lisp
> is impoverished for not having any of these.
Completely not my point at all actually. My point was that "this is what
I normally use to do it, I hear that these things are a better way to go,
can someone give me a hint as to why they are so good to leap me make the
conceptual leap"
In fact, I'm starting to think that lisp is a better match for what I'd
like to do. I do not care for arguments about which is better or mine is
bigger than yours, or I can piss higher than you can and use a fancy name
for it - if it works better for me, then I'll use it. While I'm a bit
slowed down going thru the examples people have given me due to other
commitments, I do appreciate them and will study them.
> Lisp does not come with all possible notations for describing things
> as the base language, yes.
Nor should it, not did I suggest as much.
> But I daresay Lisp is more capable of morphing its syntax and
> capabilities into something suitable to people with different
> expressional needs than other languages.
I am appreciating this more now, and is exactly what I want to look into.
> A programmer with enough desire can make a .lisp file look like
> (and have underlying support for) pretty much any language; I cite CGOL
> as an example of it having been done. I doubt Haskell is as accomodating.
I think there are Haskell programmers who probably could get close.
I just don't care to do so.
thanks for your comments,
niall
Niall Dalton <·······@ics.uci.edu> writes:
> > Yes, other languages have strong static typing, but all that's doing is
> > telling you what programs will not be permitted to run. It isn't telling
> > you what the programs will do when they do run.
>
> But, if I understand correctly, I can also statically type Lisp by
> declaring the types, which the compiler will then check and use during
> compilation.
Well, some implementations will check it. Most won't do much or any checking.
The purpose of declarations in CL is not primarily to say something the
compiler could figure out anyway, so for the compiler to try checking is
really "optimistic", probably often overly so. The real purpose of
declarations in CL is to make a promise to the compiler which you imagine
the compiler can't figure out. In most cases where the compiler could figure
it out and check, most people don't bother making the declaration explicit.
> If that's true, then this is much closer to how I would like
> to program. Initially not declare types, do some exploratory bottom-up
> design and coding.
You can certainly do this part. And you can add types later, but largely
it won't buy you anything except in a couple of implementations. (CMU CL
does some pretty aggressive type inferencing, checking, and so on.)
> People tend to tackle tasks opportunistically, and
> I've noted this myself while programming. Dynamically typed languages seem
> to support this much better than statically types ones. Allowing me to
> declare types if it turns out some part needs to be better optimized by
> the compiler,
Now the issue of optimization is orthogonal to the issue of
"checking". People do declare things for the sake of compiler
optimization, but that doesn't oblige a compiler to do the
optimization. If you think you're not getting a good optimization,
that's a vendor issue, not a language issue.
> or just for my peace of mind appeals as well - and fits
> perfectly with the way I tend to refine code as I develop it.
>
> I was not arguing that static typing is the way to go, in fact I did not
> think I was arguing in either direction.
Sorry, my mistake there. Most FP people other than Scheme folks seem
to get really into static typing with their functions.
> I simply was saying the
> background I was coming from and what I was trying to think about doing
> in case I was coming at it completely assways.
I see. I have no way to say for sure on that.
> > I'm not math-literate enough to know what a monad is, so I guess I'm
> > missing out there, but I didn't find myself stopping at any point and
> > saying "gee, there's this thing I want to do and I wish I had a more
> > powerful language that could help me over the hump", so I conclude
> > that it's either an incidental syntactic issue related to a particular
> > coding style. (I tried a web search, but that found me only either
> > non-CS/math references or Haskell doc references that assumed I
> > already knew.)
> I'm not math-literate enough either to go into gory details on what
> a monad is when it comes to the real they were introduced (as I recall
> Moggi found that they were especially useful to structure denotational
> semantics of languages). In Haskell, monads were found to be useful as
> a tool in structuring programs, pulling out the common patterns and
> allowing people to write general functions which operate on code
> in these patterns.
>
> Rather than attempt a fumbling explanation myself, I'd suggest:
> http://www.dcs.gla.ac.uk/~nww/Monad.html
> which shows monads from the programmer's rather than the math-freak's
> point of view.
Thanks for the pointer.
> I'm quite certain that monads are easily used in Lisp, and who knows,
> may even be an interesting little pattern with some uses for lisp
> programmers.
>
> > in it, it just makes the notation painful to Lispers. So if some Haskell
> > guy says "oh, we can do that, all you have to do is <insert something
> > that looks awful and painful to lispers but probably feels natural to
> > haskell folks>" I won't be that surprised.
> I wouldn't be surprised either, but that's really not the point.
> The point is, in lisp, which is the better or more natural way to do it
> (or preferably both).
Best to be concrete about the problem in order to get a concrete answer.
> > We just do it in a more engineering way, calling them when we
> > actually need them, just not religiously as part of everything we do
> > in order to create some mystic style.
> Ah, that's what the parentheses in lisp are for, right?:-)
That's not a religious issue, it's a practical one.
It makes coding easier. I'd give it up in a minute if I thought I was
getting nothing from it.
> I'm a pretty promiscious programmer, I'll use anyting that gets the job
> done.
Ah, that's good.
> > And, on the other hand, I think Niall's apparent point that somehow Lisp
> > is impoverished for not having any of these.
> Completely not my point at all actually.
Thanks for the detailed correction.
> My point was that "this is what
> I normally use to do it, I hear that these things are a better way to go,
> can someone give me a hint as to why they are so good to leap me make the
> conceptual leap"
>
> In fact, I'm starting to think that lisp is a better match for what I'd
> like to do. I do not care for arguments about which is better or mine is
> bigger than yours, or I can piss higher than you can and use a fancy name
> for it - if it works better for me, then I'll use it. While I'm a bit
> slowed down going thru the examples people have given me due to other
> commitments, I do appreciate them and will study them.
That's good.
> > Lisp does not come with all possible notations for describing things
> > as the base language, yes.
> Nor should it, not did I suggest as much.
>
> > But I daresay Lisp is more capable of morphing its syntax and
> > capabilities into something suitable to people with different
> > expressional needs than other languages.
> I am appreciating this more now, and is exactly what I want to look into.
>
> > A programmer with enough desire can make a .lisp file look like
> > (and have underlying support for) pretty much any language; I cite CGOL
> > as an example of it having been done. I doubt Haskell is as accomodating.
> I think there are Haskell programmers who probably could get close.
> I just don't care to do so.
>
> thanks for your comments,
> niall
Ok. I'll go back if I get some time and fish out the start of this
conversation and see if you asked a concrete question I should have answered
instead. But you didn't, the right way to proceed might be for you to just
propose a specific scenario that you're wondering about and maybe some
sketch of how you would go about doing it, and then we can nudge you in the
right direction if it seems offtrack. Sounds like that will work better
than all this philosophical back and forth.
Sorry for having contributing to keeping things off track.
> Now the issue of optimization is orthogonal to the issue of
> "checking". People do declare things for the sake of compiler
> optimization, but that doesn't oblige a compiler to do the
> optimization. If you think you're not getting a good optimization,
> that's a vendor issue, not a language issue.
Fair enough, to me its nice that the language gives the option of
declaring types in the places that I want to, and having dynamic type
checking of values. It feels confortable to code in. I like compiler work,
and am a believer in typed intermediate languages both for allowing
stronger optimizations and also in making sure the compiler doesn't "go
wrong" and turn a correct program into an incorrect one (some of which you
can catch as type errors.
Right now, high performance implementation is not an issue for me, though
its nice to know its available when I need it.
> > Ah, that's what the parentheses in lisp are for, right?:-)
> That's not a religious issue, it's a practical one.
> It makes coding easier. I'd give it up in a minute if I thought I was
> getting nothing from it.
Having looked at some lisp even for a short while, I'm finding they
begin to fade into the background are are wonderfully consistent
compared to other languages.
> conversation and see if you asked a concrete question I should have answered
> instead. But you didn't, the right way to proceed might be for you to just
> propose a specific scenario that you're wondering about and maybe some
> sketch of how you would go about doing it, and then we can nudge you in the
> right direction if it seems offtrack. Sounds like that will work better
> than all this philosophical back and forth.
Sure enough it probably would. Fits with the feel of lisp as a pragmatic
language as well. I'm shopping for a language for part of a compiler
project. Its my own project, and I'm not limiting my options for the
language - I'll happily learn whichever is best suited for the task.
Lisp seems perfectly suited to the task, the other contender being
Objective Caml. I'm not looking for flame war, so please no one start one.
If you have an opinion on this, perhaps mail me directly rather than
bother the group. I'll take up the option of actual coding scenarios
a little later, when I have time to put a decent effort in first..roll
on December and the end of my quarter:-)
Thanks,
niall
Niall Dalton <·······@ics.uci.edu> writes:
...
> Sure enough it probably would. Fits with the feel of lisp as a pragmatic
> language as well. I'm shopping for a language for part of a compiler
> project. Its my own project, and I'm not limiting my options for the
> language - I'll happily learn whichever is best suited for the task.
> Lisp seems perfectly suited to the task, the other contender being
> Objective Caml. I'm not looking for flame war, so please no one start one.
> If you have an opinion on this, perhaps mail me directly rather than
> bother the group. I'll take up the option of actual coding scenarios
> a little later, when I have time to put a decent effort in first..roll
> on December and the end of my quarter:-)
At least I have an inkling of what Haskell is about, but I don't know
the first thing about OCaml. You're still at a fairly abstract level here
that makes it hard to know what to say. Lisp is a great language for
writing compielrs, so if that's all you wanted to know, you're all set. ;-)
Is there some specific thing where you're worried it might not be able to
work or where you can't visualize what you'd do?
It's not just an issue of the pragmatic, at least for me. I prefer to
speak about concrete things even when speaking in the abstract because
concrete things come with details. Often a discussion about abstracts
omits details. It's true that the details often vary among instances
of the same abstract, but it's easier for me to keep in mind that
details might change than that they might materialize... Plus I have
mostly no good text-based notations for talking about abstract
instances, only for concrete code. (It's not that I don't think there
are people who have notations for theory; I just don't trust they can
link the theory back to practice in every instance.)
> that makes it hard to know what to say. Lisp is a great language for
> writing compielrs, so if that's all you wanted to know, you're all set. ;-)
Well, that's me converted ;-)
> Is there some specific thing where you're worried it might not be able to
> work or where you can't visualize what you'd do?
Right now, no. Until I start digging into the code in the next while,
I can't give you a concrete example I'm afraid. At the moment I'm
stuck doing other things, but looking forward to the project ahead..
and trying to figure out what kinda tools I want to take along.
> instances, only for concrete code. (It's not that I don't think there
> are people who have notations for theory; I just don't trust they can
> link the theory back to practice in every instance.)
As another poster pointed out, one can (overly) formalize anything.
It need not actually mean anything, right?:-)
In article <········································@godzilla.ics.uci.edu>,
Niall Dalton <·······@ics.uci.edu> wrote:
>The crux of my question is not "can lisp do this" or "haskell rules" or
>anything like that. Simply that many of my initial thoughts on macros
>could also be formulated as functions, and I wanted to appreciate the
>real power and elegance of macros and the situations where one would
>prefer them over functions.
The crux of the answer is that macros don't really provide too much that
you can't do with higher-order functions. Mostly what they provide is
syntactic sugar, so that it's easier to read and write these higher-order
constructs.
For instance, you could define IF as:
(defun if (test then-fun &optional (else-fun (lambda () nil)))
(funcall (cond (test then-fun)
(t else-fun))))
and then it would be used as:
(if (> x 100)
(lambda () (princ "X is really big!"))
(lambda () (princ "X isn't so big.")))
Long ago most Lisp programmers decided that having lots of lambda
expressions all over the place like this was confusing, and macros allow
more concise syntax to be used. Haskell uses a more compact notation for
its anonymous functions, so it may be less of an issue there. And some
functional languages solve the problem using lazy evaluation.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
> The crux of the answer is that macros don't really provide too much that
> you can't do with higher-order functions. Mostly what they provide is
> syntactic sugar, so that it's easier to read and write these higher-order
> constructs.
Well I like that they allow embedding a new language in a way such that
much of the work is done at compile time. The wonderfully consistent
syntax makes live easier as well it seems, and on the whle it seems far
easier to follow code using macros than over abundant use of higher order
functions (heresy from a Haskeller I know, but I also have other
language habits I dare not mention in public;-).
In article <·······································@godzilla.ics.uci.edu>,
Niall Dalton <·······@ics.uci.edu> wrote:
>> The crux of the answer is that macros don't really provide too much that
>> you can't do with higher-order functions. Mostly what they provide is
>> syntactic sugar, so that it's easier to read and write these higher-order
>> constructs.
>Well I like that they allow embedding a new language in a way such that
>much of the work is done at compile time. The wonderfully consistent
>syntax makes live easier as well it seems, and on the whle it seems far
>easier to follow code using macros than over abundant use of higher order
>functions (heresy from a Haskeller I know, but I also have other
>language habits I dare not mention in public;-).
I think it's mostly a matter of what you're used to.
If Lisp used higher-order functions instead of macros, I suspect that all
the lambda's would start to disappear as noise, much as the parentheses
already do. You'd see (if ...) and know that a couple of functions are
coming. (lambda () ...) would be viewed as something like a "lazy" progn.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
> If Lisp used higher-order functions instead of macros, I suspect that all
> the lambda's would start to disappear as noise, much as the parentheses
> already do. You'd see (if ...) and know that a couple of functions are
> coming. (lambda () ...) would be viewed as something like a "lazy" progn.
Especially if it was so common that there was environment support for
making the lambdas less offensive!
From: Juliusz Chroboczek
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <87vghboi6d.fsf@pps.jussieu.fr>
Barry Margolin:
BM> The crux of the answer is that macros don't really provide too
BM> much that you can't do with higher-order functions.
There is a fundamental reason why macros are more powerful than
higher-order functions. What a higher-order function gets at is a
closure: an *opaque* piece of code together with its environment. A
macro, on the other hand, gets actual, unmodified source code.
This has two consequences:
1. there is no way a higher-order function can walk its argument, and
do e.g. peephole optimisation or partial evaluation; by the time you
get your hands at the closure, the source code may very well have been
macroexpanded, converted to an intermediate representation, or
outright compiled.
2. there is no way a higher-order function can escape the binding
discipline (which, in CL, may be lexical or dynamic, which helps
somewhat).
Note that (2) is the very reason why reliable macros are so painful
to write. In short, when writing a macro you have to perform
alpha-conversion (renaming of variables so as to avoid name
clashes) by hand; hence, you need to write code such as
(defmacro foo (expr)
(let ((var-name (gensym)))
`(let ((,var-name ,expr))
... ,var-name ...)))
rather than the following, which is more intuitive but vulnerable to
name clashes (and therefore incorrect):
(defmacro foo (expr)
`(let ((var ,expr))
... var ...))
For example, the following FLUID-LET macro fundamentally relies on the
ability to pass a variable *name*:
(defmacro fluid-let (((var expr)) &body body)
(let ((temp (gensym "TEMP"))
(result (gensym "RESULT")))
`(let ((,temp ,var) ,result)
(unwind-protect
(setq ,var ,expr
,result (progn ,@body))
(setq ,var ,temp))
,result)))
In order to implement something similar using higher-order functions,
you'd need to pass both a getter and a setter to FLUID-LET; however,
there's no way you could prevent the user from passing two arbitrary
unrelated functions, and so you'd lose an important invariant.
(FLUID-LET is not very useful in CL due to the lack of global lexicals
and the availability of dynamically scoped variables. As far as I
know, FLUID-LET is due to Dybvig.)
Juliusz
Juliusz Chroboczek <···@pps.jussieu.fr> writes:
> (FLUID-LET is not very useful in CL due to the lack of global lexicals
> and the availability of dynamically scoped variables. As far as I
> know, FLUID-LET is due to Dybvig.)
I think FLUID-LET comes up in the original or Revised Report on Scheme,
actually.
In article <··············@pps.jussieu.fr>,
Juliusz Chroboczek <···@pps.jussieu.fr> wrote:
>1. there is no way a higher-order function can walk its argument, and
> do e.g. peephole optimisation or partial evaluation; by the time you
> get your hands at the closure, the source code may very well have been
> macroexpanded, converted to an intermediate representation, or
> outright compiled.
This is true, but since it's so hard to write a reliable code walker it's
rarely relevant. A long-time wishlist item has been a standard interface
to the implementation's code walker, but X3J13 never devoted any resources
to this. The closest we got were the environment-related functions that
are in CLTL2, but which we ended up removing from the standard because
there were problems with them.
>2. there is no way a higher-order function can escape the binding
> discipline (which, in CL, may be lexical or dynamic, which helps
> somewhat).
I never claimed that macros were totally replaceable with higher-order
functions, merely that *most* common uses of macros have straightforward
H-O analogues. LOOP is obviously not something I'd try to convert to H-O
functions, but Series probably could be (because it's based on nested
expressions, not a mini-language).
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> In article <··············@pps.jussieu.fr>,
> Juliusz Chroboczek <···@pps.jussieu.fr> wrote:
> >1. there is no way a higher-order function can walk its argument, and
> > do e.g. peephole optimisation or partial evaluation; by the time you
> > get your hands at the closure, the source code may very well have been
> > macroexpanded, converted to an intermediate representation, or
> > outright compiled.
>
> This is true, but since it's so hard to write a reliable code walker it's
> rarely relevant. A long-time wishlist item has been a standard interface
> to the implementation's code walker, but X3J13 never devoted any resources
> to this. The closest we got were the environment-related functions that
> are in CLTL2, but which we ended up removing from the standard because
> there were problems with them.
>
> >2. there is no way a higher-order function can escape the binding
> > discipline (which, in CL, may be lexical or dynamic, which helps
> > somewhat).
>
> I never claimed that macros were totally replaceable with higher-order
> functions, merely that *most* common uses of macros have straightforward
> H-O analogues. LOOP is obviously not something I'd try to convert to H-O
> functions, but Series probably could be (because it's based on nested
> expressions, not a mini-language).
I would really like a code walker. As a matter of fact I cooked up my
own (called AMBLER - it should be in the CLOCC).
However, the only thing I am really interested is in S-expr walking
with the possibility to carry around some state in a user controllable
way. I.e. I think that the code walkers that you see around try to do
too much; essentially because they try to exploit too much reflective
information from the underlying Lisp.
It's either that, or the code walkers I say around were too crufty to
be used reliably and portably.
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
In article <···············@octagon.mrl.nyu.edu>,
Marco Antoniotti <·······@cs.nyu.edu> wrote:
>I would really like a code walker. As a matter of fact I cooked up my
>own (called AMBLER - it should be in the CLOCC).
>
>However, the only thing I am really interested is in S-expr walking
>with the possibility to carry around some state in a user controllable
>way. I.e. I think that the code walkers that you see around try to do
>too much; essentially because they try to exploit too much reflective
>information from the underlying Lisp.
The reason why the code walker needs to be provided by the implementation
is so that it can handle implementation-specific special operators
properly. Once you encounter a special form, you have to know its syntax
so you can figure out which subforms are S-expressions that should be
recursed into. And if the special operator introduces changes to the
environment, you need to know about them.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> In article <···············@octagon.mrl.nyu.edu>,
> Marco Antoniotti <·······@cs.nyu.edu> wrote:
> >I would really like a code walker. As a matter of fact I cooked up my
> >own (called AMBLER - it should be in the CLOCC).
> >
> >However, the only thing I am really interested is in S-expr walking
> >with the possibility to carry around some state in a user controllable
> >way. I.e. I think that the code walkers that you see around try to do
> >too much; essentially because they try to exploit too much reflective
> >information from the underlying Lisp.
>
> The reason why the code walker needs to be provided by the implementation
> is so that it can handle implementation-specific special operators
> properly. Once you encounter a special form, you have to know its syntax
> so you can figure out which subforms are S-expressions that should be
> recursed into. And if the special operator introduces changes to the
> environment, you need to know about them.
I don't think this is the true reason.
3.1.2.1.2.2 Macro Forms
...
An implementation is free to implement any macro operator as a special
operator, but only if an equivalent definition of the macro is also
provided.
...
I have always taken this to mean that, in general, implementations cannot
add new special operators without the same restriction, though I could see
how someone might not read it that way. If you read it the more permissive
way, though, then it makes the above-quoted sentence pointless since the
only purpose of this sentence above is to make sure code-walkers work, and
since such code-walkers would still be broken if other special operators
could exist. Given that you do have macros for every system special form,
you're mostly all set for code-walking EXCEPT:
The real reason a code walker is hard to write, in my experience (since
I've tried to write a fully general, portable one more than once), is that
when one codewalks a macrolet correctly, one has to augment the system's
true repository for macro definitions, since any macro function that is
invoked might try to do MACROEXPAND-1 or MACROEXPAND during the expansion
given the &ENVIRONMENT argument from the system, and since if you're
squirreling away the codewalked macrolet bindings in a place of your own
devising, you're going to make them opaque to MACROEXPAND[-1]. Unfortunately,
though, there is really no way to do this. The closest I've been able to
do, which works in a lot of implementations but may not work in all of them
is by code-walking (MACROLET <bindings> . <body>) as something like the
following (which I'm doing from memory but is hopefully enough right that you
get the nature of the kludge):
(defun code-walk (exp env)
(cond ((atom exp)
(code-walk-atom exp env))
((consp (car exp))
(code-walk-lambda-exp exp env))
(t
(case (car exp)
((macrolet)
(destructuring-bind (bindings &rest body) exp
(let ((continuation (gensym "CONTINUATION")))
(macroexpand-1 `(macrolet ,bindings
(macrolet ((,continuation (&rest body
&environment env)
(code-walk body env)))
(,continuation ,@body)))
env))))
... other special operators, macros, functions ...))))
This is a pretty gross abuse of MACROEXPAND-1 to do an action that really does
not relate to macroexpanding at all, but instead is FUNCALL'ing something in
a context that has the environment augmented in a way I don't know any other
way to do.
You also have to do FLET and LABELS the same way, btw, using again
MACROEXPAND-1 of `(flet ,bindings (macrolet ((,continuation ...)) ...))
or `(labels ,bindings (macrolet ((,continuation ...)) ...)), as appropriate,
with the relevant ENV argument, to make sure it shadows any macro bindings.
Caveat emptor. I just typed this in from memory and didn't test it. But
I'm pretty sure this trick, or something vaguely like it, was what I used
to when I implemented New Flavors for Symbolics CLOE. I'm pretty sure
that implementation, while proprietary, was completely portable CLTL1-level
Common Lisp.
Kent M Pitman <······@world.std.com> writes:
> <much deleted>
This is, as usual, very interesting. However, I believe that a simple
S-expression walker, may be very useful as well.
Most of the time I found myslf traversing a S-expr tree which is some
representation of a syntax I have complete control over. In this case
a simpler code walker (read: one that does not do any introspection,
e.g. at the macro expansion level) can be written portably and without
much problems. It's a trade-off.
Cheers
PS. As far as complexity is concerned, DEFINE-SYMBOL-MACRO is always
lurking in the background :)
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
From: Kent M Pitman
Subject: Re: Code Walkers (Re: Macros versus higher order functions.)
Date:
Message-ID: <sfw1yjzrkqj.fsf@world.std.com>
Marco Antoniotti <·······@cs.nyu.edu> writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > <much deleted>
>
> This is, as usual, very interesting. However, I believe that a simple
> S-expression walker, may be very useful as well.
>
> Most of the time I found myslf traversing a S-expr tree which is some
> representation of a syntax I have complete control over. In this case
> a simpler code walker (read: one that does not do any introspection,
> e.g. at the macro expansion level) can be written portably and without
> much problems. It's a trade-off.
I don't understand what you're talking about. Sure, you can do this for
a language other than Lisp. But Lisp has the ability for "s-exprs" (i.e.,
forms) to include MACROLET and the macros created by those MACROLETs. You
can't make any usable product for use on someone else's code subject to the
constraint that you yourself have control over the syntax that other party
uses.
> Cheers
>
> PS. As far as complexity is concerned, DEFINE-SYMBOL-MACRO is always
> lurking in the background :)
It works the same way. I deliberately hid the complexity of walking atoms
in another function for my sample code, but it's just as doable.
Kent M Pitman <······@world.std.com> writes:
> The real reason a code walker is hard to write, in my experience (since
> I've tried to write a fully general, portable one more than once), is that
> when one codewalks a macrolet correctly, one has to augment the system's
> true repository for macro definitions, since any macro function that is
> invoked might try to do MACROEXPAND-1 or MACROEXPAND during the expansion
> given the &ENVIRONMENT argument from the system, and since if you're
> squirreling away the codewalked macrolet bindings in a place of your own
> devising, you're going to make them opaque to MACROEXPAND[-1]. Unfortunately,
> though, there is really no way to do this.
Sounds like you need functions allows you to manipulate the environment.
Hey, looks like environment functions like augment-environment were
added in CLtL2. (I just now looked, it was an obvious oversight in the original)
The xerox pcl code walker that I have used defines the following
functions for each implementation variant:
(defmacro with-augmented-environment((new-env old-env &key functions macros) &body body) ...)
(defun environment-function (env fn) ...)
(defun environment-macro (env macro) ...)
So, it seems like cltl2 has everything you need for portable code walkers?
Even if you didn't have the cltl2 functions, you could just smash (eg
advise) macroexpand-1 to look at the environment to see if it is one
of your squirreled bindings?
-jeff
Jeff Mincy <····@delphioutpost.com> writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > The real reason a code walker is hard to write, in my experience
> > (since I've tried to write a fully general, portable one more than
> > once), is that when one codewalks a macrolet correctly, one has to
> > augment the system's true repository for macro definitions, since
> > any macro function that is invoked might try to do MACROEXPAND-1
> > or MACROEXPAND during the expansion given the &ENVIRONMENT
> > argument from the system, and since if you're squirreling away the
> > codewalked macrolet bindings in a place of your own devising,
> > you're going to make them opaque to MACROEXPAND[-1].
> > Unfortunately, though, there is really no way to do this.
>
> Sounds like you need functions allows you to manipulate the
> environment. Hey, looks like environment functions like
> augment-environment were added in CLtL2. (I just now looked, it was
> an obvious oversight in the original)
They were going to be in ANSI, but myriad problems came up with the
way we had specified them and as quickly as we patched those problems
new ones arose. We finally decided to remove the facility not because
we didn't like the idea but because we felt it would be better for
vendors to proceed with experimentation unimpeded by a broken spec.
> The xerox pcl code walker that I have used defines the following
> functions for each implementation variant:
>
> (defmacro with-augmented-environment((new-env old-env &key functions macros) &body body) ...)
> (defun environment-function (env fn) ...)
> (defun environment-macro (env macro) ...)
>
> So, it seems like cltl2 has everything you need for portable code walkers?
Yes, it SEEMS that way. No, it doesn't in practice. It has the right
general functionality but practical problems.
> Even if you didn't have the cltl2 functions, you could just smash (eg
> advise) macroexpand-1 to look at the environment to see if it is one
> of your squirreled bindings?
No, you couldn't. This was a standard issue that came up all the time with
the Lisp Machine. You are making the unwarranted assumption that just
because CL exports a facility like MACROEXPAND-1 that the system will call
that facility. The Lisp Machine system, in nearly every case, used to call
its own functionality (written in Zetalisp) and the CL exports were just
a way in for users. Bashing that would leave the world inconsistent since
system-level macros wouldn't use CL:MACROEXPAND-1. LispM's are probably
the most extreme example, since they had multiple co-resident Lisp dialects
competing for an "obvious" way of doing things, but even in other dialects,
there is no requirement that system interfaces be the only way that a
functionality is accessed and so you just can't do what you're thinking
without exposing internals; and if you're going to make assumptions about
internals, there are better ways to implement the functionality.
All of these comments are good and in line with my reasoning that you
really do not want to mess with MACROLET and friends.
My reasoning goes this way (trying to answer KMP as well). Either you
control the syntax and you know what your "parse tree" will look like, or
you do not even want to try to parse code that contains extension
macros. This may be extreme, but I feel it is a good
compromise. I.e. if you are given a piece of code that looks like
(define-extension whatever (something)
(let ((one 1))
(declare (extension t))
....))
and I only give you a piece of documentation that states that I am
doing "something magic" whenever DEFINE-EXTENSION contains a DECLARE
(EXTENSION t), I do not think that it is really safe for a code walker
to try to guess the syntax resulting from macroexpansion. To state it
differently, IMHO, if you want to walk over the DEFINE-EXTENSION code,
you must only be concerned with the "source" s-expr and deal with it
according to its documented semantics.
This calls for a code walker that does not macroexpand subforms
without user control. I think this is the best that we can do.
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
Barry Margolin <······@genuity.net> writes:
> I never claimed that macros were totally replaceable with higher-order
> functions, merely that *most* common uses of macros have straightforward
> H-O analogues. LOOP is obviously not something I'd try to convert to H-O
> functions, but Series probably could be (because it's based on nested
> expressions, not a mini-language).
Series replaces CL:DEFUN because it wants to analyse the whole
function definition for possible optimisations. It recognizes a
declaration of its own and it tries to convert the series expressions
to loops. It even uses CLtL1's COMPILER-LET in a substantial way.
--
Lieven Marchand <···@wyrd.be>
She says, "Honey, you're a Bastard of great proportion."
He says, "Darling, I plead guilty to that sin."
Cowboy Junkies -- A few simple words
In article <··············@localhost.localdomain>,
Lieven Marchand <···@wyrd.be> wrote:
>Series replaces CL:DEFUN because it wants to analyse the whole
>function definition for possible optimisations. It recognizes a
>declaration of its own and it tries to convert the series expressions
>to loops. It even uses CLtL1's COMPILER-LET in a substantial way.
It still does that? I remember that the original CSS implementation did
this, both for the optimizations and because it was written in Maclisp
which doesn't have real closures. But I thought I heard that it was later
changed to use ordinary macros and functions, perhaps because most CL
implementations were able to perform the optimizations themselves or had
good enough performance when calling closures.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> In article <··············@localhost.localdomain>,
> Lieven Marchand <···@wyrd.be> wrote:
> >Series replaces CL:DEFUN because it wants to analyse the whole
> >function definition for possible optimisations. It recognizes a
> >declaration of its own and it tries to convert the series expressions
> >to loops. It even uses CLtL1's COMPILER-LET in a substantial way.
>
> It still does that? I remember that the original CSS implementation did
> this, both for the optimizations and because it was written in Maclisp
> which doesn't have real closures. But I thought I heard that it was later
> changed to use ordinary macros and functions, perhaps because most CL
> implementations were able to perform the optimizations themselves or had
> good enough performance when calling closures.
The code that now is maintained at sourceforge still uses compiler-let
to communicate between macros and it's not immediately clear to me how
to remove it.
--
Lieven Marchand <···@wyrd.be>
She says, "Honey, you're a Bastard of great proportion."
He says, "Darling, I plead guilty to that sin."
Cowboy Junkies -- A few simple words
From: Juliusz Chroboczek
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <87sncboawp.fsf@pps.jussieu.fr>
>> 1. there is no way a higher-order function can walk its argument, and
>> do e.g. peephole optimisation or partial evaluation; by the time you
>> get your hands at the closure, the source code may very well have been
>> macroexpanded, converted to an intermediate representation, or
>> outright compiled.
Barry Margolin:
BM> This is true, but since it's so hard to write a reliable code walker it's
BM> rarely relevant.
Granted in the fully general case. But special-case code-walking and
partial evaluation can often be done.
I once found myself using a function
(defun wherep (m c)
... hairy code to test whether m and c are in the WHERE relationship ...
)
Initial testing showed that the program was spending a significant
fraction within WHEREP. However, in most (but not all) occurrences,
the C argument was a literal constant; hence, I rewrote this as a
partial evaluator:
(defmacro wherep (m c)
(if (and (consp c) (eql 'quote (car c)))
... hairy code to generate inline special-case code ...
`(wherep* m c)))
(defun wherep* (m c)
...)
This lead to a 20% speed boost with no changes to the calling code.
This sort of thing I can do with macros, but probably with no other
mechanism.
(My office neighbour is claiming that much the same thing can be done
with the ocamlm4 preprocessor for O'Caml. According to him, ocamlm4
operates on the parse tree and gives you the full power of the O'Caml
programming language at preprocessing time. I don't know ocamlm4
myself, and can only repeat what he's saying.)
I'm also making the dual claim: DEFMACRO is overkill for much of what
we use it for. Still, I think that other mechanisms (hygienic macros,
higher-order functions, etc.), desirable as they are, have failed for
much the same reason as Algol-like notations for Lisp: in the time it
took to have them developed, a generation of hackers willing to deal
with the arcana of writing reliable DEFMACRO macros has been brought up.
Juliusz
Juliusz Chroboczek <···@pps.jussieu.fr> writes:
> I once found myself using a function
>
> (defun wherep (m c)
> ... hairy code to test whether m and c are in the WHERE relationship ...
> )
>
> Initial testing showed that the program was spending a significant
> fraction within WHEREP. However, in most (but not all) occurrences,
> the C argument was a literal constant; hence, I rewrote this as a
> partial evaluator:
>
> (defmacro wherep (m c)
> (if (and (consp c) (eql 'quote (car c)))
> ... hairy code to generate inline special-case code ...
> `(wherep* m c)))
>
> (defun wherep* (m c)
> ...)
>
> This lead to a 20% speed boost with no changes to the calling code.
> This sort of thing I can do with macros, but probably with no other
> mechanism.
BTW, you can use compiler-macros for this sort of stuff:
(defun wherep (m c)
...)
(define-compiler-macro wherep (&whole form m c)
(if (and (consp c) (eql 'quote (car c))) ; Prefer constantp here?
... (call to) hairy code to generate inline special-case code ...
form))
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
From: Juliusz Chroboczek
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <87vgh6qrb9.fsf@pps.jussieu.fr>
Pierre R Mai:
PM> BTW, you can use compiler-macros for this sort of stuff:
Yes, I know, but I don't know how to debug them, so I tend to avoid
them. I don't see a way to macroexpand them, or to test them without
recompiling. Do you have any tricks to share?
Juliusz
Juliusz Chroboczek <···@pps.jussieu.fr> writes:
> Pierre R Mai:
>
> PM> BTW, you can use compiler-macros for this sort of stuff:
>
> Yes, I know, but I don't know how to debug them, so I tend to avoid
> them. I don't see a way to macroexpand them, or to test them without
> recompiling. Do you have any tricks to share?
Well, you can gain access to the compiler-macro-function via
compiler-macro-function. Macroexpansion consists of calling that
function with the form and the environment. While the nature of the
object representing the environment is implementation-dependant, most
implementations will allow the use of nil as that argument, in order
to specify the null-lexical environment (certain functions are
specified to take an environment object, and those are required to
allow the use of nil, but AFAICS, compiler-macro-functions are not
really required to let you use nil, since they take an environment).
So
(defun compiler-macroexpand-1 (form &optional environment)
"If form is a function-call or macro form with a compiler-macro
definition, expands it once. Returns two values, the expanded form
and a T-or-NIL flag indicating whether the form was, in fact,
expanded. Environment is the lexical environment to expand in,
which defaults to the null environment."
(if (and (consp form) (symbolp (car form)))
(let ((c-m-f (compiler-macro-function (car form) environment)))
(if c-m-f
(let* ((hook (coerce *macroexpand-hook* 'function))
(result (funcall hook c-m-f form environment)))
(if (eq result form)
(values form nil)
(values result t)))
(values form nil)))
(values form nil)))
should provide a first stepping-stone for debugging and using
compiler-macros. compiler-macroexpand can be defined accordingly.
Actually I just looked at the CMU CL source, and it has commented-out
definitions of those, with a comment that they were somehow dropped
from the Spec... Hmmm.
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
From: Juliusz Chroboczek
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <87lmibt8jw.fsf@pps.jussieu.fr>
KMP> I hate it when people use mathy terms like "higher order functions" or
KMP> "types, monads, and functions" in arguments about how to write code.
I don't think that's fair. In the same way as the term ``function''
has lost its ``mathy'' flavour for Lisp programmers and is fully part
of everyday shop talk, the terms above feel common to Haskell
programmers. (Actually, many Haskell programmers do not know Category
Theory, nor how the Category-theoretic notion of monad is linked to
the Haskell construct of the same name -- and why should they?)
On the other hand, I agree that the OP has used terms familiar to the
Haskell community but not necessarily so to Lisp programmers, and has
done so on a Lisp newsgroup without providing suitable definitions (or
at least intuitions).
KMP> Am I confused or are MAPCAR, FUNCALL, etc. not higher order
KMP> functions. What would it mean to say either that you wouldn't
KMP> use them in Lisp or that you only have them in other languages.
I feel that what the OP meant is that the simpler applications of
macros can be replaced with suitable use of higher-order functions.
Given a SSC, you can replace
(defmacro with-foo (var (&rest args) &body body)
`(let ((,var (make-foo ,@args)))
,@body))
with
(defun call-with-foo (body args)
(funcall body (apply #'make-foo args)))
the main difference being that in the latter case BODY is a closure
rather than an actual piece of source code.
While this is true, the fact remains that the fully general use of
DEFMACRO is vastly more powerful (and, to be fair, vastly more
error-prone) than that. Marc's macro may be a good example of such a
case, although there are ways of implementing something equally
convenient in Haskell.
KMP> And, on the other hand, I think Niall's apparent point that
KMP> somehow Lisp is impoverished for not having any of these.
I took it as an honest query about what it is that makes DEFMACRO
preferable to what he (with his particular background) perceived as
cleaner ways of achieving the same.
[Warning: creative out-of-context quoting follows]
KMP> Yes, other languages have strong static typing, but all that's
KMP> doing is telling you what programs will not be permitted to run.
[...]
KMP> [...] for all I can tell, Haskell's got a lot of Lispy notions in
KMP> it [...]
[...]
KMP> But I daresay Lisp is more capable of morphing its syntax and
KMP> capabilities into something suitable to people with different
KMP> expressional needs than other languages.
I would tend to agree with all of the above.
A construct analogue to Marc's macro should not be too difficult to
implement in Haskell (but don't ask me to prove it -- I'd have to dust
my Haskell manuals). Of course, it would not be a macro, and would be
best described with those ``mathy'' terms that you so dislike.
Many of the designers of Haskell have excellent knowledge of Lisp. To
me, much of their work feels like (often surprisingly successful)
efforts to make Haskell as flexible as Lisp without giving up what
they perceive as the safety of a strongly statically typed purely
functional language. While many may not agree with their ideology
(and I certainly don't), I think this is a worthwile goal and I find
their work extremely interesting.
Think of language communities as political... er, sorry, I was getting
carried away.
Juliusz
Juliusz Chroboczek <···@pps.jussieu.fr> writes:
> I feel that what the OP meant is that the simpler applications of
> macros can be replaced with suitable use of higher-order functions.
> Given a SSC, you can replace
>
> (defmacro with-foo (var (&rest args) &body body)
> `(let ((,var (make-foo ,@args)))
> ,@body))
>
> with
>
> (defun call-with-foo (body args)
> (funcall body (apply #'make-foo args)))
>
> the main difference being that in the latter case BODY is a closure
> rather than an actual piece of source code.
Heh. Where possible, I usually do:
(defmacro with-foo ((var &rest args) &body body)
(let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
(flet ((,body-fn (,var) ,@body))
(declare (dynamic-extent #',body-fn))
(call-with-foo #',body-fn ,@args))))
(defun call-with-foo (thunk args)
(funcall thunk (apply #'make-foo args)))
so that clients of most of my macros can use either style, and,
more importantly, so that I can patch CALL-WITH-FOO if I need to change
its behavior, without updating all the callers.
> While this is true, the fact remains that the fully general use of
> DEFMACRO is vastly more powerful (and, to be fair, vastly more
> error-prone) than that. Marc's macro may be a good example of such a
> case, although there are ways of implementing something equally
> convenient in Haskell.
Right. One certainly doesn't write CALL-WITH-LOOPISMS taking a bunch
of funargs...
> Many of the designers of Haskell have excellent knowledge of Lisp.
I've seen only a small amount of Haskell code and I don't doubt this
claim. It seems like a fine language for its target audience. But I
do think its mathy-ness is a barrier to its widespread acceptance.
And I take seriously the various criticisms of Lisp which observe its
mathy-ness as a weakness in the same regard. Sometimes a language
can't change its spots without losing its audience, and I bet that's
true of both Haskell and Lisp. But it's worth continuing to ask
questions...
> To
> me, much of their work feels like (often surprisingly successful)
> efforts to make Haskell as flexible as Lisp without giving up what
> they perceive as the safety of a strongly statically typed purely
> functional language. While many may not agree with their ideology
> (and I certainly don't), I think this is a worthwile goal and I find
> their work extremely interesting.
I take no issue with this.
> Think of language communities as political... er, sorry, I was getting
> carried away.
Heh.
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <669c4w4p.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> Heh. Where possible, I usually do:
>
> (defmacro with-foo ((var &rest args) &body body)
> (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
> (flet ((,body-fn (,var) ,@body))
> (declare (dynamic-extent #',body-fn))
> (call-with-foo #',body-fn ,@args))))
I wouldn't put in the dynamic extent declaration. What if a client
created a closure in the body? (Oh, and you forgot a backquote.)
(defmacro with-foo ((var &rest args) &body body)
(let ((body-fn (gensym (symbol-name :with-foo-body-function-))))
`(flet ((,body-fn (,var) ,@body))
(call-with-foo #',body-fn ,@args))))
···@itasoftware.com writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > Heh. Where possible, I usually do:
> >
> > (defmacro with-foo ((var &rest args) &body body)
> > (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
> > (flet ((,body-fn (,var) ,@body))
> > (declare (dynamic-extent #',body-fn))
> > (call-with-foo #',body-fn ,@args))))
>
> I wouldn't put in the dynamic extent declaration. What if a client
> created a closure in the body? (Oh, and you forgot a backquote.)
>
> (defmacro with-foo ((var &rest args) &body body)
> (let ((body-fn (gensym (symbol-name :with-foo-body-function-))))
> `(flet ((,body-fn (,var) ,@body))
> (call-with-foo #',body-fn ,@args))))
The dynamic extent refers to the CLOSURE, not to the ARGUMENT. The
only consumer of the closure is CALL-WITH-FOO, which does not create a
closure.
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <1yk04s66.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> ···@itasoftware.com writes:
>
> > Kent M Pitman <······@world.std.com> writes:
> >
> > > Heh. Where possible, I usually do:
> > >
> > > (defmacro with-foo ((var &rest args) &body body)
> > > (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
> > > (flet ((,body-fn (,var) ,@body))
> > > (declare (dynamic-extent #',body-fn))
> > > (call-with-foo #',body-fn ,@args))))
> >
> > I wouldn't put in the dynamic extent declaration. What if a client
> > created a closure in the body? (Oh, and you forgot a backquote.)
> >
> > (defmacro with-foo ((var &rest args) &body body)
> > (let ((body-fn (gensym (symbol-name :with-foo-body-function-))))
> > `(flet ((,body-fn (,var) ,@body))
> > (call-with-foo #',body-fn ,@args))))
>
> The dynamic extent refers to the CLOSURE, not to the ARGUMENT. The
> only consumer of the closure is CALL-WITH-FOO, which does not create a
> closure.
I realize that. The point is that a closure internal to body may not
be dynamic. Consider this:
(defun hack (x)
(with-foo (my-foo x)
(push (lambda () (frob my-foo)) some-list)))
Which expands to
(FLET ((#:WITH-FOO-BODY-FUNCTION-329022 (MY-FOO)
(PUSH (LAMBDA NIL (FROB MY-FOO)) SOME-LIST)))
<possible dynamic-extent decl>
(CALL-WITH-FOO #'#:WITH-FOO-BODY-FUNCTION-329022 X))
Notice lambda in the FLET. It closes over a variable bound by the
with-foo-body-function closure which is declared dynamic-extent.
···@itasoftware.com writes:
>
> Kent M Pitman <······@world.std.com> writes:
>
> > ···@itasoftware.com writes:
> >
> > > Kent M Pitman <······@world.std.com> writes:
> > >
> > > > Heh. Where possible, I usually do:
> > > >
> > > > (defmacro with-foo ((var &rest args) &body body)
> > > > (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
> > > > (flet ((,body-fn (,var) ,@body))
> > > > (declare (dynamic-extent #',body-fn))
> > > > (call-with-foo #',body-fn ,@args))))
> > >
> > > I wouldn't put in the dynamic extent declaration. What if a client
> > > created a closure in the body? (Oh, and you forgot a backquote.)
> > >
> > > (defmacro with-foo ((var &rest args) &body body)
> > > (let ((body-fn (gensym (symbol-name :with-foo-body-function-))))
> > > `(flet ((,body-fn (,var) ,@body))
> > > (call-with-foo #',body-fn ,@args))))
> >
> > The dynamic extent refers to the CLOSURE, not to the ARGUMENT. The
> > only consumer of the closure is CALL-WITH-FOO, which does not create a
> > closure.
>
> I realize that. The point is that a closure internal to body may not
> be dynamic. Consider this:
>
> (defun hack (x)
> (with-foo (my-foo x)
> (push (lambda () (frob my-foo)) some-list)))
>
> Which expands to
>
> (FLET ((#:WITH-FOO-BODY-FUNCTION-329022 (MY-FOO)
> (PUSH (LAMBDA NIL (FROB MY-FOO)) SOME-LIST)))
> <possible dynamic-extent decl>
> (CALL-WITH-FOO #'#:WITH-FOO-BODY-FUNCTION-329022 X))
>
> Notice lambda in the FLET. It closes over a variable bound by the
> with-foo-body-function closure which is declared dynamic-extent.
I think we had this discussion before. I think the compiler is responsible
for creating code to evacuate such data structures to the heap. Partly because
I don't think you can actually know what closures are and are not created.
And also because environments are not slots and the question of where this
data is stored is beyond the knoweldge of the programmer.
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <sncf3gei.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> I think we had this discussion before.
I'm pretty sure it wasn't with me.
> I think the compiler is responsible for creating code to evacuate
> such data structures to the heap. Partly because I don't think you
> can actually know what closures are and are not created. And also
> because environments are not slots and the question of where this
> data is stored is beyond the knoweldge of the programmer.
Well, I agree that the compiler *ought* to make it work correctly, but
I'm unsure if the dynamic-extent declaration gives the compiler
license to stack allocate the lexical environment. In any case, I
prefer to live on the safe side and not put it in by default.
···@itasoftware.com writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > I think we had this discussion before.
>
> I'm pretty sure it wasn't with me.
>
> > I think the compiler is responsible for creating code to evacuate
> > such data structures to the heap. Partly because I don't think you
> > can actually know what closures are and are not created. And also
> > because environments are not slots and the question of where this
> > data is stored is beyond the knoweldge of the programmer.
>
> Well, I agree that the compiler *ought* to make it work correctly, but
> I'm unsure if the dynamic-extent declaration gives the compiler
> license to stack allocate the lexical environment.
Let's put it this way: the lexical environment is not a first-class object;
that is, it is not user-manipulable data. The DYNAMIC-EXTENT declaration,
by contrast, IS a promise about user-manipulable data. So if the compiler
infers something from this promise beyond information about user-manipulable
data, it has inferred too much and is in error.
> In any case, I
> prefer to live on the safe side and not put it in by default.
You will cons too much as a result of this assumption, but you are welcome
to do that.
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <k7xr396x.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> Let's put it this way: the lexical environment is not a first-class object;
> that is, it is not user-manipulable data. The DYNAMIC-EXTENT declaration,
> by contrast, IS a promise about user-manipulable data. So if the compiler
> infers something from this promise beyond information about user-manipulable
> data, it has inferred too much and is in error.
When you have this particular form:
(flet ((foo (x) (+ x y)))
(declare (dynamic-extent foo))
(some-function #'foo))
What first-class user object is it that I am declaring dynamic?
> > In any case, I
> > prefer to live on the safe side and not put it in by default.
>
> You will cons too much as a result of this assumption, but you are welcome
> to do that.
How much consing is `too much'? (Modern Zen question)
Thanks for letting me cons, though.
···@itasoftware.com writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > Let's put it this way: the lexical environment is not a first-class object;
> > that is, it is not user-manipulable data. The DYNAMIC-EXTENT declaration,
> > by contrast, IS a promise about user-manipulable data. So if the compiler
> > infers something from this promise beyond information about user-manipulable
> > data, it has inferred too much and is in error.
>
> When you have this particular form:
>
> (flet ((foo (x) (+ x y)))
> (declare (dynamic-extent foo))
You mean (declare (dynamic-extent #'foo))
> (some-function #'foo))
>
> What first-class user object is it that I am declaring dynamic?
The function that results from #'foo.
> > > In any case, I
> > > prefer to live on the safe side and not put it in by default.
> >
> > You will cons too much as a result of this assumption, but you are welcome
> > to do that.
>
> How much consing is `too much'? (Modern Zen question)
>
> Thanks for letting me cons, though.
?
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <bsj332qc.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> ···@itasoftware.com writes:
>
> > When you have this particular form:
> >
> > (flet ((foo (x) (+ x y)))
> > (declare (dynamic-extent foo))
>
> You mean (declare (dynamic-extent #'foo))
Yes, that's what I meant.
> > (some-function #'foo))
> >
> > What first-class user object is it that I am declaring dynamic?
>
> The function that results from #'foo.
I knew that! What I was asking is what is the nature of that object,
and what exactly changes by declaring it to be `dynamic-extent'.
Now in the most general case, it will be a closure of some sort, and
it may have a lexical environment associated with it. But as you
pointed out, the lexical environment is not a user visible object, and
as I pointed out, it may need indefinite extent, so is the
dynamic-extent declaration only referring to the environment/code
pair?
···@itasoftware.com writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > ···@itasoftware.com writes:
> >
> > > When you have this particular form:
> > >
> > > (flet ((foo (x) (+ x y)))
> > > (declare (dynamic-extent foo))
> >
> > You mean (declare (dynamic-extent #'foo))
>
> Yes, that's what I meant.
>
> > > (some-function #'foo))
> > >
> > > What first-class user object is it that I am declaring dynamic?
> >
> > The function that results from #'foo.
>
> I knew that! What I was asking is what is the nature of that object,
> and what exactly changes by declaring it to be `dynamic-extent'.
The objects which are its parts. It has no documented parts. What if
you found that an integer you'd declared dynamic-extent had backing store
that was shared with other integers of the same value? Would you be obliged
therefore not to declare it dynamic-extent? No, of course not. The reason
is that the implementation, in choosing its representations, must take these
issues into account.
> Now in the most general case, it will be a closure of some sort, and
> it may have a lexical environment associated with it. But as you
> pointed out, the lexical environment is not a user visible object, and
> as I pointed out, it may need indefinite extent, so is the
> dynamic-extent declaration only referring to the environment/code
> pair?
I believe it is a remark about the pointer, the actual identity of the
container which is the function. I do not believe it is a remark about
anything internal, since you cannot know what there IS internally, and
since it would hurt all programs and all programming for you to be guessing.
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <y9m71i7f.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> ···@itasoftware.com writes:
>
> > Kent M Pitman <······@world.std.com> writes:
> >
> > > ···@itasoftware.com writes:
> > > > What first-class user object is it that I am declaring dynamic?
> > >
> > > The function that results from #'foo.
> >
> > I knew that! What I was asking is what is the nature of that object,
> > and what exactly changes by declaring it to be `dynamic-extent'.
>
> The objects which are its parts. It has no documented parts. What if
> you found that an integer you'd declared dynamic-extent had backing store
> that was shared with other integers of the same value? Would you be obliged
> therefore not to declare it dynamic-extent? No, of course not. The reason
> is that the implementation, in choosing its representations, must take these
> issues into account.
So the dynamic-extent declaration doesn't do anything.
···@itasoftware.com writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > ···@itasoftware.com writes:
> >
> > > Kent M Pitman <······@world.std.com> writes:
> > >
> > > > ···@itasoftware.com writes:
> > > > > What first-class user object is it that I am declaring dynamic?
> > > >
> > > > The function that results from #'foo.
> > >
> > > I knew that! What I was asking is what is the nature of that object,
> > > and what exactly changes by declaring it to be `dynamic-extent'.
> >
> > The objects which are its parts. It has no documented parts.
> > What if you found that an integer you'd declared dynamic-extent
> > had backing store that was shared with other integers of the same
> > value? Would you be obliged therefore not to declare it
> > dynamic-extent? No, of course not. The reason is that the
> > implementation, in choosing its representations, must take these
> > issues into account.
>
> So the dynamic-extent declaration doesn't do anything.
I just don't understand you sometimes.
When you FLET something and use #' to access it, it conses something.
The DYNAMIC-EXTENT declaration tells the compiler to allocate it on the
stack. If it contains an internal closure, whoever creates that internal
closure is responsible for evacuating the relevant parts of the stack-consed
closure to the heap so that they will survive any dynamic extent of the parent
object. All of this is invisible to the user. What is unclear about
this?
Btw, last time around in this discussion, I took the position you did.
But this argument I am now regurgitating is the one that was presented to
me and I found it to be compelling.
Kent M Pitman <······@world.std.com> writes:
> When you FLET something and use #' to access it, it conses
> something. The DYNAMIC-EXTENT declaration tells the compiler to
> allocate it on the stack.
Okay, I understand that I'm being a bit pedantic here, since the
context of this discussion was what is/isn't on the stack; but, that's
not what DYNAMIC-EXTENT means. It tells the compiler that the object
will only have dynamic extent. Which gives it license to allocate it
on the stack, but if it doesn't want to, or if it can come up with
some better optimization, it can do whatever it wants.
I just bring this up because a couple months ago, didn't someone bring
up some optimization that ACL *wasn't* doing, because it was seeing
DYNAMIC-EXTENT, but reading "stack allocate". I think I remember that
they said they fixed it so the better optimization would take
precedence. (I realize this is a hazy recollection, but I don't use
ACL, so it was only interesting to me in the abstract).
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
···@whirlwind.OCF.Berkeley.EDU (Thomas F. Burdick) writes:
> I just bring this up because a couple months ago, didn't someone bring
> up some optimization that ACL *wasn't* doing, because it was seeing
> DYNAMIC-EXTENT, but reading "stack allocate". I think I remember that
> they said they fixed it so the better optimization would take
> precedence. (I realize this is a hazy recollection, but I don't use
> ACL, so it was only interesting to me in the abstract).
Sure, but that's why it's called dynamic-extent. What you're really
saying is "I'm giving this back quickly - please optimize that if you
can". Indeed it is not a way of demanding a specific optimization; if
it were, it would say so.
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <k7xn22m2.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> ···@itasoftware.com writes:
>
> > Kent M Pitman <······@world.std.com> writes:
> >
> > > ···@itasoftware.com writes:
> > >
> > > > Kent M Pitman <······@world.std.com> writes:
> > > >
> > > > > ···@itasoftware.com writes:
> > > > > > What first-class user object is it that I am declaring dynamic?
> > > > >
> > > > > The function that results from #'foo.
> > > >
> > > > I knew that! What I was asking is what is the nature of that object,
> > > > and what exactly changes by declaring it to be `dynamic-extent'.
> > >
> > > The objects which are its parts. It has no documented parts.
> > > What if you found that an integer you'd declared dynamic-extent
> > > had backing store that was shared with other integers of the same
> > > value? Would you be obliged therefore not to declare it
> > > dynamic-extent? No, of course not. The reason is that the
> > > implementation, in choosing its representations, must take these
> > > issues into account.
> >
> > So the dynamic-extent declaration doesn't do anything.
>
> I just don't understand you sometimes.
I get that a lot. Go figure.
In this case I'm being deliberately naive.
> When you FLET something and use #' to access it, it conses something.
Right.
> The DYNAMIC-EXTENT declaration tells the compiler to allocate it on the
> stack. If it contains an internal closure, whoever creates that internal
> closure is responsible for evacuating the relevant parts of the stack-consed
> closure to the heap so that they will survive any dynamic extent of the parent
> object. All of this is invisible to the user. What is unclear about
> this?
I'll stop being perverse and try to be more perspicuous. When one
uses FLET and passes the function object to an unknown function, in
general the compiler must create the closure in the heap because the
extent is unknown. Presumably, the DYNAMIC-EXTENT declaration is a
hint to the compiler that extent of the closure is dynamic. But if
there is a closure internal to the function, and that closure may
outlast the containing closure, then there is a problem.
So do you ignore the dynamic-extent declaration or do you assume it
applies to the internal closure as well? (or do you assume that the
dynamic-extent declaration applies only to those elements unaffected
by sharing?) If you choose to ignore the dynamic-extent declaration
(or ignore it when an internal closure is created), it seems to me you
are defeating the purpose of the declaration altogether. If you
assume that the declaration is `correct' (i.e., the compiler trusts
your declaration despite the danger that the internal closure may
escape), then you should be very careful where you use it. In
particular, you shouldn't blindly expand WITH-FOOBAR- macros to:
(flet ((body (x) (...)))
(declare (dynamic-extent #'body))
(call-with-foobar #'body ...))
You really do know that no part of body may be needed outside the
extent of the call.
> Btw, last time around in this discussion, I took the position you did.
> But this argument I am now regurgitating is the one that was presented to
> me and I found it to be compelling.
From: Gabe Garza
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <itd7aaxx.fsf@kynopolis.org>
···@itasoftware.com writes:
> I'll stop being perverse and try to be more perspicuous. When one
> uses FLET and passes the function object to an unknown function, in
> general the compiler must create the closure in the heap because the
> extent is unknown. Presumably, the DYNAMIC-EXTENT declaration is a
> hint to the compiler that extent of the closure is dynamic. But if
> there is a closure internal to the function, and that closure may
> outlast the containing closure, then there is a problem.
Please excuse me if my observations seem naive, I'm mainly asking
them to try and understand this thread better. Assume an implicit
question mark after every sentence...
I don't see the problem here. If a closure declared to have dynamic
extent creates a closure that doesn't have dynamic extent, where's the
problem? Assuming the compiler supports the DYNAMIC-EXTENT declaration,
the DYNAMIC-EXTENT closure will be allocated on the stack. The closure
that it creates isn't declared to have dynamic extent and must not be
in an "otherwise inaccessible part of" the first closure; thus wouldn't
it be heap-allocated and therefore available after the first closure
has been obliterated?
> So do you ignore the dynamic-extent declaration or do you assume it
> applies to the internal closure as well? (or do you assume that the
> dynamic-extent declaration applies only to those elements unaffected
> by sharing?)
Again, please correct me if I'm confused, but doesn't the "otherwise
inaccessible part" clause in the CLHS mandate the last option?
Gabe Garza
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <lmi3ziwe.fsf@itasoftware.com>
> ···@itasoftware.com writes:
>
> > So do you ignore the dynamic-extent declaration or do you assume it
> > applies to the internal closure as well? (or do you assume that the
> > dynamic-extent declaration applies only to those elements unaffected
> > by sharing?)
Gabe Garza <·······@ix.netcom.com> writes:
> Again, please correct me if I'm confused, but doesn't the "otherwise
> inaccessible part" clause in the CLHS mandate the last option?
I think you are right.
···@itasoftware.com writes:
> I'll stop being perverse and try to be more perspicuous. When one
> uses FLET and passes the function object to an unknown function, in
> general the compiler must create the closure in the heap because the
> extent is unknown. Presumably, the DYNAMIC-EXTENT declaration is a
> hint to the compiler that extent of the closure is dynamic. But if
> there is a closure internal to the function, and that closure may
> outlast the containing closure, then there is a problem.
>
> So do you ignore the dynamic-extent declaration or do you assume it
> applies to the internal closure as well?
The two are unrelated objects. The closure is created by code inside
but it is not itself inside of bar.
In
(defun foo ()
(flet ((bar (x y)
#'(lambda (z)
#'(lambda () (+ x y z)))))
(declare (dynamic-extent #'bar))
(baz #'bar)))
there is for each closure one of two possibilities: either it's going
onto the stack, in which case either of stack or heap pointers is ok,
or else it's going into the heap, in which case only heap pointers are ok.
So if you peruse your parent chain and find it's stack-allocated, you
need to first copy the stack-allocated part to the heap and then leave a
pointer behind e.g., you might allocate a single extra cell on the stack
to point to the stack-allocated environment and then if you move the thing
to the heap, you just redirect that indirection cell to the heap. so it
starts off:
#<stack closure>
|
| stack heap
| +---------------+ +-------------+
| | | | foo a,b |
+----->| code ------------>| bar b |
| | | baz c,5 |
+---------------+ | ...code |
| | +-------------+
| *------------+
| | |
+---------------+ |
| | |
| X | <--+
| |
+---------------+
| |
| Y |
| |
+---------------+
| |
| parent |
| |
+---------------+
but if the "environment of record" needs to be heap-consed, all the
storage for those cells can be moved (leaving useless space on the
stack, but allowing included closures to be created).
#<stack closure> #<heap closure>
| |
| stack heap | +-------+
| +---------------+ +---------------+ | | |
| | | | foo a,b | +-->| code ---->
+----->| code ------------->| bar b | | |
| | | baz c,5 | +-------+
+---------------+ | ...code | | |
| | +---------------+ | Z |
| *-----------+ | |
| | | +---------------+ +-------+
+---------------+ | | | | |
| | +--->| X |<------- parent|
| (abandoned X) | | | | |
| | +---------------+ +-------+
+---------------+ | |
| | | Y |
| (abandoned Y) | | |
| | +---------------+
+---------------+ | |
| | | parent |
| (abandoned | | |
| parent) | +---------------+
+---------------+
This process is what is called evacuation, and it can be done on a demand
basis so that the original closure can be stack-consed and only heap-consed
if need be, while still not confusing the various closures involved, since
they would reference all environments indirect through a cell that enabled
the migration. (Note only the initially-stack-allocated envs need indirect
pointers, the heap ones don't, since they don't get evacuated. The code
vectors would have this info procedurally wired into them. You probably
wouldn't allocate a parent that was going to be null, since it's statically
known, but I put it here to remind you that there has to be such a pointer
in the more general case.)
There may be other ways to do this as well, but this is at least one way.
In article <···············@world.std.com>,
Kent M Pitman <······@world.std.com> wrote:
>···@itasoftware.com writes:
>
>> Kent M Pitman <······@world.std.com> writes:
>>
>> > Heh. Where possible, I usually do:
>> >
>> > (defmacro with-foo ((var &rest args) &body body)
>> > (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
>> > (flet ((,body-fn (,var) ,@body))
>> > (declare (dynamic-extent #',body-fn))
>> > (call-with-foo #',body-fn ,@args))))
>>
>> I wouldn't put in the dynamic extent declaration. What if a client
>> created a closure in the body? (Oh, and you forgot a backquote.)
>>
>> (defmacro with-foo ((var &rest args) &body body)
>> (let ((body-fn (gensym (symbol-name :with-foo-body-function-))))
>> `(flet ((,body-fn (,var) ,@body))
>> (call-with-foo #',body-fn ,@args))))
>
>The dynamic extent refers to the CLOSURE, not to the ARGUMENT. The
>only consumer of the closure is CALL-WITH-FOO, which does not create a
>closure.
I think he's thinking about the body containing code that creates a
closure:
(setq *foo*
(with-foo (v)
(lambda () v)))
(funcall *foo*)
Does this closure contain any references to the body function? Probably
not, but a more complex case would:
(setq *foo*
(with-foo (v)
(let ((first-time t))
(tagbody
(when first-time
(progn (setq first-time nil)
(lambda () (go tag))))
tag
v))))
However, calling this closure would violate the rule that TAGBODY tags have
dynamic extent. I suspect that all the cases that would require references
to the containing function would have this property -- Common Lisp's
closures are strictly limited to having variable bindings in their
environments, not any control flow bindings.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
In article <············@itasoftware.com>, <···@itasoftware.com> wrote:
>Kent M Pitman <······@world.std.com> writes:
>
>> Heh. Where possible, I usually do:
>>
>> (defmacro with-foo ((var &rest args) &body body)
>> (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
>> (flet ((,body-fn (,var) ,@body))
>> (declare (dynamic-extent #',body-fn))
>> (call-with-foo #',body-fn ,@args))))
>
>I wouldn't put in the dynamic extent declaration. What if a client
>created a closure in the body?
Most WITH-foo types of macros are not designed to be re-enterable.
CALL-WITH-foo probably destroys its foo before returning, so that closure
probably wouldn't be usable unless it never referrenced the variable that
WITH-foo binds. Also, if CALL-WITH-foo makes use of UNWIND-PROTECT to
destroy the foo, you're not allowed to re-enter it.
So in most cases, the procedure really is useless outside the dynamic
extent of the macro invocation, so it might as well be declared so.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
> I've seen only a small amount of Haskell code and I don't doubt this
> claim. It seems like a fine language for its target audience. But I
> do think its mathy-ness is a barrier to its widespread acceptance.
This is definitely true, even when one has some practice in the language.
Its still at the stage where it is used mostly by academics who are
suprised at the mathy way something can be done at all in Haskell, and
publish on it, including sufficient gore to scare off people simply
wishing to use, say, a parsing combinator library.
> And I take seriously the various criticisms of Lisp which observe its
> mathy-ness as a weakness in the same regard.
You mean that some people consider Lisp too mathy??
Kent M Pitman <······@world.std.com> wrote in message news:<···············@world.std.com>...
> Juliusz Chroboczek <···@pps.jussieu.fr> writes:
>
> > I feel that what the OP meant is that the simpler applications of
> > macros can be replaced with suitable use of higher-order functions.
> > Given a SSC, you can replace
> >
> > (defmacro with-foo (var (&rest args) &body body)
> > `(let ((,var (make-foo ,@args)))
> > ,@body))
> >
> > with
> >
> > (defun call-with-foo (body args)
> > (funcall body (apply #'make-foo args)))
> >
> > the main difference being that in the latter case BODY is a closure
> > rather than an actual piece of source code.
>
> Heh. Where possible, I usually do:
>
> (defmacro with-foo ((var &rest args) &body body)
> (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
> (flet ((,body-fn (,var) ,@body))
> (declare (dynamic-extent #',body-fn))
> (call-with-foo #',body-fn ,@args))))
>
> (defun call-with-foo (thunk args)
> (funcall thunk (apply #'make-foo args)))
>
> so that clients of most of my macros can use either style, and,
> more importantly, so that I can patch CALL-WITH-FOO if I need to change
> its behavior, without updating all the callers.
This is a good example of a case where a macro-generating-macro could
come in handy:
(defmacro (define-kent-style-with-macro/function-pair classname)
`(progn
(defmacro (concatenate-symbol 'with- ',classname)
`(let ...))
(defun (concatenate-symbol 'call-with- ',classname)
`(funcall ...))))
Or you could make a single "universal with" macro:
(defmacro with ((var classname &rest args) &body body)
...
(call-with ',classname #'body-fn ,@args))))
(defun call-with (classname thunk args)
(funcall thunk (apply #'make-instance classname args)))
IMO this sort of thing is even more useful if you do something like this:
(defun call-with (classname thunk args)
(let ((x (apply #'make-instance classname args)))
(unwind-protect
(funcall thunk x)
(cleanup-after-with x))))
Then you can do things like:
(defmethod cleanup-after-with ((s my-socket)) () (close s))
(with (my-socket x ...) ...)
to insure that you don't have unclosed sockets lying around.
Erann
From: ···@itasoftware.com
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <3d4f2wtg.fsf@itasoftware.com>
···@flownet.com (Erann Gat) writes:
> This is a good example of a case where a macro-generating-macro could
> come in handy:
Let's see the C++ weenies do this!
> IMO this sort of thing is even more useful if you do something like this:
>
> (defun call-with (classname thunk args)
> (let ((x (apply #'make-instance classname args)))
> (unwind-protect
> (funcall thunk x)
> (cleanup-after-with x))))
>
Slightly safer:
(defun call-with (classname thunk args)
(let ((x nil))
(unwind-protect
(progn (setq x (apply #'make-instance classname args))
(funcall thunk x))
(when x
(cleanup-after-with x)))))
In article <····························@posting.google.com>, "Erann Gat"
<···@flownet.com> wrote:
> Kent M Pitman <······@world.std.com> wrote in message
>> Heh. Where possible, I usually do:
>>
>> (defmacro with-foo ((var &rest args) &body body)
>> (let ((body-fn (gensym "WITH-FOO-BODY-FUNCTION")))
>> (flet ((,body-fn (,var) ,@body))
>> (declare (dynamic-extent #',body-fn)) (call-with-foo #',body-fn
>> ,@args))))
>>
>> (defun call-with-foo (thunk args)
>> (funcall thunk (apply #'make-foo args)))
>> so that clients of most of my macros can use either style, and, more
>> importantly, so that I can patch CALL-WITH-FOO if I need to change its
>> behavior, without updating all the callers.
> This is a good example of a case where a macro-generating-macro could
> come in handy:
> (defmacro (define-kent-style-with-macro/function-pair classname)
> `(progn
> (defmacro (concatenate-symbol 'with- ',classname)
> `(let ...))
> (defun (concatenate-symbol 'call-with- ',classname)
> `(funcall ...))))
Last time this came around I proposed:
(defmacro define-with (with-name invoke-fn vars form-vars)
"defines a macro WITH-NAME that takes (VARS* FORMS*) BODY. The BODY
is enclosed in a LAMBDA with VARS* as arguments and passed as the
first argument to INVOKE-FN; the FORMS are evaluated and passed as the
remaining arguments to INVOKE-FN."
(let ((body-var (gensym "BODY")))
`(defmacro ,with-name ((,@vars ,@form-vars) &body ,body-var)
`(,',invoke-fn #'(lambda (,,@vars) ,@,body-var) ,,@form-vars))))
Tim
> I don't think that's fair. In the same way as the term ``function''
> has lost its ``mathy'' flavour for Lisp programmers and is fully part
> of everyday shop talk, the terms above feel common to Haskell
> programmers. (Actually, many Haskell programmers do not know Category
> Theory, nor how the Category-theoretic notion of monad is linked to
> the Haskell construct of the same name -- and why should they?)
This is quite true. Actually I feel that the wieght given to the theory
behind Haskell tends to drive people away from it. I feel that way myself
and I actually like the language!
> On the other hand, I agree that the OP has used terms familiar to the
> Haskell community but not necessarily so to Lisp programmers, and has
> done so on a Lisp newsgroup without providing suitable definitions (or
> at least intuitions).
Mea culpa, though in my defense I was assuming many lisp hackers might
know the stuff better than I.
> I feel that what the OP meant is that the simpler applications of
> macros can be replaced with suitable use of higher-order functions.
Yes, and followups have shown me part of what I was looking for beyond
these simple applications.
niall
In article <··················································@lp.airnews.net>,
Marc Battyani <·············@fractalconcept.com> wrote:
>Here is a well known example of a macro that generate HTML writing code
>
>(html:html
> (:html :head
> (:body ((:div :style "width:650")
> ((:img :src "logo.jpg")) :br
> ((:p :style "font-family:verdana;font-size:14") title)
> ((:p :style "font-family:verdana;font-size:12") text)
> ((:a :href "andt.html")"back to the screenshots list") :br
>:br
> ((:img :format (:src "~a.jpg" file)))))))
Does that *really* need to be a macro? Why couldn't it be:
(html:output-html
`(:html :head
(:body ((:div :style "width:650")
((:img :src "logo.jpg")) :br
((:p :style "font-family:verdana;font-size:14") ,title)
((:p :style "font-family:verdana;font-size:12") ,text)
((:a :href "andt.html") "back to the screenshots ,list")
:br :br
((:img :format (:src "~a.jpg" ,file)))))))
The only difference is that the transformation would take place at run-time
instead of compile-time. But since this is used for I/O, it's likely that
the I/O channel is the bottleneck, not the transformation.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> Marc Battyani <·············@fractalconcept.com> wrote:
>
> >Here is a well known example of a macro that generate HTML writing code
> >
> >(html:html
> > (:html :head
> > (:body ((:div :style "width:650")
> > ((:img :src "logo.jpg")) :br
> > ((:p :style "font-family:verdana;font-size:14") title)
> > ((:p :style "font-family:verdana;font-size:12") text)
> > ((:a :href "andt.html")"back to the screenshots list") :br
> >:br
> > ((:img :format (:src "~a.jpg" file)))))))
>
> Does that *really* need to be a macro? Why couldn't it be:
>
> (html:output-html
> `(:html :head
> (:body ((:div :style "width:650")
> ((:img :src "logo.jpg")) :br
> ((:p :style "font-family:verdana;font-size:14") ,title)
> ((:p :style "font-family:verdana;font-size:12") ,text)
> ((:a :href "andt.html") "back to the screenshots ,list")
> :br :br
> ((:img :format (:src "~a.jpg" ,file)))))))
>
> The only difference is that the transformation would take place at run-time
> instead of compile-time. But since this is used for I/O, it's likely that
> the I/O channel is the bottleneck, not the transformation.
This is to some extent the question of whether one NEEDS embedded
languages and whether one couldn't instead manipulate such language
notations as program data. Lisp originally did s-expressions this
way. But there is something warm and fuzzy about having the option of
eliminating the quote and saying, subjectively, this IS my code.
Necessary? I doubt it. But it feels completely different.
Also, the use of backquote alerts one to wasted consing and in my
implementation of a notation similar to the above, there is no wasted
consing, even where commas are used. So the backquoted notation would
tend to cue one's visual recognizers to a false concern.
Barry Margolin <······@genuity.net> writes:
> In article <··················································@lp.airnews.net>,
> Marc Battyani <·············@fractalconcept.com> wrote:
> >Here is a well known example of a macro that generate HTML writing code
> >
> >(html:html
> > (:html :head
> > (:body ((:div :style "width:650")
> > ((:img :src "logo.jpg")) :br
> > ((:p :style "font-family:verdana;font-size:14") title)
> > ((:p :style "font-family:verdana;font-size:12") text)
> > ((:a :href "andt.html")"back to the screenshots list") :br
> >:br
> > ((:img :format (:src "~a.jpg" file)))))))
>
> Does that *really* need to be a macro? Why couldn't it be:
>
> (html:output-html
> `(:html :head
> (:body ((:div :style "width:650")
> ((:img :src "logo.jpg")) :br
> ((:p :style "font-family:verdana;font-size:14") ,title)
> ((:p :style "font-family:verdana;font-size:12") ,text)
> ((:a :href "andt.html") "back to the screenshots ,list")
> :br :br
> ((:img :format (:src "~a.jpg" ,file)))))))
>
> The only difference is that the transformation would take place at
> run-time instead of compile-time. But since this is used for I/O,
> it's likely that the I/O channel is the bottleneck, not the
> transformation.
Now the opposite question is more interesting -- why shoud it be done
a zillion times at runtime if it can be done once at compile time? Or
do you think that interpreting gives you something useful here?
--
Janis Dzerins
Eat shit -- billions of flies can't be wrong.
In article <··············@asaka.latnet.lv>,
Janis Dzerins <·····@latnet.lv> wrote:
>Barry Margolin <······@genuity.net> writes:
>
>> In article <··················································@lp.airnews.net>,
>> Marc Battyani <·············@fractalconcept.com> wrote:
>> >Here is a well known example of a macro that generate HTML writing code
>> >
>> >(html:html
>> > (:html :head
>> > (:body ((:div :style "width:650")
>> > ((:img :src "logo.jpg")) :br
>> > ((:p :style "font-family:verdana;font-size:14") title)
>> > ((:p :style "font-family:verdana;font-size:12") text)
>> > ((:a :href "andt.html")"back to the screenshots list") :br
>> >:br
>> > ((:img :format (:src "~a.jpg" file)))))))
>>
>> Does that *really* need to be a macro? Why couldn't it be:
>>
>> (html:output-html
>> `(:html :head
>> (:body ((:div :style "width:650")
>> ((:img :src "logo.jpg")) :br
>> ((:p :style "font-family:verdana;font-size:14") ,title)
>> ((:p :style "font-family:verdana;font-size:12") ,text)
>> ((:a :href "andt.html") "back to the screenshots ,list")
>> :br :br
>> ((:img :format (:src "~a.jpg" ,file)))))))
>>
>> The only difference is that the transformation would take place at
>> run-time instead of compile-time. But since this is used for I/O,
>> it's likely that the I/O channel is the bottleneck, not the
>> transformation.
>
>Now the opposite question is more interesting -- why shoud it be done
>a zillion times at runtime if it can be done once at compile time? Or
>do you think that interpreting gives you something useful here?
Interpreting allows it to vary arbitrarily based on information computed at
run time.
It looks like the macro does allow variables, although I can't tell how
flexible it is about where they're allowed (e.g. can the style strings be
variables?). Can arbitrary expressions be put where the variables are, and
if so how does it tell whether a list is a function call or pattern data?
Backquote, on the other hand, makes it perfectly clear which parts are
static template and which are executed (the latter have the comma prefix),
and allows them anywhere.
For example, what if you didn't want to send the logo to Lynx browsers?
With the interpreted version, you can do:
(html:output-html
`(:html :head
(:body ((:div :style "width:650")
,@(unless (string-equal http:user-agent "lynx")
'(((:img :src "logo.jpg")) :br))
((:p :style "font-family:verdana;font-size:14") ,title)
((:p :style "font-family:verdana;font-size:12") ,text)
((:a :href "andt.html") "back to the screenshots list")
:br :br
((:img :format (:src "~a.jpg" ,file)))))))
Perhaps the HTML macro has some conditional directives, but I'll wager that
it's much more limited than Lisp's general-purpose facilities.
Compare this to the recent thread about LOOP, where the OP was disappointed
to find out that it's not flexible about mixing variable assignments and
end testing. Both LOOP and HTML:HTML are mini-languages, and it's
important to remember the "mini" -- most macro writers are not going to
provide all the control structures that a real language offers.
BTW, the OP might be interested to know that towards the end of the 80's,
Symbolics started using a programming style where many of their fancy
macros were really higher-order functions in disguise. Their entire
table-formatting facility was implemented with macros something like:
(defmacro formatting-table ((stream-name &rest options) &body body)
`(formatting-table-internal (lambda (stream-name) ,@body) ,@options))
(defun formatting-table-internal (continuation &rest options)
(let (table-stream)
(unwind-protect
(progn (setq table-stream (apply #'make-table-stream options))
(funcall continuation table-stream))
(when table-stream
(finish-table table-stream)))))
This combines the convenient syntax of macros with the full power of the
Lisp language and the automatic hygiene of higher-order functions.
Using this style, the above HTML might be written as:
(html:formatting-html
(html:formatting-head)
(html:formatting-body
(html:format-divider :style "width:650")
(unless (string-equal http:user-agent "lynx")
(html:format-image :src "logo.jpg")
(html:format-break))
(html:formatting-paragraph (:style "font-family:verdana;font-size:14")
title)
(html:formatting-paragraph (:style "font-family:verdana;font-size:12")
text)
(html:format-break)
(html:format-break)
(html:format-image :src (format nil "~a.jpg" file))))
Notice that this doesn't require a :format option to html:format-image to
make it use the string as a format control string, I can just call the
standard FORMAT function directly. The HTML macro needs to have lots of
special options like this to provide the flexibility that you get for free
if you use ordinary Lisp.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
> BTW, the OP might be interested to know that towards the end of the 80's,
> Symbolics started using a programming style where many of their fancy
> macros were really higher-order functions in disguise. Their entire
> table-formatting facility was implemented with macros something like:
<code snipped>
> This combines the convenient syntax of macros with the full power of the
> Lisp language and the automatic hygiene of higher-order functions.
Cool example. I'm sure the wizards can produce some remarkable code
by combining these features. Do you happen to know why Symbolics moved
to this style, if there was a reason beyond the advantage you've
highlighted?
In article <········································@godzilla.ics.uci.edu>,
Niall Dalton <·······@ics.uci.edu> wrote:
>> BTW, the OP might be interested to know that towards the end of the 80's,
>> Symbolics started using a programming style where many of their fancy
>> macros were really higher-order functions in disguise. Their entire
>> table-formatting facility was implemented with macros something like:
><code snipped>
>
>> This combines the convenient syntax of macros with the full power of the
>> Lisp language and the automatic hygiene of higher-order functions.
>
>Cool example. I'm sure the wizards can produce some remarkable code
>by combining these features. Do you happen to know why Symbolics moved
>to this style, if there was a reason beyond the advantage you've
>highlighted?
The other advantage, which I think Kent mentioned in passing, is that you
can redefine the helper function on the fly and the callers will pick up
the change immediately. If the macro itself were redefined, all the places
where it's invoked would have to be recompiled. Patches to complex
packages like this were quite frequent, and uses were widespread in the
system and user code, so forcing everything to be recompiled would have
been quite inconvenient.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Tim Bradshaw
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <ey3pu7kez7h.fsf@cley.com>
* Barry Margolin wrote:
> Interpreting allows it to vary arbitrarily based on information computed at
> run time.
> It looks like the macro does allow variables, although I can't tell how
> flexible it is about where they're allowed (e.g. can the style strings be
> variables?). Can arbitrary expressions be put where the variables are, and
> if so how does it tell whether a list is a function call or pattern data?
> Backquote, on the other hand, makes it perfectly clear which parts are
> static template and which are executed (the latter have the comma prefix),
> and allows them anywhere.
> For example, what if you didn't want to send the logo to Lynx browsers?
> With the interpreted version, you can do:
> (html:output-html
> `(:html :head
> (:body ((:div :style "width:650")
> ,@(unless (string-equal http:user-agent "lynx")
> '(((:img :src "logo.jpg")) :br))
> ((:p :style "font-family:verdana;font-size:14") ,title)
> ((:p :style "font-family:verdana;font-size:12") ,text)
> ((:a :href "andt.html") "back to the screenshots list")
> :br :br
> ((:img :format (:src "~a.jpg" ,file)))))))
> Perhaps the HTML macro has some conditional directives, but I'll wager that
> it's much more limited than Lisp's general-purpose facilities.
With my html output macro (which I don't think is better than any of
the others, it's just the one I know) the conditional directives
available are Common Lisp. The above would be something like:
(with-html-output (s)
(:html (:head)
(:body
((:div :style "width:650")
(unless ...
(htm ((:img :src "logo.jpg")) (:br)))
...))))
The only syntactic ugliness is that once you are in embedded Lisp
code, you need to say (HTM ...) to get back to HTML - HTM is just a
local macro which is short for a WITH-HTML-OUTPUT on the original
stream. This could easily be made less verbose with a readtable hack,
but I'm averse to them since they're essentially global.
I quite commonly write stuff like this:
(with-html-output (s)
(:html (:head ...)
(:body
(:ul
(dolist (e es)
(html (:li (esc e))))))))
(ESC is another local macro which quotes html-magic chars in a
string).
There are some syntactic peculiarities remaining, but not many, and
most of them are due to the inevitable confustion between things
evaluated for-value and things evaluated for side-effect (namely
printing). For instance if you have (format s ...) you don't want to
print the result of FORMAT to the stream, you want format to do its
stuff by side-effect, but if you come across a literal string you
generally do want it to be emitted.
Another issue I found is that it's pretty hard for an intepreter to
get reasonable I/O performance, and this can matter. The thing that's
hard is to reduce the number of calls you make. A compiled
implementation can spot constant HTML and reduce it to a constant
string at compile time, compiling code which is a single call to
WRITE-SEQUENCE rather than possibly hundreds of calls. I don't think
the function call overhead is an issue but I suspect that the reduced
number of system calls is: in any case there are significant
performance differences in some cases. My html output macro does this
constant-coalescing only in rather simple cases, but it does do it.
There's basically no interpreter-compiler issue here for me: the
compiler wins hands down.
--tim
Barry Margolin <······@genuity.net> writes:
> Interpreting allows it to vary arbitrarily based on information
> computed at run time.
Compiled code will do that too.
> It looks like the macro does allow variables, although I can't tell
> how flexible it is about where they're allowed (e.g. can the style
> strings be variables?).
It's just how flexible the transforming macro is. I'm using the one
from AServe which does all you mention here. Some other people have
written their own such macros (from postings here I suppose Tim
Bradshaw has (even maybe two different ones), Erik Naggum has and Wade
Humeniuk has).
> Can arbitrary expressions be put where the variables are, and if so
> how does it tell whether a list is a function call or pattern data?
People don't usually define functions in keyword pacakage. It may also
be a restriction that they do not if they want to use the HTML
macro. And templates don't have to be parenthesis-delimited lists as
#\< and #\> can be used as the delimiters as well (Erik's macro does
something like this and Tim's second macro influeced by it as well, I
suppose).
> Backquote, on the other hand, makes it perfectly clear which parts
> are static template and which are executed (the latter have the
> comma prefix), and allows them anywhere.
It is lisp we're discussing here -- should it not be possible to make
something superior to that, at least for this very domain-specific
task?
I see how backquote makes it easy for prototyping the concept of HTML
templates (hehe -- we have had general code generating templates long
before some IBM guys "invented" them for HTML). But why stop at the
prototype?
> For example, what if you didn't want to send the logo to Lynx browsers?
> With the interpreted version, you can do:
>
> (html:output-html
> `(:html :head
> (:body ((:div :style "width:650")
> ,@(unless (string-equal http:user-agent "lynx")
> '(((:img :src "logo.jpg")) :br))
> ((:p :style "font-family:verdana;font-size:14") ,title)
> ((:p :style "font-family:verdana;font-size:12") ,text)
> ((:a :href "andt.html") "back to the screenshots list")
> :br :br
> ((:img :format (:src "~a.jpg" ,file)))))))
>
> Perhaps the HTML macro has some conditional directives, but I'll
> wager that it's much more limited than Lisp's general-purpose
> facilities.
You have full control of how limited the programs you write
are. Example using AServe's html macro:
(html
(:html
(:head)
(:body
((:div :style "width:650")
;; http header values are accessed differently but
;; that is not the point
(unless (string-equal http:user-agent "lynx")
(html ((:img :src "logo.jpg") :br))) ;; notice the html macro again
((:p :style "font-family:verdana;font-size:14") (:princ title))
((:p :style "font-family:verdana;font-size:12") (:princ text))
((:a :href "andt.html") "back to the screenshots list")
:br :br
((:img :src (format nil "~a.jpg" file)))))))
> Compare this to the recent thread about LOOP, where the OP was
> disappointed to find out that it's not flexible about mixing
> variable assignments and end testing. Both LOOP and HTML:HTML are
> mini-languages, and it's important to remember the "mini" -- most
> macro writers are not going to provide all the control structures
> that a real language offers.
But it's possible. If the macro helps -- why not use it. If it does
not help -- I either make a better macro or use "the real language"
(loop in this case is both the "mini language" and the "real
language", by the way). And extensions that move the real language
ahead are always welcome. At least that's the feeling I have.
> BTW, the OP might be interested to know that towards the end of the
> 80's, Symbolics started using a programming style where many of
> their fancy macros were really higher-order functions in disguise.
> Their entire table-formatting facility was implemented with macros
> something like:
>
> (defmacro formatting-table ((stream-name &rest options) &body body)
> `(formatting-table-internal (lambda (stream-name) ,@body) ,@options))
>
> (defun formatting-table-internal (continuation &rest options)
> (let (table-stream)
> (unwind-protect
> (progn (setq table-stream (apply #'make-table-stream options))
> (funcall continuation table-stream))
> (when table-stream
> (finish-table table-stream)))))
>
> This combines the convenient syntax of macros with the full power of the
> Lisp language and the automatic hygiene of higher-order functions.
>
> Using this style, the above HTML might be written as:
>
> (html:formatting-html
> (html:formatting-head)
> (html:formatting-body
> (html:format-divider :style "width:650")
> (unless (string-equal http:user-agent "lynx")
> (html:format-image :src "logo.jpg")
> (html:format-break))
> (html:formatting-paragraph (:style "font-family:verdana;font-size:14")
> title)
> (html:formatting-paragraph (:style "font-family:verdana;font-size:12")
> text)
> (html:format-break)
> (html:format-break)
> (html:format-image :src (format nil "~a.jpg" file))))
Looks a bit like something one has (or had last time I had a look at
it a year or so ago) to write in CL-HTTP (no stream passing all over
the place, thought). Does it look like HTML now? It's very verbose and
I would not want to use this, especially when I know it can be done in
a much better way.
> Notice that this doesn't require a :format option to
> html:format-image to make it use the string as a format control
> string, I can just call the standard FORMAT function directly. The
> HTML macro needs to have lots of special options like this to
> provide the flexibility that you get for free if you use ordinary
> Lisp.
See my example above.
--
Janis Dzerins
Eat shit -- billions of flies can't be wrong.
From: Tim Bradshaw
Subject: Re: Macros versus higher order functions.
Date:
Message-ID: <ey37ktrg6vk.fsf@cley.com>
* Janis Dzerins wrote:
> People don't usually define functions in keyword pacakage. It may also
> be a restriction that they do not if they want to use the HTML
> macro. And templates don't have to be parenthesis-delimited lists as
> #\< and #\> can be used as the delimiters as well (Erik's macro does
> something like this and Tim's second macro influeced by it as well, I
> suppose).
My `second macro' is actually just a file/stream syntax. It could be
made possible to use within CL source code, but it would need
readtable hacks which I dislike, and that wasn't really the aim - what
I was after was something less painful to type than HTML (even with
psgml in emacs I find typing HTML a pain, though psgml makes it very much
less painful than the `clever' WYSYWIG editors). Yes, it was
influenced by Erik's post mentioning a syntax, though it is probably
hackier than Erik's design.
(yes, it will be public at some point)
--tim
"Barry Margolin" <······@genuity.net> wrote
> Janis Dzerins <·····@latnet.lv> wrote:
> >Barry Margolin <······@genuity.net> writes:
> >> Marc Battyani <·············@fractalconcept.com> wrote:
> >> >Here is a well known example of a macro that generate HTML writing
code
> >> >
> >> >(html:html
> >> > (:html :head
> >> > (:body ((:div :style "width:650")
> >> > ((:img :src "logo.jpg")) :br
> >> > ((:p :style "font-family:verdana;font-size:14") title)
> >> > ((:p :style "font-family:verdana;font-size:12") text)
> >> > ((:a :href "andt.html")"back to the screenshots list")
:br
> >> >:br
> >> > ((:img :format (:src "~a.jpg" file)))))))
> >>
> >> Does that *really* need to be a macro? Why couldn't it be:
> >>
> >> (html:output-html
> >> `(:html :head
> >> (:body ((:div :style "width:650")
> >> ((:img :src "logo.jpg")) :br
> >> ((:p :style "font-family:verdana;font-size:14") ,title)
> >> ((:p :style "font-family:verdana;font-size:12") ,text)
> >> ((:a :href "andt.html") "back to the screenshots ,list")
> >> :br :br
> >> ((:img :format (:src "~a.jpg" ,file)))))))
> >>
> >> The only difference is that the transformation would take place at
> >> run-time instead of compile-time. But since this is used for I/O,
> >> it's likely that the I/O channel is the bottleneck, not the
> >> transformation.
> >
> >Now the opposite question is more interesting -- why shoud it be done
> >a zillion times at runtime if it can be done once at compile time? Or
> >do you think that interpreting gives you something useful here?
>
> Interpreting allows it to vary arbitrarily based on information computed
at
> run time.
Being compiled does not mean you can't have variable parts.
I fact the 3 main reasons why I've done what is a small HTML to Lisp
compiler are:
1 - Efficiency, if the html you give is constant, you will get only one big
write-string. As you says I/O is probably the bottleneck but in the
architecture I use I can have 1 Lisp application server behind several
Apache servers; Apache talks to the Lisp server through mod_lisp (persistent
socket) and 100Mps or 1Gbps links. So the IO bottleneck does not apply to
the Lisp server.
2 - Complicated constructs. Some tags I have need to insert styles or
JavaScript at the beginning of the HTML code and it would be difficult to do
this with an interpreter. For instance I have tabs controls that requires
this. ex (:tabs ("tab name 1" (html code for tab 1))etc...
3 - I really like writing compilers. I would have written it that way even
without the other reasons...
> It looks like the macro does allow variables, although I can't tell how
> flexible it is about where they're allowed (e.g. can the style strings be
> variables?). Can arbitrary expressions be put where the variables are,
and
> if so how does it tell whether a list is a function call or pattern data?
> Backquote, on the other hand, makes it perfectly clear which parts are
> static template and which are executed (the latter have the comma prefix),
> and allows them anywhere.
> For example, what if you didn't want to send the logo to Lynx browsers?
> With the interpreted version, you can do:
>
> (html:output-html
> `(:html :head
> (:body ((:div :style "width:650")
> ,@(unless (string-equal http:user-agent "lynx")
> '(((:img :src "logo.jpg")) :br))
> ((:p :style "font-family:verdana;font-size:14") ,title)
> ((:p :style "font-family:verdana;font-size:12") ,text)
> ((:a :href "andt.html") "back to the screenshots list")
> :br :br
> ((:img :format (:src "~a.jpg" ,file)))))))
>
> Perhaps the HTML macro has some conditional directives, but I'll wager
that
> it's much more limited than Lisp's general-purpose facilities.
My HTML macro has some specialized conditionals but can use any Lisp ones,
You just have to call the html macro again to go on in HTML. Tim Bradshaw
has explained this in another post.
In the example you give I could write it that way:
'(html:html
(:html :head
(:body ((:div :style "width:650")
(unless (string-equal user-agent "lynx")
(html:html ((:img :src "logo.jpg")) :br))
((:p :style "font-family:verdana;font-size:14") title)
((:p :style "font-family:verdana;font-size:12") text)
((:a :href "andt.html") "back to the screenshots list")
:br :br
((:img :format (:src "~a.jpg" file)))))))))
This is quite similar to what you've written but expands to :
(progn
(write-string "<HTML><HEAD></HEAD><BODY><DIV STYLE=\"width:650\">"
html:*html-stream*)
(unless (string-equal user-agent "lynx") (html:html ((:img :src
"logo.jpg")) :br))
(write-string "<P STYLE=\"font-family:verdana;font-size:14\">"
html:*html-stream*)
(princ title html:*html-stream*)
(write-string "</P><P STYLE=\"font-family:verdana;font-size:12\">"
html:*html-stream*)
(princ text html:*html-stream*)
(write-string "</P><A HREF=\"andt.html\">back to the screenshots
list</A><BR><BR><IMG SRC=\"" html:*html-stream*)
(format html:*html-stream* "~a.jpg" file)
(write-string "\"></DIV></BODY></HTML>" html:*html-stream*))
Doing constant aggregation reduces the number of write-string from 29 to 5.
> Compare this to the recent thread about LOOP, where the OP was
disappointed
> to find out that it's not flexible about mixing variable assignments and
> end testing. Both LOOP and HTML:HTML are mini-languages, and it's
> important to remember the "mini" -- most macro writers are not going to
> provide all the control structures that a real language offers.
>
> BTW, the OP might be interested to know that towards the end of the 80's,
> Symbolics started using a programming style where many of their fancy
> macros were really higher-order functions in disguise. Their entire
> table-formatting facility was implemented with macros something like:
>
> (defmacro formatting-table ((stream-name &rest options) &body body)
> `(formatting-table-internal (lambda (stream-name) ,@body) ,@options))
>
> (defun formatting-table-internal (continuation &rest options)
> (let (table-stream)
> (unwind-protect
> (progn (setq table-stream (apply #'make-table-stream options))
> (funcall continuation table-stream))
> (when table-stream
> (finish-table table-stream)))))
>
> This combines the convenient syntax of macros with the full power of the
> Lisp language and the automatic hygiene of higher-order functions.
>
> Using this style, the above HTML might be written as:
>
> (html:formatting-html
> (html:formatting-head)
> (html:formatting-body
> (html:format-divider :style "width:650")
> (unless (string-equal http:user-agent "lynx")
> (html:format-image :src "logo.jpg")
> (html:format-break))
> (html:formatting-paragraph (:style "font-family:verdana;font-size:14")
> title)
> (html:formatting-paragraph (:style "font-family:verdana;font-size:12")
> text)
> (html:format-break)
> (html:format-break)
> (html:format-image :src (format nil "~a.jpg" file))))
>
> Notice that this doesn't require a :format option to html:format-image to
> make it use the string as a format control string, I can just call the
> standard FORMAT function directly. The HTML macro needs to have lots of
> special options like this to provide the flexibility that you get for free
> if you use ordinary Lisp.
No I have a special variable called html:*html-stream* so that you can do
(format html:*html-stream* "~a.jpg" file). There is no need to create a
string when we can write directly to the stream.
I have some predefined tags like :tabs; :on-off; :menu, etc. that output
HTML and JavaScript not related to their names.
I any case, any Lisp version is much much better than the
ASP/Java/PHP/Perl/Python/C#/<future trendy languages> alternatives.
Marc
Janis Dzerins <·····@latnet.lv> writes:
> Barry Margolin <······@genuity.net> writes:
> > In article <··················································@lp.airnews.net>,
> > Marc Battyani <·············@fractalconcept.com> wrote:
> > >Here is a well known example of a macro that generate HTML writing code
> > >
> > >(html:html
> > > (:html :head
> > > (:body ((:div :style "width:650")
> > > ((:img :src "logo.jpg")) :br
> > > ((:p :style "font-family:verdana;font-size:14") title)
> > > ((:p :style "font-family:verdana;font-size:12") text)
> > > ((:a :href "andt.html")"back to the screenshots list") :br
> > >:br
> > > ((:img :format (:src "~a.jpg" file)))))))
> > Does that *really* need to be a macro? Why couldn't it be:
> > (html:output-html
> > `(:html :head
> > (:body ((:div :style "width:650")
> > ((:img :src "logo.jpg")) :br
> > ((:p :style "font-family:verdana;font-size:14") ,title)
> > ((:p :style "font-family:verdana;font-size:12") ,text)
> > ((:a :href "andt.html") "back to the screenshots ,list")
> > :br :br
> > ((:img :format (:src "~a.jpg" ,file)))))))
> > The only difference is that the transformation would take place at
> > run-time instead of compile-time. But since this is used for I/O,
> > it's likely that the I/O channel is the bottleneck, not the
> > transformation.
> Now the opposite question is more interesting -- why shoud it be
> done a zillion times at runtime if it can be done once at compile
> time? Or do you think that interpreting gives you something useful
> here?
Personally, I'd rather see some of both; a compiler to do what can be
compiled, and an interpreter to do what needs to be interpreted.
Suppose I'm going to be pulling in a set of entries from a database.
That's going to turn into an itemized list, or perhaps a table.
It would be pretty slick to be able to write:
(defun table-row (id name description price)
(html:tr
(html:td :align 'LEFT :valign 'TOP id)
(html:td :align 'LEFT :valign 'TOP name)
(html:td :align 'LEFT :valign 'TOP description)
(html:td :align 'RIGHT :valign 'TOP price)))
And then have a "mixed" data structure that results, looking rather
like Barry's:
(setf table
(list :table :border 1 ;;; this bit is interpreted
(html:thead ;;;; this bit is compiled...
(html:tr (html:th :align 'CENTER "ID"))
(html:tr (html:th :align 'CENTER "Name"))
(html:tr (html:th :align 'CENTER "Description"))
(html:tr (html:th :align 'CENTER "Price"))
(loop
for element in database
for id = (id element)
for name = (name element)
for descr = (description element)
for price = (price element)
do
append (table-row id name description price)))))
This would then get appended into the body of an HTML document.
It's simultaneously got "static, precompiled" bits as well as
"dynamic, interpreted" bits.
It seems to me that there needs to be a combination of them within
just about _any_ document:
-> Some data will obviously be dynamic, and so you need to be able to
generate dynamic queries;
-> There will be some "style" info that relates to what browser is
being used. That's not known 'til runtime, so it needs to get
_attached_ at that point. But if there are only a few choices,
it's quite possible that virtually all of the style info can be
precomputed at run time, with a "Netscape style," "IE-4" style,
"IE-5" style, and such. They'd all be precomputed; all that need
happen is that the runtime portion pick which one to insert.
--
(reverse (concatenate 'string ········@" "enworbbc"))
http://www.cbbrowne.com/info/spiritual.html
I'm not born again -- my mother got it right the first time.
<········@acm.org> wrote
>
> Personally, I'd rather see some of both; a compiler to do what can be
> compiled, and an interpreter to do what needs to be interpreted.
>
> Suppose I'm going to be pulling in a set of entries from a database.
>
> That's going to turn into an itemized list, or perhaps a table.
>
> It would be pretty slick to be able to write:
>
> (defun table-row (id name description price)
> (html:tr
> (html:td :align 'LEFT :valign 'TOP id)
> (html:td :align 'LEFT :valign 'TOP name)
> (html:td :align 'LEFT :valign 'TOP description)
> (html:td :align 'RIGHT :valign 'TOP price)))
>
> And then have a "mixed" data structure that results, looking rather
> like Barry's:
>
> (setf table
> (list :table :border 1 ;;; this bit is interpreted
> (html:thead ;;;; this bit is compiled...
> (html:tr (html:th :align 'CENTER "ID"))
> (html:tr (html:th :align 'CENTER "Name"))
> (html:tr (html:th :align 'CENTER "Description"))
> (html:tr (html:th :align 'CENTER "Price"))
> (loop
> for element in database
> for id = (id element)
> for name = (name element)
> for descr = (description element)
> for price = (price element)
> do
> append (table-row id name description price)))))
Well I do prefer
(defun table-row (id name description price)
(html:html
(:tr
((:td :align "LEFT" :valign "TOP") id)
((:td :align "LEFT" :valign "TOP") name)
((:td :align "LEFT" :valign "TOP") description)
((:td :align "RIGHT" :valign "TOP") price))))
(html:html
((:table :border "1")
(:thead
(:tr ((:th :align "CENTER") "ID")
((:th :align "CENTER") "Name")
((:th :align "CENTER") "Description")
((:tr (html:th :align "CENTER") "Price"))))
(loop
for element in database
for id = (id element)
for name = (name element)
for descr = (description element)
for price = (price element)
do (table-row id name description price)))))
This does exactly the same, it's just more readable and compiled.
It expands to:
(progn
(write-string "<TABLE BORDER=\"1\"><THEAD><TR><TH
ALIGN=\"CENTER\">ID</TH><TH ALIGN=\"CENTER\">Name</TH><TH
ALIGN=\"CENTER\">Description</TH><TH
ALIGN=\"CENTER\">Price</TH></TR></THEAD>"
html:*html-stream*)
(loop for element in database
for id = (id element)
for name = (name element)
for descr = (description element)
for price = (price element)
do (table-row id name description price))
(write-string "</TABLE>" html:*html-stream*))
Marc
In article
<········································@godzilla.ics.uci.edu>, Niall
Dalton wrote:
>Hi,
>
>I've been learning lisp recently, with a background in various imperative
>and functional (ML and Haskell) languages.
>
>Macros in lisp are certainly powerful, but many of the ways to use them
>that I think of I could equally well use higher order functions.
Do you have any examples?
>Can
>someone suggest the kinds of things macros are better for rather than high
>order functions?
There are big differences.
Macros receive the unevaluated argument forms as their parameters,
so they can control evaluation.
In a function call, the argument forms are evaluated, and the
resulting values are passed. So macros control the evaluation
of their arguments; functions do not.
A macro calls returns a form which is substituted in place of the
macro call.
A function can dynamically return another function (lexical
closure) but that closure is not automatically evaluated; it
must be explicitly funcalled, and does not have special access
to the environment in which it is funcalled---it's a ``cooked''
callable object with its own environment.
A function could return a form, just like a macro. But that
form is not implicitly evaluated. It can be explicitly evaluated
with eval, but there is no access to the lexical context.
The forms returned by macros have access to the lexical environment in
which they replace the macro call.
Functions do not have access to their calling environment, except
through other functions which capture that environment and are
passed into them, and through special variables. (E.g. you can
make a closure, then call a function and give it that closure,
and by calling that closure, the called function has some indirect
access to the captured environment).
Lastly, macros can take parameter lists which have a nested tree structure.
Lambda variables can appear at all the nesting levels; they pull out the
corresponding unevaluated form. This is called destructuring (and is
available in a separate binding construct called destructuring-bind).
Functions do not support nested parameter lists and destructuring.
All of this has implications on the kinds of things you can do with
higher-order function style programming versus macros.
> >Macros in lisp are certainly powerful, but many of the ways to use them
> >that I think of I could equally well use higher order functions.
> Do you have any examples?
The kinds of examples I was thinking of are defining macros to implement
control structures. Ignoring any standard/predefined for a moment, say
implementing a "for" or "while" loop macro.
Perhaps something like (stolen from some sample code by Paul Graham I
believe):
(defmacro for ((var start stop) &body body)
(let ((gstop (gensym)))
`(do ((,var ,start (1+ ,var))
(,gstop ,stop))
((> ,var ,gstop))
,@body)))
compared to stolen Haskell code (from Simon Peyton Jones):
for [] f = return ()
for (x:xs) f = f x >> for xs f
uses as for [1..10] (\x -> putStr (show x))
Or
repeatN 0 a = return ()
repeatN n a = a >> repeatN (n-1) a
used as: repeatN 10 (putChar 'x')
these do not do exactly the same thing as the lisp code perhaps, but
should the use of macros versus functions.
> There are big differences.
>
> Macros receive the unevaluated argument forms as their parameters,
> so they can control evaluation.
>
> In a function call, the argument forms are evaluated, and the
> resulting values are passed. So macros control the evaluation
> of their arguments; functions do not.
ah ok, I was not clear enough here. I was thinking in more general terms
than specifically lisp. For instance in Haskell the arguments will be
passed in an unevaluated form. I should separate my question into lisp
and non-lisp specific parts. I presume though, that I can wrap an argument
side a lambda, pass the funtion and later evaluate it when I want? Hence
avoiding evaluating the expression before it is passed.
> A macro calls returns a form which is substituted in place of the
> macro call.
I think I'm just not yet 'getting' the lisp style of thinking here.
A use of this that comes to mind is perhaps returning customized control
structures, but again I can do that with functions I think. Can you
suggest a different line of thought?
> Lastly, macros can take parameter lists which have a nested tree structure.
> Lambda variables can appear at all the nesting levels; they pull out the
> corresponding unevaluated form. This is called destructuring (and is
> available in a separate binding construct called destructuring-bind).
>
> Functions do not support nested parameter lists and destructuring.
this is interesting, can you provide an example?
> All of this has implications on the kinds of things you can do with
> higher-order function style programming versus macros.
Can you elaborate on what kinds of things I may want to do with these?
I am not saying macros are bad or anything, simply tying to appreciate and
understand them better.
Thanks for the response,
niall
You might be coming at this from the wrong direction.
A macro is a function which takes code as input and produces other code
to be interpreted or compiled. That means a macro lets me create any
little language I like, as long as I can code the macro to expand it
into code the compiler will understand.
Looked at this way there is no point in comparing macros to conventional
functions since the latter do not see the code of their arguments nor is
their output treated as code by the interpreter/compiler.
kenny
clinisys
> A macro is a function which takes code as input and produces other code
> to be interpreted or compiled. That means a macro lets me create any
> little language I like, as long as I can code the macro to expand it
> into code the compiler will understand.
I presume its consider ok (or good?) style to do this a lot? I can see
that be defining a whole bunch of macros that some programs could suddenly
become very succinct indeed. And if done properly, very efficiently as
well it seems.
Niall Dalton wrote:
>
> > A macro is a function which takes code as input and produces other code
> > to be interpreted or compiled. That means a macro lets me create any
> > little language I like, as long as I can code the macro to expand it
> > into code the compiler will understand.
>
> I presume its consider ok (or good?) style to do this a lot?
yes. it comes up most when one has cooked up a little embedded hack
which is going to get a lot of use and one wants to make things, as you
say, more succinct.
kenny
clinisys
In article
<········································@godzilla.ics.uci.edu>, Niall
Dalton wrote:
>> >Macros in lisp are certainly powerful, but many of the ways to use them
>> >that I think of I could equally well use higher order functions.
>> Do you have any examples?
>The kinds of examples I was thinking of are defining macros to implement
>control structures. Ignoring any standard/predefined for a moment, say
>implementing a "for" or "while" loop macro.
>
>Perhaps something like (stolen from some sample code by Paul Graham I
>believe):
>(defmacro for ((var start stop) &body body)
> (let ((gstop (gensym)))
> `(do ((,var ,start (1+ ,var))
> (,gstop ,stop))
> ((> ,var ,gstop))
> ,@body)))
Right, so as you can see, this macro simply matches a call like
(for (x 0 10) ...body forms...)
and transforms it into
(do ((x 0 (1+ x)) (#:G1 10) ((> x #:G1)) ...body forms...))
almost exactly as if you typed it in place of the call yourself.
The macro is effectively doing programming for you, performing
programmer-like tasks, such as inventing variable names via (gensym).
You specify some simple thing in the form of a macro call, and the macro
writes code which implement that thing's semantics.
>> In a function call, the argument forms are evaluated, and the
>> resulting values are passed. So macros control the evaluation
>> of their arguments; functions do not.
>
>ah ok, I was not clear enough here. I was thinking in more general terms
>than specifically lisp. For instance in Haskell the arguments will be
>passed in an unevaluated form.
And if the language is purely functional, then it doesn't make a
difference. A symbol can be replaced by the expression that computed it
or vice versa. In mathematics, function arguments are dummies; given f(x)
= x*x, we can expand f(a+1) into (a+1)*(a+1). If we know the value of
a, we can evaluate it first, then substitute, or we can substitute the
formula, then evaluate.
But what about issues of scope? I know nothing about Haskell.
In any case, in Lisp, you can have side effects, so the sequencing
of evaluation can be critical, in addition to the lexical context of
evaluation.
>> A macro calls returns a form which is substituted in place of the
>> macro call.
>
>I think I'm just not yet 'getting' the lisp style of thinking here.
>A use of this that comes to mind is perhaps returning customized control
>structures, but again I can do that with functions I think. Can you
>suggest a different line of thought?
The different line of thought is that you cannot do that with functions
in Lisp. Function calls are evaluated too late in order to be able to
replace a chunk of the program with an alternate form.
At the top level, it doesn't make much difference. It looks like
the result of a macro is a form which is evaluated immediately.
But if a macro is buried, say, in a loop, or in a function body, there
is a difference. The macro is expanded but once, when the function is
defined, or when the loop is read. No matter how many times that function
is called, or how many iterations of that loop, the macro is not expanded
more than once. That evaluation takes place using the expanded form;
the macro processing is over and done with at macro expansion time,
and leaves behind a form in its wake.
Also, you really do need control over evaluation to be able to
do control constructs. Sometimes control constructs avoid evaluating
something in some conditions.
Suppose you want to implement a form which behaves similarly to the the
Common Lisp when operator.
(defmacro my-when (expr &rest body)
`(if ,expr (progn ,@body) nil))
my-when cannot be a function, because then the body forms would be
evaluated before the if test decision is made. The forms must only
be evaluated if the expr yields a non-null value.
You could quote quote all the forms:
(my-when-func '(< x 0) '(format t "x is negative~%"))
but the whole idea is to be transparent. The real when operator doesn't
need quoting! Moreover, you still have the problem that even if the
function gets the forms quoted, it cannot evaluate them in the proper
context where the identifier x is established, unless x is a dynamically
scoped (special) variable. So this can't work because of issues of
scope.
>> Lastly, macros can take parameter lists which have a nested tree structure.
>> Lambda variables can appear at all the nesting levels; they pull out the
>> corresponding unevaluated form. This is called destructuring (and is
>> available in a separate binding construct called destructuring-bind).
>>
>> Functions do not support nested parameter lists and destructuring.
>
>this is interesting, can you provide an example?
Yes, how about
(defmacro for ((var start stop) &body body)
(let ((gstop (gensym)))
`(do ((,var ,start (1+ ,var))
(,gstop ,stop))
((> ,var ,gstop))
,@body)))
Haha! Have you seen that before?
Examine the lambda list closely, it is:
((var start stop) &body body)
Note how the var start stop are nested within a list. This specifies
a syntax. The macro must be called like this:
(for (form1 form2 form3) zero-or-more-forms)
the macro takes this syntax, and pulls out the parameters from it. This
is what is meant by destructuring: taking a structure and decomposing
it back to the elements from which it was made---the opposite of
constructing.
Destructuring is what allows a macro to succinctly analyze, so that
it can take the pieces it needs and then synthesize. Without built-in
destructuring, it would be hard, because the programmer would have
to write explicit tree processing code to hunt down the pieces.
If destructuring didn't exist, it would be necessary to invent it;
nobody would want to write tree-munging code over and over again
in each new macro. :)
In other words, the for macro could be written like this:
(defmacro for (&rest args) ...)
Then it would have to validate, all by itself, that the argument
list has the correct syntax: that it's a list whose first
element is a three-element list. Then it would have to traverse
into that three element list and pull out the values.
With destructuring, the macro expansion system itself performs
your syntax check and extraction, so you don't have to.
You can think of the macro as forming a link between two templates;
the destructuring template using which things are extracted from the
input, and the constructing template according to which which things
are assembled to form an output. The first is the destructuring
lambda list, the second by a back-quoted form. Both are artifacts which
resemble the form that they analyze or synthesize.
>> All of this has implications on the kinds of things you can do with
>> higher-order function style programming versus macros.
>Can you elaborate on what kinds of things I may want to do with these?
You can invent invent new language constructs with whatever syntax and
semantics you want them to have. The macros don't have to implement
all of the semantics; they act as a programmer interface to functions
or other macros.
> In any case, in Lisp, you can have side effects, so the sequencing
> of evaluation can be critical, in addition to the lexical context of
> evaluation.
Well Haskell is purely functional and lazy, so I was trying to apply my
Haskell style thinking too directly in this case.
> You could quote quote all the forms:
>
> (my-when-func '(< x 0) '(format t "x is negative~%"))
>
> but the whole idea is to be transparent.
Yup, I can appreciate that. Again, I was thinking too non-strictly.
> the macro takes this syntax, and pulls out the parameters from it. This
> is what is meant by destructuring: taking a structure and decomposing
> it back to the elements from which it was made---the opposite of
> constructing.
thank you! I knew I was missing something from my understanding of that
macro.
thanks for going into details on this, I'm statying to really appreciate
it now.
niall
Niall Dalton <·······@ics.uci.edu> writes:
> Macros in lisp are certainly powerful, but many of the ways to use them
> that I think of I could equally well use higher order functions. Can
> someone suggest the kinds of things macros are better for rather than high
> order functions? I guess this is part of my functional language
> indoctrination:-) Literature pointers also welcom.
>
Well, everyone who has written web applications in Lisp has macros which do
things like:
(with-html-output (s)
(:html
(:head (:title (system-state-header s)))
(:body
(:h1 (system-state-header s))
(:table
(:thead
(:tr
(:td "System")
(:td "Up?")
(:td "Load")
(:td "Comments")))
(:tbody
(dolist ...
(htm
((:tr :valign "baseline")
(loop ...)))
...))))))
And this is a good example of the kind of things that macros are often used
for: extending Lisp to allow other languages (in this case HTML) to be
embedded seamlessly into Lisp programs, or alternatively writing other
`little languages' which can be embedded into Lisp programs.
Note that the embedded bit is important: if you look at this code you can
see that it's jumping back and forth between a lispy representation of HTML
and actual Lisp code (for instance the calls to SYSTEM-STATE-HEADER, and
the DOLIST which then has something called HTM inside it which jumps back
into HTML then out again in the LOOP). Writing little languages is a common
thing to do, but embedding them in an existing language in such a way that
things just work correctly (in this example, although it's not clear, things
liike variable scope work just the way you'd expect) and you can freely
mix the languages is something that macros make very possible. This kind
of macro is essentially an embedded compiler which takes the little language
and compiles it into Lisp.
Of course, HTML generation is kind of a done thing now, but the idea is
valid for many sorts of extensions to the language.
--tim