From: Eyal Lotem
Subject: The power difference between a macro and a function
Date:
Message-ID: <435c1bf4@news.bezeqint.net>
Hello,
I am well aware of the differences between a function and a macro, but I
want to ask a different question. What extra expressive power macros give
us on top of ordinary functions, in a language that fully supports
closures?
Lets for the sake of simplicity drop the issue of performance, and focus on
expressiveness only.
The main similarity I see between macros and functions in a
closure-supportive environment:
A macro takes its arguments unevaluated
A function can take its argument unevaluated if the arguments are explicitly
wrapped within a closure.
The main differences:
DiffA. A macro does this implicitly, a function requires explicit wrapping.
DiffB. A macro has access to the AST "inside" the given arguments.
I personally think that macros have serious downsides, and as such, the
extra power they allow must justify itself. I think the first difference is
unimportant and can be made trivial with syntactic changes.
My questions are:
A. What other differences do you see between macros and functions that take
closures?
B. What practical cases of DiffB did you encounter? Were there no reasonable
alternatives?
Eyal Lotem wrote:
> Hello,
>
> I am well aware of the differences between a function and a macro, but I
> want to ask a different question. What extra expressive power macros give
> us on top of ordinary functions, in a language that fully supports
> closures?
Hi,
Ernst van Waning gave a good talk on this very topic:
http://wiki.alu.org/lisp-user-meeting-amsterdam-april-2004
(The other talks by Dirk and Arthur are also illuminating.)
You are very correct in the sort of "anarchist" viewpoint that power
must regularly justify itself. Somewhere in LL2, Dan Weinreb (who
worked for one of those random Lisp Machine companies) explained that
in his projects, people could always add functions, no problem. With
macros, there was some added scrutiny. With reader-macros, there was a
LOT of justification required.
http://ll2.ai.mit.edu/
(This is just my memory. He mentioned this in the Q&A sessions, when
people were going nuts debating over macros.)
> Lets for the sake of simplicity drop the issue of performance, and focus on
> expressiveness only.
No prob. Macros are best introduced in the light of readability, not
performance hackery. Just as with any other paradigm, macros can be
badly- and over-used.
> The main similarity I see between macros and functions in a
> closure-supportive environment:
> A macro takes its arguments unevaluated
> A function can take its argument unevaluated if the arguments are explicitly
> wrapped within a closure.
>
> The main differences:
> DiffA. A macro does this implicitly, a function requires explicit wrapping.
> DiffB. A macro has access to the AST "inside" the given arguments.
Well, they do their dirty deeds at different times. Joe Marshall once
explained, "Variables abstract over values, functions abstract over
behavior, macros abstract over syntax," which I think is an interesting
way of putting it.
> I personally think that macros have serious downsides, and as such, the
> extra power they allow must justify itself. I think the first difference is
> unimportant and can be made trivial with syntactic changes.
>
> My questions are:
> A. What other differences do you see between macros and functions that take
> closures?
> B. What practical cases of DiffB did you encounter? Were there no reasonable
> alternatives?
Here's how I look at Lisp macros. You know how it's usually pretty
simple for programmers to massage and manipulate data? They pretty much
do it all the time. Why isn't "code" modelled in such a way to lend
itself to simple data manipuation, just as you would any other domain?
Instead of writing:
for( i=0; i<arr.length; i++ ) {
int num = arr[ i ];
// ...
}
people frequently beg their language designer to allow something like:
for( int num in arr ) {
// ...
}
But the programming language user could've done that herself; it's a
simple, mechanical change, which has clear readability advantages.
What's the point of being a programmer if you can't do these mechanical
things? When you can't change these things, you become more of a user
than a programmer; increased power lies with the language designer, who
is not the expert for every domain his users program for.
Now, this is just a very simple example, no doubt there may be more
convincing ones.
Tayssir
--
"Moreover, under existing conditions, private capitalists inevitably
control, directly or indirectly, the main sources of information
(press, radio, education). It is thus extremely difficult, and indeed
in most cases quite impossible, for the individual citizen to come to
objective conclusions and to make intelligent use of his political
rights."
-- Albert Einstein
http://www.monthlyreview.org/598einst.htm
Eyal Lotem wrote:
> Hello,
>
> I am well aware of the differences between a function and a macro, but I
> want to ask a different question. What extra expressive power macros give
> us on top of ordinary functions, in a language that fully supports
> closures?
There are many macros in Common Lisp that would not be possible to write
in a language that supported first class closures but not first class
symbols. Here's a small list of the most commonly used ones:
defun
setf
defstruct
defclass
defmethod
loop
Now assume you had symbols as a first class data type in your language.
Think about the syntax that would make defun, defstruct, defclass,
and defmethod usable. Think about the efficiency lost by setf having to
do table lookups at runtime and loop having to parse its arguments at
runtime.
Macros are what allow things like CLOS to be written as portable
libraries, and not require extensive modifications to the underlying
compiler.
-- MJF
From: Peter Seibel
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <m264rnejnw.fsf@beagle.local>
Eyal Lotem <··········@gmail.com> writes:
> Hello,
>
> I am well aware of the differences between a function and a macro,
> but I want to ask a different question. What extra expressive power
> macros give us on top of ordinary functions, in a language that
> fully supports closures?
>
> Lets for the sake of simplicity drop the issue of performance, and
> focus on expressiveness only.
>
> The main similarity I see between macros and functions in a
> closure-supportive environment: A macro takes its arguments
> unevaluated A function can take its argument unevaluated if the
> arguments are explicitly wrapped within a closure.
>
> The main differences:
> DiffA. A macro does this implicitly, a function requires explicit wrapping.
> DiffB. A macro has access to the AST "inside" the given arguments.
Difference B is the big one. Because it means you can do
transformations a lot more interesting than just wrapping some forms
in a closure. I give a number of examples in my book. See in
particular
<http://www.gigamonkeys.com/book/practical-building-a-unit-test-framework.html>
<http://www.gigamonkeys.com/book/practical-parsing-binary-files.html>
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s36l8FlrmnlU1@individual.net>
Eyal Lotem wrote:
> Hello,
>
> I am well aware of the differences between a function and a macro, but I
> want to ask a different question. What extra expressive power macros give
> us on top of ordinary functions, in a language that fully supports
> closures?
>
> Lets for the sake of simplicity drop the issue of performance, and focus on
> expressiveness only.
>
> The main similarity I see between macros and functions in a
> closure-supportive environment:
> A macro takes its arguments unevaluated
> A function can take its argument unevaluated if the arguments are explicitly
> wrapped within a closure.
This is an incorrect comparison. The arguments a macro takes are
completely different from the arguments a function takes. You can't say
that the arguments to a macro are not evaluated while they are for a
function because there is nothing that could be evaluated in the case of
a macro in the first place. Macro expansion takes place at compile time
(not true in the general case, but a good approximation). At that time,
the bindings are not available that could be used to evaluate the
"arguments" for a macro because they are only available at runtime
(again not true in the general case, but again a good approximation).
Closures can indeed be used to defer evaluation of an expression. Macros
are also often characterized as being able to defer evaluation, but
that's not true in a strict sense. They are only able to rearrange the
code such that the evaluation of an expression will be deferred. That's
maybe a subtle, but important difference. (Another way to state the same
thing is that at runtime, the s-expressions that the macro can still see
and rearrange are gone, so what is supposedly not evaluated at macro
expansion time and what is evaluated at runtime are completely different
beasts.)
> The main differences:
> DiffA. A macro does this implicitly, a function requires explicit wrapping.
> DiffB. A macro has access to the AST "inside" the given arguments.
So strictly speaking, DiffA is an incorrect characterization. DiffB is
correct.
> I personally think that macros have serious downsides, and as such, the
> extra power they allow must justify itself. I think the first difference is
> unimportant and can be made trivial with syntactic changes.
>
> My questions are:
> A. What other differences do you see between macros and functions that take
> closures?
> B. What practical cases of DiffB did you encounter? Were there no reasonable
> alternatives?
Macros provide you with the ability to create syntactic abstractions
while functions allow you to build functional abstractions. That's the
only real difference. Abstractions are used to hide implementation
details, and macros allow you to hide implementation details that you
cannot hide otherwise.
Consider the following function:
(defun whilef (pred thunk)
(when (funcall pred)
(funcall thunk)
(whilef pred thunk)))
You can use this to express a while iteration:
(whilef
(lambda () (< i 10))
(lambda ()
(print i)
(incf i)))
The problem with this functional abstraction is that it leaks an
important implementation detail: It uses closures to control the
iteration. Changing the syntax for creating closures doesn't change
this, users of this function would still have to be aware that they have
to provide closures to make the function work.
A macro allows you to hide this implementation detail:
(defmacro while (expression &body body)
`(whilef
(lambda () ,expression)
(lambda () ,@body)))
Now you can say this:
(while (< i 10)
(print i)
(incf i))
This is not only nicer to read than the functional version, it also
allows you to change your mind about the implementation without changing
the interface:
(defmacro while (expression &body body)
(let ((start (gensym))
(end (gensym)))
`(tagbody
,start (unless ,expression
(go ,end))
,@body
(go ,start)
,end)))
You can still say:
(while (< i 10)
(print i)
(incf i))
This is probably more efficient than the previous version of that macro
because this version avoids the overhead of creating two closures and
calling them (and doesn't rely on a "sufficiently smart" compiler to
automagically inline such closures).
The user doesn't (or shouldn't) really care that (or whether) the while
abstraction uses closures to control the iteration. It is not possible
to hide that implementation detail without resorting to a macro. If a
functional abstraction doesn't leak such details, there is no need to
use a macro.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Ulrich Hobelmann
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s39e6Flr8njU1@individual.net>
Eyal Lotem wrote:
> The main differences:
> DiffA. A macro does this implicitly, a function requires explicit wrapping.
> DiffB. A macro has access to the AST "inside" the given arguments.
B is a huge thing: while functions operate on arguments (values), macros
operate on s-expressions, i.e. full ASTs as you say. That means that
the macro can process a whole sublanguage in a way as complex as you
want. There are simple control-flow macros, then there is stuff like
LOOP, then there are regular-expression compilers and parser generators,
stuff that other languages do via external (pre-)processors, because the
language itself is rather rigid.
> I personally think that macros have serious downsides, and as such, the
> extra power they allow must justify itself. I think the first difference is
> unimportant and can be made trivial with syntactic changes.
Not at all. In other languages (C, Java, ML) you need full extra
languages and compilers, such as lex and yacc to do certain tasks.
--
Blessed are the young for they shall inherit the national debt.
Herbert Hoover
I think you got a little over-excited here, Ulrich. ;-)
Ulrich Hobelmann wrote:
> Eyal Lotem wrote:
>> The main differences:
>> DiffA. A macro does this implicitly, a function requires explicit
>> wrapping. DiffB. A macro has access to the AST "inside" the given
>> arguments.
>
> B is a huge thing: while functions operate on arguments (values), macros
> operate on s-expressions, i.e. full ASTs as you say. That means that
> the macro can process a whole sublanguage in a way as complex as you
> want.
Embedding the sublanguage within Lisp, yes.
> There are simple control-flow macros, then there is stuff like
> LOOP, then there are regular-expression compilers and parser generators,
> stuff that other languages do via external (pre-)processors, because the
> language itself is rather rigid.
Regular expression compilers and parser generators can be implemented as
ordinary functions, they do not need to be macros.
>> I personally think that macros have serious downsides, and as such, the
>> extra power they allow must justify itself. I think the first difference
>> is unimportant and can be made trivial with syntactic changes.
>
> Not at all. In other languages (C, Java, ML) you need full extra
> languages and compilers, such as lex and yacc to do certain tasks.
Firstly, macros and yacc are not comparable: Macros can convert incoming
syntax into host code. Yacc statically checks grammars and generates host
code that implements a parser for that grammar.
Secondly, "full extra languages and compilers" like yacc are used out of
convenience. They are not a necessity. You can write parsers by hand in C,
Java, ML etc. if you wish.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
From: Ulrich Hobelmann
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s4i5uFlvoeuU1@individual.net>
Jon Harrop wrote:
> Regular expression compilers and parser generators can be implemented as
> ordinary functions, they do not need to be macros.
But if they are just functions, then how do you get efficient code? Lex
and yacc, for instance, all generate C code; they aren't just C functions.
>>> I personally think that macros have serious downsides, and as such, the
>>> extra power they allow must justify itself. I think the first difference
>>> is unimportant and can be made trivial with syntactic changes.
>> Not at all. In other languages (C, Java, ML) you need full extra
>> languages and compilers, such as lex and yacc to do certain tasks.
>
> Firstly, macros and yacc are not comparable: Macros can convert incoming
> syntax into host code. Yacc statically checks grammars and generates host
> code that implements a parser for that grammar.
A yacc macro can also check grammars statically and output errors if you
want it to. It definitely generates host code (Lisp in this case).
> Secondly, "full extra languages and compilers" like yacc are used out of
> convenience. They are not a necessity. You can write parsers by hand in C,
> Java, ML etc. if you wish.
But often you don't want to hand-code every complex piece of code.
--
Blessed are the young for they shall inherit the national debt.
Herbert Hoover
In article <··············@individual.net>,
Ulrich Hobelmann <···········@web.de> wrote:
> Jon Harrop wrote:
> > Regular expression compilers and parser generators can be implemented as
> > ordinary functions, they do not need to be macros.
>
> But if they are just functions, then how do you get efficient code?
You don't, but it's not an issue in this thread. Go back and reread the
original post, he said "drop the issue of performance, and consider
expressiveness only."
The thing about macros is that they allow you to express things
concisely. Imagine trying to do something like LOOP entirely with
lambda expressions. The whole point of LOOP is that it allows you to
abbreviate down to the essential words, so the purpose of the code is
readily understood.
If you don't consider this significant, then what's the point of
high-level languages in the first place? Turing equivalence means that
they're no more expressive than machine language, but there's clearly a
qualitative difference.
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: Eyal Lotem
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <436007f5@news.bezeqint.net>
Barry Margolin wrote:
> In article <··············@individual.net>,
> Ulrich Hobelmann <···········@web.de> wrote:
>
>> Jon Harrop wrote:
>> > Regular expression compilers and parser generators can be implemented
>> > as ordinary functions, they do not need to be macros.
>>
>> But if they are just functions, then how do you get efficient code?
>
> You don't, but it's not an issue in this thread. Go back and reread the
> original post, he said "drop the issue of performance, and consider
> expressiveness only."
>
> The thing about macros is that they allow you to express things
> concisely. Imagine trying to do something like LOOP entirely with
> lambda expressions. The whole point of LOOP is that it allows you to
> abbreviate down to the essential words, so the purpose of the code is
> readily understood.
I think that this was already done in a very nice way in Smalltalk. Forgive
me if I "twist" the language a bit, but I think it illustrates the point:
(1 to: 10) do: [ n |
...
]
Or:
[conditional code] whileTrue: [
...
]
And various other methods that work on closures make for very nice looping
capabilities, while using only the very flexible method call syntax
provided in the base language.
From: Ulrich Hobelmann
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3sbhj5Fkl0a0U1@individual.net>
Eyal Lotem wrote:
>> The thing about macros is that they allow you to express things
>> concisely. Imagine trying to do something like LOOP entirely with
>> lambda expressions. The whole point of LOOP is that it allows you to
>> abbreviate down to the essential words, so the purpose of the code is
>> readily understood.
>
> I think that this was already done in a very nice way in Smalltalk. Forgive
> me if I "twist" the language a bit, but I think it illustrates the point:
>
> (1 to: 10) do: [ n |
> ...
> ]
>
> Or:
>
> [conditional code] whileTrue: [
> ...
> ]
>
> And various other methods that work on closures make for very nice looping
> capabilities, while using only the very flexible method call syntax
> provided in the base language.
Yes, Smalltalk's block syntax is simple and beautiful. The downside is
that the compiler needs to know about the lower-level methods in order
to optimize them. A non-optimized whileTrue: or looping operation of
your choice would be really slow, not just function-call-slow, but
dynamic-method-call-slow.
Macros transform the syntax into something more efficient.
--
The road to hell is paved with good intentions.
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3scfvbFn6g7iU1@individual.net>
Eyal Lotem wrote:
>>The thing about macros is that they allow you to express things
>>concisely. Imagine trying to do something like LOOP entirely with
>>lambda expressions. The whole point of LOOP is that it allows you to
>>abbreviate down to the essential words, so the purpose of the code is
>>readily understood.
>
> I think that this was already done in a very nice way in Smalltalk. Forgive
> me if I "twist" the language a bit, but I think it illustrates the point:
>
> (1 to: 10) do: [ n |
> ...
> ]
>
> Or:
>
> [conditional code] whileTrue: [
> ...
> ]
>
> And various other methods that work on closures make for very nice looping
> capabilities, while using only the very flexible method call syntax
> provided in the base language.
Now you have made exactly one kind of application for macros nicer.
Macros have a broader applicability than that, and as soon as you have
macros in your language, it doesn't matter that much anymore how nice
your closure abstraction looks like, since you can make anything look
the way you want. As already mentioned, you can even hide the fact that
you are using closures internally which gives you the opportunity to
change your mind about the implementation of (some of) your macros, for
example to optimize them.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Ulrich Hobelmann wrote:
> Jon Harrop wrote:
>> Regular expression compilers and parser generators can be implemented as
>> ordinary functions, they do not need to be macros.
>
> But if they are just functions, then how do you get efficient code?
You write them in an efficient language. :-)
> Lex and yacc, for instance, all generate C code; they aren't just C
> functions.
Yes. You are quite right that generating code will logically improve
performance. However, facilitating code generation costs performance, so it
is not clear which approach is actually fastest.
>> Firstly, macros and yacc are not comparable: Macros can convert incoming
>> syntax into host code. Yacc statically checks grammars and generates host
>> code that implements a parser for that grammar.
>
> A yacc macro can also check grammars statically and output errors if you
> want it to. It definitely generates host code (Lisp in this case).
Yes. I was referring to direct use of macros (not libraries) vs yacc. If you
use a yacc macro then you get the best of both worlds.
>> Secondly, "full extra languages and compilers" like yacc are used out of
>> convenience. They are not a necessity. You can write parsers by hand in
>> C, Java, ML etc. if you wish.
>
> But often you don't want to hand-code every complex piece of code.
I agree. I think this is a major benefit of using BNF.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
From: Peter Seibel
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <m2pspucmzo.fsf@beagle.local>
Jon Harrop <······@jdh30.plus.com> writes:
> Yes. You are quite right that generating code will logically improve
> performance. However, facilitating code generation costs
> performance, so it is not clear which approach is actually fastest.
In what way does facilitating code generation "cost performance"?
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
Peter Seibel wrote:
> In what way does facilitating code generation "cost performance"?
It is an oversimplification but there is a trade-off between static or
closed-world optimisations and run-time code generation.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
From: Peter Seibel
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <m2ll0hczkz.fsf@beagle.local>
Jon Harrop <······@jdh30.plus.com> writes:
> Peter Seibel wrote:
>> In what way does facilitating code generation "cost performance"?
>
> It is an oversimplification but there is a trade-off between static
> or closed-world optimisations and run-time code generation.
Okay, you've been hanging out here in c.l.l. for quite some time. How
is it that you still think macros have to do with run-time code
generation? Now there are *other* Lisp features that might make
closed-world optimizations problematic but macros are not one of them.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
From: Edi Weitz
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <uacgxv8rd.fsf@agharta.de>
On Tue, 25 Oct 2005 16:48:28 GMT, Peter Seibel <·····@gigamonkeys.com> wrote:
> Okay, you've been hanging out here in c.l.l. for quite some
> time. How is it that you still think macros have to do with run-time
> code generation?
Peter, how is it that you still think Dr. Harrop would listen to
rational arguments?
:)
Cheers,
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Peter Seibel wrote:
> Jon Harrop <······@jdh30.plus.com> writes:
>> It is an oversimplification but there is a trade-off between static
>> or closed-world optimisations and run-time code generation.
>
> Okay, you've been hanging out here in c.l.l. for quite some time. How
> is it that you still think macros have to do with run-time code
> generation? Now there are *other* Lisp features that might make
> closed-world optimizations problematic but macros are not one of them.
I didn't mention macros?!
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
From: Peter Seibel
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <m2hdb5ccaw.fsf@beagle.local>
Jon Harrop <······@jdh30.plus.com> writes:
> Peter Seibel wrote:
>> Jon Harrop <······@jdh30.plus.com> writes:
>>> It is an oversimplification but there is a trade-off between static
>>> or closed-world optimisations and run-time code generation.
>>
>> Okay, you've been hanging out here in c.l.l. for quite some time. How
>> is it that you still think macros have to do with run-time code
>> generation? Now there are *other* Lisp features that might make
>> closed-world optimizations problematic but macros are not one of them.
>
> I didn't mention macros?!
No, that's just the subject at hand. But never mind. Hopefully the OP
will ignore this subthread or at least recognize that it has no
bearing on his question.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
Jon Harrop wrote:
> However, facilitating code generation costs performance, so it
> is not clear which approach is actually fastest.
Actually, the only cost is having a code generator available at
run-time and using it. Said run-time code generator can do everything
that an in-advance code generator can do, so the only "performance"
cost is the time to generate code at run-time. The ability to generate
code at run-time does not restrict the implementation.
You'd think that someone who so strongly favored static analysis would
know when it could be used.
> I agree. I think this is a major benefit of using BNF.
It would be if BNF was a sane way to express things or if the types of
languages that folks expressable with BNF were sane. (Yup, you can do
better than BNF for exactly the same languages, but that's only been
known since the 70s.)
But, go ahead believing that 25 levels of precedence and associativity
is reasonable.
······@earthlink.net wrote:
> Jon Harrop wrote:
>> However, facilitating code generation costs performance, so it
>> is not clear which approach is actually fastest.
>
> Actually, the only cost is having a code generator available at
> run-time and using it. Said run-time code generator can do everything
> that an in-advance code generator can do, so the only "performance"
> cost is the time to generate code at run-time. The ability to generate
> code at run-time does not restrict the implementation.
It does if you want everything to be statically typed (at the first compile
time), e.g. MetaOCaml.
> But, go ahead believing that 25 levels of precedence and associativity
> is reasonable.
A 7 year old learning maths can understand a lot of precedence and
associativity.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> ······@earthlink.net wrote:
> > Actually, the only cost is having a code generator available at
> > run-time and using it. Said run-time code generator can do everything
> > that an in-advance code generator can do, so the only "performance"
> > cost is the time to generate code at run-time. The ability to generate
> > code at run-time does not restrict the implementation.
>
> It does if you want everything to be statically typed (at the first compile
> time), e.g. MetaOCaml.
No, it doesn't. If MetaOCaml adds restrictions, it does so for its own
reasons. They must be good reasons because everything associated with
ML is perfect, but the restrictions are not required by static
analysis.
But, you'd have to actually know something about code-generation to
understand why run-time code generation is not more limited than
"before run-time" code generation.
> > But, go ahead believing that 25 levels of precedence and associativity
> > is reasonable.
>
> A 7 year old learning maths can understand a lot of precedence and
> associativity.
When concentrating on precedence, most people can get 7-8 levels right
consistently. However, "concentrating on precedence" is not a goal.
In fact, said concentration is actually an obstacle. To put it another
way, "understanding precedence" isn't the goal or a good thing, it is a
cost/overhead/a bad thing.
One can see the same thing in natural language utterances. There are
many syntactically and semantically correct sentences that are too
convoluted to use if one's goal is communication/understanding with
people. With NL, we can't choose different rules. With computer
languages we can.
······@earthlink.net wrote:
>> A 7 year old learning maths can understand a lot of precedence and
>> associativity.
>
> When concentrating on precedence, most people can get 7-8 levels right
> consistently. However, "concentrating on precedence" is not a goal.
> In fact, said concentration is actually an obstacle. To put it another
> way, "understanding precedence" isn't the goal or a good thing, it is a
> cost/overhead/a bad thing.
You must concede that the vast majority of people choose to use
associativity and precedence. We could all maximally bracket our
mathematics to remove ambiguity but we don't because we know the
expressions are much more comprehensible when written in conventional
style.
> One can see the same thing in natural language utterances. There are
> many syntactically and semantically correct sentences that are too
> convoluted to use if one's goal is communication/understanding with
> people. With NL, we can't choose different rules. With computer
> languages we can.
Mathematics is a better analogy. The mathematics seen at degree level
probably has as many associativities and precedences associated with it as
the average programming language.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> ······@earthlink.net wrote:
> >> A 7 year old learning maths can understand a lot of precedence and
> >> associativity.
> >
> > When concentrating on precedence, most people can get 7-8 levels right
> > consistently. However, "concentrating on precedence" is not a goal.
> > In fact, said concentration is actually an obstacle. To put it another
> > way, "understanding precedence" isn't the goal or a good thing, it is a
> > cost/overhead/a bad thing.
>
> You must concede that the vast majority of people choose to use
> associativity and precedence. We could all maximally bracket our
> mathematics to remove ambiguity but we don't because we know the
> expressions are much more comprehensible when written in conventional
> style.
>
> > One can see the same thing in natural language utterances. There are
> > many syntactically and semantically correct sentences that are too
> > convoluted to use if one's goal is communication/understanding with
> > people. With NL, we can't choose different rules. With computer
> > languages we can.
>
> Mathematics is a better analogy. The mathematics seen at degree level
> probably has as many associativities and precedences associated with it as
> the average programming language.
Mathematics is a relatively poor analogy. There is a great deal of
ambiguity in it that is overcome by the fact it's read by intelligent
humans.
When I was at university I was taught different mathematical
terminology by the maths department and electronics department. For
example do the beginning squiggly S and the terminating "dx" in an
integral sign imply a bracketing of the expression inside it?
Mathematics is a language invented by humans for humans, and one that
has grown over time, replete with many unnecessarily, confusing and
unnecessarily confusing aspects. There's no reason why programming
should continue to propogate them.
Also, expressions in mathematics don't relate precisely to expressions
in programming. In integer math there is more than one possible
concept of division, in FP & integer math associative laws always don't
hold precisely.
When I program in languages with infix syntax I only use ~3 of the
precedence levels (those of +, -, *, /, ->, ! and unary *). Most other
programmers I've met seem to do the same.
Jon Harrop wrote:
> ······@earthlink.net wrote:
> > When concentrating on precedence, most people can get 7-8 levels right
> > consistently. However, "concentrating on precedence" is not a goal.
> > In fact, said concentration is actually an obstacle. To put it another
> > way, "understanding precedence" isn't the goal or a good thing, it is a
> > cost/overhead/a bad thing.
Note that Harrop doesn't dispute that concentrating on precedence isn't
a goal, that it's actually overhead.
Note that the 7-8 levels of precedence that Harrop argues that people
are capable of is a long way from the 25 levels required. That's why
people don't actually use precedence in the way that Harris argues for.
> You must concede that the vast majority of people choose to use
> associativity and precedence.
I'll happily concede all irrelevant facts.
However, "vast majority" isn't usefully true. The "vast majority of
people" don't use arithmetic expressions, so they're not choosing
anything.
Of the fraction that do use arithmetic expressions, the vast majority
use at most 2 operators and 1 level of precedence. The next large
chunk
use at most 4 operators with 2 levels of precedence. Neither of those
communities support Harrop's claim.
The above rarely involves communication, so ambiguity is resolved
outside
the expression.
But, let's look at math communities. Collectively, there are more
operators, but within communities/actual documents, the set of
operators
is fairly small. Again, that's not at all like programming.
In the small fraction of math that involves communication, there are
humans on both side, and humans resolve ambiguity using semantics and
"must have meant", two options that are not available to programming
languages.
However, let's look at programming. If Harrop's claim was true, all
infix languages would have the same precedence and associtivity rules.
Oops - they don't. They mostly agree on a few, but ....
Finally, so let's look at programmers. They tend to use parens as soon
as there are more than 3 different operators in an expression, and
they're more likely to use parens in expressions involving operators
that the programmer doesn't use as often.
This is especially true of programs with multiple authors. Yes,
there's
occasionally someone who uses 5-6 operator expressions without
parentheses,
but as other people work on that code, parens get added because their
fellow humans are not interested in thinking about precedence.
> We could all maximally bracket our
> mathematics to remove ambiguity but we don't because we know the
> expressions are much more comprehensible when written in conventional
> style.
Most arithmetic is interpreted by its author, so ambiguity is resolved
outside the text. Math papers are interpreted by humans who will
unknowingly "fix" precedence errors. Programming languages don't work
that way.
>I think
>typesetting makes infix much more attractive because it can augment
>expressions with subtle changes to indicate associativity and precedence,
>e.g. slightly bigger spaces between low-precedence operators.
That augmentation isn't necessary or even valuable if associativity and
precedence actually worked. Moreover, the typesetter has to know the
source language because the rules vary.
> > One can see the same thing in natural language utterances. There are
> > many syntactically and semantically correct sentences that are too
> > convoluted to use if one's goal is communication/understanding with
> > people. With NL, we can't choose different rules. With computer
> > languages we can.
>
> Mathematics is a better analogy. The mathematics seen at degree level
is practiced by folks who have no relevance to typical programmers.
The natural language example is quite on point as it involves
communication
and lots of operators. It's an easier problem than programming
languages because it can use semantics and "must have meant" to resolve
things.
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s8s82Fn7q76U1@individual.net>
Jon Harrop wrote:
> ······@earthlink.net wrote:
>
>>>A 7 year old learning maths can understand a lot of precedence and
>>>associativity.
>>
>>When concentrating on precedence, most people can get 7-8 levels right
>>consistently. However, "concentrating on precedence" is not a goal.
>>In fact, said concentration is actually an obstacle. To put it another
>>way, "understanding precedence" isn't the goal or a good thing, it is a
>>cost/overhead/a bad thing.
>
> You must concede that the vast majority of people choose to use
> associativity and precedence.
A majority uses indeed associativity and precedence. That doesn't
necessarily mean that they choose to use them. It seems to me that a
considerably large number of people doesn't know about prefix or postfix
languages. So they can't make a choice. Another group of people reject
the notion of prefix or postfix languages without having tried them, out
of a gut feeling to stick with what they are used to. Of course, there
are also people who tried both and indeed choose infix languages, but
others make the switch.
> We could all maximally bracket our
> mathematics to remove ambiguity but we don't because we know the
> expressions are much more comprehensible when written in conventional
> style.
I always had a tendency to maximally bracket mathematical expressions in
the various infix languages I encountered because I never really
understood all the precedence rules. It always seemed to me that there
were strange unintuitive exceptions in those rules, and I never bothered
to understand the full rules. Using parentheses always seemed to me to
be the better solution that is guaranteed to be unambiguous. I think I
am by far not the only one to think that way.
Admittedly there seems to be value in infix notation for mathematical
expressions even if they are fully parenthesized. At least I regularly
catch myself to want to use infix notation for mathematical stuff when I
program in Lisp. Maybe it's just a bad habit from the old days, who
knows... ;)
>>One can see the same thing in natural language utterances. There are
>>many syntactically and semantically correct sentences that are too
>>convoluted to use if one's goal is communication/understanding with
>>people. With NL, we can't choose different rules. With computer
>>languages we can.
>
> Mathematics is a better analogy.
I don't think so. Lots of code I haven seen and that I write doesn't
contain a lot of mathematics. What is neat about Lisp is that it allows
me to write code that is closer to "natural" language than in other
programming languages. (Consider the LOOP macro, for example.)
That's another reason why I think that the ray tracer code is not very
representative of what is typically done in programs, at least in my
experience...
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Well, you and Rob have both given very good counter arguments. :-)
Pascal Costanza wrote:
> I always had a tendency to maximally bracket mathematical expressions in
> the various infix languages I encountered because I never really
> understood all the precedence rules. It always seemed to me that there
> were strange unintuitive exceptions in those rules, and I never bothered
> to understand the full rules. Using parentheses always seemed to me to
> be the better solution that is guaranteed to be unambiguous. I think I
> am by far not the only one to think that way.
In Mathematica you can use write the AST (prefix) or use infix operators to
do the same thing:
1 + 2
Plus[1, 2]
I think Mathematica users tend to use the former. However, the latter is not
as succinct as it is in Lisp so this isn't such a fair comparison. I think
typesetting makes infix much more attractive because it can augment
expressions with subtle changes to indicate associativity and precedence,
e.g. slightly bigger spaces between low-precedence operators.
>> Mathematics is a better analogy.
>
> I don't think so. Lots of code I haven seen and that I write doesn't
> contain a lot of mathematics. What is neat about Lisp is that it allows
> me to write code that is closer to "natural" language than in other
> programming languages. (Consider the LOOP macro, for example.)
>
> That's another reason why I think that the ray tracer code is not very
> representative of what is typically done in programs, at least in my
> experience...
Ray tracers are certainly not representative of many other types of program
in terms of style and construction of code. However, my ray tracer is not
totally dissimilar to much of the code that I write, and it uses a variety
of different constructs (e.g. loops and recursion) and algorithms (e.g.
folding over lists and searching trees) that I think makes it more
representative of other types of programs than most benchmarks.
There are still pathological cases, like SBCL being completely GC bound.
However, I think this only applies to SBCL in this case (only SBCL-compiled
Lisp shows a huge performance improvement when consing is eliminated).
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s9ja0Fn3evrU1@individual.net>
Jon Harrop wrote:
> Well, you and Rob have both given very good counter arguments. :-)
>
> Pascal Costanza wrote:
>
>>I always had a tendency to maximally bracket mathematical expressions in
>>the various infix languages I encountered because I never really
>>understood all the precedence rules. It always seemed to me that there
>>were strange unintuitive exceptions in those rules, and I never bothered
>>to understand the full rules. Using parentheses always seemed to me to
>>be the better solution that is guaranteed to be unambiguous. I think I
>>am by far not the only one to think that way.
>
> In Mathematica you can use write the AST (prefix) or use infix operators to
> do the same thing:
[...]
Whatever.
>>>Mathematics is a better analogy.
>>
>>I don't think so. Lots of code I haven seen and that I write doesn't
>>contain a lot of mathematics. What is neat about Lisp is that it allows
>>me to write code that is closer to "natural" language than in other
>>programming languages. (Consider the LOOP macro, for example.)
>>
>>That's another reason why I think that the ray tracer code is not very
>>representative of what is typically done in programs, at least in my
>>experience...
>
> Ray tracers are certainly not representative of many other types of program
> in terms of style and construction of code. However, my ray tracer is not
> totally dissimilar to much of the code that I write, and it uses a variety
> of different constructs (e.g. loops and recursion) and algorithms (e.g.
> folding over lists and searching trees) that I think makes it more
> representative of other types of programs than most benchmarks.
That's nonsense. (Or to be more constructive: What empirical evidence do
you have apart from your own personal experiences to substantiate your
claim, i.e., that your ray tracer is representative of more types of
programs than other benchmarks?)
On a related note,
http://www.zedshaw.com/blog/programming/programmer_stats.html is an
interesting point of view on statistics, and it seems to me that you
make quite a lot of the mistakes that are mentioned on that page.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Jon Harrop wrote:
> ······@earthlink.net wrote:
>
>>>A 7 year old learning maths can understand a lot of precedence and
>>>associativity.
>>
>>When concentrating on precedence, most people can get 7-8 levels right
>>consistently. However, "concentrating on precedence" is not a goal.
>>In fact, said concentration is actually an obstacle. To put it another
>>way, "understanding precedence" isn't the goal or a good thing, it is a
>>cost/overhead/a bad thing.
>
>
> You must concede that the vast majority of people choose to use
> associativity and precedence.
Germans and old time Romans choose reverse polish notation. :)
Cheers
--
Marco
From: Ulrich Hobelmann
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s9hr1Fn030eU3@individual.net>
Marco Antoniotti wrote:
>> You must concede that the vast majority of people choose to use
>> associativity and precedence.
>
> Germans and old time Romans choose reverse polish notation. :)
That I know did yet not.
Forth people some do though.
--
The road to hell is paved with good intentions.
Jon Harrop wrote:
> ······@earthlink.net wrote:
> >> A 7 year old learning maths can understand a lot of precedence and
> >> associativity.
> >
> > When concentrating on precedence, most people can get 7-8 levels right
> > consistently. However, "concentrating on precedence" is not a goal.
> > In fact, said concentration is actually an obstacle. To put it another
> > way, "understanding precedence" isn't the goal or a good thing, it is a
> > cost/overhead/a bad thing.
>
> You must concede that the vast majority of people choose to use
> associativity and precedence. We could all maximally bracket our
> mathematics to remove ambiguity but we don't because we know the
> expressions are much more comprehensible when written in conventional
> style.
>
> > One can see the same thing in natural language utterances. There are
> > many syntactically and semantically correct sentences that are too
> > convoluted to use if one's goal is communication/understanding with
> > people. With NL, we can't choose different rules. With computer
> > languages we can.
>
> Mathematics is a better analogy. The mathematics seen at degree level
> probably has as many associativities and precedences associated with it as
> the average programming language.
1. Mathematics notations are two-dimensional. Typesetting them properly
requires pretty sophisticated software.
2. The two-dimensions often reveal what the precedence is without the
use of parentheses. For instance, consider complicated nested
fractions. Many symbols which are not parentheses in fact form
two-dimensional enclosures which parenthesize. Even the innocuous
superscript is a kind of parenthesis, indicated by levels of vertical
indentation:
yz
X
The relationship (<operator> X (<operator> y z)) is not indicated by
implicit precedence, but by the use of graphical layout.
Even multiplication uses visual clues. No operator is written precisely
so that the two operands could be put close together, visually
indicating their tighter binding compared to addition and subtraction.
Given an expression like:
2y
x + z
------------
3x + y
-----
z - x
there is absolutely no need to invoke any precedence rule to parse it.
Spatial positioning and whitespace make the structure obvious.
Multiplicative terms are almost lexical tokens, rather than syntactic
phrases! Concessions are made to make that work. The numbers go on the
left, and all variables involved have to be one letter long. You
freaking switch to Greek when you run out of names in the Roman
alphabet, rather than the use of two or more letters to name a
quantity. Or diacritical marks: apostrophe's, superscript asterisks,
etc. 3xy cannot mean ``three times the quantity named xy'', and there
is no escape mechanism to even do that.
In economics or finance, there do occur quantities that have names more
than one letter long. The notation has to adjust, or else live with
ambiguities which are resolved lexically:
FVn = PV (1 + k)n
What is that?? F is not multiplied by V. Rather, The Future Value at
the end of term n, is the Present Value and so forth.
Not to be confused with PV = nRT, relating the temperature, pressure
and volume of a quanitity of gas. :)
3. Scientific and mathematics papers typically use a limited subset of
the notation. There is no complete set of rules which would guide you
in combining every known mathematical notation in the same expression
such that its syntactic structure would be universally agreed upon.
Moreover, many notations are re-invented for different meanings in
different contexts and probably with different rules for combination.
Kaz Kylheku wrote:
> 1. Mathematics notations are two-dimensional.
Sometimes.
> Typesetting them properly requires pretty sophisticated software.
Nothing like as sophisticated as a compiler.
> 2. The two-dimensions often reveal what the precedence is without the
> use of parentheses. For instance, consider complicated nested
> fractions. Many symbols which are not parentheses in fact form
> two-dimensional enclosures which parenthesize. Even the innocuous
> superscript is a kind of parenthesis, indicated by levels of vertical
> indentation:
>
> yz
> X
That example requires parentheses with conventional operator precedence:
x^(y z)
> The relationship (<operator> X (<operator> y z)) is not indicated by
> implicit precedence, but by the use of graphical layout.
>
> Even multiplication uses visual clues. No operator is written precisely
> so that the two operands could be put close together, visually
> indicating their tighter binding compared to addition and subtraction.
No. LaTeX spaces the infix symbols +, -, x and \div equally.
> Given an expression like:
>
> 2y
> x + z
> ------------
> 3x + y
> -----
> z - x
>
> there is absolutely no need to invoke any precedence rule to parse it.
> Spatial positioning and whitespace make the structure obvious.
For that particular expression, yes. If you swap the z and x^(2y), that is
no longer true.
> Multiplicative terms are almost lexical tokens, rather than syntactic
> phrases! Concessions are made to make that work. The numbers go on the
> left, and all variables involved have to be one letter long. You
> freaking switch to Greek when you run out of names in the Roman
> alphabet, rather than the use of two or more letters to name a
> quantity.
Or use sub/superscripts.
> Or diacritical marks: apostrophe's, superscript asterisks,
> etc. 3xy cannot mean ``three times the quantity named xy'', and there
> is no escape mechanism to even do that.
Use x_y.
> 3. Scientific and mathematics papers typically use a limited subset of
> the notation.
Yes.
> There is no complete set of rules which would guide you
> in combining every known mathematical notation in the same expression
> such that its syntactic structure would be universally agreed upon.
Due to types, nothing to do with precedence.
> Moreover, many notations are re-invented for different meanings in
> different contexts and probably with different rules for combination.
Some notations are, yes.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> Kaz Kylheku wrote:
> > Typesetting them properly requires pretty sophisticated software.
>
> Nothing like as sophisticated as a compiler.
>
WTF do you think TeX is? It's a tex-layout-language to DVI compiler,
for crying out loud!
> > 2. The two-dimensions often reveal what the precedence is without the
> > use of parentheses. For instance, consider complicated nested
> > fractions. Many symbols which are not parentheses in fact form
> > two-dimensional enclosures which parenthesize. Even the innocuous
> > superscript is a kind of parenthesis, indicated by levels of vertical
> > indentation:
> >
> > yz
> > X
>
> That example requires parentheses with conventional operator precedence:
>
> x^(y z)
"conventional" meaning "infix computer programming languages"? Because
people were writing it without parentheses for more than a hundred
years before FORTRAN. Kaz is obviously talking about printing on paper
or writing on chalkboard.
> > The relationship (<operator> X (<operator> y z)) is not indicated by
> > implicit precedence, but by the use of graphical layout.
> >
> > Even multiplication uses visual clues. No operator is written precisely
> > so that the two operands could be put close together, visually
> > indicating their tighter binding compared to addition and subtraction.
>
> No. LaTeX spaces the infix symbols +, -, x and \div equally.
You miss the point (how many times am I going to say that to you, I
wonder?)---usually the infix multiplication sign is *omitted* entirely,
in which case the spacing is much tighter.
> > Given an expression like:
> >
> > 2y
> > x + z
> > ------------
> > 3x + y
> > -----
> > z - x
> >
> > there is absolutely no need to invoke any precedence rule to parse it.
> > Spatial positioning and whitespace make the structure obvious.
>
> For that particular expression, yes. If you swap the z and x^(2y), that is
> no longer true.
What? Swapping the two terms in the numerator? Changing the structure?
What are you talking about?
> > etc. 3xy cannot mean ``three times the quantity named xy'', and there
> > is no escape mechanism to even do that.
>
> Use x_y.
That's not named "xy", that's "x" with a y "diacritic" attached. People
don't name their variables T_{ime} or T_{emperature}, they call them
T_0, T_1 for "Time number 0" and "Time number 1" or T_i "The ith Time".
> > There is no complete set of rules which would guide you
> > in combining every known mathematical notation in the same expression
> > such that its syntactic structure would be universally agreed upon.
>
> Due to types, nothing to do with precedence.
You must have some magical definition of "types" if you think it can
accomodate every impulse that a mathematician meeting chalkboard or
paper has had when he needed to write what he was thinking. Lots of
notation has nothing to do with conventional function application or
array indexing that computer languages use.
Consider old-fashioned general relativity tensor notation, with all
their summation conventions, use of greek vs. roman indices, subscripts
vs. superscripts, commas for partial derivatives, and conventional
notations for various special tensors, assumptions about metric
signature, and so on.
You think all of that is mandated by some theory of types? Or is it
just human convention, based on leaving out whatever they got tired of
writing when communicating within the group?
··············@hotmail.com wrote:
> Jon Harrop wrote:
>> Kaz Kylheku wrote:
>> > Typesetting them properly requires pretty sophisticated software.
>>
>> Nothing like as sophisticated as a compiler.
>
> WTF do you think TeX is? It's a tex-layout-language to DVI compiler,
> for crying out loud!
Typesetting does not require TeX.
> You miss the point (how many times am I going to say that to you, I
> wonder?)---usually the infix multiplication sign is *omitted* entirely,
> in which case the spacing is much tighter.
Yes, that is a special case. This rule does not apply to division, cross
product. The converse applies to many other constructs, e.g. integrals,
sums, products.
>> For that particular expression, yes. If you swap the z and x^(2y), that
>> is no longer true.
>
> What? Swapping the two terms in the numerator? Changing the structure?
> What are you talking about?
Swap the terms -> precedence is no longer implied by the typesetting. Kaz'
statement is true for his particular expressions but it is not true in
general. You still need operator precedences and associativies even in the
presence of typesetting.
>> Due to types, nothing to do with precedence.
>
> You must have some magical definition of "types" if you think it can
> accomodate every impulse that a mathematician meeting chalkboard or
> paper has had when he needed to write what he was thinking. Lots of
> notation has nothing to do with conventional function application or
> array indexing that computer languages use.
>
> Consider old-fashioned general relativity tensor notation, with all
> their summation conventions, use of greek vs. roman indices, subscripts
> vs. superscripts, commas for partial derivatives, and conventional
> notations for various special tensors, assumptions about metric
> signature, and so on.
>
> You think all of that is mandated by some theory of types? Or is it
> just human convention, based on leaving out whatever they got tired of
> writing when communicating within the group?
Mathematical operators are typed. You cannot apply numeric addition to the
result of a cross product. So there is never any need to define relative
precedences between them.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> ··············@hotmail.com wrote:
>>Jon Harrop wrote:
>>>Kaz Kylheku wrote:
>>>
>>>>Typesetting them properly requires pretty sophisticated software.
>>>
>>>Nothing like as sophisticated as a compiler.
>>
>>WTF do you think TeX is? It's a tex-layout-language to DVI compiler,
>>for crying out loud!
>
> Typesetting does not require TeX.
Of course not!
But I'd like to see you show an example of a typesetting program (sans
parser) featured enough to render simple mathematical equations that is
significantly simpler than a compiler (sans parser and optimizer).
-- MJF
M Jared Finder wrote:
> Jon Harrop wrote:
>> ··············@hotmail.com wrote:
>>>WTF do you think TeX is? It's a tex-layout-language to DVI compiler,
>>>for crying out loud!
>>
>> Typesetting does not require TeX.
>
> Of course not!
>
> But I'd like to see you show an example of a typesetting program (sans
> parser) featured enough to render simple mathematical equations that is
> significantly simpler than a compiler (sans parser and optimizer).
Presenta renders animated typeset mathematics via OpenGL:
http://www.ffconsultancy.com/products/presenta
The mathematical typesetting code is under 600LOC in OCaml.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> M Jared Finder wrote:
>>Jon Harrop wrote:
>>
>>But I'd like to see you show an example of a typesetting program (sans
>>parser) featured enough to render simple mathematical equations that is
>>significantly simpler than a compiler (sans parser and optimizer).
>
> Presenta renders animated typeset mathematics via OpenGL:
>
> http://www.ffconsultancy.com/products/presenta
>
> The mathematical typesetting code is under 600LOC in OCaml.
For comparison, here is a little Scheme in under 6000LOC of x86
assembly. I bet porting the code to a higher level language (say, C),
would reduce it to about 600 LOC. A real high level language could do
even better.
<http://www.stripedgazelle.org/joey/dream.html>
-- MJF
M Jared Finder wrote:
> For comparison, here is a little Scheme in under 6000LOC of x86
> assembly. I bet porting the code to a higher level language (say, C),
> would reduce it to about 600 LOC. A real high level language could do
> even better.
>
> <http://www.stripedgazelle.org/joey/dream.html>
Yes. I'm sure I could write an interpreter in well under 600LOC. However,
you did say "compiler". Given that 600LOC gets you most of TeX's
mathematical typesetting, we should be comparing with a complete compiler
for a non-trivial language. I may be wrong, but I doubt you can get one in
under 600LOC, even if it is compiling to bytecode.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> M Jared Finder wrote:
>
>>For comparison, here is a little Scheme in under 6000LOC of x86
>>assembly. I bet porting the code to a higher level language (say, C),
>>would reduce it to about 600 LOC. A real high level language could do
>>even better.
>>
>><http://www.stripedgazelle.org/joey/dream.html>
>
> Yes. I'm sure I could write an interpreter in well under 600LOC. However,
> you did say "compiler". Given that 600LOC gets you most of TeX's
> mathematical typesetting, we should be comparing with a complete compiler
> for a non-trivial language. I may be wrong, but I doubt you can get one in
> under 600LOC, even if it is compiling to bytecode.
I don't see how it's any more complicated. Instead of immediately
executing the code, just copy one of the assembly templates you have.
It won't be super fast, but it's simple to implement. Here's a few
templates. They all assume you leave the resulting value in the
register "a":
(+ a b)
<asm code to evaluate a>
push a
<asum code to evaluate b>
pop b
add a,b
(if a b c)
<asm code to evaluate a>
test a,CL_NIL
bnz else1
<asm code to evaluate b>
jp endif1
else1
<asm code to evaluate c>
endif1
If anything, this would be less code than an actual interpreter, because
all the "code" has been moved into the assembly templates.
-- MJF
M Jared Finder wrote:
> I don't see how it's any more complicated. Instead of immediately
> executing the code, just copy one of the assembly templates you have.
Yes. They also need some stitching together though.
> It won't be super fast, but it's simple to implement. Here's a few
> templates. They all assume you leave the resulting value in the
> register "a":
Yes, we must include the code generation stage of the compiler.
> (+ a b)
> <asm code to evaluate a>
> push a
> <asum code to evaluate b>
> pop b
> add a,b
>
> (if a b c)
> <asm code to evaluate a>
> test a,CL_NIL
> bnz else1
> <asm code to evaluate b>
> jp endif1
Generating and handling labels requires a little more code in the compiler.
> else1
> <asm code to evaluate c>
> endif1
The language will either need to support syntax for imperative loops or tail
call elimination.
Will the target language allocate memory or use a run-time environment with
a GC?
> If anything, this would be less code than an actual interpreter, because
> all the "code" has been moved into the assembly templates.
I assume we're counting the "assembly templates" as part of the compiler?
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Sure, Let's all these modifications and re-count the lines of code. But
we have to put the same type of requirements on Presenta, or it's not a
fair comparison.
If you make all these changes, the comparison will be fair.
Jon Harrop wrote:
> M Jared Finder wrote:
>
>>I don't see how it's any more complicated. Instead of immediately
>>executing the code, just copy one of the assembly templates you have.
>
> Yes. They also need some stitching together though.
You'll need to reipmelemnt all the OpenGL matrix functions you use other
than glGetDoublev, glLoadMatrix, and glMatrixMode. No fair using a
library to stitch together the different rendering primitives!
>>It won't be super fast, but it's simple to implement. Here's a few
>>templates. They all assume you leave the resulting value in the
>>register "a":
>
> Yes, we must include the code generation stage of the compiler.
You'll need to reimplement glutStrokeCharacter and count that in your
code. No fair using a library to generate line segments for you!
>>(+ a b)
>> <asm code to evaluate a>
>> push a
>> <asum code to evaluate b>
>> pop b
>> add a,b
>>
>>(if a b c)
>> <asm code to evaluate a>
>> test a,CL_NIL
>> bnz else1
>> <asm code to evaluate b>
>> jp endif1
>
> Generating and handling labels requires a little more code in the compiler.
>
>>else1
>> <asm code to evaluate c>
>>endif1
>
> The language will either need to support syntax for imperative loops or tail
> call elimination.
Your reimplementation must support dynamic levels of detail on the fonts.
>
> Will the target language allocate memory or use a run-time environment with
> a GC?
>
>>If anything, this would be less code than an actual interpreter, because
>>all the "code" has been moved into the assembly templates.
>
> I assume we're counting the "assembly templates" as part of the compiler?
You must include the source to the font files as part of the OCaml code.
After all these changes, is the count still 600 lines of code?
-- MJF
M Jared Finder wrote:
> Sure, Let's all these modifications and re-count the lines of code. But
> we have to put the same type of requirements on Presenta, or it's not a
> fair comparison.
>
> If you make all these changes, the comparison will be fair.
:-)
> Jon Harrop wrote:
>> M Jared Finder wrote:
>>
>>>I don't see how it's any more complicated. Instead of immediately
>>>executing the code, just copy one of the assembly templates you have.
>>
>> Yes. They also need some stitching together though.
>
> You'll need to reipmelemnt all the OpenGL matrix functions you use other
> than glGetDoublev, glLoadMatrix, and glMatrixMode. No fair using a
> library to stitch together the different rendering primitives!
The equivalent code (to compose typeset subexpressions) is already counted
in the 600 LOC.
>>>It won't be super fast, but it's simple to implement. Here's a few
>>>templates. They all assume you leave the resulting value in the
>>>register "a":
>>
>> Yes, we must include the code generation stage of the compiler.
>
> You'll need to reimplement glutStrokeCharacter and count that in your
> code. No fair using a library to generate line segments for you!
I make no use of glut's fonts. The fonts are TrueType, loaded via FreeType,
dynamically tesselated using GLU and rendered using OpenGL. That's only
10LOC extra in OCaml.
>>>(+ a b)
>>> <asm code to evaluate a>
>>> push a
>>> <asum code to evaluate b>
>>> pop b
>>> add a,b
>>>
>>>(if a b c)
>>> <asm code to evaluate a>
>>> test a,CL_NIL
>>> bnz else1
>>> <asm code to evaluate b>
>>> jp endif1
>>
>> Generating and handling labels requires a little more code in the
>> compiler.
>>
>>>else1
>>> <asm code to evaluate c>
>>>endif1
>>
>> The language will either need to support syntax for imperative loops or
>> tail call elimination.
>
> Your reimplementation must support dynamic levels of detail on the fonts.
That is another 10kLOC. :-)
>> Will the target language allocate memory or use a run-time environment
>> with a GC?
>>
>>>If anything, this would be less code than an actual interpreter, because
>>>all the "code" has been moved into the assembly templates.
>>
>> I assume we're counting the "assembly templates" as part of the compiler?
>
> You must include the source to the font files as part of the OCaml code.
That's another 5kLOC.
> After all these changes, is the count still 600 lines of code?
No. It is more like 15kLOC. However, you're now comparing a minimal compiler
with the most sophisticated vector graphics renderer in existence (AFAIK).
IMHO, that isn't fair. ;-)
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> M Jared Finder wrote:
>
>>Sure, Let's all these modifications and re-count the lines of code. But
>>we have to put the same type of requirements on Presenta, or it's not a
>>fair comparison.
>>
>>If you make all these changes, the comparison will be fair.
>
> :-)
<snip>
>>>>If anything, this would be less code than an actual interpreter, because
>>>>all the "code" has been moved into the assembly templates.
>>>
>>>I assume we're counting the "assembly templates" as part of the compiler?
>>
>>You must include the source to the font files as part of the OCaml code.
>
> That's another 5kLOC.
>
>>After all these changes, is the count still 600 lines of code?
>
> No. It is more like 15kLOC. However, you're now comparing a minimal compiler
> with the most sophisticated vector graphics renderer in existence (AFAIK).
> IMHO, that isn't fair. ;-)
I've snipped all but the killer point for me. The assembly templates
directly correspond to the font files. We should either include them
both or include neither. I suggest including neither, so we can analyze
the meat of both problems -- stitching together the text for math layout
and filling out assembly templates for the compiler.
Both of those seem like simple tasks that would take up about 500 lines
of code.
-- MJF
Jon Harrop wrote:
> Kaz Kylheku wrote:
> > 1. Mathematics notations are two-dimensional.
>
> Sometimes.
>
> > Typesetting them properly requires pretty sophisticated software.
>
> Nothing like as sophisticated as a compiler.
>
> > 2. The two-dimensions often reveal what the precedence is without the
> > use of parentheses. For instance, consider complicated nested
> > fractions. Many symbols which are not parentheses in fact form
> > two-dimensional enclosures which parenthesize. Even the innocuous
> > superscript is a kind of parenthesis, indicated by levels of vertical
> > indentation:
> >
> > yz
> > X
>
> That example requires parentheses with conventional operator precedence:
>
> x^(y z)
That's right. The point is that people, including those statistical
oddballs in the population who relish mathematics and even make it
their career, don't like operator precedence and they don't like
parentheses.
Operator precedence isn't the underlying principle in mathematics
notations; two-dimensional layout is.
We need two-dimensional layout in our programs too: that's what
indentation is.
w
yz
x uses vertical indentation to show structure similarly to the way
computer programs use horizontal indentation.
Mathematicians have invented these two-dimensional layouts in order to
avoid relying on having to mentally parse operator precedence.
Note that in Lisp we don't have precedence parentheses, only
function-calling parentheses; we thus avoid precedence in another way.
And of course we use two dimensional layout, so we can mentally filter
out the parentheses when we read code and data. Experienced Lisp coders
report that parentheses have become virtually "invisible" to them.
> > The relationship (<operator> X (<operator> y z)) is not indicated by
> > implicit precedence, but by the use of graphical layout.
> >
> > Even multiplication uses visual clues. No operator is written precisely
> > so that the two operands could be put close together, visually
> > indicating their tighter binding compared to addition and subtraction.
>
> No. LaTeX spaces the infix symbols +, -, x and \div equally.
The multiplicative x does not appear in conventional alegbraic terms.
It is implicit from juxtaposition. Division is properly represented as
a horizontal rule separating two formulas above each other.
You rarely see infix multiplication and division outside of arithmetic
books for children, "skill testing questions" in bubblegum wrapper
sweepstakes, or financial forms (e.g. tax declarations) that need to be
filled in correctly even by complete idiots.
> > Given an expression like:
> >
> > 2y
> > x + z
> > ------------
> > 3x + y
> > -----
> > z - x
> >
> > there is absolutely no need to invoke any precedence rule to parse it.
> > Spatial positioning and whitespace make the structure obvious.
>
> For that particular expression, yes. If you swap the z and x^(2y), that is
> no longer true.
No, then you get:
2y
z + x
-----------
etc. It's still obvious from the spacing and two-dimensional layout
what goes with what.
> > There is no complete set of rules which would guide you
> > in combining every known mathematical notation in the same expression
> > such that its syntactic structure would be universally agreed upon.
>
> Due to types, nothing to do with precedence.
It all has to do with precedence. You don't have a global rule set that
would integrate all of the syntax, and assign everything a precedence
relative to everything else.
Kaz Kylheku wrote:
> Operator precedence isn't the underlying principle in mathematics
> notations; two-dimensional layout is.
I was taught precedence before I came across 2D layout.
> We need two-dimensional layout in our programs too: that's what
> indentation is.
>
> w
> yz
> x uses vertical indentation to show structure similarly to the way
> computer programs use horizontal indentation.
IMHO, we need typesetting.
> Mathematicians have invented these two-dimensional layouts in order to
> avoid relying on having to mentally parse operator precedence.
Precedence is still widely taught and widely used.
>> No. LaTeX spaces the infix symbols +, -, x and \div equally.
>
> The multiplicative x does not appear in conventional alegbraic terms.
> It is implicit from juxtaposition.
Not if it means vector cross product, or if it is used between lines or...
> Division is properly represented as
> a horizontal rule separating two formulas above each other.
>
> You rarely see infix multiplication and division outside of arithmetic
> books for children, "skill testing questions" in bubblegum wrapper
> sweepstakes, or financial forms (e.g. tax declarations) that need to be
> filled in correctly even by complete idiots.
I used infix multiplication in my PhD thesis and many papers that I've read
have also used it. I just downloaded the first paper on the e-print archive
under condensed matter physics and it uses "x" for the first time on page
5. The first book I pull off my shelf, "Wavelets in Physics" by J.C. van
den Berg, uses it on p.175.
>> For that particular expression, yes. If you swap the z and x^(2y), that
>> is no longer true.
>
> No, then you get:
>
> 2y
> z + x
> -----------
>
> etc. It's still obvious from the spacing and two-dimensional layout
> what goes with what.
No, it isn't. You need to know that the superscript takes precedence over
the horizontal operator.
>> > There is no complete set of rules which would guide you
>> > in combining every known mathematical notation in the same expression
>> > such that its syntactic structure would be universally agreed upon.
>>
>> Due to types, nothing to do with precedence.
>
> It all has to do with precedence. You don't have a global rule set that
> would integrate all of the syntax, and assign everything a precedence
> relative to everything else.
You don't need to assign precedence relative to everything else because most
such expressions are not well typed, i.e. they are meaningless anyway.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> IMHO, we need typesetting.
If precedence worked, you wouldn't.
> Precedence is still widely taught and widely used.
And the source of no end of bugs.
> You don't need to assign precedence relative to everything else because most
> such expressions are not well typed, i.e. they are meaningless anyway.
"most" isn't good enough. If two infix operators can be used with an
operand
between them, you must have a precedence rule.
Note that Harrop still hasn't figured out that figuring out precedence
isn't
the goal.
-andy
Jon Harrop <······@jdh30.plus.com> writes:
...typical incompetent drivel snipped...
Just out of curiosity, do you do any real work? I mean other than
just post nonsense to usenet groups all day?
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Ulrich Hobelmann
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s78qtFn2pdaU2@individual.net>
Jon Harrop wrote:
> A 7 year old learning maths can understand a lot of precedence and
> associativity.
But over the years, not over the course of a few hours. And the
operators are all fixed, too, and mostly apply to arithmetic (and maybe
groups, rings, fields, but there the precedence rules are usually
defined from scratch).
--
Blessed are the young for they shall inherit the national debt.
Herbert Hoover
Jon Harrop <······@jdh30.plus.com> writes:
> A 7 year old learning maths can understand a lot of precedence and
> associativity.
Too bad C programmers are older than a 7-yo, and tend to forget the
weird precendence and associativity rules of C. When I was younger, I
knew the 'Pascal Report' by heart. Not anymore, but I'm still a
better programmer than then.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Cats meow out of angst
"Thumbs! If only we had thumbs!
We could break so much!"
From: Peter Seibel
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <m2y84icpp0.fsf@beagle.local>
Jon Harrop <······@jdh30.plus.com> writes:
> I think you got a little over-excited here, Ulrich. ;-)
>
> Ulrich Hobelmann wrote:
>> Eyal Lotem wrote:
>>> The main differences:
>>> DiffA. A macro does this implicitly, a function requires explicit
>>> wrapping. DiffB. A macro has access to the AST "inside" the given
>>> arguments.
>>
>> B is a huge thing: while functions operate on arguments (values), macros
>> operate on s-expressions, i.e. full ASTs as you say. That means that
>> the macro can process a whole sublanguage in a way as complex as you
>> want.
>
> Embedding the sublanguage within Lisp, yes.
>
>> There are simple control-flow macros, then there is stuff like
>> LOOP, then there are regular-expression compilers and parser generators,
>> stuff that other languages do via external (pre-)processors, because the
>> language itself is rather rigid.
>
> Regular expression compilers and parser generators can be
> implemented as ordinary functions, they do not need to be macros.
But that doesn't mean there still aren't advantages to implementing
them as macros, both in terms of runtime efficiency and ease of
expression. For instance here are a few productions from a grammar I
wrote for parsing Java:
(defprod floating-point-literal ()
(/
((+ digit)
(/
(#\. (* digit) (? exponent-part) (? float-type-suffix))
(exponent-part (? float-type-suffix))
float-type-suffix))
(#\. (+ digit) (? exponent-part) (? float-type-suffix))))
(defprod exponent-part () (exponent-indicator signed-integer))
(defchartype exponent-indicator '(member #\e #\E))
(defprod signed-integer () ((? sign) (+ digit)))
(defchartype sign '(member #\+ #\-))
(defchartype float-type-suffix '(member #\f #\F #\d #\D))
These form are expanded into Common Lisp code which is then compiled
down to native code by the Lisp compiler. The generated code is likely
more efficient than what I might write by hand (since I don't have to
worry about keeping the generated code comprehensible) and certainly
the code above more clearly expresses my intent than the equivalent
hand written recursive-descent functions.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
From: Peter Seibel
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <m2u0f6cobj.fsf@beagle.local>
Peter Seibel <·····@gigamonkeys.com> writes:
> Jon Harrop <······@jdh30.plus.com> writes:
>
>> I think you got a little over-excited here, Ulrich. ;-)
>>
>> Ulrich Hobelmann wrote:
>>> Eyal Lotem wrote:
>>>> The main differences:
>>>> DiffA. A macro does this implicitly, a function requires explicit
>>>> wrapping. DiffB. A macro has access to the AST "inside" the given
>>>> arguments.
>>>
>>> B is a huge thing: while functions operate on arguments (values), macros
>>> operate on s-expressions, i.e. full ASTs as you say. That means that
>>> the macro can process a whole sublanguage in a way as complex as you
>>> want.
>>
>> Embedding the sublanguage within Lisp, yes.
>>
>>> There are simple control-flow macros, then there is stuff like
>>> LOOP, then there are regular-expression compilers and parser generators,
>>> stuff that other languages do via external (pre-)processors, because the
>>> language itself is rather rigid.
>>
>> Regular expression compilers and parser generators can be
>> implemented as ordinary functions, they do not need to be macros.
>
> But that doesn't mean there still aren't advantages to implementing
> them as macros, both in terms of runtime efficiency and ease of
> expression. For instance here are a few productions from a grammar I
> wrote for parsing Java:
>
> (defprod floating-point-literal ()
> (/
> ((+ digit)
> (/
> (#\. (* digit) (? exponent-part) (? float-type-suffix))
> (exponent-part (? float-type-suffix))
> float-type-suffix))
> (#\. (+ digit) (? exponent-part) (? float-type-suffix))))
>
> (defprod exponent-part () (exponent-indicator signed-integer))
> (defchartype exponent-indicator '(member #\e #\E))
> (defprod signed-integer () ((? sign) (+ digit)))
> (defchartype sign '(member #\+ #\-))
> (defchartype float-type-suffix '(member #\f #\F #\d #\D))
>
> These form are expanded into Common Lisp code which is then compiled
> down to native code by the Lisp compiler. The generated code is likely
> more efficient than what I might write by hand (since I don't have to
> worry about keeping the generated code comprehensible) and certainly
> the code above more clearly expresses my intent than the equivalent
> hand written recursive-descent functions.
Oh, I should have mentioned that in the language of these macros '/'
means "or"--I would have used '|' except it would require escaping
unless I felt like mucking with the read table. '*', '+', and '?' mean
what you'd expect if you have any familiarity with regexps.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
Eyal Lotem wrote:
> Hello,
>
> I am well aware of the differences between a function and a macro, but I
Are you? It doesn't seem that way.
> want to ask a different question. What extra expressive power macros give
> us on top of ordinary functions, in a language that fully supports
> closures?
The expressive power to invent any syntax whatsoever and give it any
meaning whatsoever, and then instantly turn around and use it in the
same program.
> Lets for the sake of simplicity drop the issue of performance, and focus on
> expressiveness only.
>
> The main similarity I see between macros and functions in a
> closure-supportive environment:
> A macro takes its arguments unevaluated
> A function can take its argument unevaluated if the arguments are explicitly
> wrapped within a closure.
The main similarity is that macros /are/ functions.
The biggest difference between macros and ordinary functions is /what/
they operate on and, more importantly, /where/ and /when/ they do it.
Macros are functions that are installed in the Lisp compiler, to
transform the source code of the program however the programmer sees
fit. Complex macros can be broken into funtions, and those auxiliary
functions look like ordinary Lisp functions, often written using DEFUN
or LABELS or whatever. The top-level ones written with DEFUN in fact
have to be wrapped in an EVAL-WHEN form which tells the compiler to
install those functions into itself so they are available to the macro!
A macro does not take ``unevaluated'' arguments. Macros often take
arguments that aren't even supposed to be evaluated normally, because
they do not represent ordinary Lisp expressions. A macro's job isn't to
delay or rearrange evaluation, although macros can implement that.
Macros are potentially called in a different environment from the one
where the software ultimately runs. For instance, macros may be
expanded on a software company's build machine as part of creating a
system image, which is then deployed to some embedded systems at a
customer site. Those ``ordinary'' functions in the software are are
executed on those systems.
It's easy to get all mixed up when you are doing all your debugging
work with everything happening in one Lisp image.
All languages have macros, just not all languages extend their
repertoire of macros from definitions in the programs that they
process. Even those functional languages whose users claim that macros
are unnecessary still have macros: hard-coded syntactic recognizers
built into their compilers, which make sense out of the jumble of
symbols to give those programs meaning.
Eyal Lotem wrote:
> I personally think that macros have serious downsides, and as such, the
> extra power they allow must justify itself.
There are many trade-offs involved. If you ditch macros then you might as
well ditch s-exprs. Instead, you would have to define your own AST type in
order to manipulate ASTs, e.g. as a variant type in OCaml:
type expr =
| EInt of int
| EFloat of float
| EVar of string
| EIf of expr * expr * expr
...
If you want to evaluate an "expr" then you must write your own "eval"
function:
let rec eval vars = function
| EInt n -> VInt n
| EFloat x -> VFloat x
| EVar var -> List.assoc var vars
| EIf(p, t, f) -> eval vars (if unBool(eval vars p) then t else f)
...
Firstly, this requires you to write more code. Secondly, unless you're going
to write a full-blown compiler, a host Lisp compiler is likely to execute
s-exprs more efficiently.
On the other hand, you can design your language from the ground up to
support features that conflict with having macros in the language.
You can hoist macros out of the language and have them as a preprocessor
only. OCaml does this in the form of camlp4 macros. In the context of
OCaml, macros are regarded as a last resort to address deficiencies in the
language's syntax and are rarely used as a consequence.
SML doesn't have a macro preprocessor and just sticks with compiler
compilers like yacc.
Without run-time code generation, the language designer is free to make
different design decisions. For example, you can add static typing, to
validate code and safely remove unnecessary run-time type checks. Look at
some performance comparisons between safe ML and safe Lisp code, for
example.
Static typing paves the way for a variety of other static checks, such as
redundancy and exhaustiveness checks in pattern matches. This makes pattern
matching more expressive and facilitates more optimisations, resulting in
yet faster code.
Users can retrofit this kind of functionality onto Lisp using macros. Or
they can download an existing implementation from the web, such as Qi:
http://www.cliki.net/Qi
The advantage is that Qi allows you to keep the "power" of Lisp.
The disadvantages are that:
1. you are never likely to regain the lost performance
2. the vast majority of users have little to no knowledge of static typing
and type systems so they are not qualified to write such a system or even
pick one off the shelf.
3. all such add-on packages are inherently prone to being ad-hoc,
informally-specified, bug-ridden, slow implementations of functionality
that had already been implemented properly in compilers for more modern
languages.
The fact that many people use Lisp/Scheme and many people use
SML/OCaml/Haskell implies that there is no clear reason to prefer one
approach to the other. I believe Lisp and Scheme are more commonly taught
on undergrad courses in the US whereas MLs are taught in Europe.
To answer the question, you really need to ask someone well versed in both
types of language.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
> You can hoist macros out of the language and have them as a preprocessor
> only. OCaml does this in the form of camlp4 macros. In the context of
> OCaml, macros are regarded as a last resort to address deficiencies in the
> language's syntax and are rarely used as a consequence.
That's "a consequence" if there are no/few deficiencies to address. A
more likely explanation is that the pain of addressing them exceeds the
pain of tolerating them.
> Without run-time code generation, the language designer is free to make
> different design decisions. For example, you can add static typing, to
> validate code and safely remove unnecessary run-time type checks.
Interestingly enough, one can have both run-time code generation and
static
checks, such as typing.
However, lispers are clearly not worthy. They should not be exposed to
the perfection that is the ML family. Leave them to their sins.
······@earthlink.net wrote:
>> You can hoist macros out of the language and have them as a preprocessor
>> only. OCaml does this in the form of camlp4 macros. In the context of
>> OCaml, macros are regarded as a last resort to address deficiencies in
>> the language's syntax and are rarely used as a consequence.
>
> That's "a consequence" if there are no/few deficiencies to address. A
> more likely explanation is that the pain of addressing them exceeds the
> pain of tolerating them.
Do you think it is harder to write OCaml macros than Lisp macros?
>> Without run-time code generation, the language designer is free to make
>> different design decisions. For example, you can add static typing, to
>> validate code and safely remove unnecessary run-time type checks.
>
> Interestingly enough, one can have both run-time code generation and
> static checks, such as typing.
In a restricted form, like MetaOCaml.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop wrote:
> ······@earthlink.net wrote:
> > That's "a consequence" if there are no/few deficiencies to address. A
> > more likely explanation is that the pain of addressing them exceeds the
> > pain of tolerating them.
>
> Do you think it is harder to write OCaml macros than Lisp macros?
Folks write macros when the cost of doing so is less than the benefits
of doing so.
Since OCaml has perfect syntax, there's no benefit, so the fact that
OCaml macros are the easiest to write goes unused.
Lisp does not have perfect syntax but its macros are fairly easy to
write, so lispers often write macros.
> > Interestingly enough, one can have both run-time code generation and
> > static checks, such as typing.
>
> In a restricted form, like MetaOCaml.
Nope - run-time code generation need not have restrictions beyond those
imposed on programs with "in advance" code generation. (Yes, one may
not choose to use as much time for run-time code generation as one is
willing to use for in-advance code generation, but that's a choice.)
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s4ft1FmbbvnU1@individual.net>
Jon Harrop wrote:
> To answer the question, you really need to ask someone well versed in both
> types of language.
You should read the OP's question again. You don't need to refer to
other languages to explain what kind of expressive power macros provide.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Pascal Costanza wrote:
> Jon Harrop wrote:
>> To answer the question, you really need to ask someone well versed in
>> both types of language.
>
> You should read the OP's question again. You don't need to refer to
> other languages to explain what kind of expressive power macros provide.
He asked: "Were there no reasonable alternatives?".
Many people consider using other languages, that already support the syntax
you want, to be a "reasonable alternative" to retrofitting an ad-hoc syntax
onto Lisp yourself.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s4iqgFmhhmiU1@individual.net>
Jon Harrop wrote:
> Pascal Costanza wrote:
>
>>Jon Harrop wrote:
>>
>>>To answer the question, you really need to ask someone well versed in
>>>both types of language.
>>
>>You should read the OP's question again. You don't need to refer to
>>other languages to explain what kind of expressive power macros provide.
>
> He asked: "Were there no reasonable alternatives?".
>
> Many people consider using other languages, that already support the syntax
> you want, to be a "reasonable alternative" to retrofitting an ad-hoc syntax
> onto Lisp yourself.
Macros are good for implementing your own (domain-specific) languages.
The constructs in those languages are not supported yet by definition.
Syntactic abstractions give you a different kind of expressive power
than functional (or object-oriented, etc.) abstractions.
The alternatives are whether you think it's a good idea to map
everything onto a restricted set of syntactic constructs, i.e., whether
you think that a certain set of syntactic constructs is sufficient for
everything, or else whether you think it's a good idea to be able to
express solutions that are closer to your problem domain, i.e., whether
you think that the set of useful syntactic construct is open-ended.
This doesn't have anything to do with performance considerations.
As a sidenote, it should be stressed (again) that toy benchmarks, like
ray tracers, don't really tell you anything about the efficiency that
you can achieve with a programming language in large, real-world
examples because the various influencing factors are hard to pinpoint
and hard to control. There are, however, numerous examples in which
Common Lisp programs beat the performance of equivalent programs
implemented in other, supposedly more efficient languages. Jans Aasman
(of Franz Inc.) has given a nice overview of such examples in a
presentation he has given at last week's OOPSLA.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Pascal Costanza wrote:
> Jon Harrop wrote:
>> Many people consider using other languages, that already support the
>> syntax you want, to be a "reasonable alternative" to retrofitting an
>> ad-hoc syntax onto Lisp yourself.
>
> Macros are good for implementing your own (domain-specific) languages.
> The constructs in those languages are not supported yet by definition.
Yes. However, Lisp macros are also used to implement syntax already found in
a wide variety of other languages so, in those cases, you are not
implementing a DSL.
> Syntactic abstractions give you a different kind of expressive power
> than functional (or object-oriented, etc.) abstractions.
Yes.
> The alternatives are whether you think it's a good idea to map
> everything onto a restricted set of syntactic constructs, i.e., whether
> you think that a certain set of syntactic constructs is sufficient for
> everything, or else whether you think it's a good idea to be able to
> express solutions that are closer to your problem domain, i.e., whether
> you think that the set of useful syntactic construct is open-ended.
No. A language could provide both a large set of built-in constructs and be
extensible.
Here is a related question: If a new standard for Common Lisp were proposed
that included a good pattern matching implementation, would Lisp users
welcome it?
> This doesn't have anything to do with performance considerations.
No. Look at the optimisations facilitated by having pattern matching with
exhaustiveness and redundancy testing built into ML, for example.
> As a sidenote, it should be stressed (again) that toy benchmarks, like
> ray tracers, don't really tell you anything about the efficiency that
> you can achieve with a programming language in large, real-world
> examples because the various influencing factors are hard to pinpoint
> and hard to control.
In the case of my ray tracer, Juho Snellman showed that SBCL-compiled Lisp's
performance is bottlenecked almost entirely by garbage collection. By
addressing that, the Lisp code can easily be made faster than the Java.
> There are, however, numerous examples in which
> Common Lisp programs beat the performance of equivalent programs
> implemented in other, supposedly more efficient languages. Jans Aasman
> (of Franz Inc.) has given a nice overview of such examples in a
> presentation he has given at last week's OOPSLA.
If this is the appropriate page:
http://www.oopsla.org/2005/ShowEvent.do?id=692
then I agree that Lisp can beat Java on performance. I am not sure about
code size. However, Java and C# aren't really relevant as both are
notoriously slow and neither provide macros or first class functions.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s4q88Fm4ef8U1@individual.net>
Jon Harrop wrote:
> Pascal Costanza wrote:
>
>>Jon Harrop wrote:
>>
>>>Many people consider using other languages, that already support the
>>>syntax you want, to be a "reasonable alternative" to retrofitting an
>>>ad-hoc syntax onto Lisp yourself.
>>
>>Macros are good for implementing your own (domain-specific) languages.
>>The constructs in those languages are not supported yet by definition.
>
> Yes. However, Lisp macros are also used to implement syntax already found in
> a wide variety of other languages so, in those cases, you are not
> implementing a DSL.
Correct.
>>Syntactic abstractions give you a different kind of expressive power
>>than functional (or object-oriented, etc.) abstractions.
>
> Yes.
>
>>The alternatives are whether you think it's a good idea to map
>>everything onto a restricted set of syntactic constructs, i.e., whether
>>you think that a certain set of syntactic constructs is sufficient for
>>everything, or else whether you think it's a good idea to be able to
>>express solutions that are closer to your problem domain, i.e., whether
>>you think that the set of useful syntactic construct is open-ended.
>
> No. A language could provide both a large set of built-in constructs and be
> extensible.
Sure.
> Here is a related question: If a new standard for Common Lisp were proposed
> that included a good pattern matching implementation, would Lisp users
> welcome it?
Why not?
>>This doesn't have anything to do with performance considerations.
>
> No. Look at the optimisations facilitated by having pattern matching with
> exhaustiveness and redundancy testing built into ML, for example.
What does this have to do with extensibility provided by macros?
>>As a sidenote, it should be stressed (again) that toy benchmarks, like
>>ray tracers, don't really tell you anything about the efficiency that
>>you can achieve with a programming language in large, real-world
>>examples because the various influencing factors are hard to pinpoint
>>and hard to control.
>
> In the case of my ray tracer, Juho Snellman showed that SBCL-compiled Lisp's
> performance is bottlenecked almost entirely by garbage collection. By
> addressing that, the Lisp code can easily be made faster than the Java.
...and still wouldn't tell you a lot about how the language
implementation scales for large programs.
>>There are, however, numerous examples in which
>>Common Lisp programs beat the performance of equivalent programs
>>implemented in other, supposedly more efficient languages. Jans Aasman
>>(of Franz Inc.) has given a nice overview of such examples in a
>>presentation he has given at last week's OOPSLA.
>
> If this is the appropriate page:
>
> http://www.oopsla.org/2005/ShowEvent.do?id=692
>
> then I agree that Lisp can beat Java on performance. I am not sure about
> code size. However, Java and C# aren't really relevant as both are
> notoriously slow and neither provide macros or first class functions.
Java and C# aren't notoriously slow. At least in the case of Java, the
lack of performance comes mostly from its libraries. You should read
that page again, there are also other languages mentioned there.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Edi Weitz
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <uk6g2lrwv.fsf@agharta.de>
On Mon, 24 Oct 2005 19:09:35 +0200, Pascal Costanza <··@p-cos.net> wrote:
> There are, however, numerous examples in which Common Lisp programs
> beat the performance of equivalent programs implemented in other,
> supposedly more efficient languages. Jans Aasman (of Franz Inc.) has
> given a nice overview of such examples in a presentation he has
> given at last week's OOPSLA.
Is the talk available somewhere?
Thanks,
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
From: Pascal Costanza
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <3s4qa6Fm4ef8U2@individual.net>
Edi Weitz wrote:
> On Mon, 24 Oct 2005 19:09:35 +0200, Pascal Costanza <··@p-cos.net> wrote:
>
>>There are, however, numerous examples in which Common Lisp programs
>>beat the performance of equivalent programs implemented in other,
>>supposedly more efficient languages. Jans Aasman (of Franz Inc.) has
>>given a nice overview of such examples in a presentation he has
>>given at last week's OOPSLA.
>
> Is the talk available somewhere?
You should ask Jans. I don't think that the website of the Dynamic
Languages Symposium is going to host the slides of the talks (but I
could be wrong).
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Joe Marshall
Subject: Re: The power difference between a macro and a function
Date:
Message-ID: <br1eeakc.fsf@alum.mit.edu>
Eyal Lotem <··········@gmail.com> writes:
> Hello,
>
> I am well aware of the differences between a function and a macro, but I
> want to ask a different question. What extra expressive power macros give
> us on top of ordinary functions, in a language that fully supports
> closures?
>
> Lets for the sake of simplicity drop the issue of performance, and focus on
> expressiveness only.
>
> The main similarity I see between macros and functions in a
> closure-supportive environment:
> A macro takes its arguments unevaluated
> A function can take its argument unevaluated if the arguments are explicitly
> wrapped within a closure.
This comes up every term.
(defmacro counterexample (x)
`(let ((value ,x))
(format nil "~s evals to ~s" ',x value)))
(counterexample (+ 3 4))
=> "(+ 3 4) evaluates to 7"