Some of you may have noticed my last post regarding how macro expansion
is performed (i.e. the actual mechanics of it). You may have even
guessed that the reason I asked is that I'm writing my own little lisp
interpreter and am trying to implement macros. =)
So I've got a question. I've noticed that CL macros are not
first-class...i.e.:
CL> (defmacro foo ...)
FOO
CL> foo
#error, foo not bound yada yada
I've been designing my interpreter around the notion that I would be
able to create first-class macros, i.e. (Scheme notation, sorry!):
(define foo
(macro (x y)
(list 'cons x y)))
(foo (+ 1 2) (+ 3 4)) => (3 . 7)
or even
((macro (x y) (list 'cons x y)) (+ 1 2) (+ 3 4)) => (3 . 7)
[ I guess a better example would be (to show non-eval of args):
((macro (x) (list (cadr x) (caddr x))) (+ 1 2)) => (1 2) ]
Are there any huge pitfalls to this? I guess what I'm saying is, I'm
surprised that CL doesn't have first-class macros, and since it
doesn't, maybe there's a good reason and I'm going to waste a lot of
time and energy going down a path that's either extremely difficult or
maybe even impossible.
Comments?
(I hope that the multiple namespaces of CL aren't biting me here and
that foo is actually defined as a macro somewhere...again, CL ignorance
apologies...)
Thanks,
Mike
P.S. I just had a conversation with a Lisp-buddy of mine, and he said
he thinks they're not first-class because the expansion really occurs
at compile-time, and thus isn't really "required" in an environment.
Since my interpreter is really basic, and nothing is ever compiled, I
don't really have a distinction between compile and execute. Does this
change how I should think about first-class macros?
"Mike" <······@gmail.com> writes:
> I've been designing my interpreter around the notion that I would be
> able to create first-class macros, i.e. (Scheme notation, sorry!):
>
> (define foo
> (macro (x y)
> (list 'cons x y)))
>
> (foo (+ 1 2) (+ 3 4)) => (3 . 7)
>
> or even
>
> ((macro (x y) (list 'cons x y)) (+ 1 2) (+ 3 4)) => (3 . 7)
>
> [ I guess a better example would be (to show non-eval of args):
> ((macro (x) (list (cadr x) (caddr x))) (+ 1 2)) => (1 2) ]
>
> Are there any huge pitfalls to this?
You can't compile.
"Mike" <······@gmail.com> writes:
> Some of you may have noticed my last post regarding how macro expansion
> is performed (i.e. the actual mechanics of it). You may have even
> guessed that the reason I asked is that I'm writing my own little lisp
> interpreter and am trying to implement macros. =)
>
> So I've got a question. I've noticed that CL macros are not
> first-class...i.e.:
>
> CL> (defmacro foo ...)
> FOO
> CL> foo
> #error, foo not bound yada yada
No, macros are first class, but you're mixing another
feature. Common-Lisp is a "lisp-2", ie. symbols have several slots,
one for values and one for functions amongst others.
If you want to get the macro object denoted by the symbol foo, you
should use: (MACRO-FUNCTION 'foo)
> (I hope that the multiple namespaces of CL aren't biting me here and
> that foo is actually defined as a macro somewhere...again, CL ignorance
> apologies...)
Yes, they did bite you.
> P.S. I just had a conversation with a Lisp-buddy of mine, and he said
> he thinks they're not first-class because the expansion really occurs
> at compile-time, and thus isn't really "required" in an environment.
> Since my interpreter is really basic, and nothing is ever compiled, I
> don't really have a distinction between compile and execute. Does this
> change how I should think about first-class macros?
That does not prevent you to use the macro function at run time:
[1]> (defmacro foo (x) `(print 'x))
FOO
[2]> (macro-function 'foo)
#<CLOSURE FOO (SYSTEM::<MACRO-FORM> SYSTEM::<ENV-ARG>)
(DECLARE (CONS SYSTEM::<MACRO-FORM>)) (DECLARE (IGNORE SYSTEM::<ENV-ARG>))
(IF (/= (EXT:LIST-LENGTH-DOTTED SYSTEM::<MACRO-FORM>) 2)
(SYSTEM::MACRO-CALL-ERROR SYSTEM::<MACRO-FORM>)
(LET* ((X (CADR SYSTEM::<MACRO-FORM>))) (BLOCK FOO `(PRINT 'X))))>
[3]> (funcall (macro-function 'foo) '(foo x) nil)
(PRINT 'X)
[4]>
--
__Pascal Bourguignon__ http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
"Mike" <······@gmail.com> writes:
> Some of you may have noticed my last post regarding how macro expansion
> is performed (i.e. the actual mechanics of it). You may have even
> guessed that the reason I asked is that I'm writing my own little lisp
> interpreter and am trying to implement macros. =)
>
> So I've got a question. I've noticed that CL macros are not
> first-class...i.e.:
>
> CL> (defmacro foo ...)
> FOO
> CL> foo
> #error, foo not bound yada yada
>
> I've been designing my interpreter around the notion that I would be
> able to create first-class macros, i.e. (Scheme notation, sorry!):
>
> (define foo
> (macro (x y)
> (list 'cons x y)))
>
> (foo (+ 1 2) (+ 3 4)) => (3 . 7)
>
> or even
>
> ((macro (x y) (list 'cons x y)) (+ 1 2) (+ 3 4)) => (3 . 7)
This is a bad example, but there do exist occasional good ones.
Still, and importantly, in this case, the information is lexically
apparent to the compiler. It's in the opaque case of (FOO ...)
that you must know the definition of FOO at compile (or syntax) time.
> [ I guess a better example would be (to show non-eval of args):
> ((macro (x) (list (cadr x) (caddr x))) (+ 1 2)) => (1 2) ]
Not necessarily. In fact, I assumed you were evaluating the args, which
is actually sometimes helpful. Consider:
(defvar *digits-table*
((macro (array)
(dotimes (i 10)
(setf (aref array (digit-char i)) t))
array)
(make-array 256. :element-type nil)))
Then again, you could just as well write:
(defvar *digits-table*
(macrolet ((do-it ()
(let ((array (make-array 256. :element-type nil)))
(dotimes (i 10)
(setf (aref array (digit-char i)) t))
array)))
(do-it)))
It hardly ever comes up, frankly.
> Are there any huge pitfalls to this?
As others have cited: compilation.
> I guess what I'm saying is, I'm
> surprised that CL doesn't have first-class macros, and since it
> doesn't, maybe there's a good reason and I'm going to waste a lot of
> time and energy going down a path that's either extremely difficult or
> maybe even impossible.
It would encourage people not to separate compile-time from runtime programming.
> Comments?
>
> (I hope that the multiple namespaces of CL aren't biting me here and
> that foo is actually defined as a macro somewhere...again, CL ignorance
> apologies...)
No, not really. You can only have either a macro definition or a function
definition.
In MACLISP [pre-CL and pre-Macintosh--it's named after MIT's Project
MAC, the original name of their Lab for Comp Sci], you used to be able
to have both because definitions lived on the plist, and worse, it
mattered the order of what indicators were on the PLIST as to how it
was all resolved. Big mess.
I wrote a dialect of Lisp called ULISP that was used briefly [one
semester] at MIT for teaching before Sussman ran screaming from it
(because it didn't have tail call elimination) and decided to use
Scheme. ULISP had first-class macros, but no compiler.
> Thanks,
>
> Mike
>
> P.S. I just had a conversation with a Lisp-buddy of mine, and he said
> he thinks they're not first-class because the expansion really occurs
> at compile-time, and thus isn't really "required" in an environment.
> Since my interpreter is really basic, and nothing is ever compiled, I
> don't really have a distinction between compile and execute. Does this
> change how I should think about first-class macros?
Your friend is both right and wrong.
Yes, they are about compile time.
No, compile time does not cease at runtime. Consider:
(defun foo (x) (eval x))
If you compile this program and run it, it still has to know how to
do (foo '(let ((x 3)) (+ x x))) even if LET is implemented as a macro.
Compilation is not an all-or-nothing process in Lisp. A compiler is
given a set of data to compile either in-core (using COMPILE) or
externalized to a file (using COMPILE-FILE). The resulting compiled
program may have pointers to data or may create new data which itself
needs to be compiled. Logically, such compilations are not related.
Also, incidentally, even without a compiler, you still have
compilation-related issues. For example, MACLISP tended to be used a lot
interpreted, and still did things with macros. (Its language definition
was different, and allowed you to modify the source list structure from
a macro, something you aren't allowed to do in CL.)
People would write "displacing macros" like:
(DEFUN FIRST MACRO (WHOLE-FORM)
(RPLACA WHOLE-FORM 'CAR))
which in CL parlance would be vaguely like:
(defun first macro (&whole whole-form arg)
(setf (car whole-form) 'car)
whole-form)
except for the fact that such a side-effect is illegal in CL.
A commonly used macro in MACLISP was called DISPLACE, which did, if I'm
not making a mistake:
(DEFUN DISPLACE (FORM NEW-FORM)
(RPLACA FORM (CAR NEW-FORM))
(RPLACD FORM (CDR NEW-FORM))
FORM)
This allowed you to do things like:
(DEFUN LET1 MACRO (FORM)
(DISPLACE FORM (CONS 'LET (LIST (LIST (CAR FORM) (CADR FORM))) (CDDR FORM))))
The reason for this was so that after the first execution, you didn't
have to macroexpand any more.
Although it was confusing to note that if you did
(DEFUN FOO (X) (FIRST X))
(GET 'FOO 'EXPR) => (LAMBDA (X) (FIRST X))
until you first used it and then later contained
(GET 'FOO 'EXPR) => (LAMBDA (X) (CAR X))
With more sophistication, the language evolved to replace such forms instead
of with the form itself, like (CAR X) but with something more like
(DISPLACED-TO (CAR X) (FIRST X))
which still had to be macroexpanded, but which was trivially macroexpanded
by just
(DEFMACRO DISPLACED-TO (WHOLE-FORM) (CADR FORM))
but which was pretty-printable by showing only the caddr of the displaced-to
form.
And later still in time, it was also possible, with some bookkeeping,
to void the cache of DISPLACED-TO forms so that a macro redefinition
could still cause a new expansion.
CL doesn't allow this IN USER CODE, but does allow you to think about
these issues in the system code. Indeed, one might say that the
reason CL doesn't allow you to modify the definition backbone of the
user code is so that you CAN modify the internal structure of the code
you have saved for execution. (Don't modify the user's literal
structure as given to the compiler, of course, since the user may have
retained a copy he expects not to change, but maybe a copy...)
So you have plenty of implementation issues to think about related to
macros, even when compilation isn't occurring.
BUT the key thing to understand is that this syntax processing must
occur _sometime_ and it's best if you make your user aware of when
that will be and try to make as consistent a world as you can.
If you defer the expansions always until runtime, repeating them over
and over, you're technically conforming. But users may hate the amount
of runtime overhead that costs.
The reason that compilation is usually done is that you can think of
runtime as happening over and over, sort of like a loop. Compilation
you can think of as a kind of "loop invariant", where steps that don't
need repeat application are lifted out of the loop. When you blur these,
you do the same as failing to lift loop invariants from your code, and
you condemn the inner loops to pretty pessimal performance.
Also, the power of the evaluator and compiler are RARELY needed at
runtime. I won't say never. And it's a powerful feature of Lisp that
you can have them at runtime if you want. But you should not ask for
them lightly. It will GREATLY increase the application footprint to
include a compiler, and may somewhat even to include an interpreter
[though some implementations have only a compiler or interpreter, and
it may be either one]. It's not overhead you want to invite. And
there may be intellectual property issues. Some Lisp licenses charge
you lots more if you want the compiler in the runtime than if you
don't, I think in part because they know you have a full turing
machine (including compiler) available in your image, and they fear
you might be making a fully powerful language that could compete with
them.
Holy crap what a great response...thanks Kent. More insight than I
could have asked for.
I appreciate it!
P.S. The only reason I'm writing a "lisp" myself is because I can't
get one where I work. I was hoping originally to work with SISC, being
that I was primarily a Scheme advocate. Now that I'm learning much
more about lisp implementation issues (even at my basic level of
understanding), I'm starting to lean toward liking CL more. Mind you,
I've barely wrote a lick of real code in either. Anyway, what I was
getting at is that *I'm* the only user right now, so if runtime
performance is abysmal (as I'm sure it will be, given that my
implementation is written on top of a super-slow abstract machine which
is itself written in Perl), I'm the only one who will complain.
Hopefully I can get a real lisp where I work soon...but in the
meantime, I'm learning a heckuva lot about the internals...
"Mike" <······@gmail.com> writes:
> Holy crap what a great response...thanks Kent. More insight than I
> could have asked for.
>
> I appreciate it!
>
> P.S. The only reason I'm writing a "lisp" myself is because I can't
> get one where I work. I was hoping originally to work with SISC, being
> that I was primarily a Scheme advocate. Now that I'm learning much
> more about lisp implementation issues (even at my basic level of
> understanding), I'm starting to lean toward liking CL more. Mind you,
> I've barely wrote a lick of real code in either. Anyway, what I was
> getting at is that *I'm* the only user right now, so if runtime
> performance is abysmal (as I'm sure it will be, given that my
> implementation is written on top of a super-slow abstract machine which
> is itself written in Perl), I'm the only one who will complain.
> Hopefully I can get a real lisp where I work soon...but in the
> meantime, I'm learning a heckuva lot about the internals...
I wouldn't bee too cavalier about poor performance of the kind that
Kent was referring to. Depending on the types of macros you write --
and if you make the most of Lisp, you'll be writing computationally
intensive macros -- this can be an algorithmic slowdown. A slow
virtual machine isn't usually a problem in these days of fast
machines. When you only expand a macro-call once, it doesn't matter
much how slow that is, or even usually whether it's O(n^2) where it
could be O(n). If you expand the macro-call every time it gets
evaluated, you can be talking about another universe of performance
problems from that of slow VMs.
Since you seem to be building a Lisp-1, you might want to look at
syntactic closures, which are almost as easy to use as traditional
Lisp-2 macros, but avoid the "hygene" issues that can plague Lisp-1s.
Or, make it a Lisp-2. Remember there's no rule that says you can't
have arbirary compound forms in the first position of a form in a
Lisp-2. Ie, ((complement #'foop) x).
--
"Mike" <······@gmail.com> wrote on 4 Mar 2005 22:09:
> P.S. The only reason I'm writing a "lisp" myself is because I can't
> get one where I work.
> I'm starting to lean toward liking CL more.
Can you explain this more fully? Writing a Lisp yourself is a huge non-trivial
task. Especially something inspired by Common Lisp, and writing from scratch.
What prevents you from getting one where you work? Why are you allowed to
use your own Lisp that you write yourself, but not anybody else's? That
sounds like quite an odd restriction.
> Hopefully I can get a real lisp where I work soon...
I'm really curious what situation you could possibly be in where it makes
sense to create a Lisp implementation from scratch rather than simply
installing one of the existing high-quality implementations.
-- Don
_______________________________________________________________________________
Don Geddis http://don.geddis.org/ ···@geddis.org
So far as I can remember, there is not one word in the Gospels in praise of
intelligence. -- Bertrand Russell
On Sat, 05 Mar 2005 18:00:49 -0800, Don Geddis wrote:
> "Mike" <······@gmail.com> wrote on 4 Mar 2005 22:09:
>> P.S. The only reason I'm writing a "lisp" myself is because I can't
>> get one where I work.
>> I'm starting to lean toward liking CL more.
>
> Can you explain this more fully? Writing a Lisp yourself is a huge non-trivial
> task.
Depends what your definition of "a Lisp" is. People have implemented
reasonably large subsets of Scheme in a single day, for example.
> Especially something inspired by Common Lisp
This I would certainly agree with though, as well as the point that "I
can't get one where I work" seems a little unlikely.
Cheers,
Bill.
On Mon, 07 Mar 2005 22:24:00 +0000, Christopher C. Stacy wrote:
> "Mike" <······@gmail.com> writes:
>> I'm coding on a network that is separate from those networks that
>> contain all the known implementations of CL. (That I know of). I
>> can't make any (read: ANY) bridge between the two networks.
>>
>> So yes, I'm making a small Lisp by myself. Not CL. Not even Scheme.
>
> I don't understand. A CL implementation just consists of some
> files which are installed onto your computer like any other program;
> no networking is necessary for this. Do you mean that you are
> not allowed to use any software, other than what you write yourself?
>
> That's a very odd requirement. (It certainly does not lend
> itself to creating a secure computer, for example.)
I went for an interview once in an environment like this. They weren't
allowed network connections, they weren't allowed floppy disks, they
weren't allowed CD media or USB drives. Nothing.
It was horrible. I'm glad I didn't go to work there!
Cheers,
Bill.
In article <······························@abstractnonsense.com>,
William Bland <·······@abstractnonsense.com> wrote:
> On Mon, 07 Mar 2005 22:24:00 +0000, Christopher C. Stacy wrote:
>
> > "Mike" <······@gmail.com> writes:
> >> I'm coding on a network that is separate from those networks that
> >> contain all the known implementations of CL. (That I know of). I
> >> can't make any (read: ANY) bridge between the two networks.
> >>
> >> So yes, I'm making a small Lisp by myself. Not CL. Not even Scheme.
> >
> > I don't understand. A CL implementation just consists of some
> > files which are installed onto your computer like any other program;
> > no networking is necessary for this. Do you mean that you are
> > not allowed to use any software, other than what you write yourself?
> >
> > That's a very odd requirement. (It certainly does not lend
> > itself to creating a secure computer, for example.)
>
> I went for an interview once in an environment like this. They weren't
> allowed network connections, they weren't allowed floppy disks, they
> weren't allowed CD media or USB drives. Nothing.
>
> It was horrible. I'm glad I didn't go to work there!
What happens when you need to reinstall the OS, install patches, etc.?
How do *any* applications get onto the systems in that cloistered
environment?
I presume the actual restriction is that you can't bring in removable
media unless it contains approved content. And in the OP's case, I
guess CMUCL is not on the approved list.
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
William Bland <·······@abstractnonsense.com> writes:
> On Mon, 07 Mar 2005 22:24:00 +0000, Christopher C. Stacy wrote:
>
> > "Mike" <······@gmail.com> writes:
> >> I'm coding on a network that is separate from those networks that
> >> contain all the known implementations of CL. (That I know of). I
> >> can't make any (read: ANY) bridge between the two networks.
> >>
> >> So yes, I'm making a small Lisp by myself. Not CL. Not even Scheme.
> >
> > I don't understand. A CL implementation just consists of some
> > files which are installed onto your computer like any other program;
> > no networking is necessary for this. Do you mean that you are
> > not allowed to use any software, other than what you write yourself?
> >
> > That's a very odd requirement. (It certainly does not lend
> > itself to creating a secure computer, for example.)
>
> I went for an interview once in an environment like this. They weren't
> allowed network connections, they weren't allowed floppy disks, they
> weren't allowed CD media or USB drives. Nothing.
>
> It was horrible. I'm glad I didn't go to work there!
It must be good for job security no? Instead of fetching free sources
at speed of 1Mb/s, you must type them in at most at 7 B/s, usually
more like 1 b/s. Projects may last a long time at this speed $$$
--
__Pascal Bourguignon__ http://www.informatimago.com/
In deep sleep hear sound,
Cat vomit hairball somewhere.
Will find in morning.
"Mike" <······@gmail.com> writes:
> Liking CL and the "flavor" of CL more...yes. Implementing CL by
> myself...no.
>
> I'm coding on a network that is separate from those networks that
> contain all the known implementations of CL. (That I know of). I
> can't make any (read: ANY) bridge between the two networks.
There are people who learn entire books (eg in Fahrenheit 451).
You could start to learn the source of a debugged Common-Lisp
implementation, and type it in again on the secure network.
> So yes, I'm making a small Lisp by myself. Not CL. Not even Scheme.
>
> But I need some sort of macro system, and looking at the portable
> syntax-case macro expander, holy goodness. I think I'll just try to
> get CL-like unsafe macros to work.
--
__Pascal Bourguignon__ http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
Spake Pascal Bourguignon:
> There are people who learn entire books (eg in Fahrenheit 451).
ISTR an allusion to memory-enhancing drugs in that. Of course, it was
*fiction*. Memorizing verbatim a long string of text or code is
something mere humans can't do without a good deal of training.
--
Trent Buck, Student Errant
Move along, move along, nothing to see here, definitely no evil mind
control software here, move along, move along... -- Thorfinn
Trent Buck <·········@tznvy.pbz> writes:
> Spake Pascal Bourguignon:
> > There are people who learn entire books (eg in Fahrenheit 451).
>
> ISTR an allusion to memory-enhancing drugs in that. Of course, it was
> *fiction*. Memorizing verbatim a long string of text or code is
> something mere humans can't do without a good deal of training.
Yes, training. It's called: "learning by heart" (at least that's the
transliterated expression in French).
The Odyssey was not printed: people learned it by heart.
--
__Pascal Bourguignon__ http://www.informatimago.com/
In a World without Walls and Fences,
who needs Windows and Gates?
Pascal Bourguignon wrote:
> Trent Buck <·········@tznvy.pbz> writes:
>
>> Spake Pascal Bourguignon:
>> > There are people who learn entire books (eg in Fahrenheit 451).
>>
>> ISTR an allusion to memory-enhancing drugs in that. Of course, it was
>> *fiction*. Memorizing verbatim a long string of text or code is
>> something mere humans can't do without a good deal of training.
>
> Yes, training. It's called: "learning by heart" (at least that's the
> transliterated expression in French).
>
> The Odyssey was not printed: people learned it by heart.
>
Yes, and they repeatedly made slight mistakes in the retelling.
That's fine for a story, but I don't think it would work well in this
case. ;)
Trent Buck <·········@tznvy.pbz> wrote:
> Spake Pascal Bourguignon:
>> There are people who learn entire books (eg in Fahrenheit 451).
>
> ISTR an allusion to memory-enhancing drugs in that. Of course, it was
> *fiction*. Memorizing verbatim a long string of text or code is
> something mere humans can't do without a good deal of training.
An awful lot of actors do it, as well as obsessed fans. Nor are the two
always different people: I heard a radio interview recently with one of
the guys doing the Monty Python musical on Broadway, in which he
mentioned that he'd been "off book" (meaning, had memorized the entire
script) with "Monty Python and the Holy Grail" since age 15.
(On a more serious note, it is a notable feat in the Muslim world for
scholars to memorize al-Qur'an -- common enough that there is a word for
it in Arabic -- Hafiz.)
A tune, or a meter, does make it easier to memorize a work. I'm not
sure if that applies to code, though -- I seem to recall the "DeCSS"
song was not such a big hit.
--
Karl A. Krueger <········@example.edu> { s/example/whoi/ }
Pie Jesu domine, dona eis requiem. *thunk*