I'm a Lisp novice. Bumped into a conceptual question about compiling
Lisp.
I have some Common Lisp code (from Riesbeck & Schank, _Inside Case Based
Reasoning_). This code defines a macro INSIST that is a runtime
assertion -- that is, it's an error-prevention mechanism that verifies
that something is numeric or above zero or not nil or something like that
before the real processing code tries to do something with it.
This INSIST macro is then invoked inside a defun. In this particular use
of the INSIST macro it verifies that certain things aren't nil.
When I load the code using (say) CMU CL the macro isn't invoked at the
time the defun is being loaded. That is, it just does the defmacro and
the defun and everything is okay. When I load it using a Common
Lisp compiler I'm playing with, the INSIST macro is invoked during the
defun attempt and it hollers because those ought-not-to-be-nil things are
nil (since actual processing hasn't started yet) and hence the defun
fails. I've been discussing with a colleague whether the compiler ought
to behave the CMU CL way and how one would decide such a thing.
So here at last is the conceptual question: In general a "def..."
construct would be evaluated at compile time. An "eval" inside a defun
would be evaluated only at run time. It seems that a macro invoked
inside a defun could either be actually invoked during the defun (i.e.,
at compile time, as this compiler is doing) or treated as a kind of eval
(invoked to generate and execute code only at runtime).
I'm not going to ask which is "right." I'm going to ask what kind of a
decision is it? Is it a left-to-the-implementor decision or would some
standard specify this behavior for "Common Lisp" or is it wrapped up in
some big debate about language issues beyond my limited ken?
Clark Wilson
········@uchicago.edu
In article <··························@uchinews.uchicago.edu>,
Clark Wilson <········@my-dejanews.com> wrote:
>So here at last is the conceptual question: In general a "def..."
>construct would be evaluated at compile time. An "eval" inside a defun
>would be evaluated only at run time. It seems that a macro invoked
>inside a defun could either be actually invoked during the defun (i.e.,
>at compile time, as this compiler is doing) or treated as a kind of eval
>(invoked to generate and execute code only at runtime).
>
>I'm not going to ask which is "right." I'm going to ask what kind of a
>decision is it? Is it a left-to-the-implementor decision or would some
>standard specify this behavior for "Common Lisp" or is it wrapped up in
>some big debate about language issues beyond my limited ken?
Macros are expanded at compile time. Normally, macros just expand into
code, which is substituted into the function definition, and that code will
be executed when the function is called. However, if the macro function
(as opposed to the code it expands into) has side effects, they should
occur at compile time.
For example:
(defmacro insist-runtime (thing)
`(let ((temp ,thing)) ; prevent multiple evaluation
(assert (and (numberp temp) (> temp 0)))
temp)
(defmacro insist-compiletime (thing)
(assert (and numberp thing) (> thing 0))
thing)
The first will expand into code that will check at run time whether its
parameter evaluates to a number > 0. The second will check at compile time
whether its parameter is a literal number > 0; it has to be a literal,
because parameters to macros are unevaluated.
--
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Don't bother cc'ing followups to me.
Clark Wilson wrote:
> I have some Common Lisp code (from Riesbeck & Schank, _Inside Case Based
> Reasoning_). This code defines a macro INSIST that is a runtime
> assertion -- that is, it's an error-prevention mechanism that verifies
> that something is numeric or above zero or not nil or something like that
> before the real processing code tries to do something with it.
>
> This INSIST macro is then invoked inside a defun. In this particular use
> of the INSIST macro it verifies that certain things aren't nil.
So it's something like
(defmacro insist (&rest args)
`(unless (and . ,args)
(error "foo bar blah blah")))
?
> When I load the code using (say) CMU CL the macro isn't invoked at the
> time the defun is being loaded. That is, it just does the defmacro and
> the defun and everything is okay. When I load it using a Common
> Lisp compiler I'm playing with, the INSIST macro is invoked during the
> defun attempt and it hollers because those ought-not-to-be-nil things are
> nil (since actual processing hasn't started yet) and hence the defun
> fails. I've been discussing with a colleague whether the compiler ought
> to behave the CMU CL way and how one would decide such a thing.
This sounds extremely broken.
Or, when you say "invoked", do you actually mean "expanded"?
If the definition were something like
(defmacro insist (&rest args)
(unless (every #'eval args)
(error "foo bar blah blah")))
then this would make more sense (although the macro definition
would then probably be crazy).
Could you post, or make visible on the web, the file in question
and a copy of the output you get from this second CL compiler (the
one that complains)?
> So here at last is the conceptual question: In general a "def..."
> construct would be evaluated at compile time. An "eval" inside a defun
> would be evaluated only at run time. It seems that a macro invoked
> inside a defun could either be actually invoked during the defun (i.e.,
> at compile time, as this compiler is doing) or treated as a kind of eval
> (invoked to generate and execute code only at runtime).
It has to be done at compile time. Section 3.2.2.2 of the standard
defines "minimal compilation" in a way that includes the expansion
of all macros, and section 3.2.3.1 says that COMPILE-FILE does
"minimal compilation" on the top-level forms in the file being
compiled.
However, there's another issue. Under certain circumstances, some
of the contents of a file being compiled may be evaluated too. There
is a way of requesting this explicitly, but it can also happen
implicitly (when the compiler thinks it might need the result of
evaluating the form to process stuff later in the file). This
might possibly be relevant; it's hard to tell.
--
Gareth McCaughan Dept. of Pure Mathematics & Mathematical Statistics,
·····@dpmms.cam.ac.uk Cambridge University, England.
In article <··························@uchinews.uchicago.edu>,
········@my-dejanews.com says...
> I'm a Lisp novice. Bumped into a conceptual question about compiling
> Lisp.
>
This was an accurate self-description. The replies I got, and my further
futzing around, show that I don't really understand what's going on.
This message is just an acknowledgement of the replies I got, and a lack-
of-progress report.
One datum about me and the code: If at the command line in CMU CL I say
(load "mcmops.lsp")
then the only messages I get are two warnings that certain items are
"special", then at the end a T. Sounds good.
If (again in a brand new CMU CL session) I say
(compile-file "mcmops.lsp")
then I get a bunch of error messages including complaints that a certain
function that appears to have been compiled okay earlier in the compile
process isn't defined later on in it.
So, I'm pursuing this but so far the main thing that I know is that I
have a lot to learn and that I'm learning it.
Thanks to all who replied so quickly to my original posting.
Clark Wilson
In article <··························@uchinews.uchicago.edu>,
········@my-dejanews.com says...
> I'm a Lisp novice. Bumped into a conceptual question about compiling
> Lisp.
>
> I have some Common Lisp code (from Riesbeck & Schank, _Inside Case Based
> Reasoning_). This code defines a macro INSIST that is a runtime
> assertion -- that is, it's an error-prevention mechanism that verifies
> that something is numeric or above zero or not nil or something like that
> before the real processing code tries to do something with it.
>
> This INSIST macro is then invoked inside a defun. In this particular use
> of the INSIST macro it verifies that certain things aren't nil.
>
> When I load the code using (say) CMU CL the macro isn't invoked at the
> time the defun is being loaded. That is, it just does the defmacro and
> the defun and everything is okay. When I load it using a Common
> Lisp compiler I'm playing with, the INSIST macro is invoked during the
> defun attempt and it hollers because those ought-not-to-be-nil things are
> nil (since actual processing hasn't started yet) and hence the defun
> fails. I've been discussing with a colleague whether the compiler ought
> to behave the CMU CL way and how one would decide such a thing.
>
After more futzing around and with help from netizens I have learned some
really basic basics about Lisp compilers and the code I was working with.
According to my sources the code does not fit at all gracefully into the
compiled world and would probably best be rewritten from scratch.
My thanks to all who replied.
Clark Wilson
In article <··························@uchinews.uchicago.edu>,
········@uchicago.edu (Clark Wilson) wrote:
>In article <··························@uchinews.uchicago.edu>,
>········@my-dejanews.com says...
>> I have some Common Lisp code (from Riesbeck & Schank, _Inside Case Based
>> Reasoning_). This code defines a macro INSIST that is a runtime
>> assertion -- that is, it's an error-prevention mechanism that verifies
>> that something is numeric or above zero or not nil or something like that
>> before the real processing code tries to do something with it.
>>
>> This INSIST macro is then invoked inside a defun. In this particular use
>> of the INSIST macro it verifies that certain things aren't nil.
>>
>> When I load the code using (say) CMU CL the macro isn't invoked at the
>> time the defun is being loaded. That is, it just does the defmacro and
>> the defun and everything is okay. When I load it using a Common
>> Lisp compiler I'm playing with, the INSIST macro is invoked during the
>> defun attempt and it hollers because those ought-not-to-be-nil things are
>> nil (since actual processing hasn't started yet) and hence the defun
>> fails....
>
>According to my sources the code does not fit at all gracefully into the
>compiled world and would probably best be rewritten from scratch.
I didn't see this on the newsgroups till now. Old though that
code is, I have to call your sources into question. INSIST should work
fine in a compiled Lisp. The INSIST macro expands into code and
whether that happens early or late in the compile process isn't
relevant, as long as it follows CL standards. When the compiled
code is run, the expanded macro code will be code, which is as it should be.
Which compiler is giving you trouble here?
[This followup was posted to comp.lang.lisp and a copy was sent to the
cited author.]
In article <·························@riesbeck.ils.nwu.edu>,
········@ils.nwu.edu says...
> In article <··························@uchinews.uchicago.edu>,
> ········@uchicago.edu (Clark Wilson) wrote:
>
> >In article <··························@uchinews.uchicago.edu>,
> >········@my-dejanews.com says...
>
> >> I have some Common Lisp code (from Riesbeck & Schank, _Inside Case Based
> >> Reasoning_). This code defines a macro INSIST that is a runtime
> >> assertion -- that is, it's an error-prevention mechanism that verifies
> >> that something is numeric or above zero or not nil or something like that
> >> before the real processing code tries to do something with it.
> >>
[...]
> >
> >According to my sources the code does not fit at all gracefully into the
> >compiled world and would probably best be rewritten from scratch.
>
> I didn't see this on the newsgroups till now. Old though that
> code is, I have to call your sources into question. INSIST should work
> fine in a compiled Lisp. The INSIST macro expands into code and
> whether that happens early or late in the compile process isn't
> relevant, as long as it follows CL standards. When the compiled
> code is run, the expanded macro code will be code, which is as it should be.
>
> Which compiler is giving you trouble here?
>
The compilers I have tried are:
http://www.corman.net/CormanLisp.html
which is purely a compiler, so I used the "execute file" menu choice,
and the CMU CL compiler is indicated by this log header:
> CMU Common Lisp 17f, running on alisun
[...]
> Loaded subsystems:
> Python 1.0, target SPARCstation/Sun 4
> CLOS based on PCL version: September 16 92 PCL (f)
> * (compile-file "mcmops.lsp")
> Python version 1.0, VM version SPARCstation/Sun 4 on 10 DEC 98 05:00:51 pm.
> Compiling: /u7/u1/clrk/Schank/icbr/mcmops.lsp 10 DEC 98 05:00:18 pm
in which I used the compile-file command. (The load command didn't give
any error messages -- just a couple of messages about some symbols being
special.)
The code I am gnawing at is at
ftp://ftp.cs.umd.edu/pub/schank/icbr
Rather than spend the usenet bandwidth to include the actual code and
lisp logs, I will send them directly via email.
Clark Wilson
Administrative Information Systems
University of Chicago
········@uchicago.edu
From: Steve Gonedes
Subject: Re: novice's conceptual question re lisp compilers
Date:
Message-ID: <m2k8zjfk0g.fsf@KludgeUnix.com>
········@my-dejanews.com (Clark Wilson) writes:
< in which I used the compile-file command. (The load command didn't give
< any error messages -- just a couple of messages about some symbols being
< special.)
<
< The code I am gnawing at is at
<
< ftp://ftp.cs.umd.edu/pub/schank/icbr
<
< Rather than spend the usenet bandwidth to include the actual code and
< lisp logs, I will send them directly via email.
All the functions called in a macro should be defined before the macro
is used. If you compile all the files at one time then the macro is
evaluated and being expanded but the functions that it calls have not
been loaded yet. If you can compile and load files (one at a time)
this usually helps. Also you can load the project and then compile it;
which should also work.
It's kinda neat that you can program in context, but can also be a
real pain in the neck if you're not careful. Probably the best way to
deal with compilation of multiple files is to use a `defsystem' which
will compile and load things in the right order (like a makefile).
Hope this helps some...
In article <··············@KludgeUnix.com>,
Steve Gonedes <········@worldnet.att.net> wrote:
>All the functions called in a macro should be defined before the macro
>is used. If you compile all the files at one time then the macro is
>evaluated and being expanded but the functions that it calls have not
>been loaded yet. If you can compile and load files (one at a time)
>this usually helps. Also you can load the project and then compile it;
>which should also work.
EVAL-WHEN can also be useful to solve this problem. Any functions that are
called by a macro can be defined with something like:
(eval-when (:compile-toplevel :eval)
(defun ...))
If you also want the functions to be available at runtime due to loading
the compiled file, you would add :load-toplevel to the eval-when times.
--
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Don't bother cc'ing followups to me.
In article <·················@burlma1-snr1.gtei.net>, Barry Margolin
<······@bbnplanet.com> wrote:
>In article <··············@KludgeUnix.com>,
>Steve Gonedes <········@worldnet.att.net> wrote:
>>All the functions called in a macro should be defined before the macro
>>is used. If you compile all the files at one time then the macro is
>>evaluated and being expanded but the functions that it calls have not
>>been loaded yet.
>
>EVAL-WHEN can also be useful to solve this problem. Any functions that are
>called by a macro can be defined with something like:
>
>(eval-when (:compile-toplevel :eval)
> (defun ...))
>
>If you also want the functions to be available at runtime due to loading
>the compiled file, you would add :load-toplevel to the eval-when times.
I haven't heard back yet, but I think the problem is that this
pretty old code defined a LOOP-like macro called FOR (because
there was no standard CL LOOP then) which sets up a table of
keywords like this
(defmacro for ...)
...
(defmacro define-for-key ...)
(define-for-key :do ...)
(define-for-key :splice...)
...
(defun foo ()
(for (x :in l) :splice ...))
The macros were getting defined but the table wasn't
being initialized, so the expansion of FOR in FOO couldn't
find the pieces it needed.
I recommended moving the FOR stuff into a separate file to be
compiled and loaded first.
What surprises me more is how many Lisp's this still works in,
including Macintosh Common Lisp 4.1 and Allegro CL Lite v 5.0
From: Steve Gonedes
Subject: Re: novice's conceptual question re lisp compilers
Date:
Message-ID: <m21zlq571f.fsf@KludgeUnix.com>
Barry Margolin <······@bbnplanet.com> writes:
< In article <··············@KludgeUnix.com>,
< Steve Gonedes <········@worldnet.att.net> wrote:
< >All the functions called in a macro should be defined before the macro
< >is used. If you compile all the files at one time then the macro is
< >evaluated and being expanded but the functions that it calls have not
< >been loaded yet. If you can compile and load files (one at a time)
< >this usually helps. Also you can load the project and then compile it;
< >which should also work.
<
< EVAL-WHEN can also be useful to solve this problem. Any functions that are
< called by a macro can be defined with something like:
<
< (eval-when (:compile-toplevel :eval)
< (defun ...))
I always get nervous about using this though; I feel like I'm
disturbing the compiler or something. If possible I try to use a local
function - but this doesn't seem to work very often.
End of this particular segment of the Clark-and-Lisp saga, which surfaced
in comp.lang.lisp 'way back on Dec. 7, 1998. Some snippets from my msg
of Dec. 23, to orient folks:
In article <··························@uchinews.uchicago.edu>,
········@my-dejanews.com says...
> In article <·························@riesbeck.ils.nwu.edu>,
> ········@ils.nwu.edu says...
> > In article <··························@uchinews.uchicago.edu>,
> > ········@uchicago.edu (Clark Wilson) wrote:
> >
> > >In article <··························@uchinews.uchicago.edu>,
> > >········@my-dejanews.com says...
> >
> > >> I have some Common Lisp code (from Riesbeck & Schank, _Inside Case Based
> > >> Reasoning_). This code defines a macro INSIST that is a runtime
> > >> assertion -- that is, it's an error-prevention mechanism that verifies
> > >> that something is numeric or above zero or not nil or something like that
> > >> before the real processing code tries to do something with it.
> > >>
> [...]
> > >
> > >According to my sources the code does not fit at all gracefully into the
> > >compiled world and would probably best be rewritten from scratch.
> >
> > I didn't see this on the newsgroups till now. Old though that
> > code is, I have to call your sources into question. INSIST should work
> > fine in a compiled Lisp. The INSIST macro expands into code and
> > whether that happens early or late in the compile process isn't
> > relevant, as long as it follows CL standards. When the compiled
> > code is run, the expanded macro code will be code, which is as it should be.
> >
1) The INSIST macro was being handled properly by Corman's implementation
of Common Lisp and it was hollering about a true error condition. Roger
Corman traced it down to the order of arguments given to SET-FN:
(DEFMACRO DEFINE-TABLE (FN VARS PLACE)
(LET ((KEY (CAR VARS))
(SET-FN (GENTEMP "SET-FN."))
(VAL (GENTEMP "VAL.")))
`(PROGN (DEFUN ,FN (,KEY) (GETF ,PLACE ,KEY))
(DEFUN ,SET-FN (,VAL ,KEY) ; reversed val and key
(SETF (GETF ,PLACE ,KEY) ,VAL))
(DEFSETF ,FN ,SET-FN)
',FN)))
When the two arguments were reversed (see the comment) the code worked in
the Corman Lisp compiler.
2) When I shared this fix with Chris Riesbeck, he noted that the original
order of the arguments was actually valid:
"Actually, the original code was correct and the patch (and I guess
Corman Lisp) is incorrect. This code is using the short form of the old
DEFSETF macro, because (defun (setf ...) ...) didn't exist when it was
first written, and still doesn't exist in XlispStat, which many of
my students use in their intro AI class.
"The short form is (DEFSETF accessor-name updater-name). Here's the
relevant description from the CL HyperSpec (which echoes Steele)
http://www.harlequin.com/education/books/HyperSpec/Body/mac_defsetf.html
'defsetf may take one of two forms, called the ``short form'' and the
``long form,'' which are distinguished by the type of the second
argument.'"
3) In the very beginning, I could have RTFM: comp.lang.lisp FAQ question
3-16 ("My program works when interpreted but not when compiled!") and
Touretzky, _Common Lisp_ (1990), sections 14.8-14.10. (Those references
might have reduced my initial confusion but would not have eliminated the
ultimate source of my problem.)
4) More than formal thanks to all who helped solve this, most notably
Roger Corman and Chris Riesbeck.
5) FYI, the "Corman Lisp Online Community" can be found at
http://www.dejanews.com/~cormanlisp/
6) I am now very happily moving into learning about the ICBR MOP
implementation (and much more about Common Lisp).
Clark Wilson
········@uchicago.edu