I've been messing around with eval-when and compile-file and have found
a small problem seemingly without a straight forward fix. First of all,
my system looks a bit like this:
file1.lisp
(eval-when (:compile-toplevel :load-toplevel :evaluate)
(defvar *things* nil)
;; handy functions used by define-thing
) ; end eval-when
(defmacro define-thing (...)
;; add stuff to *things*
;; define classes and methods using *things*
)
file2.lisp
(define-thing 1 ...)
(define-thing 2 ...)
file3.lisp
(define-thing 3 ...)
(define-thing 4 ...)
Now if I do the following:
(with-compilation-unit ()
(compile-file "file1")
(load "file1")
(compile-file "file2")
(load "file2")
(compile-file "file3")
(load "file3"))
everything works just fine.
However if I do this:
(with-compilation-unit ()
(load "file1") ; file1 hasn't changed
(load "file2") ; file2 hasn't changed
(compile-file "file3")
(load "file3"))
when file3 is loaded, *things* is nil and the information that was put
there during the compilation of file2 is lost.
After carefully reading the hyperspec I conclude that the only way round
this is to explicitly write out *things* to a scatch file at the end of
compiling file2 and to explicitly read the file back into *things* at
the beginning of compiling file3. But I confess that my understanding of
eval-when and compile-file is wanting, so maybe there a better way?
In the meantime I've taken the easy way out and combined file2 and file3
into a single file, but as it grows large, compile times increase most
annoyingly.
Any other suggestions?
Cheers,
Andrew Smith
Andrew Smith <······@thesoftwaresmith.com.au> writes:
> I've been messing around with eval-when and compile-file and have
> found a small problem seemingly without a straight forward fix. First
> of all, my system looks a bit like this:
>
> file1.lisp
>
> (eval-when (:compile-toplevel :load-toplevel :evaluate)
You want :execute here.
:evaluate is not a defined keyword for EVAL-WHEN.
>
> (defvar *things* nil)
>
> ;; handy functions used by define-thing
>
> ) ; end eval-when
>
> (defmacro define-thing (...)
> ;; add stuff to *things*
> ;; define classes and methods using *things*
> )
Does it expand into stuff or does it do stuff as a side-effect?
If it expands into the push, that's what you want.
> file2.lisp
>
> (define-thing 1 ...)
> (define-thing 2 ...)
>
> file3.lisp
>
> (define-thing 3 ...)
> (define-thing 4 ...)
>
>
> Now if I do the following:
>
> (with-compilation-unit ()
> (compile-file "file1")
> (load "file1")
> (compile-file "file2")
> (load "file2")
> (compile-file "file3")
> (load "file3"))
>
> everything works just fine.
>
> However if I do this:
>
> (with-compilation-unit ()
> (load "file1") ; file1 hasn't changed
> (load "file2") ; file2 hasn't changed
> (compile-file "file3")
> (load "file3"))
>
> when file3 is loaded, *things* is nil and the information that was put
> there during the compilation of file2 is lost.
Shouldn't be, unless you're doing compile-time side-effects in DEFINE-THING.
You shouldn't do that.
> After carefully reading the hyperspec I conclude that the only way
> round this is to explicitly write out *things* to a scatch file at the
> end of compiling file2 and to explicitly read the file back into
> *things* at the beginning of compiling file3.
No.
> But I confess that my
> understanding of eval-when and compile-file is wanting, so maybe there
> a better way?
Expand into a set of things to be done at runtime.
On Wed, 08 Jan 2003 15:01:03 +1100, Andrew Smith
<······@thesoftwaresmith.com.au> wrote:
>
>I've been messing around with eval-when and compile-file and have found
>a small problem seemingly without a straight forward fix. First of all,
>my system looks a bit like this:
>
>file1.lisp
>
> (eval-when (:compile-toplevel :load-toplevel :evaluate)
>
> (defvar *things* nil)
>
> ;; handy functions used by define-thing
>
> ) ; end eval-when
>
> (defmacro define-thing (...)
> ;; add stuff to *things*
> ;; define classes and methods using *things*
> )
Here's a guess at your problem and a possible solution template:
You don't want DEFINE-THING to 'add stuff to *things*'
at MACROEXPANSION time (which is usually compile time), because
then once a file with a DEFINE-THING is compiled using one lisp image
and you` try to load the .fasl file so produced into a subsequent lisp
image the macro expansion won't execute again, and no stuff will be
added to *things*.
What you want is for DEFINE-THING to expand into a form
that will cause stuff to be added to *things* at LOAD time,
regardless of whether you are loading the .lisp file or the
compiled .fasl file.
So instead of
(defmacro define-thing ()
(add-stuff-to-*things*)
`(define-classes ...))
do
(defmacro define-thing ()
`(progn
(eval-when (:compile-toplevel :load-toplevel :execute)
(add-stuff-to-*things*))
(define-classes ...)
))
>
>file2.lisp
>
> (define-thing 1 ...)
> (define-thing 2 ...)
>
>file3.lisp
>
> (define-thing 3 ...)
> (define-thing 4 ...)
>
>
>Now if I do the following:
>
> (with-compilation-unit ()
> (compile-file "file1")
> (load "file1")
> (compile-file "file2")
> (load "file2")
> (compile-file "file3")
> (load "file3"))
>
>everything works just fine.
>
>However if I do this:
>
> (with-compilation-unit ()
> (load "file1") ; file1 hasn't changed
> (load "file2") ; file2 hasn't changed
> (compile-file "file3")
> (load "file3"))
>
>when file3 is loaded, *things* is nil and the information that was put
>there during the compilation of file2 is lost.
>
>After carefully reading the hyperspec I conclude that the only way round
>this is to explicitly write out *things* to a scatch file at the end of
>compiling file2 and to explicitly read the file back into *things* at
>the beginning of compiling file3. But I confess that my understanding of
>eval-when and compile-file is wanting, so maybe there a better way?
>
>In the meantime I've taken the easy way out and combined file2 and file3
> into a single file, but as it grows large, compile times increase most
>annoyingly.
>
>Any other suggestions?
>
>Cheers,
>Andrew Smith
>
From: Tim Bradshaw
Subject: Re: fun with eval-when and compile-file
Date:
Message-ID: <ey3el7nncai.fsf@cley.com>
* JP Massar wrote:
> What you want is for DEFINE-THING to expand into a form
> that will cause stuff to be added to *things* at LOAD time,
> regardless of whether you are loading the .lisp file or the
> compiled .fasl file.
And in this case you almost certainly will not need the EVAL-WHEN,
either: DEFINE-THING will expand to code which refers to *THINGS*, but
this code will not actually run until the file is loaded, when
*THINGS* and the utility functions will be defined in the normal way.
--tim
Tim Bradshaw <···@cley.com> writes:
> * JP Massar wrote:
> > What you want is for DEFINE-THING to expand into a form
> > that will cause stuff to be added to *things* at LOAD time,
> > regardless of whether you are loading the .lisp file or the
> > compiled .fasl file.
>
> And in this case you almost certainly will not need the EVAL-WHEN,
> either: DEFINE-THING will expand to code which refers to *THINGS*, but
> this code will not actually run until the file is loaded, when
> *THINGS* and the utility functions will be defined in the normal way.
The usual case is that an EVAL-WHEN is needed in the expansion if you plan
to use the macro you're defining later in the same file and not otherwise.
Kent M Pitman <······@world.std.com> wrote:
+---------------
| Tim Bradshaw <···@cley.com> writes:
| > And in this case you almost certainly will not need the EVAL-WHEN,
| > either: DEFINE-THING will expand to code which refers to *THINGS*, but
| > this code will not actually run until the file is loaded, when
| > *THINGS* and the utility functions will be defined in the normal way.
|
| The usual case is that an EVAL-WHEN is needed in the expansion if you plan
| to use the macro you're defining later in the same file and not otherwise.
+---------------
I've found that in order to compile-file files that REQUIRE other
compiled macro-containing modules, I have to wrap the REQUIREs in
EVAL-WHEN. For example, the prolog for my CGI-SQL module (that supports
CGI scripting in either CMUCL or CLISP) that uses Tim Bradshaw's HTOUT
macros & Eric Marsden's POSTGRESQL macros (except on CLISP, where for
now I'm still using the PostgreSQL binding bundled into the distribution)
had to be this to get it to compile:
(eval-when (:compile-toplevel :load-toplevel :execute) ; for the macros
(require :htout)
(require :cgi)
#-clisp ; May be removed later if move to PG for CLISP, too.
(require :pg))
(defpackage :cgi-sql
(:use :cl :ext :htout #-clisp :pg)
#+clisp ; conflicts with CLISP's EXT
(:shadowing-import-from :htout #:with-html-output)
(:export #:sql-result-html-table ; (sql-result &key stream callback)
#:list-html-table ; (lists &key stream callback)
...etc... )
(in-package :cgi-sql)
(provide :cgi-sql)
...etc...
Am I missing something obvious that would let me omit the EVAL-WHEN?
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
> Kent M Pitman <······@world.std.com> wrote:
> +---------------
> | Tim Bradshaw <···@cley.com> writes:
> | > And in this case you almost certainly will not need the EVAL-WHEN,
> | > either: DEFINE-THING will expand to code which refers to *THINGS*, but
> | > this code will not actually run until the file is loaded, when
> | > *THINGS* and the utility functions will be defined in the normal way.
> |
> | The usual case is that an EVAL-WHEN is needed in the expansion if you plan
> | to use the macro you're defining later in the same file and not otherwise.
> +---------------
>
> I've found that in order to compile-file files that REQUIRE other
> compiled macro-containing modules, I have to wrap the REQUIREs in
> EVAL-WHEN. For example, the prolog for my CGI-SQL module (that supports
> CGI scripting in either CMUCL or CLISP) that uses Tim Bradshaw's HTOUT
> macros & Eric Marsden's POSTGRESQL macros (except on CLISP, where for
> now I'm still using the PostgreSQL binding bundled into the distribution)
> had to be this to get it to compile:
> [...]
> Am I missing something obvious that would let me omit the EVAL-WHEN?
No, this is the same issue. The contents of the REQUIRE are needed later in
the same file, so you need an EVAL-WHEN to hurry up the REQUIRE, which would
otherwise not happen until load time.
Note that this isn't contradicting what I said above. In the case you
describe, though EVAL-WHEN is needed for the REQUIRE, it is probably not
needed in the file you are requiring, though it would be if the file you were
requiring made use of its own macros in that same file.
>>I've found that in order to compile-file files that REQUIRE other
>>compiled macro-containing modules, I have to wrap the REQUIREs in
>>EVAL-WHEN. ...
>>Am I missing something obvious that would let me omit the EVAL-WHEN?
>
> No, this is the same issue. The contents of the REQUIRE are needed later in
> the same file, so you need an EVAL-WHEN to hurry up the REQUIRE, which would
> otherwise not happen until load time.
There is a simple, general rule here. But first, some allegorical history.
Back in prehistoric times, when the earth's semantic crust hadn't finished
cooling and dinosaurs like Maclisp still roamed, lisp had all sorts of
irregular execution metaphor operators; there were fexprs and lexprs and
nsubrs, and even the fearsom Nlambdasaurus Lex, which would grab an
unsuspecting code body and rend parentheses asunder with its fearsome teeth.
In those days, small hairy compilers scurried almost unnoticed among these
crude, ill-behaved monsters, but there was little they could do other than
try to avoid these them and maybe suck a few eggs.
Things got a little better after an asteroid or two cleaned out the most
offensive constructs, and CLtL1 evolved. (I think Guy Steele of lexical
scoping fame starred with Raquel Welch in a film about this, side effects
by Ray Harryhausen, called "One Million Years B.C. (Before Commonlisp)"
<http://us.imdb.com/Title?0060782>, but I may be confused about some these
details. CLtL was originally published as the book version of the film,
or maybe it was the other way around.)
End of allegory, which is beginning tio tire even me.
When the X3J13 process started, there was considerable variance how CL
implementations treated certain pseudo-declarative forms when file compiling.
In particular, _functions_ like in-package, provide, and require were
treated specially (i.e. irregularly) when encountered at top level in a
file compilation. This was necessary to conform to programmer usage
and examples in CLtL1, but permitted much to much variance in _how_ different
implementations reasoned about these forms. Further, since this irregular
reasoning was opaque, it was impossible for code walkers or programmers to
reason about it, even with their post-reptilian-evolved brains. One of the
brilliant language-simplification moves of the X3J13 compiler subcommittee
was to recommend that the language remove any compile-file-top-level
special behavior from all CL _functions_. The necessary semantics of
defmumble defining macros would be implemented by expansion into transparent
eval-when forms, and (I recollect this was the only such case) in-package
was changed from a function into a macro. See CLtL2, which was a remake of
CLtL1 without Raquel Welch. Definitional use (top-level in a file
compilation) of the various package mangling functions (export, shadow,
use-package, etc.) was replaced with the new defpackage macro.
Now for the simple, general rule: "No CL function operators have any
irregular interpretation by the file compiler at top-level." That's why
the eval-when is necessary, to give the call to the require function its
special behavior at compile time. Compilers are free to provide (and
should provide, since there is a lot of old code out there) warnings
and a CLtL1 compatibility mode, but elimination of special-operator
dinosaur functions from the official language was a really good thing.
Steven M. Haflich <·················@alum.mit.edu> wrote:
+---------------
| One of the brilliant language-simplification moves of the X3J13
| compiler subcommittee was to recommend that the language remove
| any compile-file-top-level special behavior from all CL _functions_.
| The necessary semantics of defmumble defining macros would be
| implemented by expansion into transparent eval-when forms, and
| (I recollect this was the only such case) in-package was changed
| from a function into a macro.
+---------------
Thanks! After reading your comment, I also found it quite
helpful to pass the following forms through CMUCL & CLISP:
(macroexpand '(defmacro foo (x) `(+ ,x 1)))
(macroexpand '(in-package "foo"))
"Very interesting..."
+---------------
| Now for the simple, general rule: "No CL function operators have any
| irregular interpretation by the file compiler at top-level." That's
| why the eval-when is necessary, to give the call to the require function
| its special behavior at compile time.
+---------------
Thanks again.
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
In article <·······················@newssvr21.news.prodigy.com>,
Steven M. Haflich <·················@alum.mit.edu> wrote:
> See CLtL2, which was a remake of
>CLtL1 without Raquel Welch.
To be more precise, she's still in there, but there are black bars and
dotted lines obscuring many of her features.
Sorry, I just couldn't resist extending the metaphor more than it
deserves. :)
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin wrote:
> Sorry, I just couldn't resist extending the metaphor more than it
> deserves. :)
Thanks for the support.
BTW, anyone who didn't fully understand the dinosaur metaphor
for nlambda and related species, please see the visual
explanation on this web page:
<http://www.franz.com/~smh/nlambda.html>
From: Andrew Smith
Subject: Re: fun with eval-when and compile-file
Date:
Message-ID: <ht8hva.ic.ln@192.168.1.111>
Andrew Smith wrote:
> (defmacro define-thing (...)
> ;; add stuff to *things*
> ;; define classes and methods using *things*
> )
Thanks for your help everyone, I've got it working now. As you
suggested, the way to do it was to change
(defmacro define-thing (...)
;; add stuff to *things*
`(prog
;; define classes and methods using *things*
)
to
(defmacro define-thing (...)
`(prog
(eval-when (:compile-toplevel :load-toplevel :execute)
;; add ',stuff to *things*
)
;; define classes and methods using *things*
)
The eval-when statement is necessary because without it, successive
invocations of define-thing in the same file cannot use the cumulative
value of *things* at compile time to construct the class definitions.
Cheers,
Andrew Smith