The application I am writing is divided up into several packages,
which I then tie together in another source file.
Example :
(load "/file/1")
(load "/file/2")
(load "/file/3")
(use-package "PACKAGE-1")
(use-package "PACKAGE-2")
;;; Add code here which uses exported interfaces from used
packages.
However, I think that I may be wrong here to add the loading of the
files in this source file, because when I try to compile this file,
this fails. Using CLISP as my example, when I then let CLISP execute
the load functions (using -x "(load "/file/1")"), and then compile it
(compile-file "source"), it succeeds partially. When I additionally
load the source file, and then execute (compile-file), the compilation
goes through completely.
If the way I express it seems a bit awkward, it is because the way the
system construction works is a bit counter-intuitive, for me at least,
but also because there seems to be no good explicit description of how
it all should tie together.
The way this all works is also tied in with toolkits like SLIME and
ILISP, which then give for me the same counterintuitive results.
E.g. if I load the file in Emacs, which contains the load and use-
package statements, and I execute a command to compile the whole file
(C-c C-k), I get errors when external objects are referenced. When I
execute the statements in the source file one by one (C-c C-c), then I
get the desired results.
By trying to write this post, I seem to get the understanding that CL
compilation is just a translation of a representation in source code,
to a representation in byte-code or assembler, and that the resulting
(.fas, .fasl) files only contain the things that where stated in the
source files.
I probably should see them more as object files, where the unresolved
dependencies must be resolved by the programmer, by using an
overarching system construction file which loads the necessary files
and then calls from the file which is the application, the main
routine.
Is this reasoning correct ? Are there other sources than the CLHS
about system construction ?
Jurgen
In article <························@p10g2000cwp.googlegroups.com>,
"jurgen_defurne" <··············@pandora.be> wrote:
> The application I am writing is divided up into several packages,
> which I then tie together in another source file.
>
> Example :
> (load "/file/1")
> (load "/file/2")
> (load "/file/3")
>
> (use-package "PACKAGE-1")
> (use-package "PACKAGE-2")
>
> ;;; Add code here which uses exported interfaces from used
> packages.
>
> However, I think that I may be wrong here to add the loading of the
> files in this source file, because when I try to compile this file,
> this fails. Using CLISP as my example, when I then let CLISP execute
> the load functions (using -x "(load "/file/1")"), and then compile it
> (compile-file "source"), it succeeds partially. When I additionally
> load the source file, and then execute (compile-file), the compilation
> goes through completely.
>
> If the way I express it seems a bit awkward, it is because the way the
> system construction works is a bit counter-intuitive, for me at least,
> but also because there seems to be no good explicit description of how
> it all should tie together.
>
> The way this all works is also tied in with toolkits like SLIME and
> ILISP, which then give for me the same counterintuitive results.
>
> E.g. if I load the file in Emacs, which contains the load and use-
> package statements, and I execute a command to compile the whole file
> (C-c C-k), I get errors when external objects are referenced. When I
> execute the statements in the source file one by one (C-c C-c), then I
> get the desired results.
>
Remember that LOAD is just a function. So the LOAD calls at the top of
your file aren't executed when the file is compiled. At compile-time,
Lisp hasn't yet loaded the information in 1.lisp, 2.lisp, and 3.lisp, so
referring to that information later in the file is a mistake.
You could resolve this with:
(eval-when (:execute :load-toplevel :compile-toplevel)
(load "/file/1")
(load "/file/2")
;; ...
)
so that those loads will be evaluated at compile-time as well as
load-time.
However, what you're talking about isn't what I would call "system
construction." One constructs a system using a facility like ASDF or
MK:DEFSYSTEM, etc.
With ASDF:
(defpackage :test.system (:use :asdf :cl))
(in-package :test.system)
(defsystem test
:components ((:module "file"
:serial t
:components ((:file "1")
(:file "2")
(:file "3")))))
You put this definition in a separate file and use ASDF to load and
compile it. Google can assist with the details. I think this is a
better solution, as it allows incremental recompilation and removes
clutter from your actual code.
> By trying to write this post, I seem to get the understanding that CL
> compilation is just a translation of a representation in source code,
> to a representation in byte-code or assembler, and that the resulting
> (.fas, .fasl) files only contain the things that where stated in the
> source files.
>
> I probably should see them more as object files, where the unresolved
> dependencies must be resolved by the programmer, by using an
> overarching system construction file which loads the necessary files
> and then calls from the file which is the application, the main
> routine.
>
> Is this reasoning correct ? Are there other sources than the CLHS
> about system construction ?
>
> Jurgen
CL compilation is not necessarily just a translation to another format.
As you saw above, compilation can involve the execution of code, e.g.
through the LOAD-TIME-VALUE and EVAL-WHEN forms. Several other forms
(like DEFPACKAGE and IN-PACKAGE) expand into EVAL-WHEN forms and so have
similar behavior.
"jurgen_defurne" <··············@pandora.be> writes:
> By trying to write this post, I seem to get the understanding that CL
> compilation is just a translation of a representation in source code,
> to a representation in byte-code or assembler, and that the resulting
> (.fas, .fasl) files only contain the things that where stated in the
> source files.
More or less, but it's more sophisticated.
You have "phases" that are more or less independent. In each phase,
you have the opportunity to have code executed to produce the data
(the "source") used by the phase.
read time
---------
At read time, the data consumed is the characters in the source
stream. It is parsed according to the lisp reader algorithm
described chapter 2 and 23. Reading (with CL:READ) produces ONE
lisp object, possibly compound, like a list of sublists and atoms.
This involves reader macros, either standard ones, or used defined
reader macros. These macros may or may not consume characters from
the input stream, and may or may not return one lisp object to be
considered as read. In doing so, these reader macro may do any
kind of processing, having access to a whole Common Lisp
environment.
macro-expansion time
--------------------
At macro expansion time, the input data is a form, whose CAR is the
name of the macro, and the macro can do whatever it wants with this
form and should produce a form as result, to be used in place of
the original form. You can also have access to the
macro-expansion time thru the MACRO-EXPANSION-HOOK, which allows a
user defined function to be called for each macro expansion. In
doing so, these macros and macro-expansion hooks may do any kind of
processing, having access to a whole Common Lisp environment.
compilation time
----------------
At compilation time, the input is forms, and the output is anything
adequat for the implementation. It might be a simplier forms, if
only the minimal compilation is implemented, or any kind of
"processor" code. You can define compiler-macros that are called
to substitute an _function_ call form by another form, before the
object code is generated by the compiler. In doing so, these
compiler macros may do any kind of processing, having access to a
whole Common Lisp environment.
execution time
--------------
At execution time, well, your program is executing with whatever
input it wants, and produces any output it wants. In doing so,
your functions may do any kind of processing, having access to a
whole Common Lisp environment.
Now the real time when these phases are executed can overlap, and in
the case of macro-expansion time, usually overlap either compilation
time or execution time.
Notice how I write "A whole Common Lisp environment". This is not
necessarily the same CL environment used in the various phases. If
you execute these different phases in the same lisp image, usually
it's the same CL environment. But if you read and compile a file in
one CL image, and then read and execute the .fasl file in another CL
image, then you'll have two distinct CL environments, and changes you
made in the CL environment while compiling won't be preserved in the
CL environment while executing, if you don't do anything special to
this effect.
The definitions of the CL:DEF* operators ensure that the changes
_they_ make to the CL environment will be in effect at execution time,
but usually not necessarily at compilation or macro-expansion time,
much less at read time. As mentionned, in other answers, you can use
EVAL-WHEN to have these effect taken into account in a different
phase.
> I probably should see them more as object files, where the unresolved
> dependencies must be resolved by the programmer, by using an
> overarching system construction file which loads the necessary files
> and then calls from the file which is the application, the main
> routine.
>
> Is this reasoning correct ?
Not really. You could rather see the .fasl files as scripts built by
the compilation phase to produce in the execution environment the
changes specified by the "source". It's already a running program,
used to build the eventual program.
(See for example the description of LOAD-TIME-VALUE in CLHS).
> Are there other sources than the CLHS
> about system construction ?
I don't know. I'd have to use google. I'll let you do so, there may
be some paper on this subject...
And now, for some fun:
C/USER5[57]> (cat "/tmp/p.lisp")
(eval-when (:compile-toplevel :load-toplevel :execute)
(defparameter *counter* 0))
(set-macro-character #\^ (lambda (stream char)
(declare (ignore stream char))
(incf *counter*)
(format *trace-output*
"~&Reading the caret number ~A~%"
*counter*)
*counter*))
(defvar *fun* (list (quote ^) (quote ^)))
(defmacro m ()
(format *trace-output* "~&At macro expansion time~%")
(when ^
`(progn
(format *trace-output* "~&At run time from (m)~%")
^)))
(print (m))
(defun f ()
(format *trace-output* "~&At run time from (f)~%")
(format *trace-output* "~&Read: ~S~%" (read-from-string "(^ ^)"))
(list (m) ^))
(print (f))
C/USER5[58]> (load"/tmp/p.lisp")
;; Loading file /tmp/p.lisp ...
Reading the caret number 1
Reading the caret number 2
Reading the caret number 3
Reading the caret number 4
At macro expansion time
At run time from (m)
4
Reading the caret number 5
At macro expansion time
At run time from (f)
Reading the caret number 6
Reading the caret number 7
Read: (6 7)
At run time from (m)
(4 5)
;; Loaded file /tmp/p.lisp
T
C/USER5[59]> (compile-file "/tmp/p.lisp")
;; Compiling file /tmp/p.lisp ...
Reading the caret number 1
Reading the caret number 2
Reading the caret number 3
Reading the caret number 4
At macro expansion time
Reading the caret number 5
At macro expansion time
;; Wrote file /tmp/p.fas
0 errors, 0 warnings
#P"/tmp/p.fas" ;
NIL ;
NIL
C/USER5[60]> (load"/tmp/p.fas")
;; Loading file /tmp/p.fas ...
At run time from (m)
4
At run time from (f)
Reading the caret number 1 ; here you can see that we have a different
Reading the caret number 2 ; environment, since the *counter* have been
Read: (1 2) ; reset by loading the .fas (defparameter)
At run time from (m) ; but the object read by ^ in the macro
; and function were built in another
(4 5) ; environment where it was already 4 and 5.
;; Loaded file /tmp/p.fas
T
C/USER5[61]>
--
__Pascal Bourguignon__ http://www.informatimago.com/
Un chat errant
se soulage
dans le jardin d'hiver
Shiki
On Feb 14, 5:56 pm, "jurgen_defurne" <··············@pandora.be>
wrote:
> The application I am writing is divided up into several packages,
> which I then tie together in another source file.
>
> Example :
> (load "/file/1")
> (load "/file/2")
> (load "/file/3")
>
> (use-package "PACKAGE-1")
> (use-package "PACKAGE-2")
>
> ;;; Add code here which uses exported interfaces from used
> packages.
>
> However, I think that I may be wrong here to add the loading of the
> files in this source file, because when I try to compile this file,
> this fails. Using CLISP as my example, when I then let CLISP execute
> the load functions (using -x "(load "/file/1")"), and then compile it
> (compile-file "source"), it succeeds partially. When I additionally
> load the source file, and then execute (compile-file), the compilation
> goes through completely.
>
> If the way I express it seems a bit awkward, it is because the way the
> system construction works is a bit counter-intuitive, for me at least,
> but also because there seems to be no good explicit description of how
> it all should tie together.
>
> The way this all works is also tied in with toolkits like SLIME and
> ILISP, which then give for me the same counterintuitive results.
>
> E.g. if I load the file in Emacs, which contains the load and use-
> package statements, and I execute a command to compile the whole file
> (C-c C-k), I get errors when external objects are referenced. When I
> execute the statements in the source file one by one (C-c C-c), then I
> get the desired results.
>
> By trying to write this post, I seem to get the understanding that CL
> compilation is just a translation of a representation in source code,
> to a representation in byte-code or assembler, and that the resulting
> (.fas, .fasl) files only contain the things that where stated in the
> source files.
>
> I probably should see them more as object files, where the unresolved
> dependencies must be resolved by the programmer, by using an
> overarching system construction file which loads the necessary files
> and then calls from the file which is the application, the main
> routine.
>
> Is this reasoning correct ? Are there other sources than the CLHS
> about system construction ?
>
> Jurgen
I also noted that this behavious IS implementation dependant.
If CLISP is used, I do need to load the package files separately
before I can compile without errors, SBCL however, executes everything
what is in the file when I do (compile-file "path").
That is of course a strong reason to use asdf, because it eliminates
these differences for the user.
Jurgen