From: Steven D. Majewski
Subject: more package madness...
Date: 
Message-ID: <E5wy7s.IE2@murdoch.acc.Virginia.EDU>
 Thanks for everyone who responded to my previous message and 
explained the reasoning behind Lisp's reader interning symbols
that I didn't want interned.  I now understand why it's implemented
that way, but I can't say it's a desirable design feature. 
( However, I suspect that Lisp packages were designed for larger
  grained packaging than I've been trying to do. How many packages
  are typical in a large lisp application ? ) 

Another example I recently ran into is that I find I can't say:

( if ( find-package "XXX" )
     ( defun function that uses XXX:func  ... ) 
     ( defun function that simulates XXX:func without reference to 
	that package  ... ))
 

  In this case, the external package has interfaces to drivers to
talk to a spectrometer and acquire data. The replacement functions
are so that I can test the graphics update and interface of the 
program on a machine without the data acquisition hardware. 

  However, even when the 'else' condition holds, I still get errors
from the reader reading the first clause and complaining about no 
package named "XXX" . 


 I first changed this conditional to use *features*, and right now 
I'm changing things so that I define an entire fake "XXX" simulation
package. This latter is probably the better solution to the problem,
however, now that I uncovered this problem in a simple example, I 
recognize that this sort of use of undefined symbols in a conditional
was probably the source of some similar errors that I had in other modules.
I those cases, the problem was disguised a bit more subtly. I believe
I just gave up on using conditional code in some of those cases and just
hardwired in the requirement that requires the other packages. 


I'm not convinced that the fact that the lisp reader does partial 
evaluation of statements with side effects in (possibly) the global
environment is an unmixed blessing as a feature. 


 I also find I've having trouble dealing with compile-time vs eval-time
issues: typically, I find I have to load all of my code, and then 
compile it to get it to compile without error. Perhaps it will compile
if I compile it in the "proper" order, but what I *can't* do is compile
each file in the directory in directory order.  


 "Load everything and compile" is an acceptable solution, but I wonder
whether my problems are a sign of my missing some other subtle word-
of-mouth, undocumented Lisp structuring principle! 



---|  Steven D. Majewski   (804-982-0831)  <·····@Virginia.EDU>  |---
---|  Department of Molecular Physiology and Biological Physics  |---
---|  University of Virginia             Health Sciences Center  |---
---|  P.O. Box 10011            Charlottesville, VA  22906-0011  |---
         By doing just a little every day, you can gradually 
                let the task completely overwhelm you.

From: Thomas A. Russ
Subject: Re: more package madness...
Date: 
Message-ID: <ymizpwzb0po.fsf@hobbes.isi.edu>
In article <...> ·····@elvis.med.Virginia.EDU (Steven D. Majewski) writes:
 >
 > Another example I recently ran into is that I find I can't say:
 > 
 > ( if ( find-package "XXX" )
 >      ( defun function that uses XXX:func  ... ) 
 >      ( defun function that simulates XXX:func without reference to 
 > 	that package  ... ))

Yes, this is indeed a problem.  There are three ways around it:

  (1)  Use a feature :XXX and the code conditionalization:

       #+:XXX  ( defun function that uses XXX:func  ... ) 
       #-:XXX  ( defun function that simulates XXX:func without reference to 
           	that package  ... ))

	Since #+ and friends are read time constructs, this will get you
	past reader-related package problems.

        I think this is the best solution.

  (2) Instead of calling XXX:func, have your code "use" package XXX.
      Then there won't be any package prefix to worry about since
      XXX:func can be referenced simply as func in the first function.
      This may, however not be practical because of name conflicts.
      Also, you will need to make sure that the use-package call is
	suitably conditionalized.

  (3) Gross Hack:  Construct the symbol at runtime using intern and then
      funcall it.
      This is really only something that makes sense for limited
      applications.  I've used it in my init file, but that's about the
      only place I would think of using it.

 >  I first changed this conditional to use *features*, and right now 
 > I'm changing things so that I define an entire fake "XXX" simulation
 > package. This latter is probably the better solution to the problem,

Yes.

...

 >  I also find I've having trouble dealing with compile-time vs eval-time
 > issues: typically, I find I have to load all of my code, and then 
 > compile it to get it to compile without error. Perhaps it will compile
 > if I compile it in the "proper" order, but what I *can't* do is compile
 > each file in the directory in directory order.  
 > 
 >  "Load everything and compile" is an acceptable solution, but I wonder
 > whether my problems are a sign of my missing some other subtle word-
 > of-mouth, undocumented Lisp structuring principle! 

Well, the only places where you can get into trouble have to do with
variable special declarations and declamations, macros, structures and
in-line code.  If you define methods that reference classes not defined
in the same file, then that can cause problems also.

One general principle is to organize code spread across several files as
follows:

  variables.lisp    ;; All the defvars and defparameters
  macros.lisp       ;; All the macro definitions and small in-line functions
  structures.lisp   ;; Defstruct forms.
  classdefs.lisp    ;; CLOS class definitions.
  code-1.lisp
  code-2.lisp
etc.

Then you compile and load the variables, compile and load the macros,
compile and load the structures, compile and load the CLOS classdefs and
then compile and load the rest of the code.

There are also some tools around that attempt to figure out dependencies
among files.  These are often used with DEFSYSTEM utilities to set up
proper compilation dependencies.


-- 
Thomas A. Russ,  USC/Information Sciences Institute          ···@isi.edu    
From: Marco Antoniotti
Subject: Re: more package madness...
Date: 
Message-ID: <s087mk25cai.fsf@crawdad.ICSI.Berkeley.EDU>
As Rainer said: USE A DEFSYSTEM.

I will also add:  use Mark Kantrowitz DEFSYSTEM.  It runs on every
common lisp platform and is "declarative" in nature.

-- 
Marco Antoniotti - Resistente Umano
From: Rainer Joswig
Subject: Re: more package madness...
Date: 
Message-ID: <joswig-ya023180002102970005030001@news.lavielle.com>
In article <··········@murdoch.acc.Virginia.EDU>,
·····@elvis.med.Virginia.EDU (Steven D. Majewski) wrote:

> ( However, I suspect that Lisp packages were designed for larger
>   grained packaging than I've been trying to do. How many packages
>   are typical in a large lisp application ? ) 
> Another example I recently ran into is that I find I can't say:

My MCL image uses 30 packages, my ACL at home comes up
with 40 packages, and my Symbolics has 120 packages preloaded.

> 
> ( if ( find-package "XXX" )
>      ( defun function that uses XXX:func  ... ) 
>      ( defun function that simulates XXX:func without reference to 
>         that package  ... ))
>  
> 
>   In this case, the external package has interfaces to drivers to
> talk to a spectrometer and acquire data. The replacement functions
> are so that I can test the graphics update and interface of the 
> program on a machine without the data acquisition hardware. 
> 
>   However, even when the 'else' condition holds, I still get errors
> from the reader reading the first clause and complaining about no 
> package named "XXX" .

This is a usual problem.

Package handling is best being done in a separate file.
Maybe you can setup a new package and import/export symbols
selectively.

> recognize that this sort of use of undefined symbols in a conditional
> was probably the source of some similar errors that I had in other modules.
> I those cases, the problem was disguised a bit more subtly. I believe
> I just gave up on using conditional code in some of those cases and just
> hardwired in the requirement that requires the other packages. 

Make sure that your symbols have valid functions, etc. Once
you are writing your software and you configure your
software, you can map over packages and look whether
symbols have values, name a package or actually name
a function. Remember, in Lisp these are informations
you can actually use and compute with.

>  I also find I've having trouble dealing with compile-time vs eval-time
> issues: typically, I find I have to load all of my code, and then 
> compile it to get it to compile without error. Perhaps it will compile
> if I compile it in the "proper" order, but what I *can't* do is compile
> each file in the directory in directory order.

No, you can't that in general. So, use a defsystem to 
write down the dependencies of your code. Simple defsystems
just will compile and load your code in sequence.
Others will use more information and compile only
a small subset of files, if you have made changes to the source.
Others will let you add patches to your system, and load
th patches individual. On Genera, for example, you
can say "I like to load the released version", or
"I like to load the system in version 60.103 (the current devo version of
CL-HTTP)". A real wizard with a real Lisp system can
add patches to a software system for some months until
he needs a complete recompile. With some clever
coding a Lisp machine **development** environment can stay up for
weeks and months. Remember how often people tend to restart
their development environments. Every time they need to do
it they lose state and time. Don't exit Lisp, restart Lisp,
recompile the code, rerun your application.

>  "Load everything and compile" is an acceptable solution, but I wonder
> whether my problems are a sign of my missing some other subtle word-
> of-mouth, undocumented Lisp structuring principle! 

Use a defsystem. Not using defsystem is a sign of poor practice.
The more you state declaratively about your
code, the more handling can be done by the Lisp system.
The Lisp environment is being built to make this
possible. If I would write some critical code, especially
with hard constraints on development time, add
automatic regression testing to ensure that your code
survives your changes and patches. Lisp is good for
projects where you have *no* time, because it makes
it possible to add your special purpose support code
that reduces the edit/compile/load/run/test/debug/document
cycles to a minimum. You want to use an editor
which lets you compile only the changes in the editor
windows, etc.

-- 
http://www.lavielle.com/~joswig/
From: Barry Margolin
Subject: Re: more package madness...
Date: 
Message-ID: <5ejgms$3vk@pasilla.bbnplanet.com>
In article <··········@murdoch.acc.Virginia.EDU>,
Steven D. Majewski <·····@elvis.med.Virginia.EDU> wrote:
> Thanks for everyone who responded to my previous message and 
>explained the reasoning behind Lisp's reader interning symbols
>that I didn't want interned.  I now understand why it's implemented
>that way, but I can't say it's a desirable design feature. 

In fact, many Lisp wizards agree that if we had it to do over, it probably
would not be done the same way.  And if you look at the Lisp-like languages
that have been designed since (e.g. Dylan) you'll see that they did not
include anything like the Common Lisp package system, but instead have
module systems that implement multiple mappings from names to
function/variable bindings.

>( However, I suspect that Lisp packages were designed for larger
>  grained packaging than I've been trying to do. How many packages
>  are typical in a large lisp application ? ) 

Yes, packages are intended to be very coarse-grained.  The entire Symbolics
Genera OS had 20-30 packages.  Applications tend to have 1 or 2 packages,
and rarely more than 5.  The general idea is to keep your application's
internal symbols from conflicting with other applications, since you have
no control over developers of other applications.  Within an application
development team you can use non-language techniques to prevent conflicts,
so you can all work in the same package.

>Another example I recently ran into is that I find I can't say:
>
>( if ( find-package "XXX" )
>     ( defun function that uses XXX:func  ... ) 
>     ( defun function that simulates XXX:func without reference to 
>	that package  ... ))

In the then-clause, you need to use

(funcall (find-symbol "FUNC" "XXX") ...)

rather than

(XXX:func ...)

This way, the symbol lookup occurs at run time only when that function is
called, rather than at read time regardless of the result of the
FIND-PACKAGE.

> I also find I've having trouble dealing with compile-time vs eval-time
>issues: typically, I find I have to load all of my code, and then 
>compile it to get it to compile without error. Perhaps it will compile
>if I compile it in the "proper" order, but what I *can't* do is compile
>each file in the directory in directory order.  

This is why DEFSYSTEM was invented.  We didn't succeed in agreeing on a
version for inclusion in the ANSI standard, but there are some public
versions available in the Lisp archives.  The only thing that came out of
that was the WITH-COMPILATION-UNIT macro, which allows you to compile a
bunch of files that are mutually recursive and avoid unknown
function/variable warnings from the files that make forward references.

> "Load everything and compile" is an acceptable solution, but I wonder
>whether my problems are a sign of my missing some other subtle word-
>of-mouth, undocumented Lisp structuring principle! 

In most cases you should be able to compile then load each file in order.
You generally need to compile/load package definition files first, then
files containing macro definitions, and finally function definitions.  The
only reason you might need to load function definitions before compiling
something is if a macro calls one of its functions at expansion time.  This
is usually handled by putting those "macro helper" functions in the macro
definition file, and wrapping them with (eval-when (:compile-toplevel
:load-toplevel :execute) ...), which will make the definitions available at
macro expansion time.
-- 
Barry Margolin
BBN Corporation, Cambridge, MA
······@bbnplanet.com
(BBN customers, call (800) 632-7638 option 1 for support)