From: Stephen Intille
Subject: Packages and file organization tips
Date: 
Message-ID: <5vmp9r$clc$1@hpns1.media.mit.edu>
Hi,

I'm doing a fair amount of lisp programming on a large system and I'm
finding organizing my subsystems into components where package works
correctly and the compiler doesn't spit out warnings to be somewhat
annoying.

I'm curious if people have  package/file naming organizational 
conventions they use that work well.

For example, say I have three packages where P2 depends on P1 and P3
depends on P1 and P2. Each package has macro definitions,
class/structure definitions, global variables (within package), and
function definitions (several files worth, where some functions in
file A refer to functions defined in file B and visa versa). Let's say
there are also a few globals to import from cl-user (e.g. stream
path). There is also the system definition itself to worry about for
each package and then for the entire system.

Starting from cl-user, is there a preferred way to lump this stuff
into files so that I can easily specify dependencies in defsystem and
not get errors on loading/compiling while still keeping related
components together?

For instance, it seems the way to avoid errors is have a packages.lisp
file that contains all package definitions and for each system a
system definition file. Load the packages then the system
definition. Then for each package, load an imports file, then a macros
file, then class file, then globals, then functions and to specify
that all the function files depend upon one another and that each
function file depends on the macro, class, and globals files.

My problem with this is that some components aren't together. E.g.
the package definitions are not in the packages and some class access
functions are in different files from the class definitions. The
functions for a particular class are spread out in several files,
making it harder to use just that class functionality in other
code. As I work, more circularity tends to be introduced between the
function files, which causes more warnings.

My instinct is to divide the code up into even smaller package units,
but then I'm typing package names all over the place. 

Suggestions (and even better a defsystem example!) appreciated.

Stephen





-- 
Stephen Intille
Research Assistant
MIT Media Lab
E15-368C, 20 Ames St. 

From: David Gadbois
Subject: Re: Packages and file organization tips
Date: 
Message-ID: <6016d6$q8o$1@news3.texas.net>
Stephen Intille <·······@ongar.media.mit.edu> wrote:
>I'm doing a fair amount of lisp programming on a large system and I'm
>finding organizing my subsystems into components where package works
>correctly and the compiler doesn't spit out warnings to be somewhat
>annoying.

The package system is one of the more hazardous parts of Common Lisp.
One of the big dangers is getting carried away with package.  I
generally see packages as very coarse-grained.  They just don't work
well at defining fine units of modularity.  For example, it would work
OK to use a separate package for a whole application program.  It
doesn't work so well to put something smaller like a queue abstract
data type in a package of its own.

If your package use and import relations have cycles, I'd rethink the
organization so as to break the cycles.  With enough care, it is
possible to set up cyclic package relations properly, but it is hard
to imagine a situation where it would be worth the effort and the
maintenance hassle.  If you can't do it with just DEFPACKAGE, maybe it
shouldn't be done.

As to how to set up a system to use packages, I always put all the
package definitions in a separate subsystem that gets compiled and
loaded before anything else.  The files containing the package and
system definitions go into the CL-USER package.

--David Gadbois
From: Marco Antoniotti
Subject: Re: Packages and file organization tips
Date: 
Message-ID: <scfwwkbzrt8.fsf@infiniti.PATH.Berkeley.EDU>
·······@cyc.com (David Gadbois) writes:

> 
> Stephen Intille <·······@ongar.media.mit.edu> wrote:
> >I'm doing a fair amount of lisp programming on a large system and I'm
> >finding organizing my subsystems into components where package works
> >correctly and the compiler doesn't spit out warnings to be somewhat
> >annoying.
> 
> The package system is one of the more hazardous parts of Common Lisp.
> One of the big dangers is getting carried away with package.  I
> generally see packages as very coarse-grained.  They just don't work
> well at defining fine units of modularity.  For example, it would work
> OK to use a separate package for a whole application program.  It
> doesn't work so well to put something smaller like a queue abstract
> data type in a package of its own.

I beg to differ.  I believe this is one of the levels of granularity
that is best supported by the Common Lisp package system.  I feel that
the "application level" use of packages is way too coarse and leads
to poor reuse of code and resources.

As a matter of fact I did write a queue and a priority-queue ADT in CL
(they are avilable in the CMU AI.Repository).  I probably should look
back at them, but they use the package/adt organization and work like
a charm.  As a matter of fact I also tend to shadow COMMON-LISP
functions (e.g. "DELETE") and force their use only through the proper
package qualifier.

My suggestion is that the time spent into understanding the package
system is time well spent.  Not doing so, would be like learning Ada
without reading about packages.

-- 
Marco Antoniotti
==============================================================================
California Path Program - UC Berkeley
Richmond Field Station
tel. +1 - 510 - 231 9472