I'm in the process of trying to put my files into packages to keep my
code modular. I have a file called util which has many general utility
functions. So, if I want to make this into a package called util, I need
to have a statement saying export ... (where ... is every function
defined in the file), don't I? This is quite tedious and also seems
fragile - if I add a function but forget to add it to the export
statement, then I won't be able to access it from outside the package; I
also have to make sure if I remove any functions, that I remove them
from the export statement... Is there a better way to do this?
Lowell
Lowell wrote:
> I'm in the process of trying to put my files into packages to keep my
> code modular. I have a file called util which has many general utility
> functions. So, if I want to make this into a package called util, I need
> to have a statement saying export ... (where ... is every function
> defined in the file), don't I? This is quite tedious and also seems
> fragile - if I add a function but forget to add it to the export
> statement, then I won't be able to access it from outside the package;
you'll know in about two seconds when you get a compiler warning from
code trying to call the function.
> also have to make sure if I remove any functions, that I remove them
> from the export statement...
but they are harmless if left behind.
--
kenny tilton
clinisys, inc
http://www.tilton-technology.com/
---------------------------------------------------------------
"Career highlights? I had two. I got an intentional walk from
Sandy Koufax and I got out of a rundown against the Mets."
-- Bob Uecker
Kenny Tilton <·······@nyc.rr.com> writes:
> Lowell wrote:
> > I'm in the process of trying to put my files into packages to keep
> > my code modular. I have a file called util which has many general
> > utility functions. So, if I want to make this into a package called
> > util, I need to have a statement saying export ... (where ... is
> > every function defined in the file), don't I? This is quite tedious
> > and also seems fragile - if I add a function but forget to add it to
> > the export statement, then I won't be able to access it from outside
> > the package;
>
> you'll know in about two seconds when you get a compiler warning from
> code trying to call the function.
>
> > also have to make sure if I remove any functions, that I remove them
> > from the export statement...
>
> but they are harmless if left behind.
I concur with Kenny's remarks, but would add:
Although package definitions change a lot during initial development,
it's a bad idea to casually add new symbols to a package once you have
let anyone else use it. People who use packages may depend on them
exporting a precise set of symbols and may not take it kindly if you
just add more on every software release.
You should think hard about your interface and try to design it so
that it is robust against future change; if it has to change, you
should consider whether a new package name would work better.
Kent M Pitman wrote:
>You should think hard about your interface and try to design it so
>that it is robust against future change;
>
If you can do this then you have my admiration. In my experience trying
to design for future change is near impossible, and leads to very
brittle code as I try to shoehorn the stuff I actually need into what I
thought the design should be some time ago.
For some applications backward compatibility may be absolutely
essential, but I tend to find this comes at the cost of rapid devlopment
and a clean design.
Am I right in thinking that the linux kernel developers don't maintain
backward compatability?
Alex McGuire <····@alexmcguire.com> writes:
> Am I right in thinking that the linux kernel developers don't maintain
> backward compatability?
Depends.
Backward compatability in the kernel-user interface is maintained for
a long time.
Backward compatability in the internal interfaces (device drivers,
etc) is broken regularly.
--
Lars Brinkhoff, Services for Unix, Linux, GCC, PDP-10, HTTP
Brinkhoff Consulting http://www.brinkhoff.se/
Lars Brinkhoff <·········@nocrew.org> writes:
> Alex McGuire <····@alexmcguire.com> writes:
> > Am I right in thinking that the linux kernel developers don't maintain
> > backward compatability?
>
> Depends.
>
> Backward compatability in the kernel-user interface is maintained for
> a long time.
hahahahahahahahahahahahahaha!
(Uh oh, you weren't serious, were you?)
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
Alex McGuire <····@alexmcguire.com> writes:
> Kent M Pitman wrote:
>
> >You should think hard about your interface and try to design it so
> > that it is robust against future change;
> If you can do this then you have my admiration. In my experience
> trying to design for future change is near impossible,
To be clear, this isn't the entirety of what I said.
What I said was to think carefully about what you're doing.
Not for tomorrow but for today.
Just for example, don't release a package with a bunch of structure
accessors but missing the creator, or vice versa. This doesn't require
insight, just common sense.
During the design of ISLISP (www.islisp.info), there was a brief period of
time during which some of the participants advocated having only a subset
of the six operators >, <, /=, =, <=, and >= ... as if this kind of
minimality was somehow good. IMO, it just begs some user to convince you
that you goofed and to make you add more later, probably breaking someone's
package that added it in as a stop-gap. Better to not be silly and just to
add them from the outset. Similar near-design-errors were made by the
same group wanting to have only SIN and COS but not TAN because some
claimed it was computable from the other two. Ditto WHEN and UNLESS,
WHILE and UNTIL--don't just add one without the other because it's well-known
both are popular. If you fail to add both, some of your users _will_ add
their own and they will hate you later when you finally see the light and
add the missing symbol yourself because they'll get symbol clashes and have
to conditionalize their code.
I'm not talking about ESP, just common sense.
And the other side of what I said is that if you make a mistake,
consider making a new package to fix it. So when you decide you need
UNLESS along with WHEN, don't just re-release the JDOE\'S-CONDITIONALS
package, but instead consider a JDOE\'S-EXTENDED-CONDITIONALS or
JDOE\'S-CONDITIONALS-2004 package that contains the larger set, and
continue to maintain the JDOE\'S-CONDITIONALS package with the old
interface, if there is no compelling reason to break it.
> > also have to make sure if I remove any functions, that I remove them
> > from the export statement...
>
> but they are harmless if left behind.
Otoh it's not uncommon for an exports list to substitute for
documentation (i.e. if it's exported you can use it), so it pays to
update the exports.
I'd recommend that you maintain all your packages in a single
"package.lisp" file, using the defpackage macro rather than calls to
the function export. The result will be much more readable.
- nick
By this, do you mean to use (defpackage .... (:export ...)) rather than
having the export following the defpackage form?
Lowell
Nick Levine wrote:
> I'd recommend that you maintain all your packages in a single
> "package.lisp" file, using the defpackage macro rather than calls to
> the function export.
Lowell <······@cs.ubc.ca> wrote in message news:<············@mughi.cs.ubc.ca>...
> By this, do you mean to use (defpackage .... (:export ...)) rather than
> having the export following the defpackage form?
Yes.
- n
Nick Levine wrote:
>>>also have to make sure if I remove any functions, that I remove them
>>>from the export statement...
>>
>>but they are harmless if left behind.
>
>
> Otoh it's not uncommon for an exports list to substitute for
> documentation (i.e. if it's exported you can use it), so it pays to
> update the exports.
But the OP was about a personal utilities collection, so I decided not
to add:
"hey, don't complain about having to maintain (:export...) manually.
Deciding what in a package is part of the intended interface and what is
internal-use-only is not something the compiler can help with. The big
decision is which is which, the easy part is tidying up the export list."
ie, we agree.
--
kenny tilton
clinisys, inc
http://www.tilton-technology.com/
---------------------------------------------------------------
"Career highlights? I had two. I got an intentional walk from
Sandy Koufax and I got out of a rundown against the Mets."
-- Bob Uecker
Lowell wrote:
> I'm in the process of trying to put my files into packages to keep my
> code modular. I have a file called util which has many general utility
> functions. So, if I want to make this into a package called util, I need
> to have a statement saying export ... (where ... is every function
> defined in the file), don't I? This is quite tedious and also seems
> fragile - if I add a function but forget to add it to the export
> statement, then I won't be able to access it from outside the package; I
> also have to make sure if I remove any functions, that I remove them
> from the export statement... Is there a better way to do this?
>
> Lowell
>
First of all, check DEFPACKAGE. Secondly I organize my code using the
following scheme:
One DEFPACKAGE, one file.
In your cased, I'd have a file "util-pkg.lisp" that contains the
DEFPACKAGE form. Your "util.lisp" file will then just start with
(in-package "LOWELL.CL.UTIL")
The rest of the modularization is left to you.
If the system is big enough then I'd rather partition code in several
files and let DEFSYSTEM (MK, ASDF, your implementation one) handle the rest.
--
Cheers
Marco
Marco Antoniotti <·······@cs.nyu.edu> writes:
> First of all, check DEFPACKAGE. Secondly I organize my code using the
> following scheme:
>
> One DEFPACKAGE, one file.
Why? Just curious.
> In your cased, I'd have a file "util-pkg.lisp" that contains the
> DEFPACKAGE form. Your "util.lisp" file will then just start with
>
> (in-package "LOWELL.CL.UTIL")
Given your one-file rule, why not just put the DEFPACKAGE form in
util.lisp itself? (Again, just curious--I'm sure you have some
reason.)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel wrote:
> Marco Antoniotti <·······@cs.nyu.edu> writes:
>
>
>>First of all, check DEFPACKAGE. Secondly I organize my code using the
>>following scheme:
>>
>>One DEFPACKAGE, one file.
>
>
> Why? Just curious.
>
>
>>In your cased, I'd have a file "util-pkg.lisp" that contains the
>>DEFPACKAGE form. Your "util.lisp" file will then just start with
>>
>> (in-package "LOWELL.CL.UTIL")
>
>
> Given your one-file rule, why not just put the DEFPACKAGE form in
> util.lisp itself? (Again, just curious--I'm sure you have some
> reason.)
You can. I just like to have things separate. Note that it is a matter
of balance. If you have packages exporting only a few symbols, then it
may be an overkill, however, packages are usually "mid sized" (whatever
that means) and the factoring helps a lot in the long run. Personally I
like looking at a directory listing and figure out how many packages are
there.
As per your example, suppose that you keep adding utilities to your
file, to the point where it becomes more useful splitting it (say,
"list-utils" and "string-utils", just to name what seems a common
thing). If you had your package file separated then the split is -
quote - easier - unquote (quotes necessary as YMMV).
Finally, another thing I do is to use separated :EXPORT lists in
DEFPACKAGE to group related symbols. It works nicely for me and it adds
modularity. E.g.
(defpackage "LOWELL.CL.UTIL" (:use "COMMON-LISP")
(:export ; List related exports
#:my-list-append)
(:export ; String related exports
#:yet-another-string-split
#:strcat))
Cheers
--
Marco
> > Given your one-file rule, why not just put the DEFPACKAGE form in
> > util.lisp itself? (Again, just curious--I'm sure you have some
> > reason.)
>
> You can. I just like to have things separate.
There's a stronger reason than this, namely that it's not usually
expected to see more than one (in-package) form per file. It's not
actually wrong, but programmers taking a brief look through a file
might not notice a switch in *package*, and at least one integrated
editor (lispworks) would have to be hand-held past the switch as it
does not parse the whole file looking for such.
> Finally, another thing I do is to use separated :EXPORT lists in
> DEFPACKAGE to group related symbols. It works nicely for me and it adds
> modularity.
That's good, as is Joe's advice not to "over-package".
- n
···@ravenbrook.com (Nick Levine) writes:
> > > Given your one-file rule, why not just put the DEFPACKAGE form in
> > > util.lisp itself? (Again, just curious--I'm sure you have some
> > > reason.)
> >
> > You can. I just like to have things separate.
>
> There's a stronger reason than this, namely that it's not usually
> expected to see more than one (in-package) form per file. It's not
> actually wrong, but programmers taking a brief look through a file
> might not notice a switch in *package*, and at least one integrated
> editor (lispworks) would have to be hand-held past the switch as it
> does not parse the whole file looking for such.
Emacs (ILISP and, I think, ELI) also have that limitation. But I was
talking about putting the single DEFPACKAGE in the file with the IN-PACKAGE:
(defpackage #:foo
(:use #:common-lisp)
(:export #:bar #:baz))
(in-package #:foo)
;; code ...
My inclination, based on not a ton of real-world experience, and
probably over influenced by my Java-induced brain-damage, has been to
have one file per "module" where each module gets its own package
which is defined in the file containing the code for that package as
shown above. Which doesn't run afoul of the one IN-PACKAGE per file
either.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> My inclination, based on not a ton of real-world experience, and
> probably over influenced by my Java-induced brain-damage, has been to
> have one file per "module" where each module gets its own package
> which is defined in the file containing the code for that package as
> shown above. Which doesn't run afoul of the one IN-PACKAGE per file
> either.
It's not a bad guidline for simple modules, but I'd leave it as just
that: a *guidline* for *simple* modules. Leaving yourself the option
of factoring some portion of a module into its own file (or even a
crossection of more than one module) is a good thing. Just ask the
AOP folks :-)
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
> Emacs (ILISP and, I think, ELI) also have that limitation. But I was
> talking about putting the single DEFPACKAGE in the file with the IN-PACKAGE:
>
> (defpackage #:foo
> (:use #:common-lisp)
> (:export #:bar #:baz))
>
> (in-package #:foo)
>
> ;; code ...
Right you are. But - somewhat pedantically - in what package is the defpackage form?
-nick
Nick Levine <···@ravenbrook.com> wrote:
+---------------
| > Emacs (ILISP and, I think, ELI) also have that limitation. But I was
| > talking about putting the single DEFPACKAGE in the file with the IN-PACKAGE:
| >
| > (defpackage #:foo
| > (:use #:common-lisp)
| > (:export #:bar #:baz))
| >
| > (in-package #:foo)
| >
| > ;; code ...
|
| Right you are. But - somewhat pedantically - in what package is the
| defpackage form?
+---------------
Well, it had *better* be in a package that already USEs COMMON-LISP,
of course, or DEFPACKAGE might not be accessible (or be the wrong thing!).
But almost always at the beginning of loading files into a system you're
in COMMON-LISP-USER, although I have occasionally seen some code put an
explicit (cl:in-package :cl-user) before the DEFPACKAGE, just to be sure.
Which is also why everybody likes to use uninterned symbols to specify
the symbol names, to avoid polluting the COMMON-LISP-USER package (or
whatever package you're in). But you knew that... ;-}
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
>
> Which is also why everybody likes to use uninterned symbols to specify
> the symbol names, to avoid polluting the COMMON-LISP-USER package (or
> whatever package you're in). But you knew that... ;-}
Actually, I prefer to use strings instead.
--
Thomas A. Russ, USC/Information Sciences Institute
Thomas A. Russ <···@sevak.isi.edu> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > Which is also why everybody likes to use uninterned symbols to specify
| > the symbol names, to avoid polluting the COMMON-LISP-USER package (or
| > whatever package you're in). But you knew that... ;-}
|
| Actually, I prefer to use strings instead.
+---------------
Then how do you make your code portable between implementations that
have different default READTABLE-CASE conventions?!? The advantage
of uninterned symbols is that the strings they get interpreted as
(in DEFPACKAGE forms) are automatically of the "right" case, no matter
what the default READTABLE-CASE is...
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: Joe Marshall
Subject: Re: basic question about packages
Date:
Message-ID: <ekzfhya7.fsf@ccs.neu.edu>
····@rpw3.org (Rob Warnock) writes:
> Thomas A. Russ <···@sevak.isi.edu> wrote:
> +---------------
> | ····@rpw3.org (Rob Warnock) writes:
> | > Which is also why everybody likes to use uninterned symbols to specify
> | > the symbol names, to avoid polluting the COMMON-LISP-USER package (or
> | > whatever package you're in). But you knew that... ;-}
> |
> | Actually, I prefer to use strings instead.
> +---------------
>
> Then how do you make your code portable between implementations that
> have different default READTABLE-CASE conventions?!?
Why would anyone be concerned with porting code to non-compliant
Lisp implementations?
Joe Marshall <···@ccs.neu.edu> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > Thomas A. Russ <···@sevak.isi.edu> wrote:
| > +---------------
| > | Actually, I prefer to use strings instead.
| > +---------------
| >
| > Then how do you make your code portable between implementations that
| > have different default READTABLE-CASE conventions?!?
|
| Why would anyone be concerned with porting code to non-compliant
| Lisp implementations?
+---------------
Apparently some do. See the flamewars here last Fall (and several times
before that) about "modern mode" vs. readtable-case :invert, etc., etc.
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
>
> Thomas A. Russ <···@sevak.isi.edu> wrote:
> +---------------
> | ····@rpw3.org (Rob Warnock) writes:
> | > Which is also why everybody likes to use uninterned symbols to specify
> | > the symbol names, to avoid polluting the COMMON-LISP-USER package (or
> | > whatever package you're in). But you knew that... ;-}
> |
> | Actually, I prefer to use strings instead.
> +---------------
>
> Then how do you make your code portable between implementations that
> have different default READTABLE-CASE conventions?!? The advantage
> of uninterned symbols is that the strings they get interpreted as
> (in DEFPACKAGE forms) are automatically of the "right" case, no matter
> what the default READTABLE-CASE is...
Short answer: I don't.
But my code would be hosed anyway by having a different READTABLE-CASE,
since the "defpackage" operator itself is written in lower-case. If it
didn't get translated into upper-case by the reader, loading the file
wouldn't work anyway: It wouldn't be recognized as the correct symbol
anyway.
That does bring up an interesting omission from the ANSI CL standard,
namely a nice, convenient way to establish the readtable case for the
file in question. Something along the lines of IN-PACKAGE, which would
do the right thing in the file loader and compiler. I'm not quite sure
what I would want to call it, though.
Hmm, exactly how would a paranoid programmer handle something like this?
I though I could try something (ugly) with EVAL-WHEN and judicious use
of vertical bars:
(|CL|:|IN-PACKAGE| "COMMON-LISP-USER")
(|EVAL-WHEN| (:|EXECUTE| :|LOAD-TOPLEVEL| :|COMPILE-TOPLEVEL|)
(|SETQ| |*READTABLE*| (|COPY-READTABLE| |NIL|)))
At least *READTABLE* is bound by the LOAD and COMPILE-FILE functions so
that replacing it in this way is safe. One just needs to be careful not
to do (setf (readtable-case *readtable*) :upcase) inside the eval-when,
since that will change (mutate) the current readtable. That's why it
would be nice to have some clean syntax for doing this, maybe something
like
(use-readtable-case :preserve)
which one would, presumably need to write
(|USE-READTABLE-CASE| :|PRESERVE|)
for safety.
--
Thomas A. Russ, USC/Information Sciences Institute
Thomas A. Russ <···@sevak.isi.edu> wrote:
+---------------
| But my code would be hosed anyway by having a different READTABLE-CASE,
| since the "defpackage" operator itself is written in lower-case. If it
| didn't get translated into upper-case by the reader, loading the file
| wouldn't work anyway: It wouldn't be recognized as the correct symbol
| anyway.
+---------------
Uh... Have you not ever tried readtable-case :INVERT? A lot of people
seem to like it as a default. And it doesn't mess up the reading of
"defpackage". [Of course, in :invert mode you can't say DEFPACKAGE...
well, you can *say* it, but..]
+---------------
| That's why it would be nice to have some clean syntax for
| doing this, maybe something like
| (use-readtable-case :preserve)
+---------------
I think you will find :preserve to be very inconvenient compared to :invert.
See CLHS "23.1.2.1 Examples of Effect of Readtable Case on the Lisp Reader",
and note that :invert *preserves* mixed-case symbols.
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
···@ravenbrook.com (Nick Levine) writes:
> > Emacs (ILISP and, I think, ELI) also have that limitation. But I was
> > talking about putting the single DEFPACKAGE in the file with the IN-PACKAGE:
> >
> > (defpackage #:foo
> > (:use #:common-lisp)
> > (:export #:bar #:baz))
> >
> > (in-package #:foo)
> >
> > ;; code ...
>
> Right you are. But - somewhat pedantically - in what package is the defpackage form?
Yeah, yeah. And IN-PACKAGE too. I think I'm going to leave it to my
larger development/build process to ensure that *package* is properly
bound when I call LOAD or COMPILE-FILE. In fact by leaving it this way
I leave the option of playing games of loading these files with
*package* set to something where DEFPACKAGE and IN-PACKAGE aren't
necessarily CL:DEFPACKAGE, etc. Hmmmm. Not sure that's a great idea.
But it's an idea. ;-)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> Given your one-file rule, why not just put the DEFPACKAGE form in
> util.lisp itself? (Again, just curious--I'm sure you have some
> reason.)
I use the foo-package.lisp + foo.lisp scheme myself, so I can at least
tell you why I do it: in addition to just feeling "cleaner", it lets
you load the entire package scheme of a project, without having to
load all the (potentially buggy) component modules. This gives you
two benefits: it's easier to develop when you have all the packages
and their exported symbols loaded; and you can head off impending
clashes more easily.
To be concrete, in the development of the Testa project, there was a
time when the c_header_reader module was under heavy development (and
had to be loaded with magick incantations), the LF module was broken
(but the interface was approximately correct), and I was working on
the ir-ssa module, which used both of the above. Because one person
was working on the c_header_reader module, and another was working on
the ir-ssa module, it was really nice to see when we were both laying
claim to the same names, and generally stomping on eachother, even if
I couldn't load Eddy's work, nor he mine.
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
···@famine.OCF.Berkeley.EDU (Thomas F. Burdick) writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
> > Given your one-file rule, why not just put the DEFPACKAGE form in
> > util.lisp itself? (Again, just curious--I'm sure you have some
> > reason.)
>
> I use the foo-package.lisp + foo.lisp scheme myself, so I can at least
> tell you why I do it: in addition to just feeling "cleaner", it lets
> you load the entire package scheme of a project, without having to
> load all the (potentially buggy) component modules. This gives you
> two benefits: it's easier to develop when you have all the packages
> and their exported symbols loaded; and you can head off impending
> clashes more easily.
>
> To be concrete, in the development of the Testa project, there was a
> time when the c_header_reader module was under heavy development (and
> had to be loaded with magick incantations), the LF module was broken
> (but the interface was approximately correct), and I was working on
> the ir-ssa module, which used both of the above. Because one person
> was working on the c_header_reader module, and another was working on
> the ir-ssa module, it was really nice to see when we were both laying
> claim to the same names, and generally stomping on eachother, even if
> I couldn't load Eddy's work, nor he mine.
So I assume you don't use a single packages.lisp file containing all
the package declarations because it would be a point of contention
between multiple developers?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> ···@famine.OCF.Berkeley.EDU (Thomas F. Burdick) writes:
>
> > Peter Seibel <·····@javamonkey.com> writes:
> >
> > > Given your one-file rule, why not just put the DEFPACKAGE form in
> > > util.lisp itself? (Again, just curious--I'm sure you have some
> > > reason.)
> >
> > I use the foo-package.lisp + foo.lisp scheme myself, so I can at least
> > tell you why I do it: in addition to just feeling "cleaner", it lets
> > you load the entire package scheme of a project, without having to
> > load all the (potentially buggy) component modules. This gives you
> > two benefits: it's easier to develop when you have all the packages
> > and their exported symbols loaded; and you can head off impending
> > clashes more easily.
> >
> > To be concrete, in the development of the Testa project, there was a
> > time when the c_header_reader module was under heavy development (and
> > had to be loaded with magick incantations), the LF module was broken
> > (but the interface was approximately correct), and I was working on
> > the ir-ssa module, which used both of the above. Because one person
> > was working on the c_header_reader module, and another was working on
> > the ir-ssa module, it was really nice to see when we were both laying
> > claim to the same names, and generally stomping on eachother, even if
> > I couldn't load Eddy's work, nor he mine.
>
> So I assume you don't use a single packages.lisp file containing all
> the package declarations because it would be a point of contention
> between multiple developers?
Oh god no! That sounds like hell for two people working kind of
closely -- I'd hate to think of the merging problems for five people
working mostly independantly. I use foo-package.lisp files to defile
packages [which might have more than one file (in-package :foo)], and
a system.lisp file that puts it all together.
( People who haven't yet invented their own system definition tools
are highly encouraged to learn defsystem. I'm still working on my
new years resolution to switch from tfb-system (my homegrown thang)
to defsystem... )
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
Thomas F. Burdick <···@famine.ocf.berkeley.edu> wrote:
> I use foo-package.lisp files to defile packages
Interesting choise of words. ,)=
Cheers,
-- Nikodemus
Thomas F. Burdick <···@famine.OCF.Berkeley.EDU> wrote:
+---------------
| Peter Seibel <·····@javamonkey.com> writes:
| > So I assume you don't use a single packages.lisp file containing all
| > the package declarations because it would be a point of contention
| > between multiple developers?
|
| Oh god no! That sounds like hell for two people working kind of
| closely -- I'd hate to think of the merging problems for five people
| working mostly independantly.
+---------------
Heh-heh! Take a look at the file "cmucl-18e/src/code/exports.lisp"
some time:
(defpackage "ANSI-LOOP")
(defpackage "C-CALL"
(:export "C-STRING" "CHAR" "DOUBLE" "FLOAT" "INT" "LONG" "SHORT"
"UNSIGNED-CHAR" "UNSIGNED-INT" "UNSIGNED-LONG" "UNSIGNED-SHORT"
"VOID"))
(defpackage "INSPECT"
(:export "*INTERFACE-STYLE*" "REMOVE-ALL-DISPLAYS"
"REMOVE-OBJECT-DISPLAY" "SHOW-OBJECT"))
(defpackage "BIGNUM"
(:export "%ADD-WITH-CARRY" "%ALLOCATE-BIGNUM" "%ASHL" "%ASHR"
"%BIGNUM-LENGTH" "%BIGNUM-REF" "%BIGNUM-SET" "%BIGNUM-SET-LENGTH"
"%DIGIT-0-OR-PLUSP" "%DIGIT-LOGICAL-SHIFT-RIGHT"
...and so on...))
...and so on...
(defpackage "FORMAT")
(defpackage "COMMON-LISP"
(:nicknames "CL" "LISP")
(:export "&ALLOW-OTHER-KEYS" "&AUX" "&BODY" "&ENVIRONMENT" "&KEY"
"&OPTIONAL" "&REST" "&WHOLE" "*" "**" "***"
"*BREAK-ON-SIGNALS*"
"*COMPILE-FILE-PATHNAME*" "*COMPILE-FILE-TRUENAME*"
"*COMPILE-PRINT*" "*COMPILE-VERBOSE*" "*DEBUG-IO*"
...and so on...))
...and so on...
...and so on...
They must have had a very different style of working together.
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
From: Joe Marshall
Subject: Re: basic question about packages
Date:
Message-ID: <fzjxei3x.fsf@ccs.neu.edu>
Lowell <······@cs.ubc.ca> writes:
> I'm in the process of trying to put my files into packages to keep my
> code modular.
Another bit of advice: don't `overpackage' your code.
Packages can help make it possible for your code to exist peacefully
alongside other code, but they *add* interdependecies rather than
remove them. Rather than put every little module in its own package,
I would divide things up along large functional boundaries.
Also: define a `package' file and LOAD IT FIRST. Do NOT try to
incrementally define them.
In one big project I'm working on I take the raw Lisp image from the
vendor, add only the new packages I define, then dump the world. I
start from the dumped world when developing.
In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu>
wrote:
> Also: define a `package' file and LOAD IT FIRST. Do NOT try to
> incrementally define them.
I am very interested in this part of your statement. Could you elaborate
more on why you think that this is the right thing to do? (What do you
especially mean by incremental definition, and why should one avoid it?)
Thanks in advance!
Pascal
Pascal Costanza <········@web.de> writes:
> In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu>
> wrote:
>
> > Also: define a `package' file and LOAD IT FIRST. Do NOT try to
> > incrementally define them.
>
> I am very interested in this part of your statement. Could you elaborate
> more on why you think that this is the right thing to do? (What do you
> especially mean by incremental definition, and why should one avoid it?)
I mentioned the advantages of this scheme in my response to Peter. As
for "incremental definition", I'm pretty sure that refers to something
like:
(defpackage "TFB-UTILS" ; better not collaborate with Tim Bradshaw
(:use "CL"))
;;; Foos
(export '(foo foo*))
(defun foo () ...)
(defun foo* () ...)
;; ...
;;; Bars
(export '(bar))
(defclass bar () ...)
;; ...
I used to do this, and thought it was nice, because the top of each
logical section had export forms, which effectively documented what
was interface, and what was internal.
I'm now a convert to the define-the-package-in-its-own-file scheme,
after learning the hard way.
(FWIW, I got the incremental style from the CMUCL source)
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
Thomas Burdick wrote:
> (defpackage "TFB-UTILS" ; better not collaborate with Tim Bradshaw
I think he prefers to be TFEB, and he uses Java-style long
dotted package names anyway :-).
--
Gareth McCaughan
.sig under construc
From: Joe Marshall
Subject: Re: basic question about packages
Date:
Message-ID: <isosfe4d.fsf@ccs.neu.edu>
Pascal Costanza <········@web.de> writes:
> In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu>
> wrote:
>
>> Also: define a `package' file and LOAD IT FIRST. Do NOT try to
>> incrementally define them.
>
> I am very interested in this part of your statement. Could you elaborate
> more on why you think that this is the right thing to do? (What do you
> especially mean by incremental definition, and why should one avoid it?)
I'm working on a fairly large system that has several hundred files.
The package hacking is rather elaborate, but let me outline it and the
rationale behind it.
First of all, with this many files, it'd be madness to put each one in
its own package. It'd also be madness to try to put all the code
relevant to a single package into one file. So the code is divided
into large `subsystems'. Each subsystem has a package, and each
subsystem provides an abstract API through the symbols it exports.
Each subsystem also expects a certain amount of functionality to be
provided to it by the abstract layers below.
There are several `final products' or `deliverables'. Each one is
built out of a number of selected subsystems. Some subsystems are
included in virtually every deliverable, but some are not needed by
some products. To maximize flexibility, therefore, there is a `system
configuration' phase that selects the subsystems and `plugs them
together'. In order for this to work, it is important that a
subsystem avoid `package qualified' symbols. Here's why:
suppose subsystem `foo' depends on two functional units that provide
database services and network services. Suppose further, that
there is an `enterprise' edition of the product and a `standalone'
edition. Subsystem `foo' should not need to know whether it belongs
to the enterprise or standalone edition, so it only interfaces to the
database and network API. In the enterprise edition, the database and
network APIs each reside in their own packages. In the standalone
edition, however, the database and network API are simple wrappers
around file-system calls. So the different products have different
package topologies:
+--------------+ +----------------+
| `foo' | | `foo' |
+--------------+ +----------------+
| | |
| | |
+--------------+ | +----------------+
| `database' | | | `file db' |
+--------------+ | +----------------+
| |
+--------------+
| `network' |
+--------------+
In the enterprise topology, the DATABASE-QUERY function is provided by
the database subsystem, which exports the symbol from its package, and
the REMOTE-FILE class is provided by the network subsystem which
exports the appropriate symbol from *its* package. But in the
standalone topology, *both* symbols are exported from the file-db
subsystem. (And the standalone REMOTE-FILE class is somewhat misnamed
in that it does not understand actual remote files, but only local
ones. But subsystem FOO doesn't need to know that.)
You can see that this would break down if `foo' explicitly names
symbols in other packages. The other packages might not be there.
It also breaks down if `foo' explicitly *uses* another package. That
is to say, it would be wrong to have a file in foo that says
(defpackage "FOO" (:use "DATABASE"))
or even
(use-package "DATABASE")
Note that this restriction applies to use, import-from, and
shadowing-import-from. It does *not* apply to export, or shadow.
Export is ok because it does not refer to other packages, and shadow
is ok because an `unnecessary' shadow is harmless.
The way to look at it is this: The `subsystems' are completely
ignorant of where they fit into the big picture. The configuration
knowledge is *outside* this level of abstraction.
This doesn't mean that the files are ignorant of the subsystem that
they are in, however. Every file in the system has this prelude:
an emacs parsable `mode line' in a comment
standard copyright & legal bullshit
(in-package "CSF.CORE")
(proclaim (standard-optimizations))
(eval-when (:compile-toplevel :load-toplevel :execute)
(export '( .... )))
The IN-PACKAGE form is *always* the *very first* form in any CL code I
write. I never have a problem with loading code `in the wrong
package'.
Yes, proclaim is deprecated, but it looks nicer than
(declaim #.(standard-optimizations))
this ensures that forgetting to reset the optimization settings in the
REPL won't be a problem. The function (STANDARD-OPTIMIZATIONS) gets
its declarations from some variables, so you can easily override the
defaults in your init file if you are a developer.
The exports are readily available as sort of an API declaration.
The code base includes about 20 `subsystems'. Each subsystem is
kept in its own subdirectory and is comprised of anywhere from 1 to
about 50 files.
The Common Lisp vendor's interpretation of the spec does not always
match mine. There are also a few parts of the spec I disagree with,
a few enhancements I'd like to see, and sometimes it's necessary to
interpose some code between what Common Lisp exports and a subsystem's
use of the same. But it's just bad policy to redefine the symbols in
the Common Lisp package (it doesn't matter to me whether that's
`legal' common lisp, but redefining certain symbols tends to break the
system), so we shadow a few of the symbols in the CL package.
This means that packages should not USE the common lisp package until
the shadowing symbols are in place. Some of the replacements (e.g.
unwind-protect) won't have the right effect unless *every* use in the
system is through the shadowed replacement. It is easy enough to
arrange this in code we write, but impossible to arrange in the
third-party code that we use UNLESS we get the replacement symbols
into the third-party packages BEFORE any of the third party code is
even read. So we cannot rely on the DEFSYSTEM provided in the
third-party code to appropriately initialize those packages as it is
loaded (incrementally).
Our solution, therefore is to perform the package configuration as
early on in the build process as possible. In fact, we do the package
configuration *before* running any code at all by running the package
configuration code and dumping an empty image. Development is done by
booting the pre-configured image and loading code into it. Because
the packages have already been laid out, we can pretty much depend on
the right things ending up in the right places.
There are a couple of drawbacks to this method. When modifying the
package topology, or making new shadows for basic symbols, you have to
revisit the configuration code and redump the image. This can be
painful if you are making a lot of changes. However, we have
relatively few packages, we don't attempt to package every file
separately, and the development cycle has matured so the major
subsystems have all been identified. Redumping everybody's image is not a
frequent operation. The advantages definitely outweigh this
disadvantage.
The pre-build script starts in the cl-user package, which is the only
one you can depend on being there. It instantiates a `config' package
and a `utility' package *explicitly excluding* the CL (and any other
default) packages. There is a list of about 10 CL symbols that we
override, so they are interned into the UTILITY package with SHADOW.
The appropriate values, functions, and macro definitions of the
original symbols are then copied into the corresponding utility
symbols so that the utility code can bootstrap. Utility is then
modified to USE `config' and `CL'. The pre-build script then LOADs
the PACKAGE-SETUP file (which will load into the UTILITY package) and
funcalls a known symbol there. This step of the configuration is done
within a big LET statement at top level, so no functions, variables,
or macros have been introduced into the CL package.
The PACKAGE-SETUP file is run interpreted. It would be a pain to
configure the compiler (because the compiler configuration variables
aren't loaded yet) and performance on this just doesn't matter at
all. Every package in our system is enumerated here along with their
nicknames, USE list, and any exceptional symbols that need to be
handled specially. The packages are either created fresh, or located
as is (so that we can reload this file again later to `repair' things
if necessary). The replacement Common Lisp symbols in the UTILITY
package are shadowing-imported into each of the other packages, and
any other necessary shadows are created. (There are *very* few, maybe
one or two. We've used some code developed by people who started with
a different vendor and some `unfortunate' names are occasionally
used.) The packages are then linked together with USE-PACKAGE and the
image is dumped.
When developing, we start from this dumped image. Since every file
knows what package it is in, and since no file manipulates the package
system, there are very few difficulties. The exports occur
incrementally, so it is important that the subsystems load in the
correct order. The order of file loading *within* the subsystem
doesn't matter as much, so the symbol interdependenices are between 20
subsystems rather than several hundred files. If a file *is* loaded
in the wrong order, or if a new symbol is exported with the same name
as a symbol in the packages using this one, we'll get a conflict.
Resolving the conflict is easy: unintern the existing symbol and
re-load the file that contained it. It's pretty hard to screw up the
package system so much that re-booting is necessary.
This is obviously *far* too elaborate a process for something simple,
and I'm not recommending it for that purpose. If you have a big
project spanning hundreds of files and dozens of packages, though, it
will save you a lot of headaches. If you are starting to
modularize your system, I highly recommend that you prepare in this
way:
1. Always, always, always put an IN-PACKAGE form as the *very
first thing* at the start of every file. Don't *ever* switch
packages in the middle of the file.
2. Separate the implementation from the configuration. Put the
DEFPACKAGE forms all in one place and make sure they get
loaded before anything else. Don't start files with
DEFPACKAGE.
3. Don't use explicit package qualifiers you don't need.
4. Keep the module count fairly low.
If you do these things, you'll have fewer package `headaches'. If
your system becomes big enough that you're wanting multiple
configurations, several packages, and a custom bootstrap process,
you'll already have everything factored out right.
Joe Marshall <···@ccs.neu.edu> writes:
> There are several `final products' or `deliverables'. Each one is
> built out of a number of selected subsystems. Some subsystems are
> included in virtually every deliverable, but some are not needed by
> some products. To maximize flexibility, therefore, there is a
> `system configuration' phase that selects the subsystems and `plugs
> them together'. In order for this to work, it is important that a
> subsystem avoid `package qualified' symbols. Here's why: suppose
> subsystem `foo' depends on two functional units that provide
> database services and network services. Suppose further, that there
> is an `enterprise' edition of the product and a `standalone'
> edition. Subsystem `foo' should not need to know whether it belongs
> to the enterprise or standalone edition, so it only interfaces to
> the database and network API. In the enterprise edition, the
> database and network APIs each reside in their own packages. In the
> standalone edition, however, the database and network API are simple
> wrappers around file-system calls. So the different products have
> different package topologies:
>
>
> +--------------+ +----------------+
> | `foo' | | `foo' |
> +--------------+ +----------------+
> | | |
> | | |
> +--------------+ | +----------------+
> | `database' | | | `file db' |
> +--------------+ | +----------------+
> | |
> +--------------+
> | `network' |
> +--------------+
>
> In the enterprise topology, the DATABASE-QUERY function is provided
> by the database subsystem, which exports the symbol from its
> package, and the REMOTE-FILE class is provided by the network
> subsystem which exports the appropriate symbol from *its* package.
> But in the standalone topology, *both* symbols are exported from the
> file-db subsystem. (And the standalone REMOTE-FILE class is somewhat
> misnamed in that it does not understand actual remote files, but
> only local ones. But subsystem FOO doesn't need to know that.)
>
> You can see that this would break down if `foo' explicitly names
> symbols in other packages. The other packages might not be there.
So I'm confused. How does FOO get hooked up with one set or another of
the exported symbols? Does the system configuration code switch to the
FOO package and do a USE-PACKAGE of the appropriate subsystem packages
before loading FOO? Is this an exception to your rule about not
switching packages mid-file?
Assuming it's something like that, I'm curious, is this kind of
"symbol punning" a common technique for swapping implementations of an
single interface.
Presumably you could achieve similar effects though in via a
completely different mechanism if the DATABASE-QUERY was a generic
function and the REMOTE-FILE class was in a package that FOO did a
USE-PACKAGE on and the two different implementations of it that you
already have were both subclasses of the "public" class with
appropriate methods defined on DATABASE-QUERY. Of course then you
might have to have a factory method that was also public to create the
right kind of REMOTE-FILE instances depending on whether the code was
running in enterprise or standalone configuration.
Is there some advantage that lead you to use what I'm calling symbol
punning? Or some disadvantage to the polymorphism via subclasses and
generic functions that I'm missing?
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Joe Marshall
Subject: Re: basic question about packages
Date:
Message-ID: <65ksf97w.fsf@ccs.neu.edu>
Peter Seibel <·····@javamonkey.com> writes:
> Joe Marshall <···@ccs.neu.edu> writes:
>
>> There are several `final products' or `deliverables'. Each one is
>> built out of a number of selected subsystems. Some subsystems are
>> included in virtually every deliverable, but some are not needed by
>> some products. To maximize flexibility, therefore, there is a
>> `system configuration' phase that selects the subsystems and `plugs
>> them together'. In order for this to work, it is important that a
>> subsystem avoid `package qualified' symbols. Here's why: suppose
>> subsystem `foo' depends on two functional units that provide
>> database services and network services. Suppose further, that there
>> is an `enterprise' edition of the product and a `standalone'
>> edition. Subsystem `foo' should not need to know whether it belongs
>> to the enterprise or standalone edition, so it only interfaces to
>> the database and network API. In the enterprise edition, the
>> database and network APIs each reside in their own packages. In the
>> standalone edition, however, the database and network API are simple
>> wrappers around file-system calls. So the different products have
>> different package topologies:
>>
>>
>> +--------------+ +----------------+
>> | `foo' | | `foo' |
>> +--------------+ +----------------+
>> | | |
>> | | |
>> +--------------+ | +----------------+
>> | `database' | | | `file db' |
>> +--------------+ | +----------------+
>> | |
>> +--------------+
>> | `network' |
>> +--------------+
>>
>> In the enterprise topology, the DATABASE-QUERY function is provided
>> by the database subsystem, which exports the symbol from its
>> package, and the REMOTE-FILE class is provided by the network
>> subsystem which exports the appropriate symbol from *its* package.
>> But in the standalone topology, *both* symbols are exported from the
>> file-db subsystem. (And the standalone REMOTE-FILE class is somewhat
>> misnamed in that it does not understand actual remote files, but
>> only local ones. But subsystem FOO doesn't need to know that.)
>>
>> You can see that this would break down if `foo' explicitly names
>> symbols in other packages. The other packages might not be there.
>
> So I'm confused. How does FOO get hooked up with one set or another of
> the exported symbols? Does the system configuration code switch to the
> FOO package and do a USE-PACKAGE of the appropriate subsystem packages
> before loading FOO?
Yes. The entire package topology is set up before any code is loaded.
The config code (roughly) invokes
(MAKE-PACKAGE "DATABASE" :USE '("NETWORK"))
(MAKE-PACKAGE "FOO" :USE '("DATABASE" "NETWORK"))
and does this well before any code for database or network or foo is
read.
> Is this an exception to your rule about not switching packages mid-file?
It is different. The configuration code is read in one package. It
manipulates the package system, but there are no files anywhere in the
system that have more than one (in-package) form. There are also
times during the subsequent load that a package is newly created or
where the package topology is modified. It is created once by the
config code and never ever changed.
> Assuming it's something like that, I'm curious, is this kind of
> "symbol punning" a common technique for swapping implementations of an
> single interface.
I don't know.
> Presumably you could achieve similar effects though in via a
> completely different mechanism if the DATABASE-QUERY was a generic
> function and the REMOTE-FILE class was in a package that FOO did a
> USE-PACKAGE on and the two different implementations of it that you
> already have were both subclasses of the "public" class with
> appropriate methods defined on DATABASE-QUERY. Of course then you
> might have to have a factory method that was also public to create the
> right kind of REMOTE-FILE instances depending on whether the code was
> running in enterprise or standalone configuration.
>
> Is there some advantage that lead you to use what I'm calling symbol
> punning? Or some disadvantage to the polymorphism via subclasses and
> generic functions that I'm missing?
You can't use generic functions for special variables, and sometimes
we use third-party code where we are very reluctant to change
anything. If the third-party code didn't define a generic function,
then we're out of luck.
In any case, you sort of *have* to use some kind of symbol punning.
In order for code in REMOTE-FILE to be defined, there *has* to be a
symbol to attach the function to. In order for FOO to invoke the
function, it *has* to use the same symbol.
Now it's true we could make a base package and get both FOO and
REMOTE-FILE to import that symbol, but now we've added an extra
dependency. Or we could factor out the FILE api definitions into a
their own file, then load the appropriate methods later. This would
be a better solution, but it would mean re-organizing the code.
Generally, development might involve noticing that some new code needs
to be written and just adding the new file to the system with its own
set of exports.
Maybe we should have done it differently, I don't know.
But I'd still set up the packages beforehand to have a static
framework for reading, compiling, and loading.
In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu>
wrote:
> I'm working on a fairly large system that has several hundred files.
> The package hacking is rather elaborate, but let me outline it and the
> rationale behind it.
>
Thanks a lot for your excellent post. Another one of those c.l.l
postings that should be archived in a best-of collection...
Pascal