From: Jeff Dalton
Subject: Re: Common Lisp Package System Considered Harmful
Date:
Message-ID: <3646@skye.ed.ac.uk>
In article <··················@cdollin.hpl.hp.com> ····@hplb.hpl.hp.com (Chris Dollin) writes:
> >(c) packageing seems to be done in the wrong place - as part of the
> >lexical and syntactic analysis, not in the compiler [*2].
> >[*2] Or interpreter. For these purposes the difference is irrelevant.
>Jeff posts quite a lot, from which I extract below.
>
> Packages are handled by the reader. Thus they apply to data
> as well as code. This approach has some disadvantages (see
> one of my earlier articles for an example), but it also has
> advantages.
>I have found it inconvenient that the packaging system applies to
>data. I have resorted to using keywords rather than quoted "plain"
>symbols in several places just to ensure that packaging doesn't
>confuse me!
You have to arrange for the right data symbols to be imported,
exported, etc. There are times when this can be a pain, but there
are other times when it would be a pain to have all symbols in one
package.
> There are several ways in which data structures can end up
> being interpreted as code:
>I must declare my colours. In general I don't like to treat data as code.
>This bias will show up below.
Lisp has traditionally been a language in which you can do such
things. Scheme has not. I think it's a good thing that both
approaches are represented.
> 1. A symbol that has a global function definition can be
> used as a function.
>
>I would claim that the symbol "should" have been converted to its functional
>value at the point it was passed. That conversion must, of course, respect
>packaging.
There are, no doubt, ways to do this without losing track of what
symbol it was. (Just taking the functional value would be to lose
track.) You suggest, below, that the symbol might be converted to
a "variable". Maybe that would be a good solution, but it's hard
to be sure without implementing it and seeing how it turns out.
> 2. A list, symbol, or other object can be given to EVAL.
>
>This would require that sufficient information be given to EVAL for it to
>resolve packaging in the same way that the compiler (or interpreter) would.
>That seems reasonable to me.
I agree. Indeed, that's how the Scheme implementations that
have EVAL tend to handle it.
> 3. A list, symbol, or other object can result from macro
> expansion.
>
>I don't think that matters; if packaging is a property of the compiler, then
>macros would have to respect it; they might have to embed additional package
>*syntax* in their result, but I don't think this is a big deal.
It is difficult to make this work in a reasonable way. Suppose, for
example, you write a macro that defines a macro. It may need to put
in some package information but not be able to tell what package
it's being expanded in.
Here's an example. DEFINE-FEXPR is used to define a "function"
that operates on its unevaluated arguments (as a single list).
What it actually defines is a macro that expands to call a
function with all the arguments in a QUOTE.
(defmacro define-fexpr (name (var) &body forms)
(let ((fn-name
(gentemp (concatenate 'string (string name) "-FUNCTION"))))
`(progn
(defun ,fn-name (,var)
,@forms)
(defmacro ,name (&rest ,var)
`(,',fn-name ',,var)))))
And now the question is: how does DEFINE-FEXPR know what module
the function is in so that the macro it defines can expand to
a call to *that* function. (Despite our GENTEMP, there may
later be another function with that name.)
You also have to consider how macro calls are written. The caller
shouldn't have to add more (explicit) package information than would
be needed when calling a function or a built-in special form.
There are ways to deal with these problems, but it requires
work to figure them out, and some of the mechanisms required
look like they will be complicated and hard to understand.
> Let's look at the first example. If the package information
> were not part of the symbol, and the symbol were given as a
> function to MAPCAR, MAPCAR would have no way to determine
> which package held the function definition.
>
>I've hand-waved over that one above. I think if one were to take compiler
>packaging seriously, then the symbol would have been converted at an earlier
>appropriate moment.
I agree. But that would be doing something else rather than making
this case work. So I think we'd have to try implementing the
something else ans see how well it worked.
>A more serious problem is that the symbol gives you a layer of indirection;
>between passing the symbol in, and applying it with MAPCAR (strictly, I
>suppose, FUNCALL or APPLY), the global binding might change. If the "old"
>value has been captured, but the new one intended, then I'm stuffed.
>
>A solution is to allow *variables* as function-objects, and to pass the
>variable corresponding to the symbol as argument; the package
>information would be resolved at this point.
Note that it's much easier to do this if the user has to write
something like (VARIABLE <name>) than if it's supposed to happen
automatically.
> There are several solutions that might be applied to these problems:
>
> S-1. Eliminate the offending parts of the language.
> S-2. Adopt a convention such as: always use the current package.
> S-3. Have the user explicitly specify the package.
> S-4. Invent a zowie new mechanism that makes (almost) everything
> work as we (ought to) expect.
>
>I suspect Jeff would say I've picked S-4. Chopping context ...
>
> ... That is, I should be able to write a macro like this one:
> (defmacro mac (x) `(some-package:some-function ,x))
> or one like this:
> (defmacro mac (x) `(some-function ,x))
>
>Well, I'd write them the same way! But the reader item "package:id" would be
>translated as (eg) "(inpackage package id)", where "inpackage" is a special
>form which resolves to the variable called "id" in the package "package".
That handles the first definition of MAC, but not the second. A
number of macros that are easy to express in Common Lisp will have to
be cluttered with package information (unless there was a clever
implementation that figured it all out automatically). And the
macros's author may not know all of the package information when
writing the macro. (See above.) You can start adding mechanisms to
handle all this, but it's difficult to devise good ones.
> So that leaves S-4. It is clearly the best solution, but
> rather difficult to devise. The Scheme world will eventually
> do it for modules and macros in Scheme. Scheme won't support
> symbols as functions, and it won't have EVAL.
>
>I think I've sketched an interim solution which would work as effectively as
>pacakges do now, and gives what I regard as several advantages: eg, data is
>immune from the package system (so [modulo UNINTERN and its ilk] there's only
>one symbol with a given printname; one can write macros which do package
>manipulation, such as "(in-a-package Package Form*)" which compiles the Form's
>in the context of that package. I'm not seriously suggesting that it's a
>candidate to go into CL; it's far too late for that. But I think that it
>demonstrates that there is a rational alternative to read-time packaging.
Well, I don't like that solution. I've tried to write macros that way
recently, in an "Uncommon Lisp", and it was a pain. I don't think it
works as effectively as packages, because packages often get it right
for you, and the cases that aren't addressed by packages (certain name
capture problems) have been around for so long that people have gotten
used to dealing with them.
Moreover, I think uninterned symbols take us further from "only one
symbol with a given name" than "only one symbol with a given name in
a given package" does. (Of course, it depends on which cases one
has in mind.)
-- Jeff
PS -- You asked me to mail you a copy of an earlier message.
I don't know which one you want. (I mention this here, because
mail doesn't seem to be working very well these days.)