Hi list,
I wonder why package.lisp in asdf systems is a separate file? Is it
required for portability due to some bugs in some of implementations
or is there some other reason? I'd like to have defpackage form inside
the source file sometimes. This way it is easier to achieve
modularity.
On Feb 27, 11:18 am, budden <···········@mail.ru> wrote:
> Hi list,
> I wonder why package.lisp in asdf systems is a separate file? Is it
> required for portability due to some bugs in some of implementations
> or is there some other reason? I'd like to have defpackage form inside
> the source file sometimes. This way it is easier to achieve
> modularity.
I like to have the package file separate. In that way it is easier to
achieve modularity.
Cheers
--
Marco
On Feb 27, 10:18 am, budden <···········@mail.ru> wrote:
> I'd like to have defpackage form inside
> the source file sometimes. This way it is easier to achieve
> modularity.
It really isn't. The package definition is (part of) the specification
of the interface to your system. You don't want this tangled up with
the implementation.
Tim Bradshaw <··········@tfeb.org> writes:
> On Feb 27, 10:18 am, budden <···········@mail.ru> wrote:
>> I'd like to have defpackage form inside
>> the source file sometimes. This way it is easier to achieve
>> modularity.
>
> It really isn't. The package definition is (part of) the specification
> of the interface to your system. You don't want this tangled up with
> the implementation.
That's one of the reasons I don't like macros that combine defun+export,
and like naked EXPORT calls even less.
Zach
Yeah, I know I'm a blub programmer.
In Pascal:
- interface and implementation are in one file, but they are separated
- dependencies of interface and implementation are declared
separatedly
- interface dependency should be D.A.G while implementation dependency
may not
- every unit usually exports a small amount of symbols
- module can import dozen of namespaces in an implementation
This is a very convinient approach. Module dependencies are
declarative and
can be derived from sources, no system definition is usually required.
If you fail to
declare dependency, compilation would fail at the start. All what I
need is
a list of paths to a source file locations and a name of header file.
What goes to lisp, I still unsuccessfully struggle to make things
convinient. What are my enemies?
- package management is painful, so I tend to use large packages (I
found a workaround for that and it looks like it works)
- asdf is painful too, so I tend to use large systems with :serial t.
This way, things become slow.
- When I fail to specify dependencies correctly, lisp silently interns
symbols into wrong places. Ok, I should use package locks, I know. But
it depends on implementation if they really help.
What are obstacles to adapt Pascalish practice to lisp?
1. The way lisp merges namespaces. You can't use two namespaces unless
they're clash-free. I have a workaround for it and it looks like it
helps.
2. Dynamic nature of package system. I could avoid or isolate implicit
interning, use package locks to make it mostly static. I could use
forbidden names lists to fight this. I could trace some of package
system functions or replace them with my own versions (this is why I
need unlock cl package). I tried some of these techniques and they
help in part.
3. Macros and eval-whens. I can forward-reference a function, but I
cant forward-reference a macro. It looks like it can be solved with
code re-organization and/or declaring dependencies for macros in a
special way.
4. Asdf. I need to integrate to it somehow. But I think this can be
solved too.
But still noone answered my initial question.
On Feb 27, 10:12 am, budden <···········@mail.ru> wrote:
> But still noone answered my initial question.
On the contrary, you got several answers to your question.
They were just all of the form, "don't do that."
reasons given *not* to put the package forms in the same file:
Marco: to achieve modularity
(btw, it's a much stronger argument, as Marco makes,
that you achieve modularity when you *separate* different
things, rather than combining them, as you state.)
Pascal B: 1. when your project grows its dependencies may no longer be
a DAG.
2. why do things two different ways for small and large
projects;
just use one system from the start.
Tim Bradsaw
and
Zach Beane: separation of interface (i.e., package forms) and
implementation
So to summarize, there is no requirement of conforming
implementations that package.lisp be a separate file. But by doing so:
1. you'll save yourself trouble down the road if/when your project
grows
in complexity
2. you'll gain the modularity that comes of having different things
in different files
3. you'll achieve a separation of interface and implementation.
> > But still noone answered my initial question.
>
> On the contrary, you got several answers to your question.
> They were just all of the form, "don't do that."
Thanks, maybe I've put my question wrong.
I mean, are there bugs in implementations which would cause
incorrect behaviour of the sequence:
(defpackage :foo (:use ...) (:export ...) ...)
(in-package :foo)
....
in one file?
On Feb 27, 12:28 pm, budden <···········@mail.ru> wrote:
> > > But still noone answered my initial question.
>
> > On the contrary, you got several answers to your question.
> > They were just all of the form, "don't do that."
>
> Thanks, maybe I've put my question wrong.
>
> I mean, are there bugs in implementations which would cause
> incorrect behaviour of the sequence:
> (defpackage :foo (:use ...) (:export ...) ...)
> (in-package :foo)
> ....
> in one file?
There is nothing in the standard that disallows the above. I have seen
a good deal of simple, single file lisp code that does just what
you're suggesting. Ironically, asdf.lisp itself uses precisely this
idiom! Well, it kind of has to since it's defining the system
definition facility so best to keep that all in one file since we
don't yet have a system definition facility in place when we're
defining the system definition facility to give us rules for how to
load other files ... you get the idea.
If you do so, you'll probably want to use #: to prevent the package
we're in when the file is read from being polluted with interned
symbols from the reading of your file, thus:
----- foo.lisp ---------
(defpackage #:foo (:use :common-lisp) (:export #:bar))
(in-package #:foo)
(defun bar ()
(format t "you're inside the bar function.
the symbol bar's package is ~a~%" (symbol-package 'bar)))
this way when you go:
(use-package :foo)
you won't get any conflicts merely due to reading foo.lisp
Nor am I am not aware of any implementation that has a bug that would
cause you a problem using this idiom.
A couple of points:
1. In general, you shouldn't write code to possible bugs, but to the
standard.
2. Still might not be the best idea to put the package form(s) in the
same file 'cause it might make life more difficult for you down the
road. In addition, if you're using asdf as your original question
suggests you are, why not avail yourself of its ability to load
separate files in a well defined way?
> I have seen a good deal of simple, single file lisp code that does just what
you're suggesting. Ironically, asdf.lisp itself uses precisely this
idiom!
Oops... I should have recall it before. It looks like all this thread
is redundant...
Sorry for that...
> If you do so, you'll probably want to use #: to prevent the package
> we're in when the file is read from being polluted with interned
> symbols from the reading of your file, thus:
Yes, I do that since I have first seen it somewhere. But no reason to
> (defpackage #:foo (:use :common-lisp) (:export #:bar))
I use defpackage :foo instead as I use keywords to find a package
definition.
I just type :foo at SLIME prompt and press M-. Also, it is useful to
do
(find-package :foo). Thus, it is likely that :foo will be created soon
anyway.
> Nor am I am not aware of any implementation that has a bug that would
> cause you a problem using this idiom.
Ok, thanks.
> 1. In general, you shouldn't write code to possible bugs, but to the
> standard.
I've seen some ingenious code which was written to the bugs. It is
screamer and ap5.
> 2. Still might not be the best idea to put the package form(s) in the
> same file 'cause it might make life more difficult for you down the
> road.
It depends on the situation. Some files are useful even when they are
not a part
of any system, e.g. net.hexapodia.toposort. And I see no reason why it
should ever
grow. I think if such a file depends on CL only, it is reasonable to
keep it single.
Adding a package file implies creation of .asd file too. So, we get
three files
instead of one without any serious reason. Then we would need a folder
for them, symlink
to asd file, and I think it is a great redundancy. On the other hand,
if we use clbuild,
asdf-install, or want to make asd systems depend on the file, we would
likely need asd
system too. But if I had resources, I'd better subclass asdf::system
to support such
single files. As there is other annoying problem: I still know no way
in SLIME to
type :foo, press magic key and visit foo.asd.
> In addition, if you're using asdf as your original question
> suggests you are, why not avail yourself of its ability to load
> separate files in a well defined way?
Mmm, maybe this is one other good reason to make asd system for every
file.
On Feb 27, 9:58 am, Raffael Cavallaro <················@gmail.com>
wrote:
> If you do so, you'll probably want to use #: to prevent the package
> we're in when the file is read from being polluted with interned
> symbols from the reading of your file, thus:
>
> ----- foo.lisp ---------
>
> (defpackage #:foo (:use :common-lisp) (:export #:bar))
>
> (in-package #:foo)
This is good advice. Furthermore, whenever i do this, i always use
fully qualified symbols to reference CL:DEFPACKAGE and CL:IN-PACKAGE.
This way if you forget to (:USE :CL), which i do all the time, you can
still recompile the file without the weird ':USE is not a function'
error.
ie:
(cl:defpackage #:foo (:use :common-lisp) (:export #:bar))
(cl:in-package #:foo)
Cheers,
drewc
On Feb 27, 6:28 pm, budden <···········@mail.ru> wrote:
> I mean, are there bugs in implementations which would cause
> incorrect behaviour of the sequence:
> (defpackage :foo (:use ...) (:export ...) ...)
> (in-package :foo)
> ....
> in one file?
That will work. But it is not always advisable, as others have
already noted. One reason for separating DEFPACKAGE forms into a
separate file is that it allows me to pull in the package structure
without loading any code. This allows me to twiddle the project's
configuration *before* loading any code:
(defparameter some-package:*foo-knob* ...)
(Suddenly, the defvar/defparameter difference makes a lot of sense,
doesn't it?)
Some time ago, Joe Marshal posted advice on structuring large lisp
projects:
http://groups.google.com/group/comp.lang.lisp/msg/d23a942023da202c
I do not agree with everything (in particular for smaller-scale
projects), but the summary is good.
From: Tobias C. Rittweiler
Subject: Re: reason for having separate package.lisp?
Date:
Message-ID: <87zlg7efyw.fsf@freebits.de>
Michael Weber <·········@foldr.org> writes:
> ... One reason for separating DEFPACKAGE forms into a separate file is
> that it allows me to pull in the package structure without loading any
> code. This allows me to twiddle the project's configuration *before*
> loading any code:
>
> (defparameter some-package:*foo-knob* ...)
>
> (Suddenly, the defvar/defparameter difference makes a lot of sense,
> doesn't it?)
There are also some ASDF extensions by Ian Eslick at
http://web.media.mit.edu/~eslick/asdf-ext/
to facilitate such parametrization. I do not know whether he has ever
sent that work upstream, however.
-T.
Thanks. My projects are complex enough and they're
somewhat similar to that described in the Joe Marhsal's
post, e.g. there are three different SQL interfaces which
are partially unified at a higher layer. I'm not
sure I can take the approach he suggests (dumping an
empty image with defpackage forms). What goes to summary,
> 3. Don't use explicit package qualifiers you don't need.
I agree to this. And this is why I dislike package system
design. In Pascal, SQL, C++, Java symbol clashes are reported
when you refer to clashing symbols. In CL, it is reported
when you use-package. This forces user to avoid using packages
and to use qualified symbols very frequently (more than
necessary, indeed; I was criticised for stating it at c.l.l).
Also it forces to
> 4. Keep the module count fairly low.
It would be very easy to fix if we would extend signature
of in-package form.
(in-package package &rest more-packages)
More-packages are not used to package, but symbols are
looked up in more-packages too. New unqualified symbols found by
reader
are interned to package. If some symbols clash, error is reported
when the reader tries to read this symbol by its unqualified
name from the stream. If there is internal symbol in package
and external symbol in some of more-packages with the same
name, error is reported when reader tries to read this symbol
by its unqualified name from the stream - this is a protection
against trash symbols.
Or, alternatevely, symbol with that name from leftmost
package is taken. So, one could have an unlimited number of
small packages and don't be afraid about clashes.
I have made some experiments. I managed to implement this
macro using Pascal J. Bourguignon's GNU CL reader. The reader
still has bugs and it is licensed under GNU, so this can't be
final solution for me. Patching of SLIME is required too to
make completion work well. I didn't touch printer, but changing
printer is not cruicial, all would work correctly with existing
printer.
So it looks like it would take me about 5 full-time days to
implement a portable extension to an in-package form. Additionnaly,
portable hierarchical packages and portable package locks could be
implemented. Unfortunately, I have no resources for that. So I
have implemented quick-and-dirty solution, merge-packages-and-
reexport.
http://paste.lisp.org/display/74832#2
But this gives no help against interning trash symbols. I have
developed
implementation-dependent approach to this, changing some cl functions.
I hold a list of *forbidden-symbol-names* and make CL warn when the
symbol
is being interned to some package. But I still can't run it under
SBCL.
Now I think it'd be reasonable to hook up into asdf and search for a
trash symbols _after_ finishing load-op on file. Consider we're
loading
foo.lisp with single (in-package :bar) form in it.
Warning is issued after load-op when:
i) some new symbol is interned by load process to the package other
than :bar
or
ii) some new symbol is interned to :bar with the same name as name of
some external
symbol from other package.
Both warnings can be avoided by adding some declarations on a per name
basis.
This solution is still much worse than having portable package locks,
and it will slow down
loading essentiallly, but it seems to be easier to implement and
safer.
From: Pascal J. Bourguignon
Subject: Re: reason for having separate package.lisp?
Date:
Message-ID: <871vti7nwu.fsf@galatea.local>
budden <···········@mail.ru> writes:
> I have made some experiments. I managed to implement this
> macro using Pascal J. Bourguignon's GNU CL reader. The reader
> still has bugs and it is licensed under GNU,
I can assure you that the reader has absolutely NO bug.
The proof: I have received no bug report.
--
__Pascal Bourguignon__
On Feb 28, 7:21 am, ····@informatimago.com (Pascal J. Bourguignon)
wrote:
> budden <···········@mail.ru> writes:
> > I have made some experiments. I managed to implement this
> > macro using Pascal J. Bourguignon's GNU CL reader. The reader
> > still has bugs and it is licensed under GNU,
>
> I can assure you that the reader has absolutely NO bug.
> The proof: I have received no bug report.
>
> --
> __Pascal Bourguignon__
or...
it has one really large bug (i.e., it is GPL) so no one wants to use
it because doing so would mean licensing all the other code in any
project that used it under the GPL as well (e.g., CLISP and readline).
Hence the lack of bug reports.
From: Pascal J. Bourguignon
Subject: Re: reason for having separate package.lisp?
Date:
Message-ID: <877i3a5u7b.fsf@galatea.local>
Raffael Cavallaro <················@gmail.com> writes:
> On Feb 28, 7:21�am, ····@informatimago.com (Pascal J. Bourguignon)
> wrote:
>> budden <···········@mail.ru> writes:
>> > I have made some experiments. I managed to implement this
>> > macro using Pascal J. Bourguignon's GNU CL reader. The reader
>> > still has bugs and it is licensed under GNU,
>>
>> I can assure you that the reader has absolutely NO bug.
>> The proof: I have received no bug report.
>>
>> --
>> __Pascal Bourguignon__
>
> or...
>
> it has one really large bug (i.e., it is GPL) so no one wants to use
> it because doing so would mean licensing all the other code in any
> project that used it under the GPL as well (e.g., CLISP and readline).
> Hence the lack of bug reports.
If that's a problem, commercial licenses are available too.
--
__Pascal Bourguignon__
···@informatimago.com (Pascal J. Bourguignon) writes:
> budden <···········@mail.ru> writes:
>> I have made some experiments. I managed to implement this
>> macro using Pascal J. Bourguignon's GNU CL reader. The reader
>> still has bugs and it is licensed under GNU,
>
> I can assure you that the reader has absolutely NO bug.
> The proof: I have received no bug report.
The best feature that could be added to CL readers would be the
ability to have local nicknames.
By this I mean something like this:
(defpackage :org.cl-user.rdm-util
(:use :common-lisp)
(:local-nicknames :cl-who :de.weitz.cl-who))
Then package authors should use long names, but package users get to
use short names, with no possibility of nickname collisions.
-russ
On Feb 28, 7:40 pm, Russell McManus <···············@yahoo.com> wrote:
>
> The best feature that could be added to CL readers would be the
> ability to have local nicknames.
>
> By this I mean something like this:
>
> (defpackage :org.cl-user.rdm-util
> (:use :common-lisp)
> (:local-nicknames :cl-who :de.weitz.cl-who))
>
You mean, like the per-package alias system that conduits can
support? (yes yes, this is non-portable code, which it kind of has to
be)
Tim Bradshaw <··········@tfeb.org> writes:
> On Feb 28, 7:40�pm, Russell McManus <···············@yahoo.com> wrote:
>>
>> The best feature that could be added to CL readers would be the
>> ability to have local nicknames.
>>
>> By this I mean something like this:
>>
>> (defpackage :org.cl-user.rdm-util
>> � (:use :common-lisp)
>> � (:local-nicknames :cl-who :de.weitz.cl-who))
>>
>
> You mean, like the per-package alias system that conduits can
> support? (yes yes, this is non-portable code, which it kind of has to
> be)
I haven't looked at the conduits package, but it sure sounds like the
same idea.
Based on earlier posts of yours about conduits however, I think that
conduits can do lots more. I just want this one little feature!
-russ
I see no portable way to do it w/o modifying a reader. Modifying
reader would destroy completion. We can restore completion in EMACS,
but this won't help to commercial IDE users.
I think this could be done dynamically with package-nicknames and
rename-package.
This way, it'd be nice to develop a per-project map of short aliases
and put something like
(set-package-nicknames
:cl-who $
:clsql @
:weblocks w
)
to the system load script.
and then use smth like
(w:defapp ...
($:who ... (@:query ))
> On Feb 28, 7:40 pm, Russell McManus <···············@yahoo.com> wrote:
> > The best feature that could be added to CL readers would be the
> > ability to have local nicknames.
Hmmm.... This can be done with readmacros. Nothing is shorter than
character.
I know no way to change readtable on a per-file bases. There are two
ways
to approach this:
- SLIME has a package-to-readtable map.
- readtable can be specified in a .asd file.
Say
my-system.asd
(defreadtable my-readtable
(#\$ (lambda (c s) (let ((*package* (find-package :de.weitz.cl-who))
(read s))))
...)
(defsystem my-system
:readtable my-readtable
...
)
We have enough free characters: ·@$%^_~?
If that is not enough, we could arrange two-character sequences.
This would make lisp a bit perlish, but much terser.
From: Pascal J. Bourguignon
Subject: Re: reason for having separate package.lisp?
Date:
Message-ID: <87ocwn7orm.fsf@galatea.local>
budden <···········@mail.ru> writes:
>> > But still noone answered my initial question.
>>
>> On the contrary, you got several answers to your question.
>> They were just all of the form, "don't do that."
> Thanks, maybe I've put my question wrong.
>
> I mean, are there bugs in implementations which would cause
> incorrect behaviour of the sequence:
> (defpackage :foo (:use ...) (:export ...) ...)
> (in-package :foo)
> ....
> in one file?
Absolutely none.
I've been doing that up until recently for most of my library packages
(since they're relatively independant, small and self contained).
(However you might be interested to know that in my most recent
programs, I found it easier to define the packages (not a lot)
separately, and just put a IN-PACKAGE form in the various source
files.)
--
__Pascal Bourguignon__
budden <···········@mail.ru> wrote:
> > > But still noone answered my initial question.
> >
> > On the contrary, you got several answers to your question.
> > They were just all of the form, "don't do that."
> Thanks, maybe I've put my question wrong.
>
> I mean, are there bugs in implementations which would cause
> incorrect behaviour of the sequence:
> (defpackage :foo (:use ...) (:export ...) ...)
> (in-package :foo)
> ....
> in one file?
In gcl compiled package definitions do not work, so if you want
to compile content of the package you need to first laod interpreted
definition of the package.
--
Waldek Hebisch
·······@math.uni.wroc.pl
On Fri, 27 Feb 2009 07:12:58 -0800, budden wrote:
> What goes to lisp, I still unsuccessfully struggle to make things
> convinient. What are my enemies?
Your biggest enemy is yourself. You get hung up on minor,
inconsequential things and then you keep harping about them.
> But still noone answered my initial question.
Because this is a no-issue. It is not a big deal to have a separate
packages.lisp file so no one cares.
CL has so much to offer, but you keep dwelling on silly issues. Your
loss.
Tamas
budden wrote:
> Yeah, I know I'm a blub programmer.
Nah; your human. Lisp package management (and ASDF) have their
faults; but managing packages at runtime is a hard problem. Static
solutions (as in other languages) are generally simpler because they
tackle a simpler problem.
> In Pascal:
> - interface and implementation are in one file, but they are separated
> - dependencies of interface and implementation are declared
> separatedly
Interface dependencies go in ASDF files. Implementation dependencies
are specified as (require :package) in source files.
> - interface dependency should be D.A.G while implementation dependency
> may not
> - every unit usually exports a small amount of symbols
That probably should be true in Lisp as well.
> - module can import dozen of namespaces in an implementation
> This is a very convinient approach. Module dependencies are
> declarative and
> can be derived from sources, no system definition is usually required.
> If you fail to
> declare dependency, compilation would fail at the start. All what I
> need is
> a list of paths to a source file locations and a name of header file.
You can achieve the same effect by putting all symbols into cl-user,
directly LOADing source files, and watching SBCL's unknown-function
warnings. No?
> What goes to lisp, I still unsuccessfully struggle to make things
> convinient. What are my enemies?
> - package management is painful, so I tend to use large packages (I
> found a workaround for that and it looks like it works)
I generally doodle in cl-user and only later realize I have enough to
justify a package.
> - asdf is painful too, so I tend to use large systems with :serial t.
> This way, things become slow.
ASDF never loads packages in parallel; so :serial t is no loss...
> - When I fail to specify dependencies correctly, lisp silently interns
> symbols into wrong places. Ok, I should use package locks, I know. But
> it depends on implementation if they really help.
Same issue with C++ (namespaces), IIRC Perl, and pretty much every
other system I've used. The only difference being that you usually
"reload the world" so frequently in the non-lisp languages that
package clashes "aren't a problem".
> What are obstacles to adapt Pascalish practice to lisp?
> 1. The way lisp merges namespaces. You can't use two namespaces unless
> they're clash-free. I have a workaround for it and it looks like it
> helps.
Same issue with C++. My (least) favorite being boost clobbering the
Qt signals and slots keywords.
> 2. Dynamic nature of package system. I could avoid or isolate implicit
> interning, use package locks to make it mostly static. I could use
> forbidden names lists to fight this. I could trace some of package
> system functions or replace them with my own versions (this is why I
> need unlock cl package). I tried some of these techniques and they
> help in part.
Other languages generally don't support this feature.
> 3. Macros and eval-whens. I can forward-reference a function, but I
> cant forward-reference a macro. It looks like it can be solved with
> code re-organization and/or declaring dependencies for macros in a
> special way.
IIUC, that is a major goal of XCVB.
http://common-lisp.net/project/xcvb/
At first I thought the Lisp package system was @#$(·@&.
Then I realized that development in a running system changes the equation.
Finally lisp's package mechanisms started making more sense.
I dunno. For me, most of the pain was simply in learning a new
system. At work, I'll blow hours copying function prototypes from
source files to "header" files with no complaints. At home, I gripe
about having to simply list the symbol names in a lisp package form.
Double standard? Probably. And as I get used to Lisp, I'm starting
to see this as "standard procedure" instead of "another annoyance".
- Daniel
budden <···········@mail.ru> writes:
> Hi list,
> I wonder why package.lisp in asdf systems is a separate file? Is it
> required for portability due to some bugs in some of implementations
> or is there some other reason? I'd like to have defpackage form inside
> the source file sometimes. This way it is easier to achieve
> modularity.
I usually agree with you. For simple projects.
However, you may have packages big enough that you want to implement
them in several source files. In this case, it's better to put the
defpackage in a separate file.
Or you may have a graph of dependencies between the packages that is
not a D.A.G, and therefore it would be difficult to load the files in
the right order. In this case too it's better to put the defpackage
forms in a separate file.
--
__Pascal Bourguignon__
> However, you may have packages big enough that you want to implement
> them in several source files... Or you may have a graph of dependencies
> between the packages that is not a D.A.G, ...
Yes, this is why word "sometimes" was in my question.
budden <···········@mail.ru> writes:
>> However, you may have packages big enough that you want to implement
>> them in several source files... Or you may have a graph of dependencies
>> between the packages that is not a D.A.G, ...
> Yes, this is why word "sometimes" was in my question.
Well then the reason why you'd still want to put the defpackages in
the separate file, even in these "sometimes" simple cases, is that you
would then always do the same thing, for all your projects.
Since packages.lisp works in all cases, it's simplier. You don't need
to think, you don't have more work the day the project grows and the
graph stops being a D.A.G., tools and programmers have only one case
to deal with.
--
__Pascal Bourguignon__