I have some general thoughts about defsystems that I'd like to float
here for comment. As a relative Lisp newbie, having reacquainted
myself with the language over the last few months, they've given me a
few headaches. I've been trying to figure out how I would structure a
large system, and I'm working primarily on Windows. (I don't have a
large system yet, but hope to eventually. I've also had, throughout my
programming career, a bizarre and troublsome obsession with build and
source control systems. It hasn't affected my drinking habits yet, but
it does often scare colleagues.)
The fact that both ASDF and MK-DEFSYSTEM documentation recommend using
symbolic links caused me a lot of grief for a while, until I realized
that (at least in ASDF) it's easier to just heavily populate the
registry list. (I actually came up with a function that uses external
Cygwin programs to follow Cygwin's simulated symlinks.) The
directory-search-based solution for ASDF suggested in ASDF-INSTALL is
unpalatable, because of its ill handling of multiple system versions.
Having gotten past that hurdle, I next ran into a problem when using a
mix of ASDF and MK-DEFSYSTEM. This happens quickly when trying to use
freely available software, I guess. The first example was the CLOCC
port library. I finally hit on the solution of creating an .asd file
that forwards compile and load operations to MK-DEFSYSTEM by providing
a version of asdf:perform specialized using (eql (find-system
:port)). I'm sure this could be done more elegantly and generally by
subclassing asdf:system, but the general approach of layering an ASDF
definition seems sound.
I got sidetracked for a little while by REQUIRE, which is still used
by some software. Borrowing something from the Common Lisp Cookbook, I
redefined it, having it use ASDF where possible but improving on that
site's suggestion by forwarding requests to the system's built-in
REQUIRE when the optional pathname arguments are provided, or the
module doesn't appear to be an ASDF system. Overriding REQUIRE was
probably not necessary, but this was an interesting excercise.
So I guess I'm all squared away at this point. But one thing that
keeps bugging me is the need to edit a possibly complex set of
dependencies in a separate file. Other languages usually have
directives (#include or import) that make it relatively easy for the
programmer to specify dependencies piecemeal, making each file
self-describing in terms of its dependencies. It's easy and common for
tools to gather the dependencies together for use by a build
system. The deprecated REQUIRE and PROVIDE seem closer to this, albeit
lacking descriptive power for some aspects of systems. If I understand
correctly, it's common for people to define their own defsystems or
extensions of existing ones. Is this the sort of thing that they're
doing?
I understand that there are advantages to keeping defsystem details
_separate_ from the rest of a system, but convenience isn't one of
them. It would seem desirable for a more powerful self-describing
dependency declaration syntax to catch on and become a de facto
standard, no? If this happened, then people could even use different
defsystems with a given package, letting the dependency declarations
translate to whatever is needed by their defsystem of choice.
Any comments are welcome. I'm sure there are issues with Lisp
development that I'm not considering due to lack of familiarity;
perhaps my thinking on self-describing dependencies doesn't fit well
for some reason. Struggling with the defsystems has been educational,
but also distracting from what I'd really like to be working on with
Lisp.
Some of the references that were helpful, although not complete:
http://www.cliki.net/Getting%20Started -- especially the example setup
linked from the bottom of the page.
Common Lisp Cookbook, particularly
http://cl-cookbook.sourceforge.net/systems.html#cross
Dan Muller wrote:
> I understand that there are advantages to keeping defsystem details
> _separate_ from the rest of a system, but convenience isn't one of
> them. It would seem desirable for a more powerful self-describing
> dependency declaration syntax to catch on and become a de facto
> standard, no? If this happened, then people could even use different
> defsystems with a given package, letting the dependency declarations
> translate to whatever is needed by their defsystem of choice.
>
I thought about this for a while. One issue you need to deal with is
that there are various times at which things can be needed, and system
construction tools need to take this into account. PROVIDE & REQUIRE
on their own are not up to this because they are functions, so they
don't have efect at compile-time unless you wrap them in EVAL-WHENs.
I don't see why it should not be possible though. A system I worked
on had a partial implementation (not used for building the core
system, so I didn't have to deal with compile-time issues, but for
loading modules at runtime) which allowed various mechanisms for
describing versions and so on. You could say things like `I am
com.cley.weld.nbs.low version 1.2' and `I need com.cley.weld.nbs.low
version 1.>=1'.
--tim
"Tim Bradshaw" <··········@tfeb.org> writes:
> Dan Muller wrote:
>> I understand that there are advantages to keeping defsystem details
>> _separate_ from the rest of a system, but convenience isn't one of
>> them. It would seem desirable for a more powerful self-describing
>> dependency declaration syntax to catch on and become a de facto
>> standard, no? If this happened, then people could even use different
>> defsystems with a given package, letting the dependency declarations
>> translate to whatever is needed by their defsystem of choice.
>>
>
> I thought about this for a while. One issue you need to deal with is
> that there are various times at which things can be needed, and system
> construction tools need to take this into account. PROVIDE & REQUIRE
> on their own are not up to this because they are functions, so they
> don't have efect at compile-time unless you wrap them in EVAL-WHENs.
Yes, exactly! But it certainly seems possible to address this with the
tools in Common Lisp.
> I don't see why it should not be possible though. A system I worked
> on had a partial implementation (not used for building the core
> system, so I didn't have to deal with compile-time issues, but for
> loading modules at runtime) which allowed various mechanisms for
> describing versions and so on. You could say things like `I am
> com.cley.weld.nbs.low version 1.2' and `I need com.cley.weld.nbs.low
> version 1.>=1'.
Thanks, this is exactly the sort of thing I was curious about: How
many people write their own defsystems, or augment existing ones with
additional layers?
I really like the concept of "programmers scatter, tools gather". It
seems like programmers should be able to describe dependencies close
to where they create them, and the tools should gather than
information as needed.
Dan Muller wrote:
>
> Thanks, this is exactly the sort of thing I was curious about: How
> many people write their own defsystems, or augment existing ones with
> additional layers?
I'm hoping that ASDF is general enough that it has essentially a
meta-defsystem-protocol using which I could build a distributed
defsystem slightly more like the ones I (and maybe you) would like to
have - for instance I like to have fairly clever version matching
stuff, so I can have several versions of a system lying around and the
appropriate one will get loaded (`newest 1.x version', say). Really,
so long as whatever the normal user macros expand into is documented,
then I think you could use ASDF to build your own facility. But, like
everyone, I don't really have any time unfortunately: I'd rather spend
my free time playing the guitar...
> I really like the concept of "programmers scatter, tools gather". It
> seems like programmers should be able to describe dependencies close
> to where they create them, and the tools should gather than
> information as needed.
Yes, that's one of the things that bugs me about defsystems.
--tim
OK, good news, found the problem:
Dan Muller wrote:
> ... I've also had, throughout my
> programming career, a bizarre and troublsome obsession with build and
> source control systems. It hasn't affected my drinking habits yet, but
> it does often scare colleagues.)
..and...
> Struggling with the defsystems has been educational,
> but also distracting from what I'd really like to be working on with
> Lisp.
The distraction, as you generously concede, is self-inflicted. Doctor
Kenny prescribes deferring the defsystem angst until you have written
enough Lisp to need a defsystem. By then you will be like any other
Lispnik, having too much fun to bother much over the tedious stuff.
:)
fwiw, yes, under MCL I wrote my own poor-man's project builder, under
AllegroCL I use their project manager/defsystem-like system.
kenny
--
Cells? Cello? Celtik?: http://www.common-lisp.net/project/cells/
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
Kenny Tilton <·······@nyc.rr.com> writes:
> OK, good news, found the problem:
>
> Dan Muller wrote:
>
>> ... I've also had, throughout my
>> programming career, a bizarre and troublsome obsession with build and
>> source control systems. It hasn't affected my drinking habits yet, but
>> it does often scare colleagues.)
>
> ..and...
>
>> Struggling with the defsystems has been educational,
>> but also distracting from what I'd really like to be working on with
>> Lisp.
>
> The distraction, as you generously concede, is self-inflicted. Doctor
> Kenny prescribes deferring the defsystem angst until you have written
> enough Lisp to need a defsystem. By then you will be like any other
> Lispnik, having too much fun to bother much over the tedious stuff.
>
> :)
LOL. Good advice, I'm sure. But I keep bumping into defsystem issues
as I try to explore existing open-source software on the net. Perhaps
I'm missing simple techniques of working around this? Eh, in any case,
I think I've finally got ASDF and MK-DEFSYSTEM configured well enough
that I won't run into this as often now.
> fwiw, yes, under MCL I wrote my own poor-man's project builder, ...
Aha! Another one!
> ...under
> AllegroCL I use their project manager/defsystem-like system.
Do you run into portability issues?
Dan Muller wrote:
> Kenny Tilton <·······@nyc.rr.com> writes:
>
>
>>OK, good news, found the problem:
>>
>>Dan Muller wrote:
>>
>>
>>>... I've also had, throughout my
>>>programming career, a bizarre and troublsome obsession with build and
>>>source control systems. It hasn't affected my drinking habits yet, but
>>>it does often scare colleagues.)
>>
>>..and...
>>
>>
>>>Struggling with the defsystems has been educational,
>>>but also distracting from what I'd really like to be working on with
>>>Lisp.
>>
>>The distraction, as you generously concede, is self-inflicted. Doctor
>>Kenny prescribes deferring the defsystem angst until you have written
>>enough Lisp to need a defsystem. By then you will be like any other
>>Lispnik, having too much fun to bother much over the tedious stuff.
>>
>>:)
>
>
> LOL. Good advice, I'm sure. But I keep bumping into defsystem issues
> as I try to explore existing open-source software on the net.
Yes, "Coming to America" was a comedy, "Coming to Lisp" is a horror
movie. I think Franz and Xanalys offer friendlier edits IDE-wise, but
there is a barrier to entry.
Perhaps
> I'm missing simple techniques of working around this? Eh, in any case,
> I think I've finally got ASDF and MK-DEFSYSTEM configured well enough
> that I won't run into this as often now.
>
>
>>fwiw, yes, under MCL I wrote my own poor-man's project builder, ...
>
>
> Aha! Another one!
Sure, half a day's work. That's one of the recurring themes with Lisp.
It is so powerful one just tosses of their own solution because it is
faster to write from scratch than learn Other People's Code.
>
>
>>...under
>>AllegroCL I use their project manager/defsystem-like system.
>
>
> Do you run into portability issues?
Ya think? <g> Actually, Thomas Burdick has done some lovely work to
teach ASDF how to eat AllegroCL LPR project files. This after I
announced I was no longer going to maintain ASDF files for Cells et al.
When I move to the Mac version of my current project I will likely write
my own defsystem because both ASDF and mk::defsystem share a behavior I
cannot abide. Well, I'll look at whether I can /extend/ either of the
systems to get what I need.
kt
--
Cells? Cello? Celtik?: http://www.common-lisp.net/project/cells/
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film