In my slow journey along the Lisp highway, I now need to start using
ASDF. Any recommendations on how to organize ASDF packages, e.g. put
them in /usr/local/lisp/asdf? What has worked for you? What works well
with multiple Lisps (e.g. sbcl and LispWorks)?
On 18 Mai, 04:19, ····@stablecross.com (Bob Felts) wrote:
> In my slow journey along the Lisp highway, I now need to start using
> ASDF. Any recommendations on how to organize ASDF packages, e.g. put
> them in /usr/local/lisp/asdf? What has worked for you? What works well
> with multiple Lisps (e.g. sbcl and LispWorks)?
I do not use symbolic links to
register ASDF packages. Using symbolic links doesn't really work out
on MS Windows
which is one of my target platforms (I never liked the symbolic link
approach on other platforms too). Instead of the links I use a
function
which scans all directories in my Libraries directory remembering
those
which have .asd files. I do a check if there are conflicting ASDF
systems
(same system-name on different locations) and if not I just push all
those
directories in the ASDF:*CENTRAL-REGISTRY* list. This allows me to
install a new library by just dropping it in my libraries folder and
double-click it to unarchive it.
The directory layout of my setup is as follows:
.../Lisp/Branches/ ; e.g. copies of old releases of
lisp systems
.../Config/asdf-config.lisp ; load asdf.lisp setup ASDF:*CENTRAL-
REGISTRY*
.../editor.lisp ; editor commands such as "Load ASDF
System"
.../key-binds.lisp ; ... well ;-) as the name suggests
.../start.lisp ; load all config-files in the right
order
.../Systems/IDE/ ; Lisp-Systems which extend the IDE
.../Libraries/ ; Libraries without changes from
myself
.../Projects/ ; Libraries and applications I work
on
.../Lisp/Branches/ ; e.g. For copies of old releases of
lisp systems
.../Config/asdf-config.lisp
.../editor.lisp
.../key-binds.lisp
.../start.lisp
.../Systems/IDE/
.../Libraries/
.../Projects/
I use a function called SETUP-ASDF-CENTRAL-REGISTRY to include all
directories in
my Systems folder which have .asd files. It uses a function FIND-ASDF-
DEFINITION-PLACES
which is a bit slow, but since it is not called that often it was not
really a problem for me.
If I want to install a new library I just download the system tarball
onto my Desktop, double-click
it to unarchive, and drop the unarchived folder to ../Systems/
Libraries/. In the listener I type
(setup-asdf-central-registry)
and then
(asdf:oos 'asdf:load-op :the-new-libraries-name)
(Actually I just use the editor command "Load ASDF System" which does
the ASDF:OOS for me
and tries SETUP-ASDF-CENTRAL-REGISTRY if it doesn't find the system)
Here is the code for SETUP-ASDF-CENTRAL-REGISTRY and friends:
(defvar *system-root* #+mac "/Users/jsc/Documents/Entwicklung/Lisp/
Systems/"
#+win32 "D:/Lisp/Systems/")
(defvar *asdf-system-directories*
(mapcar (lambda (d) (merge-pathnames d *system-root*))
'("Libraries/"
"IDE/" ;; Plugins
"Projects/")))
(define-condition asdf-error (error)())
(define-condition asdf-system-definition-conflict (asdf-error)
((conflicts :accessor asdf-system-definition-conflicts
:initarg :conflicts)
(unique-files :accessor asdf-system-definition-unique-files
:initarg :unique-files))
(:report (lambda (condition stream)
(format stream "Found conflicting system-definition~P
~A."
(length (asdf-system-definition-conflicts
condition))
(asdf-system-definition-conflicts condition)))))
(defun setup-asdf-central-registry (&optional ignore-errors)
(handler-bind ((asdf-system-definition-conflict
(lambda (c)
(when ignore-errors
(use-value
(append (asdf-system-definition-conflicts c)
(asdf-system-definition-unique-files
c)))))))
(setf asdf:*central-registry*
(union
(find-asdf-definition-places *asdf-system-directories*)
asdf:*central-registry*
:test #'equal))))
(defun find-asdf-definition-places
(directories &key (test (lambda (p)
(not (find "_darcs" (pathname-
directory p)
:test #'equal)))))
(restart-case
(let ((asd-files
(delete-if-not
test
(mapcan (lambda (directory)
(directory (merge-pathnames
(make-pathname
:name :wild
:type "asd"
:directory '(:relative :wild-
inferiors))
directory)))
directories))))
(let* ((asd-files\\duplicates (remove-duplicates asd-files
:key
#'pathname-name
:test
#'string-equal))
(duplicates (set-difference asd-files asd-files\
\duplicates))
(unique-files (set-difference asd-files\\duplicates
duplicates
:key #'pathname-name
:test #'string-equal))
(conflicts (set-difference asd-files unique-files)))
(remove-duplicates
(mapcar (lambda (p) (make-pathname :name nil
:type nil
:defaults
p))
(if conflicts
(restart-case
(error 'asdf-system-definition-conflict
:conflicts conflicts
:unique-files unique-files)
(use-value (v)
v))
unique-files))
:test #'equal)))
(try-again ()
(find-asdf-definition-places directories :test test))))
ciao,
Jochen
On 2007-05-18, ··@codeartist.org <··@codeartist.org> wrote:
> I do not use symbolic links to register ASDF packages. Using symbolic
> links doesn't really work out on MS Windows which is one of my target
> platforms (I never liked the symbolic link approach on other platforms
> too).
Personally I still find Windows shortcuts to be an attractive solution
for ASDF on Windows. (But I also like symbolic links.)
d.
On 18 May 2007 06:07:28 -0700, <··@codeartist.org> wrote:
>
> I do not use symbolic links to register ASDF packages. Using symbolic
> links doesn't really work out on MS Windows which is one of my target
> platforms (I never liked the symbolic link approach on other platforms
> too). Instead of the links I use a function which scans all
> directories in my Libraries directory remembering those which have
> .asd files. I do a check if there are conflicting ASDF systems (same
> system-name on different locations) and if not I just push all those
> directories in the ASDF:*CENTRAL-REGISTRY* list. This allows me to
> install a new library by just dropping it in my libraries folder and
> double-click it to unarchive it.
Symlinks are just too much effort for too little gain. A simple file
searcher that finds files with asd types works great, and version
control is simple via compression tools. As you note, this way means
simply placing sourcecode in the appropriate directory does all.
By keeping source in one location, changes to vendor lisp are simple
too. That's one location perl lisp image, since it is easy to copy
the source, and then remove the old FASLs.
--
There are no average Common Lisp programmers
Reply-To: email is ignored.
--
Posted via a free Usenet account from http://www.teranews.com
Bob Felts wrote:
> In my slow journey along the Lisp highway, I now need to start using
> ASDF. Any recommendations on how to organize ASDF packages, e.g. put
> them in /usr/local/lisp/asdf? What has worked for you? What works well
> with multiple Lisps (e.g. sbcl and LispWorks)?
I load ASDF in my initialization files, and afterwards a compiled
version of a file containing the following code.
The first method ensures that, in case a compiled file cannot be loaded
because it has been generated by an old version of the respective
implementation, it will be automatically recompiled. (I have stolen this
idea from somewhere else, but I don't recall where from.)
The second methods ensures that compiled files are stored in
subdirectories with names that reflect the name (and sometimes version)
of the respective CL implementation. This way, the directories remain
"clean" and the extensions for compiled files cannot clash.
The third form tells ASDF that all system files are stored in a central
locations (which contains symbolic links to the actual system definition
files in the respective project directories).
Overall, this works pretty well. The only manual changes I have to make
is when a new version of a CL implementation comes out that changes
compiled files in an incompatible way - then I have to manually
recompile the file below for that version. (I put the compiled version
of that file again in subdirectories with names reflection the
implementation + version and conditionalize the initialization files for
the various CL implementations accordingly.)
I hope this helps. Maybe other people have simpler solutions...
Pascal
(defmethod asdf:perform :around
((o asdf:load-op) (c asdf:cl-source-file))
(handler-case (call-next-method o c)
(error ()
(asdf:perform (make-instance 'asdf:compile-op) c)
(call-next-method))))
(defmethod asdf:output-files :around
((op asdf:compile-op) (src asdf:source-file))
(let ((paths (call-next-method)))
(mapcar (lambda (path)
(merge-pathnames
(make-pathname
:directory
(append (pathname-directory path)
(list #+allegro-v6.2 ".acl62"
#+allegro-v7.0 ".acl70"
#+allegro-v8.0 ".acl80"
#+clisp ".clisp"
#+cmu ".cmu"
#+ecl ".ecl"
#+gcl ".gcl"
#+lispworks4.3 ".lispworks4.3"
#+lispworks4.4 ".lispworks4.4"
#+lispworks5.0 ".lispworks5.0"
#+(and mcl (not openmcl)) ".mcl"
#+openmcl ".openmcl"
#+sbcl ".sbcl")))
path))
paths)))
(setf asdf:*central-registry*
`(,@asdf:*central-registry*
#-(and mcl (not openmcl))
"/Users/costanza/lisp/systems/"
#+(and mcl (not openmcl))
"Macintosh HD:Users:costanza:lisp:systems:"))
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Pascal Costanza <··@p-cos.net> wrote:
> Bob Felts wrote:
> > In my slow journey along the Lisp highway, I now need to start using
> > ASDF. Any recommendations on how to organize ASDF packages, e.g. put
> > them in /usr/local/lisp/asdf? What has worked for you? What works well
> > with multiple Lisps (e.g. sbcl and LispWorks)?
>
> I load ASDF in my initialization files, and afterwards a compiled
> version of a file containing the following code.
Could you show me how you do this?
Bob Felts wrote:
> Pascal Costanza <··@p-cos.net> wrote:
>
>> Bob Felts wrote:
>>> In my slow journey along the Lisp highway, I now need to start using
>>> ASDF. Any recommendations on how to organize ASDF packages, e.g. put
>>> them in /usr/local/lisp/asdf? What has worked for you? What works well
>>> with multiple Lisps (e.g. sbcl and LispWorks)?
>> I load ASDF in my initialization files, and afterwards a compiled
>> version of a file containing the following code.
>
> Could you show me how you do this?
Here is my initialization file for SBCL:
(require "ASDF")
(load #P"/Users/costanza/lisp/asdf/.sbcl/fix-asdf")
Here is an excerpt of the one for LispWorks:
#+lispworks4.3
(progn
(load #P"/Users/costanza/lisp/asdf/.lispworks4.3/asdf")
(load #P"/Users/costanza/lisp/asdf/.lispworks4.3/fix-asdf"))
#+lispworks4.4
(progn
(load #P"/Users/costanza/lisp/asdf/.lispworks4.4/asdf")
(load #P"/Users/costanza/lisp/asdf/.lispworks4.4/fix-asdf"))
#+lispworks5.0
(progn
(load #P"/Users/costanza/lisp/asdf/.lispworks5.0/asdf")
(load #P"/Users/costanza/lisp/asdf/.lispworks5.0/fix-asdf"))
Similar for all the other systems. So load asdf first (depending on
which implementation you are using it may simply be already there, or
you may have to load it explicitly), then load fix-asdf (the compiled
file I have talked about in my previous posting).
If you encounter a problem loading fix-asdf, this probably means that
you have a new version of the respective CL implementation. This is easy
to resolve: Just start the CL implementation in the folder that contains
fix-asdf, and then say (compile-file "fix-asdf"). Exit your lisp again,
and put the file in the right folder.
Does this help?
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Pascal Costanza <··@p-cos.net> wrote:
> Bob Felts wrote:
> > Pascal Costanza <··@p-cos.net> wrote:
> >
> >> Bob Felts wrote:
> >>> In my slow journey along the Lisp highway, I now need to start using
> >>> ASDF. Any recommendations on how to organize ASDF packages, e.g. put
> >>> them in /usr/local/lisp/asdf? What has worked for you? What works well
> >>> with multiple Lisps (e.g. sbcl and LispWorks)?
> >> I load ASDF in my initialization files, and afterwards a compiled
> >> version of a file containing the following code.
> >
> > Could you show me how you do this?
>
> Here is my initialization file for SBCL:
>
> (require "ASDF")
> (load #P"/Users/costanza/lisp/asdf/.sbcl/fix-asdf")
>
> Here is an excerpt of the one for LispWorks:
>
> #+lispworks4.3
> (progn
> (load #P"/Users/costanza/lisp/asdf/.lispworks4.3/asdf")
> (load #P"/Users/costanza/lisp/asdf/.lispworks4.3/fix-asdf"))
> #+lispworks4.4
> (progn
> (load #P"/Users/costanza/lisp/asdf/.lispworks4.4/asdf")
> (load #P"/Users/costanza/lisp/asdf/.lispworks4.4/fix-asdf"))
> #+lispworks5.0
> (progn
> (load #P"/Users/costanza/lisp/asdf/.lispworks5.0/asdf")
> (load #P"/Users/costanza/lisp/asdf/.lispworks5.0/fix-asdf"))
>
> Similar for all the other systems. So load asdf first (depending on
> which implementation you are using it may simply be already there, or
> you may have to load it explicitly), then load fix-asdf (the compiled
> file I have talked about in my previous posting).
>
> If you encounter a problem loading fix-asdf, this probably means that
> you have a new version of the respective CL implementation. This is easy
> to resolve: Just start the CL implementation in the folder that contains
> fix-asdf, and then say (compile-file "fix-asdf"). Exit your lisp again,
> and put the file in the right folder.
>
> Does this help?
>
Sort of. It helps fill in holes in my knowledge, but actually getting
this to work is a pain. First, is there any special reason why "." is
used in directory names, e.g. /.sbcl? That makes the directory
invisible so I'd like it to be just "sbcl".
In either case, now I'm getting all sorts of file permssion problems.
I supposed that /usr/local/lisp would be a good place for downloaded
code, but slime is having all sorts of problems writing to various
subdiretories. I'm fixing this as I go along. The alternative is to
put these files somewhere in my user directory but that seems
counterintuitive in a multi-user system.
I'm now also getting " Condition: The CC environment variable has not
been set in SB-GROVEL..." Sigh. One more thing to track down.
I love the language but getting the environment set up isn't as
pleasant. I don't know where I'd be without your help. Thanks.
Bob Felts wrote:
> Pascal Costanza <··@p-cos.net> wrote:
>
>> Bob Felts wrote:
>>> Pascal Costanza <··@p-cos.net> wrote:
>>>
>>>> Bob Felts wrote:
>>>>> In my slow journey along the Lisp highway, I now need to start using
>>>>> ASDF. Any recommendations on how to organize ASDF packages, e.g. put
>>>>> them in /usr/local/lisp/asdf? What has worked for you? What works well
>>>>> with multiple Lisps (e.g. sbcl and LispWorks)?
>>>> I load ASDF in my initialization files, and afterwards a compiled
>>>> version of a file containing the following code.
>>> Could you show me how you do this?
>> Here is my initialization file for SBCL:
>>
>> (require "ASDF")
>> (load #P"/Users/costanza/lisp/asdf/.sbcl/fix-asdf")
>>
>> Here is an excerpt of the one for LispWorks:
>>
>> #+lispworks4.3
>> (progn
>> (load #P"/Users/costanza/lisp/asdf/.lispworks4.3/asdf")
>> (load #P"/Users/costanza/lisp/asdf/.lispworks4.3/fix-asdf"))
>> #+lispworks4.4
>> (progn
>> (load #P"/Users/costanza/lisp/asdf/.lispworks4.4/asdf")
>> (load #P"/Users/costanza/lisp/asdf/.lispworks4.4/fix-asdf"))
>> #+lispworks5.0
>> (progn
>> (load #P"/Users/costanza/lisp/asdf/.lispworks5.0/asdf")
>> (load #P"/Users/costanza/lisp/asdf/.lispworks5.0/fix-asdf"))
>>
>> Similar for all the other systems. So load asdf first (depending on
>> which implementation you are using it may simply be already there, or
>> you may have to load it explicitly), then load fix-asdf (the compiled
>> file I have talked about in my previous posting).
>>
>> If you encounter a problem loading fix-asdf, this probably means that
>> you have a new version of the respective CL implementation. This is easy
>> to resolve: Just start the CL implementation in the folder that contains
>> fix-asdf, and then say (compile-file "fix-asdf"). Exit your lisp again,
>> and put the file in the right folder.
>>
>> Does this help?
>>
>
> Sort of. It helps fill in holes in my knowledge, but actually getting
> this to work is a pain. First, is there any special reason why "." is
> used in directory names, e.g. /.sbcl? That makes the directory
> invisible so I'd like it to be just "sbcl".
I prefer them to be invisible, but that's all. No other specific reasons.
> In either case, now I'm getting all sorts of file permssion problems.
> I supposed that /usr/local/lisp would be a good place for downloaded
> code, but slime is having all sorts of problems writing to various
> subdiretories. I'm fixing this as I go along. The alternative is to
> put these files somewhere in my user directory but that seems
> counterintuitive in a multi-user system.
I have put my stuff in my user directory. First attempts to put things
centrally failed for me as well, but since I don't have any multi-user
requirements, I haven't looked into this any further. (Maybe someone
else can chime in here...)
> I'm now also getting " Condition: The CC environment variable has not
> been set in SB-GROVEL..." Sigh. One more thing to track down.
No idea what's that about.
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Pascal Costanza <··@p-cos.net> wrote:
> Bob Felts wrote:
> > I'm now also getting " Condition: The CC environment variable has not
> > been set in SB-GROVEL..." Sigh. One more thing to track down.
>
> No idea what's that about.
>
... sbcl/sb-grovel/def-to-lisp.lisp has the following bit of code:
(let* ((cc (or (sb-ext:posix-getenv "CC")
;; It might be nice to include a CONTINUE or
;; USE-VALUE restart here, but ASDF seems to insist
;; on handling the errors itself.
(error "The CC environment variable has not been set
in SB-GROVEL. Since this variable should always be set during the SBCL
build process, this might indicate an SBCL with a broken contrib
installation.")))
I wimped out and changed it to:
(let* ((cc (or (sb-ext:posix-getenv "CC")
"gcc" ; wrf3 23 may 2007
;; It might be nice to include a CONTINUE or
;; USE-VALUE restart here, but ASDF seems to insist
;; on handling the errors itself.
(error "The CC environment variable has not been set
in SB-GROVEL. Since this variable should always be set during the SBCL
build process, this might indicate an SBCL with a broken contrib
installation.")))
So slime/sbcl is back up.
Pascal Costanza wrote:
>Bob Felts wrote:
>> The alternative is to
>> put these files somewhere in my user directory but that seems
>> counterintuitive in a multi-user system.
> I have put my stuff in my user directory. First attempts to put things
> centrally failed for me as well, but since I don't have any multi-user
> requirements, I haven't looked into this any further. (Maybe someone
> else can chime in here...)
Not sure if this is the chiming you had in mind, but here's a
chime re Bob's "counterintuitive":
On large multiuser unix/linux systems in research environments, it is in
fact common for a user to be designated a maintainer of a dataset, and
maintain a by-other-users-read-accessible copy of a dataset (including
source trees or binaries or whatever) in a subdir of their their home
dir or another designated dir that's been chowned to them, for use by
all other users on the system, or maybe just ones in their research
group (expressed as a unix group).
So in this case, Bob could be the owner of /usr/local/lisp,
and be in charge of installing lisp stuff there for all other users
to use... :-)
(just in case: just because /usr/local is owned by root doesn't mean
that /usr/local/lisp has to be - wondering as to the source of Bob's
permission problems)
(In a more extravagant scheme, you might be given another role (or role
emulated with a secondary identity) to switch to to do installs
to /usr/local/lisp, so that in normal use, your normal role can't
trash /usr/local/lisp and so that when you then go back and test with
your normal role, your read/write access doesn't mask a problem other
users with readonly access might experience. Still not need for actual
root access though, once /usr/local/lisp has been chowned to
ultrabob... )
Another example, to totally labour the point, a meteorologist might be
the builder of the official binaries for simulation runs, and just make
them "meteo" group or "world" readable in a subdir of his home dir,
along with group or world readable up-to-date land-use tables.
Other mets in his group on the supercomputer just run the binaries from
his homedir, using input data partly from his homedir, but outputting to
working subdirs of their own home dirs.
It's at least partly for this reason that /home/jsmith etc. - i.e. the
home directory itself, is often 0755 - users can choose to share or not
share data then, they can still set a non-world-readable-by-default
umask and use private subdirs of the home (e.g. an
0700 /home/jsmith/not_porn_really_honest_its_not/ ).
That way, users can ad-hoc collaborate without bothering
lazy sysadmins too much looking for additional groups, shared
storage areas and extra privileges.
These days of course you have ACLs, but the traditional
relatively-anaemic-looking unix permissioning scheme is
quite flexible enough to meet the needs of many smaller
organisations.
And I'm so not saying this model is perfect, but for many non-secret
multiuser systems, it serves well enough.
Pascal Costanza <··@p-cos.net> writes:
> Bob Felts wrote:
>> I'm now also getting " Condition: The CC environment variable has not
>> been set in SB-GROVEL..." Sigh. One more thing to track down.
>
> No idea what's that about.
SBCL ships with a number of "contrib modules", which are structured as
asdf systems, and which have their fasls placed beside the
corresponding source files. Adding a method to ASDF:OUTPUT-FILES that
changes the location of fasl files causes asdf to have to recompile
any generalized instance of cl-source-file in every system you
subsequently try to load, so after adding such a method, the contribs
need to be recompiled.
Unfortunately, SBCL's contrib modules are a bit weird: the
infrastructure that builds SBCL distributions is ultimately driven by
Unix shell scripts and Makefiles that set up some environment
variables, and so sb-grovel, an asdf extension that's a prerequisite
for some other contrib modules, depends on an environment variable
that's likely to be unset in an ordinary SBCL instance.
Gary King's asdf-binary-locations does approximately the same as your
fix-asdf, but has provisions for not altering the fasl file location
for some components (I think it does something like pathname-match on
the component's pathname).
So to summarize: don't mess with ASDF:OUTPUT-FILES, or special case
some things if you do, or don't special case things if you do and
carefully move some fasls around.
--
RmK
Bob Felts wrote:
> I love the language but getting the environment set up isn't as
> pleasant. I don't know where I'd be without your help. Thanks.
You're welcome. Another note: If anyone wants to do a writeup for some
website of all the bits and pieces to make this work, please feel free
to include (and modify) what I posted.
Cheers,
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
On Fri, 18 May 2007 04:19:55 +0200, Bob Felts <····@stablecross.com> wrote:
> In my slow journey along the Lisp highway, I now need to start using
> ASDF. Any recommendations on how to organize ASDF packages, e.g. put
> them in /usr/local/lisp/asdf? What has worked for you? What works well
> with multiple Lisps (e.g. sbcl and LispWorks)?
For LispWorks. Load the whole thing with Edi Weits's starter pack.
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/