Is it my imagination, or is cffi the single hardest library to provide a
single working version across a large range of dependent libraries. I
seem to end up with ghettoized projects where I have to be careful to not
include certain wrapped libraries together. Quite a few projects follow
cffi with the new datatype additions, but newer cffi's don't seem terribly
backward compatible.
Though providing compatibility or consistency would be nice, it's just
wallpaper over what seems to be a more serious problem: there seems to
be a fundamental breakage in the way asdf and the package system usually
provides name-space management.
That is to say it would be nice to provide versioned libraries in a way
that allows me effectively provide a version of a package to the code that
consumes it. Control over what package is imported for the version of a
library seems doable, but hard coded fully qualified symbols (eg
cffi:blah) seems more problematic.
Anyway, kind of a rant, but it seems like a problem with no easy answer.
Matt
--
"You do not really understand something unless you
can explain it to your grandmother." - Albert Einstein.
On Sat, 14 Jul 2007 17:09:29 -0500, <································@c.net> wrote:
> Is it my imagination, or is cffi the single hardest library to provide a
> single working version across a large range of dependent libraries.
If you look at CLSQL, you'll see that it has it's own UFFI. I think
there is another package I use that does similar, but a glance over
the folders doesn't find it. Package a version with your stuff.
--
There are no average Common Lisp programmers
Reply-To: email is ignored.
--
Posted via a free Usenet account from http://www.teranews.com
On Sun, 15 Jul 2007 10:30:03 +0530, <·······@net.meer> wrote:
> * GP lisper <·······················@phoenix.clouddancer.com> :
>| On Sat, 14 Jul 2007 17:09:29 -0500, <································@c.net> wrote:
>|> Is it my imagination, or is cffi the single hardest library to
>|> provide a single working version across a large range of dependent
>|> libraries.
>|
>| If you look at CLSQL, you'll see that it has it's own UFFI. I think
>| there is another package I use that does similar, but a glance over
>| the folders doesn't find it. Package a version with your stuff.
>
> No UFFI doesn't ship with CLSQL, you have to download and install it
> separately.
Funny, a glance at the package shows clearly a directory named uffi,
and you can read the instructions about adding pointers to it. I'd
say you don't know what you are talking about. There has never been a
a second separate download for clsql in the 2-3 years I've happily
employed it.
--
There are no average Common Lisp programmers
Reply-To: email is ignored.
--
Posted via a free Usenet account from http://www.teranews.com
On Sun, 15 Jul 2007 17:39:22 -0700, GP lisper <········@CloudDancer.com> wrote:
> On Sun, 15 Jul 2007 10:30:03 +0530, <·······@net.meer> wrote:
>
>> No UFFI doesn't ship with CLSQL, you have to download and install
>> it separately.
>
> Funny, a glance at the package shows clearly a directory named uffi,
> and you can read the instructions about adding pointers to it. I'd
> say you don't know what you are talking about. There has never been
> a a second separate download for clsql in the 2-3 years I've happily
> employed it.
CLSQL contains a directory called "uffi", but that doesn't contain the
UFFI library, only the "glue" code for the clsql_uffi shared library.
I've used CLSQL since 2003 and the tarballs never contained UFFI - it
was always a separate download. Maybe you'd like to go to
http://clsql.b9.com/download.html
and check before you accuse other people of not knowing what they're
talking about.
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Matthew Swank <································@c.net> writes:
> Is it my imagination, or is cffi the single hardest library to
> provide a single working version across a large range of dependent
> libraries. I seem to end up with ghettoized projects where I have
> to be careful to not include certain wrapped libraries together.
> Quite a few projects follow cffi with the new datatype additions,
> but newer cffi's don't seem terribly backward compatible.
>
> Though providing compatibility or consistency would be nice, it's
> just wallpaper over what seems to be a more serious problem: there
> seems to be a fundamental breakage in the way asdf and the package
> system usually provides name-space management.
>
> That is to say it would be nice to provide versioned libraries in a
> way that allows me effectively provide a version of a package to the
> code that consumes it. Control over what package is imported for the
> version of a library seems doable, but hard coded fully qualified
> symbols (eg cffi:blah) seems more problematic.
Actually, the package-qualified-symbol problem is moderately easy to
solve, sort of: modify PERFORM on (COMPILE-OP, LOAD-OP, or
LOAD-SOURCE-OP) and CL-SOURCE-FILE) so to rearrange the package
namespace before and after calls to CL:COMPILE-FILE or CL:LOAD. The
easiest way to do this is with an :AROUND method on PERFORM
specializing on those operation classes and CL-SOURCE-FILE. (It would
be better if all inter-package name usage were isolated into
DEFPACKAGE-like forms, but that's not how people do things.)
It's also fairly easy to track which packages get created by which
systems, by comparing the package namespace before and after compiling
or loading a CL-SOURCE-FILE. With this information, it's easy enough
to rename packages created by a system, say, by appending the system's
version to the name of each package created as a side-effect of
compiling or loading the system. An :AFTER method on PERFORM
specializing on one of the default operation classes and
CL-SOURCE-FILE should do the trick here.
So then all you need to do is attach versioned dependency information
to systems that need specific versions of their dependencies. You
might do this without subclassing anything by using the PROPERTIES
slot available via the COMPONENT-PROPERTY function.
The problems are several:
* ASDF's mechanism for finding systems is customizable, but not
generic: FIND-SYSTEM first converts its argument to a string, and
then invokes a customizable sequence of functions on the string
until one returns a pathname. So it's effectively necessary to
encode version information in system names in order to provide
multiple system versions in the search space. This means renaming
.asd files and rewriting :depends-on information in defsystem forms.
* A few ASDF systems extend add some subclasses of ASDF's classes, by
having code in the .asd file. Some extra work is necessary to
ensure that the contents of those .asd files don't conflict with
each other. Usually, changing some package names in the .asd files
suffices, but a few systems contain components that hard-code the
names of symbols in the package used in the .asd file. So it's
necessary, in general, to include the package used in the .asd file
as part of the package namespace rearranging. It turns out to be
impossible to automatically register the package used in the .asd
file as belonging to the system, however, so that means more
metadata has to go into the defsystem form.
* In principle, any program might assume that the run-time package
namespace is the same as the compile-time or load-time package
namespace. For example, a program might do
(with-standard-io-syntax
(let* ((*package* (find-package "P"))
(thing (read-from-string string)))
(if (eql thing 'p:foo)
...)))
This code will lose if the package that was named "P" when the code
was loaded has been renamed by the time the code is run. This has
nothing to do with ASDF per se; the program should say something
like
(find-symbol "FOO" "P")
but there's some code in the wild that does the wrong thing
analogous to the above.
* Some features of SLIME don't work correctly when the run-time
package namespace has been rearranged, for reasons like the ones
above.
* Various systems in the wild don't have version numbers, at least not
in the .asd file. This is a minor nit, since you can always make up
some numbers.
* Finally, it's probably a violation of encapsulation to add methods
to ASDF's protocol functions that specialize on its default classes,
but there's no other way to customize ASDF's site-wide behavior.
Since you have to edit .asd files that you want to be part of this
pseudo-versioning system in order for ASDF to find them, you might
subclass CL-SOURCE-FILE and add the methods described above as
specializing on your subclass, but then you'll have to do extra work
for those systems whose .asd files actually extend ASDF.
So what you're talking about is technically possible without much hard
work, though it's clearly not supported, and by inspection it's not
worth the effort. I implemented the above about a year ago as an
experiment, only to find that the annoyances involved in maintaining a
mess of locally-tweaks all over the place outweighed the benefits of
having multiple versions of a system in one image.
> Anyway, kind of a rant, but it seems like a problem with no easy
> answer.
Agreed.
I wonder what people did to support multiple versions of a system on
the LispM. Were packages versioned there?
--
RmK
In article <··············@tan-ru.localdomain>,
Richard M Kreuter <·······@progn.net> wrote:
> I wonder what people did to support multiple versions of a system on
> the LispM. Were packages versioned there?
Systems and files are versioned.
The file system stores multiple versions of a file.
About versions systems you can read here:
http://common-lisp.net/project/bknr/static/lmman/patch.xml
You might need a browser similar to Firefox to read
that page.
There was also some complicated trickery, when
you wanted to work with a piece of software
that is already loaded in some version and you
want to load the software also in another versions.
--
http://lispm.dyndns.org
Rainer Joswig <······@lisp.de> writes:
> In article <··············@tan-ru.localdomain>,
> Richard M Kreuter <·······@progn.net> wrote:
>
>> I wonder what people did to support multiple versions of a system
>> on the LispM. Were packages versioned there?
>
> Systems and files are versioned.
>
> The file system stores multiple versions of a file.
>
> About versions systems you can read here:
> http://common-lisp.net/project/bknr/static/lmman/patch.xml
> You might need a browser similar to Firefox to read
> that page.
Wow, this is an astounding facility, though it looks quite different
from what we have to work with today. IIUC, versioning was applied to
sets of fasls, not to source code snapshots. How did people refer to
versions of collections of sources? And did this facility help with
cases where system A depended on system B, and a new version of system
B exposed a not-backward-compatible API?
> There was also some complicated trickery, when
> you wanted to work with a piece of software
> that is already loaded in some version and you
> want to load the software also in another versions.
I think the functionality of the complicated trickery what the OP was
wondering about. In particular, how could two versions of a system
reside in one image? Did they frob the package names, or similar?
And what about dependent systems and packages created by dependent
systems? Or did people not want to do this as much then as now?
--
RmK
In article <··············@tan-ru.localdomain>,
Richard M Kreuter <·······@progn.net> wrote:
> Rainer Joswig <······@lisp.de> writes:
> > In article <··············@tan-ru.localdomain>,
> > Richard M Kreuter <·······@progn.net> wrote:
> >
> >> I wonder what people did to support multiple versions of a system
> >> on the LispM. Were packages versioned there?
> >
> > Systems and files are versioned.
> >
> > The file system stores multiple versions of a file.
> >
> > About versions systems you can read here:
> > http://common-lisp.net/project/bknr/static/lmman/patch.xml
> > You might need a browser similar to Firefox to read
> > that page.
>
> Wow, this is an astounding facility, though it looks quite different
> from what we have to work with today. IIUC, versioning was applied to
> sets of fasls, not to source code snapshots.
No, it was applied to all files that make up a system.
Source files, FASL file, documentation files,
font files, whatever.
The patch facility also provided 'delta' files.
So, when one loads software, the major version
is loaded and then the patches piece by piece.
If you edit this version, then you edit the
corresponding source files (the file system
stores multiple versions of files).
> How did people refer to
> versions of collections of sources? And did this facility help with
> cases where system A depended on system B, and a new version of system
> B exposed a not-backward-compatible API?
IIRC it did, but I'm not sure if it was this version of the
system facility or a later. I would have to look if you
could add those requirements to the defsystem here.
>
> > There was also some complicated trickery, when
> > you wanted to work with a piece of software
> > that is already loaded in some version and you
> > want to load the software also in another versions.
>
> I think the functionality of the complicated trickery what the OP was
> wondering about. In particular, how could two versions of a system
> reside in one image? Did they frob the package names, or similar?
> And what about dependent systems and packages created by dependent
> systems? Or did people not want to do this as much then as now?
The Lisp Machine had features in the package system for that.
One such feature are hierarchical packages.
I have never used it that way, such that I tried to
work with two different versions of a system. So I can't
say how much effort it was and whether it really helped.
>
> --
> RmK
--
http://lispm.dyndns.org
Matthew Swank <································@c.net> writes:
> That is to say it would be nice to provide versioned libraries in a
> way that allows me effectively provide a version of a package to the
> code that consumes it. Control over what package is imported for the
> version of a library seems doable, but hard coded fully qualified
> symbols (eg cffi:blah) seems more problematic.
For some possibly out-of-reach inspiration here, see the OSGi� system
for Java. In particular, see sections 3.2 and 3.4--3.7 in the core
specification� for a discussion of versioned package dependency
resolution.
Footnotes:
� http://www.osgi.org/osgi_technology/index.asp?section=2
� http://www2.osgi.org/Download/Release4V41
--
Steven E. Harris