Hi,
It seems that DIRECTORY gives different results depending on the
implementation (see below). Are there various possible interpretations
of the specs ? Should implementations converge in order to give the
same result ?
I am not using the latest version of each implementation so the
problem may have been solved in the meantime.
Thibault
$ cd /tmp
$ mkdir d1
$ touch d1/.a
$ touch d1/b.
$ touch d1/c.d
$ touch d1/e
$ sbcl
This is SBCL 1.0.9, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (directory "/tmp/d1")
(#P"/tmp/d1/")
* (directory "/tmp/d1/")
(#P"/tmp/d1/")
* (directory "/tmp/d1/*")
(#P"/tmp/d1/.a" #P"/tmp/d1/e")
* (directory "/tmp/d1/.*")
(#P"/tmp/d1/.a")
* (directory "/tmp/d1/*.*")
(#P"/tmp/d1/.a" #P"/tmp/d1/b." #P"/tmp/d1/c.d" #P"/tmp/d1/e")
$ cmucl19c
; Loading #P"/home/tl/init.lisp".
;; Loading #P"/home/tl/comp/lang/lisp/code/asdf/asdf.x86f".
;; Loading #P"/home/tl/root/lisp/pdoc.lisp".
CMU Common Lisp 19c (19C), running on di-tl
With core: /home/tl/comp/lang/lisp/cmucl-19c/lib/cmucl/lib/lisp.core
Dumped on: Thu, 2005-11-17 14:12:58Z on lorien
See <http://www.cons.org/cmucl/> for support information.
Loaded subsystems:
Python 1.1, target Intel x86
CLOS based on Gerd's PCL 2004/04/14 03:32:47
* (directory "/tmp/d1")
(#P"/tmp/d1/")
* (directory "/tmp/d1/")
(#P"/tmp/d1/.a" #P"/tmp/d1/b." #P"/tmp/d1/c.d" #P"/tmp/d1/e")
* (directory "/tmp/d1/*")
(#P"/tmp/d1/.a" #P"/tmp/d1/b." #P"/tmp/d1/c.d" #P"/tmp/d1/e")
* (directory "/tmp/d1/.*")
(#P"/tmp/d1/.a")
* (directory "/tmp/d1/*.*")
(#P"/tmp/d1/.a" #P"/tmp/d1/b." #P"/tmp/d1/c.d" #P"/tmp/d1/e")
*
$ clisp
Copyright (c) Bruno Haible, Michael Stoll 1992, 1993
Copyright (c) Bruno Haible, Marcus Daniels 1994-1997
Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998
Copyright (c) Bruno Haible, Sam Steingold 1999-2000
Copyright (c) Sam Steingold, Bruno Haible 2001-2006
;; Loading file /home/tl/.clisprc.lisp ...
;; Loading file /home/tl/code/lisp/asdf.fas ...
;; Loaded file /home/tl/code/lisp/asdf.fas
;; Loaded file /home/tl/.clisprc.lisp
[1]> (directory "/tmp/d1")
*** - DIRECTORY: "/tmp/d1" names a directory, not a file
The following restarts are available:
ABORT :R1 ABORT
Break 1 [2]>
[3]> (directory "/tmp/d1/")
(#P"/tmp/d1/")
[4]> (directory "/tmp/d1/*")
(#P"/tmp/d1/.a" #P"/tmp/d1/e" #P"/tmp/d1/c.d" #P"/tmp/d1/b.")
[5]> (directory "/tmp/d1/.*")
(#P"/tmp/d1/.a")
[6]> (directory "/tmp/d1/*.*")
(#P"/tmp/d1/.a" #P"/tmp/d1/c.d" #P"/tmp/d1/b.")
[7]> (lisp-implementation-version)
"2.39 (2006-07-16) (built on guru.build.karan.org)"
Thibault Langlois <·················@gmail.com> writes:
> Hi,
> It seems that DIRECTORY gives different results depending on the
> implementation (see below). Are there various possible interpretations
> of the specs ?
Yes. The problem you are having is with the mapping of pathname
components into the file system concepts, and that can vary:
From the Hyperspec:
19.1.2 Pathnames as Filenames
...
The mapping of the pathname components into the concepts peculiar to
each file system is implementation-defined
...
--
Thomas A. Russ, USC/Information Sciences Institute
On Feb 27, 6:34 pm, ····@sevak.isi.edu (Thomas A. Russ) wrote:
> Thibault Langlois <·················@gmail.com> writes:
> > Hi,
> > It seems that DIRECTORY gives different results depending on the
> > implementation (see below). Are there various possible interpretations
> > of the specs ?
>
> Yes. The problem you are having is with the mapping of pathname
> components into the file system concepts, and that can vary:
>
> From the Hyperspec:
>
> 19.1.2 Pathnames as Filenames
>
> ...
>
> The mapping of the pathname components into the concepts peculiar to
> each file system is implementation-defined
>
> ...
>
OK thanks for your answer. We all have to live with that.
Nevertheless I think it would be more understandable if it was file
system-specific instead of implementation-specific.
> --
> Thomas A. Russ, USC/Information Sciences Institute
From: Kent M Pitman
Subject: Re: DIRECTORY behavior with /= implementations
Date:
Message-ID: <uir09vu5q.fsf@nhplace.com>
Thibault Langlois <·················@gmail.com> writes:
> > The mapping of the pathname components into the concepts peculiar to
> > each file system is implementation-defined
>
> OK thanks for your answer. We all have to live with that.
> Nevertheless I think it would be more understandable if it was file
> system-specific instead of implementation-specific.
I'm not sure this is literally true. One might get some warm fuzzy
from such wording, but after a while one would ponder about what that
fuzzy wording meant.
For example, I'm not sure to what extent Linux existed at the time of
the standard went out. (I don't recall the "feature freeze" date, but
it was a a few years before the final version--perhaps as early as
1988. We made essential changes after that up to publication time, but
it was hard to make such changes.)
But in any case, operating systems come and go, and might still have
done so. For the standard to have mentioned a specific file system in
a way that required behavior (rather than, as happens now, merely to
illustrate the option to vary) it would have been for the standard to
doom itself to being unusable in a small number of years.
If the standard had said file-system specific, that would imply that
no conforming implementation could exist on a platform without a
standard in place for its file system. That would be an undue burden,
not to mention probably doing things in the wrong order, since good
standards usually come after implementation+use, not before.
Keep in mind, too, that what implementation survived was in part a
product of what answers vendors made to these questions in order to
make their product useful. For the standard to decide when there was
legitimate and active controversy in the market risked that the
standard would doom otherwise viable implementations. The vendors on
platforms where there were multiple implementations actually needed
the flexibility to disagree precisely because these were not settled
matters.
It also is not clear what "standard" would mean beyond the standard.
Would ANSI CL saying there had to be a way of specifying it per file
system mean it was ok for there to be a de facto standard? An IEEE
standard? Would an existing conforming implementation be suddenly
non-conforming if someone came out with a new implementation on the
same file system that diverged?
The standard says what is necessary to say: that it cannot control
truth beyond its own walls. The contributors to the standard, to
include myself as editor, knew that it was possible for there to be
file-system specific layered standards on of popular platforms, and
that on obscure platforms it would not matter that there was
standardization. So we left it blank, and left it to markets and
interested communities to sort out the lines of aggregation for
partial layered standards. It might be nice if the universe were
other than it is. But the standard works within existing physics.
So I stand by that particular word choice, even if I'm easily willing
to believe there are other ill-chosen wordings all around it. :)
Kent M Pitman <······@nhplace.com> writes:
> Thibault Langlois <·················@gmail.com> writes:
>
>> > The mapping of the pathname components into the concepts peculiar to
>> > each file system is implementation-defined
>>
>> OK thanks for your answer. We all have to live with that.
>> Nevertheless I think it would be more understandable if it was file
>> system-specific instead of implementation-specific.
>
> I'm not sure this is literally true. One might get some warm fuzzy
> from such wording, but after a while one would ponder about what that
> fuzzy wording meant.
>
> For example, I'm not sure to what extent Linux existed at the time of
> the standard went out. (I don't recall the "feature freeze" date, but
> it was a a few years before the final version--perhaps as early as
> 1988. We made essential changes after that up to publication time, but
> it was hard to make such changes.)
Not Operating System, but File System. And it's even worse than that:
mount options may intervene. For example, when you mount an ISO-9660
file system on Linux, you get a subdirectory tree where you have files
with names in 8.3 with a version number! That is, unless you have
some unix or "Joliet" extension which gives you normal file names, but
you can still give options to mount to see it without these
extensions.
Another example would be MacOSX, where the default file system with
the default options is not case sensitive, but where you still can
mount "normal" unix file systems that are case sensitive.
So, even in a given Operating System, you may have serveral File
Systems with different properties, and different mapping to the OS API
for FS access.
That said, if all implementations running on Linux could agree on the
mapping between ext2/ext3 file systems and CL pathnames, it already be
a nice step forward.
Similarly for all implementations running on MS-Windows to map NTFS and CL.
On the other hand, some implementations may want to present an
homogeneous view (for CL code) across various OS and FS, and this
requirement is at cross with the portability one (even if it eases
intra-'implementation' portability).
--
__Pascal Bourguignon__
On 5294 September 1993, Pascal J. Bourguignon wrote:
> That said, if all implementations running on Linux could agree on the
> mapping between ext2/ext3 file systems and CL pathnames, it already be
> a nice step forward.
> Similarly for all implementations running on MS-Windows to map NTFS
> and CL.
I think that might be a nice CDR proposal, where "that" translates to
"proposal how file systems and CL pathnames should match on the major
platforms". Now, what constitutes a major platform is probably a
matter of debate.
BTW, I think it would be nice if there would be some obvious link from
cliki and the ALU pages/wiki to CDR, although one might argue that
four final CDR documents hardly is a lot of content.
> On the other hand, some implementations may want to present an
> homogeneous view (for CL code) across various OS and FS, and this
> requirement is at cross with the portability one (even if it eases
> intra-'implementation' portability).
Well, that would have to be seen, wouldn't it? Of course, pathnames
are file system specific and some parts might not be possible to hide
beneath some homogeneous view, but I fail to see how that differs from
the situation we have today. But perhaps I'm missing something obvious
here ..
Holger
--
--- http://hillview.bugwriter.net/ ---
"Ja, und ganz sicher sind *SIE* hinter uns her und wollen uns Qt und KDE
nehmen, so wie sie es schon mit Bielefeld gemacht haben."
-- Florian Kuehnert in de.comp.os.linux.misc
Holger Schauer <··············@gmx.de> writes:
> On 5294 September 1993, Pascal J. Bourguignon wrote:
>> That said, if all implementations running on Linux could agree on the
>> mapping between ext2/ext3 file systems and CL pathnames, it already be
>> a nice step forward.
>
>> Similarly for all implementations running on MS-Windows to map NTFS
>> and CL.
>
> I think that might be a nice CDR proposal, where "that" translates to
> "proposal how file systems and CL pathnames should match on the major
> platforms". Now, what constitutes a major platform is probably a
> matter of debate.
>
> BTW, I think it would be nice if there would be some obvious link from
> cliki and the ALU pages/wiki to CDR, although one might argue that
> four final CDR documents hardly is a lot of content.
>
>> On the other hand, some implementations may want to present an
>> homogeneous view (for CL code) across various OS and FS, and this
>> requirement is at cross with the portability one (even if it eases
>> intra-'implementation' portability).
>
> Well, that would have to be seen, wouldn't it? Of course, pathnames
> are file system specific and some parts might not be possible to hide
> beneath some homogeneous view, but I fail to see how that differs from
> the situation we have today. But perhaps I'm missing something obvious
> here ..
Well, I've not studied pathnames across systems too closely, so I'll
take the example of binary files.
On a big endian platform, one could expect that files :element-type
'(unsigned-byte 32) be 'standardized' in big-endian format, so all CL
implementation on that big endian platform will write such compatible
files.
On the other hand, clisp do write its binary files always in little
endian, even on a big endian machine, so files written by clisp on one
machine can be read by clisp on another without any problem.
Since this intra-implementation cross-platform compatibility seems
important for clisp developers and users, it would be hard to make it
use big-endian encoding on big-endian processors (happily, now that
Apple switched to Intel, there remains no big-endian processor, so to
speak). Perhaps we could just way for world domination of a single
file system, and standardize on that, when it will happen.
--
__Pascal Bourguignon__
From: Richard M Kreuter
Subject: Re: DIRECTORY behavior with /= implementations
Date:
Message-ID: <87od9ziw08.fsf@progn.net>
···@anevia.com (Pascal J. Bourguignon) writes:
> Kent M Pitman <······@nhplace.com> writes:
>> Thibault Langlois <·················@gmail.com> writes:
>>
>>> > The mapping of the pathname components into the concepts
>>> > peculiar to each file system is implementation-defined
>>>
>>> OK thanks for your answer. We all have to live with that.
>>> Nevertheless I think it would be more understandable if it was file
>>> system-specific instead of implementation-specific.
>>
>> I'm not sure this is literally true. One might get some warm fuzzy
>> from such wording, but after a while one would ponder about what that
>> fuzzy wording meant.
>>
>> For example, I'm not sure to what extent Linux existed at the time of
>> the standard went out. (I don't recall the "feature freeze" date, but
>> it was a a few years before the final version--perhaps as early as
>> 1988. We made essential changes after that up to publication time, but
>> it was hard to make such changes.)
>
> Not Operating System, but File System.
Note that the CLHS's definition of "file system" permits different
interpretations:
file system n.
a facility which permits aggregations of data to be stored in named
files on some medium that is external to the Lisp image and that
therefore persists from session to session.
And "file" is defined as
file n.
a named entry in a file system, having an implementation-defined
nature.
Consequently, it would be conforming for an implementation to define
"file" in such a way that any of the following would count as "file
systems" for the purposes of that implementation:
* a network protocol that supports naming, storage, and retrieval of
data, provided there is some medium backing the protocol,
* a format for laying out data on some medium, given that the layout
supports naming of data units,
* a namespace (a "VFS") presented by some host's operating system or
library,
* a relational database schema that associates data with names in some
manner.
(Other things are possible, too.) Consequently it's not right to
suppose that "file system" in the CLHS to describes only one of the
categories above; the term is more expansive than that.
> And it's even worse than that: mount options may intervene. For
> example, when you mount an ISO-9660 file system on Linux, you get a
> subdirectory tree where you have files with names in 8.3 with a
> version number! That is, unless you have some unix or "Joliet"
> extension which gives you normal file names, but you can still give
> options to mount to see it without these extensions.
>
> Another example would be MacOSX, where the default file system with
> the default options is not case sensitive, but where you still can
> mount "normal" unix file systems that are case sensitive.
I think these two paragraphs highlight the difference between two
senses of "file system", either of them conforming with the CLHS:
ISO-9660 and Joliet are or formats, while the behaviors of some Unix
host's namespace is an instance of a VFS. The upshot is that some
propositions turn out to have different truth values under different
senses of "file system":
-- The Unix file system supports hard links.
This is true for "file system" if you're talking about customary
Unix disk formats, but not in the VFS sense, inasmuch as most
modern Unixes can mount volumes having formats that cannot
represent hard links (e.g., FAT).
-- Modern file systems do not support wildcard file specifications.
Depending on what one means by "support", this can be true about
formats or an operating system's VFS, but false for other
definitions of file system (e.g., talking to a host via some
protocol that includes wildcards, e.g., FTP).
So I think it's worthwhile to be careful to avoid equivocation when
thinking about these issues. (I don't think you are equivocating, but
discussions of this stuff can get mired in these semantic problems,
sometimes because people don't notice that they're not using terms the
same way.)
> So, even in a given Operating System, you may have serveral File
> Systems with different properties, and different mapping to the OS API
> for FS access.
ISTM that for Lisps that aren't operating systems, it's the Lisp
implementator, not the operating system, who gets to decide what "file
system" means for that implementation -- and I think this aspect of
the language was by design.
--
RmK
From: Kent M Pitman
Subject: Re: DIRECTORY behavior with /= implementations
Date:
Message-ID: <uskzbgtcn.fsf@nhplace.com>
Richard M Kreuter <·······@progn.net> writes:
> it's not right to
> suppose that "file system" in the CLHS to describes only one of the
> categories above; the term is more expansive than that.
http://groups.google.com/group/comp.lang.lisp/msg/8fd8173d335bbbbd?
> ISTM that for Lisps that aren't operating systems, it's the Lisp
> implementator, not the operating system, who gets to decide what "file
> system" means for that implementation -- and I think this aspect of
> the language was by design.
Actually, in a much more fundamental way, the LispM had the same
option. Several people wrote their own file systems (from the bits on
the disk up to the filename syntax) and there was one notable CADR
[that's the second of the MIT lisp machine models], CADR-27 I believe,
that had at least two and I think maybe three independent file systems
installed on it--not just volumes, but completely different software
administering those volumes ... not counting FEPFS (the front end file
system).
Also, there have always been systems like Eunice (a Unix substitute
for DEC's VAX VMS from a few decades back) which offered emulation of
one file system on top of another, so that it was a point of view
thing whether it was the Lisp choosing to see one of those file
systems or the operating system choosing to provide alternate
interfaces. Of course, I'm sure there are a lot of cross-OS file
system emulators today, too, including those that proxy for far away
storage, and those that try to pretend they are a kind of file system
they really aren't. I just mean to say that such ideas are not recent
and were planned into (or, at least, planned for in) the CL design.
The Symbolics variant of Lisp Machine Lisp, which finally settled on
Greenberg's LMFS file system, also had an internal variant never
released to the public which was the "version controlled" (or VC) file
system that had extra components beyond versions, including branches
and (I think) branch ids, built into the file system. We used it
internally for years, and VC files were "flattened" for distribution
before sending out public releases that included the "sources" so that
end-users didn't need to know about VC files.
From: Richard M Kreuter
Subject: Re: DIRECTORY behavior with /= implementations
Date:
Message-ID: <871w6yjqq5.fsf@progn.net>
Thibault Langlois <·················@gmail.com> writes:
> It seems that DIRECTORY gives different results depending on the
> implementation (see below).
Yes.
> Are there various possible interpretations of the specs?
Yes, indeed.
> Should implementations converge in order to give the same result?
Maybe, but don't hold your breath. Probably the shortest path between
wherever you are and wherever you want to be is to find or write your
own directory lister, or else to figure out a way to use one
implementation's DIRECTORY, and don't worry about other
implementations' until you need your programs to run on them.
(Premature porting is akin to premature optimization.)
--
RmK
On Feb 27, 4:42 pm, Richard M Kreuter <·······@progn.net> wrote:
> Thibault Langlois <·················@gmail.com> writes:
> > It seems that DIRECTORY gives different results depending on the
> > implementation (see below).
>
> Yes.
>
> > Are there various possible interpretations of the specs?
>
> Yes, indeed.
>
> > Should implementations converge in order to give the same result?
>
> Maybe, but don't hold your breath. Probably the shortest path between
> wherever you are and wherever you want to be is to find or write your
> own directory lister, or else to figure out a way to use one
> implementation's DIRECTORY, and don't worry about other
> implementations' until you need your programs to run on them.
It is precisely my case. I discovered these differences while porting
my stuff from cmucl to sbcl.
> (Premature porting is akin to premature optimization.)
>
> --
> RmK