Hi,
I need to copy a set of files from one directory to another. So far, I
have been doing this in CLISP using something like
(ext:copy-file (concatenate 'string *clisp-dir* "asdf/lispbuilder*/bin/
*.dll") *build-dir*)
which works quite nicely. However, I would like to make the program
more portable, so I have recently started converting to CL-FAD.
Unfortunately, CL-FAD's copy function doesn't support wildcards, and
my attempts so far at making walk-directory do what I need have
failed.
(defun copy-files (from wildcard to &key directories overwrite)
(fad:walk-directory
from
(lambda (pathname)
(when (pathname-match-p pathname wildcard)
(fad:copy-file
pathname
(make-pathname :directory (pathname-directory to)
:name (pathname-name pathname)
:type (pathname-type pathname))
:overwrite overwrite)))
:directories directories))
On CLISP this fails saying it the file already exists. Apparently CL-
FAD doesn't deal with the file already existing properly, because
despite the fact that :overwrite defaults to nil, this simple test
case fails:
[2]> (fad:copy-file "foo" "bar")
[3]> (fad:copy-file "foo" "bar")
*** - OPEN: file #P"C:\\User\\Programming\\Lisp\\Common Lisp\
\Blackthorn\\bar"
already exists
On SBCL, it copies a few files then breaks the GC invariant and drops
into ldb. (Not really sure why that one happens, it might be a bug on
Windows.)
I suppose I can report these failures to their respective maintainers,
but I was wondering if anyone here has experience with CL-FAD?
(Because at least the first error seems like a fairly obvious error,
I'd be somewhat surprised if no one encountered it until now.)
And second, does anyone have any other suggestions about how to write
a portable copy-files function? (Or are there any improvements that
could be made to my current code, ignoring the errors I encountered?)
Thank you for your time.
On 18 mai, 10:43, Elliott Slaughter <················@gmail.com>
wrote:
> And second, does anyone have any other suggestions about how to write
> a portable copy-files function? (Or are there any improvements that
> could be made to my current code, ignoring the errors I encountered?)
What about:
http://darcs.informatimago.com/darcs/public/lisp/common-lisp/file.lisp
(copy-file "src" "dst" :if-exists :supersede :element-type '(unsigned-
byte 8))
Note however that portably copying files from Common Lisp won't copy
native file system attributes. What you will be doing portably, is
actually only copying the data in the file, as seen from Common Lisp.
--
__Pascal Bourguignon__
On May 18, 10:08 am, ····@informatimago.com wrote:
> On 18 mai, 10:43, Elliott Slaughter <················@gmail.com>
> wrote:
>
> > And second, does anyone have any other suggestions about how to write
> > a portable copy-files function? (Or are there any improvements that
> > could be made to my current code, ignoring the errors I encountered?)
>
> What about:http://darcs.informatimago.com/darcs/public/lisp/common-lisp/file.lisp
>
> (copy-file "src" "dst" :if-exists :supersede :element-type '(unsigned-
> byte 8))
Thank for the suggestion, that would probably work. For the moment
though I managed a work-around by wrapping the call in ignore-errors
to prevent CLISP from complaining, and moving all the files I need to
copy to a flat directory to prevent SBCL from crashing. Not elegant,
but it works.
> Note however that portably copying files from Common Lisp won't copy
> native file system attributes. What you will be doing portably, is
> actually only copying the data in the file, as seen from Common Lisp.
That's good enough for me, although I would be interested if someone
did happen to know a way to accomplish a true copy.
However, that still leaves my main question, which was more about the
best way to copy a set of files denoted by wildcards than just copying
an individual file. My solution technically works, but feels really
inefficient since I have to iterate of all subdirectories to test
matches for multiple wildcards even if an intermediate directory
doesn't match a directory wildcard.
On May 19, 8:06 am, Elliott Slaughter <················@gmail.com>
wrote:
> That's good enough for me, although I would be interested if someone
> did happen to know a way to accomplish a true copy.
For POSIX see GNU Coreutils, file src/copy.c which also points to a
lot
of utility function for this purpose that you can steal.
On May 18, 4:43 am, Elliott Slaughter <················@gmail.com>
wrote:
> Hi,
>
> I need to copy a set of files from one directory to another. So far, I
> have been doing this in CLISP using something like
>
> (ext:copy-file (concatenate 'string *clisp-dir* "asdf/lispbuilder*/bin/
> *.dll") *build-dir*)
>
> which works quite nicely.
... and is a recipe for disaster. Never assume that concatenating
strings yields usable directories. Bite the bullet and learn to live
with MERGE-PATHNAMES.
Cheers
--
Marco
However, I would like to make the program
> more portable, so I have recently started converting to CL-FAD.
> Unfortunately, CL-FAD's copy function doesn't support wildcards, and
> my attempts so far at making walk-directory do what I need have
> failed.
>
> (defun copy-files (from wildcard to &key directories overwrite)
> (fad:walk-directory
> from
> (lambda (pathname)
> (when (pathname-match-p pathname wildcard)
> (fad:copy-file
> pathname
> (make-pathname :directory (pathname-directory to)
> :name (pathname-name pathname)
> :type (pathname-type pathname))
> :overwrite overwrite)))
> :directories directories))
>
> On CLISP this fails saying it the file already exists. Apparently CL-
> FAD doesn't deal with the file already existing properly, because
> despite the fact that :overwrite defaults to nil, this simple test
> case fails:
>
> [2]> (fad:copy-file "foo" "bar")
>
> [3]> (fad:copy-file "foo" "bar")
>
> *** - OPEN: file #P"C:\\User\\Programming\\Lisp\\Common Lisp\
> \Blackthorn\\bar"
> already exists
>
> On SBCL, it copies a few files then breaks the GC invariant and drops
> into ldb. (Not really sure why that one happens, it might be a bug on
> Windows.)
>
> I suppose I can report these failures to their respective maintainers,
> but I was wondering if anyone here has experience with CL-FAD?
> (Because at least the first error seems like a fairly obvious error,
> I'd be somewhat surprised if no one encountered it until now.)
>
> And second, does anyone have any other suggestions about how to write
> a portable copy-files function? (Or are there any improvements that
> could be made to my current code, ignoring the errors I encountered?)
>
> Thank you for your time.
On May 19, 2:06 am, Marco Antoniotti <·······@gmail.com> wrote:
> On May 18, 4:43 am, Elliott Slaughter <················@gmail.com>
> wrote:
> > I need to copy a set of files from one directory to another. So far, I
> > have been doing this in CLISP using something like
>
> > (ext:copy-file (concatenate 'string *clisp-dir* "asdf/lispbuilder*/bin/
> > *.dll") *build-dir*)
>
> > which works quite nicely.
>
> ... and is a recipe for disaster. Never assume that concatenating
> strings yields usable directories. Bite the bullet and learn to live
> with MERGE-PATHNAMES.
Yeah, I've been converting all my code from manipulating strings to
manipulating pathnames. The pathname code is a probably lot more
portable, but seems really bloated. For example, instead of writing
(concatenate 'string *clisp-dir* "asdf/lispbuilder/bin/")
compared to
(merge-pathnames (make-pathname :directory '(:relative "asdf"
"lispbuilder" "bin") *clisp-dir*)
which just seems overly verbose. To make that line shorter, I defined
my own append-directory function
(defun append-directories (default-pathname &rest directories)
(merge-pathnames
(make-pathname :directory (cons :relative directories))
default-pathname))
which allowed the original line to be written
(append-directories *clisp-dir* "asdf" "lispbuilder" "bin")
but still...
I also have a couple of questions about using standard pathname
utilities to do common tasks.
I found that I can use (truename (make-pathname)) seems to return the
current working directory, but I don't know if this is guaranteed to
work or if it just happened to work when I tested it.
Also, wanted to know the location of the lisp executable I was running
from. In CLISP, I found I could use (truename (directory-namestring
(aref (ext:argv) 0)))) , but this is obviously completely
implementation dependent. Is there a better way to do this? (Or at
least a library that consistently wraps this implementation specific
behavior?)
Thanks again.
> Yeah, I've been converting all my code from manipulating strings to
> manipulating pathnames. The pathname code is a probably lot more
> portable, but seems really bloated.
Well, not really, if you consider the "proper" alternative:
> For example, instead of writing
>
> (concatenate 'string *clisp-dir* "asdf/lispbuilder/bin/")
(cond ((unix-file-system-p)
(concatenate 'string *clisp-dir* "asdf/lispbuilder/bin/"))
((windows-file-system-p)
(concatenate 'string *clisp-dir* "asdf\\lispbuilder\\bin\\"))
((mac-file-system-p)
(concatenate 'string *clisp-dir* ":asdf:lispbuilder:bin:")))
not to mention what you have to do to setup *clisp-dir* in the first
place.
> compared to
>
> (merge-pathnames (make-pathname :directory '(:relative "asdf"
> "lispbuilder" "bin") *clisp-dir*)
>
> which just seems overly verbose. To make that line shorter, I defined
> my own append-directory function
>
> (defun append-directories (default-pathname &rest directories)
> (merge-pathnames
> (make-pathname :directory (cons :relative directories))
> default-pathname))
>
> which allowed the original line to be written
>
> (append-directories *clisp-dir* "asdf" "lispbuilder" "bin")
>
> but still...
Another alternative to consider is the use of logical pathnames, which
are designed to allow you to have a set of pathnames that can be
translated to physical pathnames through the use of logical pathname
translation declarations. It allows you to isolate the pathname
dependencies of your code and manage them in one place via a declarative
mechanism rather than through programming code.
Of course, you will probably still need some additional utilities to
help you compose the logical pathnames. Although it may be a bit safer
to use strings there, since the syntax of logical pathnames is
well-specified in the standard.
> I also have a couple of questions about using standard pathname
> utilities to do common tasks.
>
> I found that I can use (truename (make-pathname)) seems to return the
> current working directory, but I don't know if this is guaranteed to
> work or if it just happened to work when I tested it.
Well, it really is implementation dependent what the value of
*default-pathname-defaults* will be, although the Spec suggests that it
be the working directory:
"An implementation-dependent pathname, typically in the working
directory that was current when Common Lisp was started up."
Some quick tests locally show that some lisps (SBCL, ACL) set this to be
the full path to the working directory. Others (Lispworks, CCL/OpenMCL)
set it to be an empty pathname. In CCL this merges to get the working
directory, though. In Lispworks, I launched it as a Mac application, so
it isn't clear what a working directory would mean in those
circumstances.
> Also, wanted to know the location of the lisp executable I was running
> from. In CLISP, I found I could use (truename (directory-namestring
> (aref (ext:argv) 0)))) , but this is obviously completely
> implementation dependent. Is there a better way to do this? (Or at
> least a library that consistently wraps this implementation specific
> behavior?)
Well, this one is a bit tougher. In some lisp systems, there is a
logical pathname defined which points to the lisp implementation. But
this isn't universal. It is something that you could perhaps arrange to
have and manage separately using logical pathnames.
--
Thomas A. Russ, USC/Information Sciences Institute
From: Richard M Kreuter
Subject: Re: Writing a portable copy-files
Date:
Message-ID: <87wslne6s1.fsf@progn.net>
Elliott Slaughter <················@gmail.com> writes:
> I've been converting all my code from manipulating strings to
> manipulating pathnames. The pathname code is a probably lot more
> portable, but seems really bloated. For example, instead of writing
>
> (concatenate 'string *clisp-dir* "asdf/lispbuilder/bin/")
>
> compared to
>
> (merge-pathnames (make-pathname :directory '(:relative "asdf"
> "lispbuilder" "bin") *clisp-dir*)
>
> which just seems overly verbose. To make that line shorter, I defined
> my own append-directory function
>
> (defun append-directories (default-pathname &rest directories)
> (merge-pathnames
> (make-pathname :directory (cons :relative directories))
> default-pathname))
>
> which allowed the original line to be written
>
> (append-directories *clisp-dir* "asdf" "lispbuilder" "bin")
>
> but still...
As I read the spec, it is not actually more CLHS-portable to call
MAKE-PATHNAME with a string or a list than to use a physical pathname
namestring: namestring syntax is no less implementation-dependent than
the limits on string and list arguments to MAKE-PATHNAME, and so calls
to MAKE-PATHNAME aren't required to work any more than (possibly
implicit) namestring parsing. So IMO you might as well just say
(merge-pathnames "asdf/lispbuilder/bin/" *clisp-dir*)
and avoid the bloat you don't like.
(The CLHS prohibits conforming programs from using physical pathname
namestrings, but the reason they're prohibited is that Common Lisp
doesn't define any physical pathname namestring syntaxes. But Common
Lisp doesn't define the limits on the values that can be used as
arguments to MAKE-PATHNAME, either.)
> I also have a couple of questions about using standard pathname
> utilities to do common tasks.
>
> I found that I can use (truename (make-pathname)) seems to return the
> current working directory, but I don't know if this is guaranteed to
> work or if it just happened to work when I tested it.
Are you really interested in the current working directory?
Common Lisp doesn't standardize the notion of current working
directory. Instead, there's a standard variable,
*DEFAULT-PATHNAME-DEFAULTS*, that various pathname and file system
operations use. In particular, functions that access a file system
[*] "fill in" pathname arguments with components from
*DEFAULT-PATHNAME-DEFAULTS* as if by calling MERGE-PATHNAMES. So, for
example,
;; For a Lisp with access to a Unix file system.
(let ((*default-pathname-defaults* (pathname "/etc/")))
(probe-file "passwd"))
=> #P"/etc/passwd"
Next, a call to MAKE-PATHNAME with no arguments should result in a
pathname whose device, directory, name, type, version components are
all NIL, and whose host is the host component of
*DEFAULT-PATHNAME-DEFAULTS*.
Finally, because TRUENAME implicitly calls MERGE-PATHNAMES,
(truename (make-pathname))
is equivalent to
(truename *default-pathname-defaults*)
So if *DEFAULT-PATHNAME-DEFAULTS* names the current working directory
(which it's likely to, when Lisp starts up), then TRUENAME will return
a truename the current working directory. But
*DEFAULT-PATHNAME-DEFAULTS* is just a variable, and can be rebound or
assigned some other value without affecting the process's current
working directory (and the implementation might let you change the
working directory without side-effect on *DEFAULT-PATHNAME-DEFAULTS*),
so the code above can return a truename for some file other than the
current working directory..
Hope that helps,
RmK
[*] IIRC, that works out to DIRECTORY, TRUENAME, PROBE-FILE,
FILE-WRITE-DATE, FILE-AUTHOR, RENAME-FILE, DELETE-FILE,
ENSURE-DIRECTORIES-EXIST, OPEN, LOAD, COMPILE-FILE, ED, and DRIBBLE.
Richard M Kreuter <·······@progn.net> writes:
> Elliott Slaughter <················@gmail.com> writes:
>
> As I read the spec, it is not actually more CLHS-portable to call
> MAKE-PATHNAME with a string or a list than to use a physical pathname
> namestring: namestring syntax is no less implementation-dependent than
> the limits on string and list arguments to MAKE-PATHNAME, and so calls
> to MAKE-PATHNAME aren't required to work any more than (possibly
> implicit) namestring parsing. So IMO you might as well just say
>
> (merge-pathnames "asdf/lispbuilder/bin/" *clisp-dir*)
>
> and avoid the bloat you don't like.
While in a strict CLHS sense there is no difference in portability, in a
real-world practical sense the pathname object route is more portable,
because you haven't embedded a particular file system's namestring
syntax into the code.
For example, the make-pathname directory argument will, per definition
of the spec, always accept a list of strings as pathname directory
components. So this will at least get you some compatibility across
file systems, at least if you start out with something that has the
proper default values. (Host or device are often not portable...)
Oddly enough, the examples use :RELATIVE and :ABSOLUTE, but the glossary
definition of valid pathname directory doesn't mention them at all. Is
this something that didn't get updated when the keywords were added to
the pathname syntax?
--
Thomas A. Russ, USC/Information Sciences Institute
From: Richard M Kreuter
Subject: Re: Writing a portable copy-files
Date:
Message-ID: <87skwaeaxg.fsf@progn.net>
···@sevak.isi.edu (Thomas A. Russ) writes:
> Richard M Kreuter <·······@progn.net> writes:
>> Elliott Slaughter <················@gmail.com> writes:
>>
>> As I read the spec, it is not actually more CLHS-portable to call
>> MAKE-PATHNAME with a string or a list than to use a physical pathname
>> namestring...
>
> While in a strict CLHS sense there is no difference in portability,
> in a real-world practical sense the pathname object route is more
> portable, because you haven't embedded a particular file system's
> namestring syntax into the code.
>
> For example, the make-pathname directory argument will, per
> definition of the spec, always accept a list of strings as pathname
> directory components. So this will at least get you some
> compatibility across file systems, at least if you start out with
> something that has the proper default values. (Host or device are
> often not portable...)
Slaughter's example MAKE-PATHNAME call was this
(make-pathname :directory '(:relative "asdf" "lispbuilder" "bin"))
This call assumes the following:
(1) the file system is hierarchical,
(2) it's acceptable in some sense to use lowercase alphabetic
characters in directory names,
(3) it's acceptable in some sense to use 11 character long strings for
directory names.
ISTM that when you parse "asdf/lispbuilder/bin/" as a physical
pathname namestring, you're assuming the same things as in the call to
MAKE-PATHNAME, and also that
(4) the forward slash counts as a directory delimiter.
As it turns out, all four assumptions are borne out under the two (or
two and a half) remaining operating systems' file system interfaces,
and I think it happens that extant Lisp implementations do treat
forward slashes as directory delimiters under both (or both and a
half) operating systems. So IMO (a) the de facto portability of
"asdf/lispbuilder/bin/" isn't interestingly different than that of
MAKE-PATHNAME call Slaughter wanted not to write, (b) neither the
namestring nor the MAKE-PATHNAME call can possibly be portable in the
CLHS sense, so one doesn't get extra karma points for preferring
MAKE-PATHNAME.
(Note that I don't say MAKE-PATHNAME has no use, only that using
MAKE-PATHNAME to produce physical pathnames doesn't by itself make
programs portable. Logical pathnames could help with portability, but
I don't think they're widely used and they're not well implemented
everywhere.)
--
RmK
Elliott Slaughter <················@gmail.com> wrote:
>
> I also have a couple of questions about using standard pathname
> utilities to do common tasks.
>
> I found that I can use (truename (make-pathname)) seems to return the
> current working directory, but I don't know if this is guaranteed to
> work or if it just happened to work when I tested it.
>
Using Poplog Clisp:
== (make-pathname)
#P""
== (truename (make-pathname))
; MISHAP - File does not exist
; INVOLVING: #P""
; DOING : (TRUENAME (MAKE-PATHNAME))
Break: (Error)
:H Help
<0>
For use in FriCAS I settled on the following:
#+(or :cmu :ecl :gcl :sbcl :clisp :openmcl)
(defun get-current-directory ()
(trim-directory-name (namestring (truename ""))))
#+:poplog
(defun get-current-directory ()
(let ((name (namestring (truename "."))))
(trim-directory-name (subseq name 0 (1- (length name))))))
where trim-directory-name removes trailing slash (on Windows also backslash)
from namestrings (if present). (truename ".") works not only in
Poplog, but I was unable to find a single construct which works
everywhere.
--
Waldek Hebisch
·······@math.uni.wroc.pl
Elliott Slaughter <················@gmail.com> writes:
> I found that I can use (truename (make-pathname)) seems to return the
> current working directory, but I don't know if this is guaranteed to
> work or if it just happened to work when I tested it.
It just happens that *default-pathname-defaults* refers the unix
"current working directory", but it's by chance.
In clisp, you can get the current working directory, and change it,
with EXT:CD.
(ext:cd)
(ext:cd "/some/path")
It might with worthwhile to keep the unix current working directory
and the *default-pathname-defaults* in synchronization. (eg. when you
use things like ext:run-program, etc). I do that by defining a my
own CD function that sets *default-pathname-defaults* and calls ext:cd
in clisp (and an equivalent implementation dependant function in
other implementations, when available).
> Also, wanted to know the location of the lisp executable I was running
> from. In CLISP, I found I could use (truename (directory-namestring
> (aref (ext:argv) 0)))) , but this is obviously completely
> implementation dependent. Is there a better way to do this? (Or at
> least a library that consistently wraps this implementation specific
> behavior?)
What do you need it for? Really, this questions has the same problems
as when asked for any unix program. You shouldn't care, and any
answer will inflict more pain than gain.
--
__Pascal Bourguignon__
Pascal J. Bourguignon wrote:
> What do you need it for? Really, this questions has the same problems
> as when asked for any unix program. You shouldn't care, and any
> answer will inflict more pain than gain.
>
Typically it's so that an executable can find some other resource files
distributed alongside it, under the assumption that "installation"
is copying a directory to somehere the user can click on it. Yes,
traditional unix way is to Not Do It That Way, I know.
FWIW, linux programs, unless they're running in a funny
restricted environment, can usually find their executable
by following the /proc/self/exe symlink. Linux Is Not UniX. :-)
AppDirs & Bundles: On *Step, see mainBundle method of NSBundle
Assign PROGDIR: on AmigaOS pointed to the directory
holding the currently running executable.
On May 26, 2:43 am, David Golden <············@oceanfree.net> wrote:
> Pascal J. Bourguignon wrote:
> > What do you need it for? Really, this questions has the same problems
> > as when asked for any unix program. You shouldn't care, and any
> > answer will inflict more pain than gain.
>
> Typically it's so that an executable can find some other resource files
> distributed alongside it, under the assumption that "installation"
> is copying a directory to somehere the user can click on it. Yes,
> traditional unix way is to Not Do It That Way, I know.
I wanted to distribute executables of my Lisp programs to my friends,
none of whom actually have Lisp installed. This is all fine, except
that the executables made by e.g. CLISP need additional run-time
libraries, which the distributer is responsible for providing. So my
script copied the dlls from their home in the CLISP executable
directory and copied them to the distribution directory. And
personally, I like (aref (ext:argv) 0)))) a lot better than hard
coding "C:\\Program Files\\clisp\\..." into my code, which is what I
did before I found this hack. I just wanted to know if there could be
an even better way of doing it.
Elliott Slaughter <················@gmail.com> writes:
> On May 26, 2:43 am, David Golden <············@oceanfree.net> wrote:
>> Pascal J. Bourguignon wrote:
>> > What do you need it for? Really, this questions has the same problems
>> > as when asked for any unix program. You shouldn't care, and any
>> > answer will inflict more pain than gain.
>>
>> Typically it's so that an executable can find some other resource files
>> distributed alongside it, under the assumption that "installation"
>> is copying a directory to somehere the user can click on it. Yes,
>> traditional unix way is to Not Do It That Way, I know.
>
> I wanted to distribute executables of my Lisp programs to my friends,
> none of whom actually have Lisp installed. This is all fine, except
> that the executables made by e.g. CLISP need additional run-time
> libraries, which the distributer is responsible for providing. So my
> script copied the dlls from their home in the CLISP executable
> directory and copied them to the distribution directory. And
> personally, I like (aref (ext:argv) 0)))) a lot better than hard
> coding "C:\\Program Files\\clisp\\..." into my code, which is what I
> did before I found this hack. I just wanted to know if there could be
> an even better way of doing it.
Well, perhaps not better, but here:
You could store all the resources needed by your executable in the
lisp image. For the resources that need to be external
(eg. libraries), to be loaded again at the C level, you could save
them at run-time in /tmp, and load the back from here. For others,
(eg. pictures) you would use them directly without reading them again.
--
__Pascal Bourguignon__
Elliott Slaughter wrote:
> I just wanted to know if there could be
> an even better way of doing it.
Not that I care for microsoft windows, just in the interests of
expanding the list, seems the "native" way to find windows' official
idea of the executable's path is GetModuleFileName() with first arg
either NULL or the "handle" the current process, itself returned by
GetModuleHandle(NULL) or as one of the windows startup args in
WinMain().
I didn't see GetModuleFileName() exposed in lispbuilder-windows (a cffi
binding to windows api), though i didn't look very hard - anyway, if
it's not there, presumably it's a trivial addition, it already has
GetModuleHandle().
David Golden <············@oceanfree.net> wrote:
+---------------
| FWIW, linux programs, unless they're running in a funny
| restricted environment, can usually find their executable
| by following the /proc/self/exe symlink. Linux Is Not UniX. :-)
+---------------
On FreeBSD, said symlink is called "/proc/curproc/file":
cmu> (truename "/proc/curproc/file")
#P"/u/rpw3/src/cmd/cmucl-19c/bin/lisp"
cmu>
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607