I would like to use rename-file to move files around as it seems to be
more portable than using for example the mv unix utility, and faster
than copying the file then deleting.
My case is special in that the source path and target path are not on
the same partition, but it is the same filesystem (i.e. under the root
directory / ). Nowhere does the CLHS say that rename-file should fail
in this case, but SBCL complains as follows:
couldn't rename
/tmp/test
to
/home/me/testdfi:
Invalid cross-device link
[Condition of type SB-INT:SIMPLE-FILE-ERROR]
Which by looking at the source of the function rename-file is an error
originating from a call to unix-rename which as far as I can read is a
foreign function call to the unix function rename.
So my question is, is this a bug?
Evan
evan <············@gmail.com> writes:
> I would like to use rename-file to move files around as it seems to be
> more portable than using for example the mv unix utility, and faster
> than copying the file then deleting.
>
> My case is special in that the source path and target path are not on
> the same partition, but it is the same filesystem (i.e. under the root
> directory / ). Nowhere does the CLHS say that rename-file should fail
> in this case, but SBCL complains as follows:
>
> couldn't rename
> /tmp/test
> to
> /home/me/testdfi:
> Invalid cross-device link
> [Condition of type SB-INT:SIMPLE-FILE-ERROR]
>
> Which by looking at the source of the function rename-file is an error
> originating from a call to unix-rename which as far as I can read is a
> foreign function call to the unix function rename.
>
> So my question is, is this a bug?
Yes. In your expectations.
Let's start by the end, the unix error message says flatly the facts:
you're trying to move the file test from one device to another, and
this cannot be done by 'moving' with rename(2), but you must copy the
bits from one device to the other. That's what mv(1) does for you
without saying in this case. If you don't believe it, try:
df /tmp/. ; df /home/me/.
These directories ARE on DIFFERENT file systems, even if they hang in
the same unix directory tree. On unix, usually file systems are
stored in a partition. Not all partitions contain a file system. But
you can also store file systems inside files in another file system.
When you mount a file system, you are hanging it on an entry in the
directory tree. Notwithstanding, you still have a distinct file
system. That's what the df(1) command shows you. Try:
df -T
and see how each file system may be of a different kind even.
Finally, RENAME-FILE is a portability layer over various OS and FS,
and you cannot expect it to even be able to move the file across
different directory, since this couldn't be done on some old
OS/FS. (IIRC, the MOVE command appeared only at version 6 of MS-DOS,
before you had to use COPY to move files from one directory to the
other). So you should consider yourself happy if RENAME-FILE is
implemented on unix systems with rename(2) which allows you to move
files intra-file-system.
One last word, COMMON-LISP provides so much functionnality, it makes
us forget that it's not a whole operating system. You only have the
kernel! Nothing prevents us to implement more userfriendly layers and
shells. Have a look at:
http://darcs.informatimago.com/lisp/common-lisp/interactive.lisp
where I implement a few "commands" like LS over the raw CL:DIRECTORY,
or CAT and MORE over the raw CL:READ-LINE and CL:PRINC, etc.
You could implement a MV command that would use CL:RENAME-FILE,
CL:WITH-OPEN-FILE, CL:READ-SEQUENCE, CL:WRITE-SEQUENCE, etc to do what
you expect.
--
__Pascal Bourguignon__
On Mar 4, 2:04 pm, ····@informatimago.com (Pascal J. Bourguignon)
wrote:
> evan <············@gmail.com> writes:
>
> > So my question is, is this a bug?
>
> Yes. In your expectations.
>
I don't think so. There is nothing I can see to prevent the
implementation of RENAME-FILE from doing what the implementation of mv
does. I don't think there is anything to say that it *should* do
that, but I am sure it is allowed to, and I would probably expect it
to, especially since is is probably extremely hard to tell if you need
to do a copy-and-delete using the filesystem tools CL gives you (you
could rely on getting an error and handle that, which may in fact be
what mv does).
--tim
On Mar 4, 6:27 pm, Tim Bradshaw <··········@tfeb.org> wrote:
>
> There is nothing I can see to prevent the
> implementation of RENAME-FILE from doing what the implementation of mv
> does.
And in fact, absent tools to set permissions, ACLs etc on files it is
probably not possible to implement a rename-by-copying at all. For
instance here is a trace of part of mv doing a copy-and-delete:
read(3, " # u s e r = r b s - u n".., 32768) = 78
write(4, " # u s e r = r b s - u n".., 78) = 78
read(3, 0x0803F038, 32768) = 0
close(3) = 0
close(4) = 0
pathconf("spot", 101) = 0
chown("/export/spot/spit", 1502, 1502) = 0
chmod("/export/spot/spit", 0100644) = 0
utimes("/export/spot/spit", 0x08046FC4) = 0
unlink("spot")
You just can't do half of that with portable CL, and you need to do
it. (And, amusingly, I am not sure if mv needed to do all that at
all, since these were filesystems sharing the same ZFS pool, so I
reckon it should, in theory, be able to just reglue the blocks into
the other FS, though I don't know if you can do that from user-level
code.)
From: Richard M Kreuter
Subject: Re: trouble with rename-file
Date:
Message-ID: <87r6epgqqj.fsf@progn.net>
Tim Bradshaw <··········@tfeb.org> writes:
> On Mar 4, 6:27 pm, Tim Bradshaw <··········@tfeb.org> wrote:
>
>>
>> There is nothing I can see to prevent the implementation of
>> RENAME-FILE from doing what the implementation of mv does.
>
> And in fact, absent tools to set permissions, ACLs etc on files it is
> probably not possible to implement a rename-by-copying at all. For
> instance here is a trace of part of mv doing a copy-and-delete:
>
> read(3, " # u s e r = r b s - u n".., 32768) = 78
> write(4, " # u s e r = r b s - u n".., 78) = 78
> read(3, 0x0803F038, 32768) = 0
> close(3) = 0
> close(4) = 0
> pathconf("spot", 101) = 0
> chown("/export/spot/spit", 1502, 1502) = 0
> chmod("/export/spot/spit", 0100644) = 0
> utimes("/export/spot/spit", 0x08046FC4) = 0
> unlink("spot")
>
> You just can't do half of that with portable CL, and you need to do
> it.
Additionally, it's not possible to detect whether a file is sparse
(i.e., not possible portably across CLs with access to some file
system that supports sparse files). It's trivial to construct
unpleasant worst case scenarios if you copy a sparse file using a
sufficiently naive routine.
--
RmK
On Mar 4, 11:04 pm, ····@informatimago.com (Pascal J. Bourguignon)
wrote:
> > So my question is, is this a bug?
>
> Yes. In your expectations.
>
> Let's start by the end, the unix error message says flatly the facts:
> you're trying to move the file test from one device to another, and
> this cannot be done by 'moving' with rename(2), but you must copy the
> bits from one device to the other. That's what mv(1) does for you
> without saying in this case. If you don't believe it, try:
>
> df /tmp/. ; df /home/me/.
>
> Finally, RENAME-FILE is a portability layer over various OS and FS,
> and you cannot expect it to even be able to move the file across
> different directory, since this couldn't be done on some old
> OS/FS. (IIRC, the MOVE command appeared only at version 6 of MS-DOS,
> before you had to use COPY to move files from one directory to the
> other). So you should consider yourself happy if RENAME-FILE is
> implemented on unix systems with rename(2) which allows you to move
> files intra-file-system.
Thanks for the explanation ! I had seen people use rename-file to
move files around so I thought that this function was a high-level
portability layer.
To my knowledge there is no straightforward way in common lisp to know
if the source and target paths would lie on the same file systems
(except maybe by checking for the error or looking at mtab), so that
limits the utility of rename-file to just renaming files in a single
directory.
> One last word, COMMON-LISP provides so much functionnality, it makes
> us forget that it's not a whole operating system. You only have the
> kernel! Nothing prevents us to implement more userfriendly layers and
> shells. Have a look at:http://darcs.informatimago.com/lisp/common-lisp/interactive.lisp
> where I implement a few "commands" like LS over the raw CL:DIRECTORY,
> or CAT and MORE over the raw CL:READ-LINE and CL:PRINC, etc.
Yes, it is perfectly fine to me that common lisp does not provide
portable file system functionality. I just like to know what I can
and cannot expect, and this is why I asked :).
> You could implement a MV command that would use CL:RENAME-FILE,
> CL:WITH-OPEN-FILE, CL:READ-SEQUENCE, CL:WRITE-SEQUENCE, etc to do what
> you expect.
As Tim Bradshaw points it would be more complicated than that, since
on a Unix system you would want to preserve permissions and ownership,
as well as modification time to the file. If that means accessing
chmod, chown and others, then directly calling mv would be better
after all.
Evan
On Mar 4, 11:21 pm, evan <············@gmail.com> wrote:
>
> To my knowledge there is no straightforward way in common lisp to know
> if the source and target paths would lie on the same file systems
> (except maybe by checking for the error or looking at mtab), so that
> limits the utility of rename-file to just renaming files in a single
> directory.
It does appear to be the case that this mv cops out here as well (at
least on Solaris): it tries the rename, and handles the error. You
could do that in CL too (but permissions, modtime, etc as discussed).