From: Andrea Taverna
Subject: Open :overwrite behaviour
Date: 
Message-ID: <gufg6v$sqg$1@news.motzarella.org>
Hello everyone.

To my understanding of the hyperspec, :overwrite and :supersede options are 
"similar". They are used when creating an output/io stream for an already 
existing file.
Then I can't explain why I get an error with the following piece of code when 
"foo.dat" *doesn't* exist

(with-open-file (os "foo.dat" :direction :output :if-exists :overwrite)
	(format os "foo"))

with Lisp complaining it can't find "foo.dat".

When "foo.dat" exists or :supersede is used, the code runs smoothly though.

I run the code with SBCL(a little outdate version) and ECL(latest version), same 
behaviour.

TIA

Andrea

From: Thomas A. Russ
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <ymid4acpqv6.fsf@blackcat.isi.edu>
Andrea Taverna <············@libero.it.invalid> writes:

> Hello everyone.
> 
> To my understanding of the hyperspec, :overwrite and :supersede options
> are "similar". They are used when creating an output/io stream for an
> already existing file.
> Then I can't explain why I get an error with the following piece of code
> when "foo.dat" *doesn't* exist
> 
> (with-open-file (os "foo.dat" :direction :output :if-exists :overwrite)
> 	(format os "foo"))
> 
> with Lisp complaining it can't find "foo.dat".
> 
> When "foo.dat" exists or :supersede is used, the code runs smoothly though.

If you look carefully at the hyperspec

  http://www.lispworks.com/documentation/HyperSpec/Body/f_open.htm

you will see that if you specify :OVERWRITE as the option for
:IF-EXISTS, then the default for :IF-DOES-NOT-EXIST is :ERROR.  So you 
are getting the specified behavior.  If you really want to overwrite an
existing file, the presumption is that you need that file there in order
to overwrite it.

You can fix this by being explicit about both cases.  The following should
presumably work for you:

(with-open-file (os "foo.dat" :direction :output
                              :if-exists :overwrite
                              :if-does-not-exist :create)
   (format os "foo"))



-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Kaz Kylheku
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <20090524235348.751@gmail.com>
On 2009-05-13, Andrea Taverna <············@libero.it.invalid> wrote:
> Hello everyone.
>
> To my understanding of the hyperspec, :overwrite and :supersede options are 
> "similar". They are used when creating an output/io stream for an already 
> existing file.
> Then I can't explain why I get an error with the following piece of code when 
> "foo.dat" *doesn't* exist
>
> (with-open-file (os "foo.dat" :direction :output :if-exists :overwrite)
> 	(format os "foo"))
>
> with Lisp complaining it can't find "foo.dat".

You forgot about the situation :IF-DOES-NOT-EXIST. The :DIRECTION :OUTPUT is
not enough to ask for the file to be created.

This is analogous to calling the Unix function open, specifying O_WRONLY,
but not O_CREAT.
From: ·······@eurogaran.com
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <a5b252b3-9862-46ec-a4bc-924a98650d12@o14g2000vbo.googlegroups.com>
On May 13, 11:23 pm, Andrea Taverna <············@libero.it.invalid>
wrote:
> Hello everyone.
>
> To my understanding of the hyperspec, :overwrite and :supersede options are
> "similar". They are used when creating an output/io stream for an already
> existing file.

Besides what others have told you, there is a difference:
:supersede is meant to replace the existent file with a new one
:overwrite is meant to replace the FILE CONTENTS

In other words: use supersede.
From: Tim Bradshaw
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <e5353ab1-d6ef-4cc4-95b2-1abdbdfb8295@g19g2000vbi.googlegroups.com>
On May 14, 1:56 pm, ·······@eurogaran.com wrote:

> Besides what others have told you, there is a difference:
> :supersede is meant to replace the existent file with a new one
> :overwrite is meant to replace the FILE CONTENTS
>
> In other words: use supersede.

That's not clear.  Imagine I am opening a file for output where that
file exists and where that file is very large (perhaps almost as large
as the filesystem in which it resides).  :SUPERSEDE is encouraged not
to delete the old file until the stream is closed, and that might be
quite bad news.

The actual answer is: work out what semantics you want, and use the
appropriate options.
From: George Neuner
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <svcr0515b7j6sokgfqkhb4po79occfbk18@4ax.com>
On Fri, 15 May 2009 06:02:41 -0700 (PDT), Tim Bradshaw
<··········@tfeb.org> wrote:

>On May 14, 1:56�pm, ·······@eurogaran.com wrote:
>
>> Besides what others have told you, there is a difference:
>> :supersede is meant to replace the existent file with a new one
>> :overwrite is meant to replace the FILE CONTENTS
>>
>> In other words: use supersede.
>
>That's not clear.  Imagine I am opening a file for output where that
>file exists and where that file is very large (perhaps almost as large
>as the filesystem in which it resides).  :SUPERSEDE is encouraged not
>to delete the old file until the stream is closed, and that might be
>quite bad news.
>
>The actual answer is: work out what semantics you want, and use the
>appropriate options.

The problem with that is the result might be non-portable.  The
semantics of :supersede are not clearly defined for systems that do
not support versioning.

George
From: Madhu
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <m3k54h991o.fsf@moon.robolove.meer.net>
* George Neuner <··································@4ax.com> :
Wrote on Fri, 15 May 2009 14:47:55 -0400:

| The problem with that is the result might be non-portable.  The
| semantics of :supersede are not clearly defined for systems that do
| not support versioning.

You cannot make that claim.  The semantics of SUPERSEDE are very clear
for systems which do not support versioning.  The implementation details
are not specified.

--
Madhu
From: Madhu
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <m3skj68ir9.fsf@moon.robolove.meer.net>
* ··········@tfeb.org
Wrote on Fri, 15 May 2009 06:02:41 -0700 (PDT):

|> Besides what others have told you, there is a difference: :supersede
|> is meant to replace the existent file with a new one :overwrite is
|> meant to replace the FILE CONTENTS
|>
|> In other words: use supersede.
|
| That's not clear.  Imagine I am opening a file for output where that
| file exists and where that file is very large (perhaps almost as large
| as the filesystem in which it resides).  :SUPERSEDE is encouraged not
| to delete the old file until the stream is closed, and that might be
| quite bad news.
|
| The actual answer is: work out what semantics you want, and use the
| appropriate options.

Just nits.

There is a small problem with your example.  If you want to supersede
the file with a smaller file, you cannot open it with :overwrite.
Common lisp does not provide an operator to truncate a file at a given
length, so after you close it, the file will be just as big.

In this scenario, If you HAD to use :overwrite , you'd have to resort to
no-in-portable ffi to system calls to truncate/ftruncate.

On the other hand, you probably could get away with using :supersede,
especially if your new file was much smaller, and also perhaps in some
cases where your OS' shady Virtual Memory system could subsidise you.

--
Madhu
From: Thomas A. Russ
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <ymitz3mnox5.fsf@blackcat.isi.edu>
Madhu <·······@meer.net> writes:

> There is a small problem with your example.  If you want to supersede
> the file with a smaller file, you cannot open it with :overwrite.
> Common lisp does not provide an operator to truncate a file at a given
> length, so after you close it, the file will be just as big.

I'm not sure this can be said definitively.

It isn't really specified exactly what :OVERWRITE should do.  One could
imagine that some lisp implementations might actually truncate the file
once the stream writing to it closes.  It is neither required nor
prohibited by the specification.

-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Madhu
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <m3octt995l.fsf@moon.robolove.meer.net>
* (Thomas A. Russ) <···············@blackcat.isi.edu> :
Wrote on 15 May 2009 12:31:34 -0700:

| Madhu <·······@meer.net> writes:
|
|> There is a small problem with your example.  If you want to supersede
|> the file with a smaller file, you cannot open it with :overwrite.
|> Common lisp does not provide an operator to truncate a file at a given
|> length, so after you close it, the file will be just as big.
|
| I'm not sure this can be said definitively.
|
| It isn't really specified exactly what :OVERWRITE should do.  One could
| imagine that some lisp implementations might actually truncate the file
| once the stream writing to it closes.  It is neither required nor
| prohibited by the specification.

Nooooooo!  Those lisp implementations that choose to truncate a file
when closing a file opened with :OVERWRITE should be taken out back.

Surely the intent is clear and difference between OVERWRITE and
SUPERSEDE is clear: OVERWRITE works very well in the case when you want
to change a few fixed bytes in an existing file.

Please do not shortsightedly/pervertedly change the spec to kill this
use case!
--
Madhu
From: Pascal J. Bourguignon
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <87eiuqchut.fsf@galatea.local>
Madhu <·······@meer.net> writes:

> * ··········@tfeb.org
> Wrote on Fri, 15 May 2009 06:02:41 -0700 (PDT):
>
> |> Besides what others have told you, there is a difference: :supersede
> |> is meant to replace the existent file with a new one :overwrite is
> |> meant to replace the FILE CONTENTS
> |>
> |> In other words: use supersede.
> |
> | That's not clear.  Imagine I am opening a file for output where that
> | file exists and where that file is very large (perhaps almost as large
> | as the filesystem in which it resides).  :SUPERSEDE is encouraged not
> | to delete the old file until the stream is closed, and that might be
> | quite bad news.
> |
> | The actual answer is: work out what semantics you want, and use the
> | appropriate options.
>
> Just nits.
>
> There is a small problem with your example.  If you want to supersede
> the file with a smaller file, you cannot open it with :overwrite.
> Common lisp does not provide an operator to truncate a file at a given
> length, so after you close it, the file will be just as big.

Neither does POSIX provide such an operator.

(truncate is an extension of BSD unix).


> In this scenario, If you HAD to use :overwrite , you'd have to resort to
> no-in-portable ffi to system calls to truncate/ftruncate.

Doubly not portable.  It's not portable at the CL level, it's not
portable at the POSIX level, it may even not be portable at the unix
level, after all it's just a BSD extension. You're lucky if linux
provides that syscall, another unix kernel may not.


> On the other hand, you probably could get away with using :supersede,
> especially if your new file was much smaller, and also perhaps in some
> cases where your OS' shady Virtual Memory system could subsidise you.

What about
(progn (delete-file "file")
       (with-open-file (f "file" :direction :io :if-does-not-exist :create) ...)) ?

-- 
__Pascal Bourguignon__
From: Spiros Bousbouras
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <52f58e8b-df39-4b31-837a-b225c3c2cd54@o14g2000vbo.googlegroups.com>
On 15 May, 19:59, ····@informatimago.com (Pascal J. Bourguignon)
wrote:
> Madhu <·······@meer.net> writes:
>
> > There is a small problem with your example.  If you want to supersede
> > the file with a smaller file, you cannot open it with :overwrite.
> > Common lisp does not provide an operator to truncate a file at a given
> > length, so after you close it, the file will be just as big.
>
> Neither does POSIX provide such an operator.

If by "operator" we mean function then there is
truncate() and ftruncate()
< http://www.opengroup.org/onlinepubs/000095399/functions/truncate.html
>
< http://www.opengroup.org/onlinepubs/009695399/functions/ftruncate.html
>

--
Who's your mama?
From: Pascal J. Bourguignon
Subject: Re: Open :overwrite behaviour
Date: 
Message-ID: <87preaasml.fsf@galatea.local>
Spiros Bousbouras <······@gmail.com> writes:

> On 15 May, 19:59, ····@informatimago.com (Pascal J. Bourguignon)
> wrote:
>> Madhu <·······@meer.net> writes:
>>
>> > There is a small problem with your example.  If you want to supersede
>> > the file with a smaller file, you cannot open it with :overwrite.
>> > Common lisp does not provide an operator to truncate a file at a given
>> > length, so after you close it, the file will be just as big.
>>
>> Neither does POSIX provide such an operator.
>
> If by "operator" we mean function 

Yes.

> then there is
> truncate() and ftruncate()
> < http://www.opengroup.org/onlinepubs/000095399/functions/truncate.html
>>
> < http://www.opengroup.org/onlinepubs/009695399/functions/ftruncate.html

Ok, I was wrong.  The Linux man pages for these functions mention only
BSD, and I didn't remember seeing them when I browsed the POSIX
standard a few years ago.

-- 
__Pascal Bourguignon__