From: Lisptracker
Subject: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <c23ae4f8.0410271432.2eabf86@posting.google.com>
Hi

Since the topic of resource allocation/deallocation was rised during
"Lisp vs C++" discussion I should ask here about it again.

Somebody said that the problem is solved with help of macro like
WITH-OPEN-FILE, but I didn't see solution in case when you opened file
during creation of some object and should close file immediately if
you lost all references to it in order to allow other
objects/applications to open that file.

The only thing I have thought up is to link resource holder object to
GC and assign "special free action" (finalizer) function so GC will
apply the function to the object and call proper deallocator.

Consider you have an instance which holds a slot with the AVI player
object. When you create the AVI player object it exclusively locks AVI
file. Than you assign another AVI file to this slot and the old file
will never be closed (at least until next GC). Of course I could check
slot and deallocate old resource when new resource is assigned, but
what if I've lost references to the object which holds the AVI player
(consider it was destroyed as part of another complex object) ?

So question is about deallocation of small (for GC) resources with
indefinite extent which can be shared between multiple objects in
program.

We are currently porting C++ project to Lisp (photogrammetry,
eye-tracking) and looking for analogue of destructors. May be solution
is to create something like resource managers (monitors) ?

Thanks in advance
Lisptracker

From: Duane Rettig
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <4k6tbivei.fsf@franz.com>
···········@mail.ru (Lisptracker) writes:

> Hi
> 
> Since the topic of resource allocation/deallocation was rised during
> "Lisp vs C++" discussion I should ask here about it again.
> 
> Somebody said that the problem is solved with help of macro like
> WITH-OPEN-FILE, but I didn't see solution in case when you opened file
> during creation of some object and should close file immediately if
> you lost all references to it in order to allow other
> objects/applications to open that file.
> 
> The only thing I have thought up is to link resource holder object to
> GC and assign "special free action" (finalizer) function so GC will
> apply the function to the object and call proper deallocator.
> 
> Consider you have an instance which holds a slot with the AVI player
> object. When you create the AVI player object it exclusively locks AVI
> file. Than you assign another AVI file to this slot and the old file
> will never be closed (at least until next GC). Of course I could check
> slot and deallocate old resource when new resource is assigned, but
> what if I've lost references to the object which holds the AVI player
> (consider it was destroyed as part of another complex object) ?
> 
> So question is about deallocation of small (for GC) resources with
> indefinite extent which can be shared between multiple objects in
> program.
> 
> We are currently porting C++ project to Lisp (photogrammetry,
> eye-tracking) and looking for analogue of destructors. May be solution
> is to create something like resource managers (monitors) ?

See

http://www.franz.com/support/documentation/7.0/doc/gc.htm#finalizations-2

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Kenneth Tilton
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <ktilton-01932B.18490927102004@nyctyp02-ge0.rdc-nyc.rr.com>
In article <···························@posting.google.com>,
 ···········@mail.ru (Lisptracker) wrote:

> Hi
> 
> Since the topic of resource allocation/deallocation was rised during
> "Lisp vs C++" discussion I should ask here about it again.
> 
> Somebody said that the problem is solved with help of macro like
> WITH-OPEN-FILE, but I didn't see solution in case when you opened file
> during creation of some object and should close file immediately if
> you lost all references to it in order to allow other
> objects/applications to open that file.

This comes up a lot. One should not be driving functionality off garbage 
collection stuff, eg, when the object becomes unreachable (remember, 
losing all references is just one way to become unreachable). The 
application needs to know when to close a file thru its own mechanisms.

kenny
From: Lisptracker
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <c23ae4f8.0410271914.30293008@posting.google.com>
Kenneth Tilton <·······@nyc.rr.com> wrote in message news:<·····························@nyctyp02-ge0.rdc-nyc.rr.com>...
> 
> This comes up a lot. One should not be driving functionality off garbage 
> collection stuff, eg, when the object becomes unreachable (remember, 
> losing all references is just one way to become unreachable). The 
> application needs to know when to close a file thru its own mechanisms.

So I should handle all resources except memory manually ???
From: Neo-LISPer
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <874qkfbjt5.fsf@yahoo.com>
···········@mail.ru (Lisptracker) writes:

> Kenneth Tilton <·······@nyc.rr.com> wrote in message news:<·····························@nyctyp02-ge0.rdc-nyc.rr.com>...
> > 
> > This comes up a lot. One should not be driving functionality off garbage 
> > collection stuff, eg, when the object becomes unreachable (remember, 
> > losing all references is just one way to become unreachable). The 
> > application needs to know when to close a file thru its own mechanisms.
> 
> So I should handle all resources except memory manually ???

I'm not sure I understand the problem you described. You want 

1. Sharing
2. Deterministic release of resources
3. Some sort of automation

Correct? Does the C++ app you are porting from use boost::shared_ptr
or equivalent then?

If so, and you like this paradigm, you can just implement a "shared
pointer" in Lisp - it is just a tuple of an object and a counter.
From: Kenneth Tilton
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <ktilton-2A5D00.17550228102004@nycmny-nntp-rdr-03-ge1.rdc-nyc.rr.com>
In article <····························@posting.google.com>,
 ···········@mail.ru (Lisptracker) wrote:

> Kenneth Tilton <·······@nyc.rr.com> wrote in message 
> news:<·····························@nyctyp02-ge0.rdc-nyc.rr.com>...
> > 
> > This comes up a lot. One should not be driving functionality off garbage 
> > collection stuff, eg, when the object becomes unreachable (remember, 
> > losing all references is just one way to become unreachable). The 
> > application needs to know when to close a file thru its own mechanisms.
> 
> So I should handle all resources except memory manually ???

Let us say you should use memory management only to manage memory. 
Nothing stops you from creating nifty resource management tools to 
automate that.

kenny
From: Antony Sequeira
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <i7Uhd.3901$zx1.2108@newssvr13.news.prodigy.com>
Lisptracker wrote:
> Kenneth Tilton <·······@nyc.rr.com> wrote in message news:<·····························@nyctyp02-ge0.rdc-nyc.rr.com>...
> 
>>This comes up a lot. One should not be driving functionality off garbage 
>>collection stuff, eg, when the object becomes unreachable (remember, 
>>losing all references is just one way to become unreachable). The 
>>application needs to know when to close a file thru its own mechanisms.
> 
> 
> So I should handle all resources except memory manually ???

I think handling resource allocation and deallocation is the same, be it 
C++ or CL.

In C++, you'd call
delete foo
and that calls ~foo
and ~foo has a line of code that says
releaseMyResource(resourceHandle)
(The point to take away here is that for all practical purposes the 
destructor call is explicit, even though you typed
delete foo
rather than
~FooClass(foo)
)


Well, in CL you do the same,
write a method
deleteFoo
that takes an argument
You pass the object to it (which corresponds to the implicit 'this' in 
the C++ code above)
This method then calls releaseMyResource with resourceHandle (which is a 
slot on 'this', just like 'resourceHandle' is a field in the C++ 'this')

This simply boils down to
1. C++ calls destructor implicitly in auto vars (which are useful for 
simple life spans of objects). You do this in Lisp using the macros
2. C++ requires you to write destructor code and call delete for 
explicit cleanup for non auto life spans, and you need to do the same in 
Lisp too. (May be cells/cello whatever can help automate this ?)

After having written all of this, I realize that your question above 
sounds like you already know all of this :) , I am only trying to put it 
in perspective.

-Antony

P.S.:I have been lurking here cause I am not having enough time to do 
real Lisp and contribute, so all of the above is in pseudo code.
From: Christopher C. Stacy
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <uoeinbwn0.fsf@news.dtpq.com>
···········@mail.ru (Lisptracker) writes:
> Consider you have an instance which holds a slot with the AVI player
> object. When you create the AVI player object it exclusively locks AVI
> file. Than you assign another AVI file to this slot and the old file
> will never be closed (at least until next GC). Of course I could check
> slot and deallocate old resource when new resource is assigned, but
> what if I've lost references to the object which holds the AVI player
> (consider it was destroyed as part of another complex object) ?

The GC is only for reclaiming storage, not for managing any functional
semantics of your objects.  This is not the same as a finalizer.

I am not sure if I am following your example, but it sounds like 
you have an AVI object that has a slot which contains an open file
stream.  If you lose that object, the file will never be closed.  
It's unclear to me how or why would you lose that AVI object; you have
the resource in hand at the point of creation and at the end of the
dynamic scope of execution. Maybe you want to do something like this.

(defmethod play-media-file ((avi avi-player) filename)
  (unwind-protect
    (with-open-file (stream filename)
      (actually-play-the-file-however-that-works stream))
   (do-other-cleanup-stuff avi)))

I think some implementations of Lisp do have finalizers that can 
be called by the GC, but that is not part of Common Lisp and is
not really the way that we resource management.   You're supposed
to encapsulate dynamic resources inside of methods that will clean
up after themselves using the UNWIND-PROTECT based mechanisms.
The above example shows two such usages -- the basic UNWIND-PROTECT 
and the WITH-OPEN-FILE macro that is built on top of it.
From: Lisptracker
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <c23ae4f8.0410271911.327705d2@posting.google.com>
······@news.dtpq.com (Christopher C. Stacy) wrote in message news:<·············@news.dtpq.com>...

> I am not sure if I am following your example, but it sounds like 
> you have an AVI object that has a slot which contains an open file
> stream.  If you lose that object, the file will never be closed.  
> It's unclear to me how or why would you lose that AVI object; you have
> the resource in hand at the point of creation and at the end of the
> dynamic scope of execution. Maybe you want to do something like this.
> 
> (defmethod play-media-file ((avi avi-player) filename)
>   (unwind-protect
>     (with-open-file (stream filename)
>       (actually-play-the-file-however-that-works stream))
>    (do-other-cleanup-stuff avi)))

I am working with the AVI file/player for a long time, for example
randomly accessing different frames and processing them. AVI
file/player object is
part of more complex object(s). I want kill the main object(s) and
immediately unlock file if there are no more references to them in
another (complex) object(s).
From: Tim Bradshaw
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <ey34qkfwbdt.fsf@cley.com>
* lisptracker  wrote:

> I am working with the AVI file/player for a long time, for example
> randomly accessing different frames and processing them. AVI
> file/player object is
> part of more complex object(s). I want kill the main object(s) and
> immediately unlock file if there are no more references to them in
> another (complex) object(s).

I think it depends on how things work.

If what happens is the object simply gets forgotten - there is no
point in the program where you say `I'm done with this now', but at
some point it is no longer reachable
- then you're in trouble. The very best you could hope for is some
kind of finalization which the garbage collector performs, but you
have no idea when, or if, the GC will run.  Java (which has this kind
of finalization) has these problems: people rely (or used to rely) on
GC finalization to clean up resources, but now JVMs have advanced GCs
it's not at all clear when or if the resource gets finalized.  Nothing
will really help this case but magic.

On the other hand, if there is a well-defined point in your program
when the object is finished with, then it is relatively easy to
arrange a finalization protocol which objects can obey.  For complex
networks of objects you may need some kind of reference counting or
something like that.  In the very simple case when things have dynamic
extent, it is then pretty easy to have some WITH-RESOURCE macro which
arranges to call the finalization stuff at the end of the extent.
This particular (dynamic extent) case is one which C++, say, does
slightly more automatically.

--tim
From: Rahul Jain
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <87vfcrabz2.fsf@nyct.net>
···········@mail.ru (Lisptracker) writes:

> I am working with the AVI file/player for a long time, for example
> randomly accessing different frames and processing them. AVI
> file/player object is
> part of more complex object(s). I want kill the main object(s) and
> immediately unlock file if there are no more references to them in
> another (complex) object(s).

Don't open the file when you create the AVI object. Open the file when
you read a frame, close the file when you're done reading it in. Of
course, you can batch the frame reading to improve efficiency. For
example, you could have a WITH-OPEN-MEDIA-FILE operator that expands
basically to a WITH-OPEN-FILE on the file that the media object
references. You then read frames or other information within that form. 
Outside of that form, the file's data is not accessible and you could
signal some sort of error.

-- 
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
From: Barry Margolin
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <barmar-27387A.20205427102004@comcast.dca.giganews.com>
In article <·············@news.dtpq.com>,
 ······@news.dtpq.com (Christopher C. Stacy) wrote:

> ···········@mail.ru (Lisptracker) writes:
> > Consider you have an instance which holds a slot with the AVI player
> > object. When you create the AVI player object it exclusively locks AVI
> > file. Than you assign another AVI file to this slot and the old file
> > will never be closed (at least until next GC). Of course I could check
> > slot and deallocate old resource when new resource is assigned, but
> > what if I've lost references to the object which holds the AVI player
> > (consider it was destroyed as part of another complex object) ?
> 
> The GC is only for reclaiming storage, not for managing any functional
> semantics of your objects.  This is not the same as a finalizer.
> 
> I am not sure if I am following your example, but it sounds like 
> you have an AVI object that has a slot which contains an open file
> stream.  If you lose that object, the file will never be closed.  
> It's unclear to me how or why would you lose that AVI object; you have
> the resource in hand at the point of creation and at the end of the
> dynamic scope of execution. Maybe you want to do something like this.

The general problem is that there may not be a simple "dynamic scope of 
execution" associated with the AVI object.  In a complex application, 
there may be numerous references to the object, some of which are 
intended to persist past the termination of the code that initially 
opened it.  So you don't want the resource to be reclaimed until all of 
them have finished using the object.

WITH-xxx macros, and the underlying UNWIND-PROTECT mechanism, work well 
when the object is only used within a single dynamic scope, like 
processing a file sequentially.  But when you get more complex than 
this, by passing these objects to other code that may hang onto them, 
you may have to resort to manual resource management techniques.  You 
can't just use simple assignment to update the instance slot, you need 
to determine what to do with the object that's already in the slot.  
Reference counting might work, although that doesn't handle the case 
where the object containing the reference becomes garbage, since that 
won't decrement the reference counts of the downstream objects (unless 
you make use of finalizers).

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: Christopher C. Stacy
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <uhdof32te.fsf@news.dtpq.com>
Barry Margolin <······@alum.mit.edu> writes:
> The general problem is that there may not be a simple "dynamic
> scope of execution" associated with the AVI object.

The point where you allocate the resources is the same point where 
you deallocate them, regardless of how the containing object is 
passed around after that.
From: Barry Margolin
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <barmar-CB9BD0.00460928102004@comcast.dca.giganews.com>
In article <·············@news.dtpq.com>,
 ······@news.dtpq.com (Christopher C. Stacy) wrote:

> Barry Margolin <······@alum.mit.edu> writes:
> > The general problem is that there may not be a simple "dynamic
> > scope of execution" associated with the AVI object.
> 
> The point where you allocate the resources is the same point where 
> you deallocate them, regardless of how the containing object is 
> passed around after that.

That's not necessarily the case.  You might create an object in one 
application, drag it into another application, then exit the first 
application.  The object continues to exist despite the termination of 
the creating application.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: Christopher C. Stacy
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <uzn27z7o7.fsf@news.dtpq.com>
Barry Margolin <······@alum.mit.edu> writes:

> In article <·············@news.dtpq.com>,
>  ······@news.dtpq.com (Christopher C. Stacy) wrote:
> 
> > Barry Margolin <······@alum.mit.edu> writes:
> > > The general problem is that there may not be a simple "dynamic
> > > scope of execution" associated with the AVI object.
> > 
> > The point where you allocate the resources is the same point where 
> > you deallocate them, regardless of how the containing object is 
> > passed around after that.
> 
> That's not necessarily the case.  You might create an object in one 
> application, drag it into another application, then exit the first 
> application.  The object continues to exist despite the termination of 
> the creating application.

Then the second application needs to accept responsibility and follow
the resource management protocol for the objects that it received.
But it's generally probably better to initialize the resources as 
late as possible - the program that initializes the, is in the best
position to manage them.

We did manage in the past to write entire huge operating systems
without any "finalizer" features in the language.  (One technique
was to create a seperate object, called a "resource" but really
a resource pool, to manage a few kinds of things like this.)
From: Barry Margolin
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <barmar-E9293A.20474128102004@comcast.dca.giganews.com>
In article <·············@news.dtpq.com>,
 ······@news.dtpq.com (Christopher C. Stacy) wrote:

> Barry Margolin <······@alum.mit.edu> writes:
> 
> > In article <·············@news.dtpq.com>,
> >  ······@news.dtpq.com (Christopher C. Stacy) wrote:
> > 
> > > Barry Margolin <······@alum.mit.edu> writes:
> > > > The general problem is that there may not be a simple "dynamic
> > > > scope of execution" associated with the AVI object.
> > > 
> > > The point where you allocate the resources is the same point where 
> > > you deallocate them, regardless of how the containing object is 
> > > passed around after that.
> > 
> > That's not necessarily the case.  You might create an object in one 
> > application, drag it into another application, then exit the first 
> > application.  The object continues to exist despite the termination of 
> > the creating application.
> 
> Then the second application needs to accept responsibility and follow
> the resource management protocol for the objects that it received.

But it could be handed off to arbitrary and many other applications, in 
random orders.  At any time there could be numerous applications with 
references to the object, with no clear "owner" with the responsibility 
to perform resource management.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: Christopher C. Stacy
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <uu0segwau.fsf@news.dtpq.com>
Barry Margolin <······@alum.mit.edu> writes:

> In article <·············@news.dtpq.com>,
>  ······@news.dtpq.com (Christopher C. Stacy) wrote:
> 
> > Barry Margolin <······@alum.mit.edu> writes:
> > 
> > > In article <·············@news.dtpq.com>,
> > >  ······@news.dtpq.com (Christopher C. Stacy) wrote:
> > > 
> > > > Barry Margolin <······@alum.mit.edu> writes:
> > > > > The general problem is that there may not be a simple "dynamic
> > > > > scope of execution" associated with the AVI object.
> > > > 
> > > > The point where you allocate the resources is the same point where 
> > > > you deallocate them, regardless of how the containing object is 
> > > > passed around after that.
> > > 
> > > That's not necessarily the case.  You might create an object in one 
> > > application, drag it into another application, then exit the first 
> > > application.  The object continues to exist despite the termination of 
> > > the creating application.
> > 
> > Then the second application needs to accept responsibility and follow
> > the resource management protocol for the objects that it received.
> 
> But it could be handed off to arbitrary and many other applications, in 
> random orders.  At any time there could be numerous applications with 
> references to the object, with no clear "owner" with the responsibility 
> to perform resource management.

In that setting, the application that defined the object must have
included a management protocol in the object.  It defined methods for
initializing and releasing the resources that are part of the object.
The applications that are passed the object have to follow those protocols.
From: Lisptracker
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <c23ae4f8.0410280251.2e1143bb@posting.google.com>
I am posting through Google and can not answer in time, so I post
reply to all incoming answers here. Many thanks to everyone who
replied on the topic.

"Neo-LISPer" <··········@yahoo.com> writes:
···················@yahoo.com...
> I'm not sure I understand the problem you described. You want 
> 
> 1. Sharing
> 2. Deterministic release of resources
> 3. Some sort of automation
> 
> Correct? Does the C++ app you are porting from use boost::shared_ptr
> or equivalent then?
> 
> If so, and you like this paradigm, you can just implement a "shared
> pointer" in Lisp - it is just a tuple of an object and a counter.

Yes, I think reference counting would be enough, but who will decrease
counter in case when owner object of slot is lost or owner of owner is
removed somehow ? In C++ this mechanism rely on lexical scope and
stack, if you destroy owner all member objects will be destroyed as
well, when you pass smart_ptr to function it is copied implicitly and
increased counter, when you done with smart_ptr and leave the function
body smart_ptr will be destroyed and counter is decreased. I don't
know How to implement such thing in Common Lisp, because of indefinite
extent, I can't follow all assignments and function applications. GC
finalizers (special free actions) as in Lispworks or CormanLisp can do
all bookkeeping for me, but if the program does not cons much, I don't
know when GC will run next time (and probably will never run).

>From: "Barry Margolin" <······@alum.mit.edu>
>Newsgroups: comp.lang.lisp
>Sent: Thursday, October 28, 2004 7:20 AM
>Subject: Re: Again. Resource deallocation in Common Lisp.
>The general problem is that there may not be a simple "dynamic scope
of
>execution" associated with the AVI object.  In a complex application,
>there may be numerous references to the object, some of which are 
>intended to persist past the termination of the code that initially 
>opened it.  So you don't want the resource to be reclaimed until all
of
>them have finished using the object.

Exactly! For example we operate by closures each of them is connected
to video stream. I want all connections to video streams to be droped
after I disposed "old" closure by "new" one, I can not wait until next
GC operation, and these closures are shared between objects. For
example It can be guaranteed that I've lost all references to "old"
objects but they all are part of complex object and when I remove that
object I do not want traverse through complex ownership hierarchy and
disconnect all sub-objects by hand.

"Christopher C. Stacy" <······@news.dtpq.com> writes:
··················@news.dtpq.com...
> Barry Margolin <······@alum.mit.edu> writes:
> > The general problem is that there may not be a simple "dynamic
> > scope of execution" associated with the AVI object.
> 
> The point where you allocate the resources is the same point where 
> you deallocate them, regardless of how the containing object is 
> passed around after that.


Problem is - you don't know WHEN to deallocate resource.

>We did manage in the past to write entire huge operating systems
>without any "finalizer" features in the language.  (One technique
>was to create a seperate object, called a "resource" but really
>a resource pool, to manage a few kinds of things like this.)

Could you briefly explain idea please, how to manage resources with
resource pool ? I've come to exactly same conclusion (to use resource
managers), but I don't know how they will work (basic principle) ?


Best regards,
Lisptracker
From: Christopher C. Stacy
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <uvfculvig.fsf@news.dtpq.com>
···········@mail.ru (Lisptracker) writes:
> 
> Could you briefly explain idea please, how to manage resources with
> resource pool ? I've come to exactly same conclusion (to use resource
> managers), but I don't know how they will work (basic principle) ?

This is just explicit deallocation; something still has to know
when to do it, but now they're all registered in a global pool.
I suppose you could do reference counting on them at allocation time.

But this whole thing about allocating all kinds of resources that you
are going to purposely leak just sounds like a flakey idea to me.
I was giving an example of a "complex program" -- the world's most
sophisticated operating system -- which did not need to leak.
From: Rahul Jain
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <87zn23ac8z.fsf@nyct.net>
···········@mail.ru (Lisptracker) writes:

> Somebody said that the problem is solved with help of macro like
> WITH-OPEN-FILE, but I didn't see solution in case when you opened file
> during creation of some object and should close file immediately if
> you lost all references to it in order to allow other
> objects/applications to open that file.

You wouldn't want to rely on the GC to do this. You need to provide an
operation that indicates that the file should be closed. If you look at
streams, they support being closed without being dereferenced. If you
try to use a closed stream, it simply fails. If your model really is so
complex that you can't easily tell when the stream is no longer in use,
a filesystem on current OSes is really not the kind of abstraction you
want to use.

-- 
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
From: Thomas F. Burdick
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <xcvacu2mxl8.fsf@conquest.OCF.Berkeley.EDU>
···········@mail.ru (Lisptracker) writes:

> So question is about deallocation of small (for GC) resources with
> indefinite extent which can be shared between multiple objects in
> program.
> 
> We are currently porting C++ project to Lisp (photogrammetry,
> eye-tracking) and looking for analogue of destructors. May be solution
> is to create something like resource managers (monitors) ?

Huh, interesting timing -- I've been working with C++ and Objective-C
recently, and had been thinking about resource management in Common
Lisp.  When I've had resources that need explicit management in Lisp,
I just define that as a part of the interface.  In practice, it works
just fine.  However, it would be easier to just use a
reference-counter, and not have to worry about it.

On a long flight a few days ago, I was thinking about writing a
portable refcounter in CL (and wishing I had my iBook).  Some of the
issues that come up in writing smart pointers in C++ apply to
user-space refcounting generally (intrusive vs external counters,
threading issues, cycle detection).  Since we have a garbage collector
in Lisp, we only need a refcounter to manage resources that need more
precise management -- which makes the job a lot easier, because it
frees us from the performance concerns that make life aweful for smart
pointer authors and users.  So, that said, I was thinking about an
interface along the following lines.  If anyone has any thoughts on
the interface, I'd be interested to hear them.  Consider this teaser
documentation for a currently-vaporware library :-)

  Generic functions: RETAIN, RELEASE, DESTROY

    Add a reference to an object, drop a reference to an object,
    destroy an object with no more references.

  Generic function: AUTORELEASE
  Macro: WITH-AUTORELEASE-POOL
  Function: QUEUE-FOR-DESTRUCTION, PERFORM-PENDING-DESTRUCTIONS

    If you know Objective-C, this should look familiar.
    AUTORELEASE is like RELEASE, but instead of immediately destroying
    an object with no references, it queues it for later destruction.
    For objects that can tolerate less precision from the refcounter,
    this can be used to return refcounted objects up the call chain;
    if the caller uses the object, they should retain it in the usual
    manner, otherwise it can be ignored, and will be destroyed when
    the current autorelease pool expires.

    WITH-AUTORELASE-POOL establishes a new autorelease pool for the
    dynamic extent of its body.  When control leaves it, any pending
    destructions are performed.  Program loops, new threads, etc.,
    should use this as appropriate.

  Classes: SIMPLE-REFCOUNTED, SIMPLE-REFCOUNTED-STRUCTURE,
           EXTERNAL-REFCOUNTED, EXTERNAL-REFCOUNTED-STRUCTURE

    STANDARD-CLASS mixin classes and STRUCTURE-CLASS classes that can
    be inherited from to get refcounted behavior: either using a
    simple intrusive-counter, or by maintaining an external list of
    references.

  Functions: EXTERNAL-RETAIN, EXTERNAL-RELEASE, EXTERNAL-AUTORELEASE

    These perform external refcounting on an object.  The methods on
    RETAIN, RELEASE, and AUTORELEASE for EXTERNAL-REFCOUNTED[-STRUCTURE]
    use these internally.  They can also be used for classes whose
    definition you don't control.

  Macro: RETAINING, RETAINING*

    Syntax like LET and LET*, but retain the objects before binding
    the variables, and release them after evaluating the form's body.

  Macros: DEFINE-REFCOUNTED-CLASS, DEFINE-REFCOUNTED-STRUCTURE

    Convenience macros that ensure the resulting class is reference
    counted properly.  They also add a :REFCOUNTED slot option that
    can be specified as NIL (default), T, or :AUTORELEASE.  This will
    cause writer and accessor methods to invoke RETAIN, RELEASE, and
    AUTORELEASE as appropriate.
From: Hannah Schroeter
Subject: Re: Again. Resource deallocation in Common Lisp.
Date: 
Message-ID: <cn0d5k$7cn$1@c3po.use.schlund.de>
Hello!

Lisptracker <···········@mail.ru> wrote:
>[... asking for C++-like "deinitialization" ...]

I'd suggest something like

(defgeneric deinitialize (object)
	    (:method (o t) nil))

(defmacro with-managed-instances (bindings &body body)
  (let ((symbols (mapcar #'first bindings)))
    `(let ,symbols
       (unwind-protect
	 (progn
	   (setf ,@(mapcar (lambda (b) (list (first b) `(progn ,@(rest b))))
			   bindings))
	   ,@body)
	 (progn
	   ,@(mapcar (lambda (symbol) `(deinitialize ,symbol)) symbols))))))

Usage:

(with-managed-instances ((foo (make-instance 'foo))
			 (bar (make-instance 'bar)))
  ...
  ...)

After the body, there're calls to (deinitialize foo) and
(deinitialize bar).

Then you have things like

(defclass some-connection (...)
  ((descriptor...)
   ...))

(defmethod deinitialize (c some-connection)
  (close (slot-value c 'descriptor)))

...
  (with-managed-instances ((c (make-instance 'some-connection :host foo)))
    ... do something with c ...))

The handling of declarations in the with-managed-instances body is
left as an exercise to the reader.

Kind regards,

Hannah.