·······@hotmail.com writes:
> What are weak pointers or weak references? I think it has something to do with
> gc, but I'm not sure...
See
http://www.franz.com/support/documentation/6.0/doc/gc.htm#weak-vecs-etc-1
for a discussion of weak pointers and finalizations for one implementation.
Note that weak-ness is not a standard CommonLisp concept.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Duane Rettig <·····@franz.com> writes:
> ·······@hotmail.com writes:
>
> > What are weak pointers or weak references? I think it has something to do with
> > gc, but I'm not sure...
>
> See
>
> http://www.franz.com/support/documentation/6.0/doc/gc.htm#weak-vecs-etc-1
>
> for a discussion of weak pointers and finalizations for one implementation.
> Note that weak-ness is not a standard CommonLisp concept.
Unfortunately. However most implementations do offer it.
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
>>>>> "marco" == Marco Antoniotti <·······@cs.nyu.edu> writes:
>> for a discussion of weak pointers and finalizations for one
>> implementation. Note that weak-ness is not a standard CommonLisp
>> concept.
marco> Unfortunately. However most implementations do offer it.
Actually, I don't think that this is unfortunate. I actually prefer
it not to be part of the standard. I've never seen it used (and never
used it) for anything but cache optimization. I'm not even sure how
big of a win it is when it's there, and you're using it correctly.
The point is that it's messy business, and it's gotta do with
something that's very unspecified: the GC mechanism. I think it was
not unfortunate, but wise that it's not part of the standard.
dave
David Bakhash <·····@alum.mit.edu> writes:
> >>>>> "marco" == Marco Antoniotti <·······@cs.nyu.edu> writes:
>
> >> for a discussion of weak pointers and finalizations for one
> >> implementation. Note that weak-ness is not a standard CommonLisp
> >> concept.
>
> marco> Unfortunately. However most implementations do offer it.
>
> Actually, I don't think that this is unfortunate. I actually prefer
> it not to be part of the standard. I've never seen it used (and never
> used it) for anything but cache optimization. I'm not even sure how
> big of a win it is when it's there, and you're using it correctly.
> The point is that it's messy business, and it's gotta do with
> something that's very unspecified: the GC mechanism. I think it was
> not unfortunate, but wise that it's not part of the standard.
Maybe you are right. However, the fact that most implementations do
offer them is a sign that some "common" behavior is accepted
(moreover, this becomes another area where portability becomes
cumbersome).
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
Marco Antoniotti <·······@cs.nyu.edu> writes:
>
> Maybe you are right. However, the fact that most implementations do
> offer them is a sign that some "common" behavior is accepted
> (moreover, this becomes another area where portability becomes
> cumbersome).
>
I thought about this a year or so ago, and on balance, I think I'd
quite like it if something like hashtables could have a weak flag, the
semantics of which was that this reference *might* go away if it was
the only reference to the object. This wouldn't require
implementations to support it but would provide a common name for the
functionality where it existed.
However I also think there's a flip side to weak pointers and related
things like finalisations - they're quite vulnerable to abuse by
people who don't understand the difference between storage management
and object management. I've heard stories that Java people use
finalisations to do stuff like closing files, which is really bad, I
think. It's `OK' so long as you have a rudimentary GC which looks at
all live objects frequently, but if you have a generational scheme you
really have no promise that some object will ever be detected as
garbage, so suddenly your finalisations never run and your weak
pointers never get nullified. But even when it's `OK' I think it's
not OK - managing objects - things like closing files &c - is not the
same thing as managing storage and is not a task really suitable for a
GC. I think, anyway.
--tim
Tim Bradshaw <···@tfeb.org> writes:
> Marco Antoniotti <·······@cs.nyu.edu> writes:
>
> >
> > Maybe you are right. However, the fact that most implementations do
> > offer them is a sign that some "common" behavior is accepted
> > (moreover, this becomes another area where portability becomes
> > cumbersome).
> >
>
> I thought about this a year or so ago, and on balance, I think I'd
> quite like it if something like hashtables could have a weak flag, the
> semantics of which was that this reference *might* go away if it was
> the only reference to the object. This wouldn't require
> implementations to support it but would provide a common name for the
> functionality where it existed.
These are available, again as an extension to the spec:
http://www.franz.com/support/documentation/6.0/doc/implementation.htm#cl-make-hash-table-2
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
>>>>> "Tim" == Tim Bradshaw <···@tfeb.org> writes:
Tim> and object management. I've heard stories that Java people use
Tim> finalisations to do stuff like closing files, which is really bad, I
Tim> think. It's `OK' so long as you have a rudimentary GC which looks at
Tim> all live objects frequently, but if you have a generational scheme you
Tim> really have no promise that some object will ever be detected as
Tim> garbage, so suddenly your finalisations never run and your weak
Tim> pointers never get nullified. But even when it's `OK' I think it's
Tim> not OK - managing objects - things like closing files &c - is not the
Tim> same thing as managing storage and is not a task really suitable for a
Tim> GC. I think, anyway.
CMUCL uses finalizations to close the Unix file descriptor associated
with a stream. Perhaps not quite the same as what you're saying here.
Ray
Raymond Toy <···@rtp.ericsson.se> writes:
> >>>>> "Tim" == Tim Bradshaw <···@tfeb.org> writes:
>
> Tim> and object management. I've heard stories that Java people use
> Tim> finalisations to do stuff like closing files, which is really bad, I
> Tim> think. It's `OK' so long as you have a rudimentary GC which looks at
> Tim> all live objects frequently, but if you have a generational scheme you
> Tim> really have no promise that some object will ever be detected as
> Tim> garbage, so suddenly your finalisations never run and your weak
> Tim> pointers never get nullified. But even when it's `OK' I think it's
> Tim> not OK - managing objects - things like closing files &c - is not the
> Tim> same thing as managing storage and is not a task really suitable for a
> Tim> GC. I think, anyway.
>
> CMUCL uses finalizations to close the Unix file descriptor associated
> with a stream. Perhaps not quite the same as what you're saying here.
I think it probably is the same, or similar. Using finalizations is
dangerous, because if you rely on the gc to trigger a finalization on a
stream to close its file descriptor, then any application which has a
high stream creation/destruction rate (such as a web-server, for example)
is more likely to run out of file descriptors until the next gc. This
is what I think Tim meant by managing objects, or as I would put it,
limited non-lisp resources.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
DR> [...] Using
DR> finalizations is dangerous, because if you rely on the gc to
DR> trigger a finalization on a stream to close its file
DR> descriptor, then any application which has a high stream
DR> creation/destruction rate (such as a web-server, for example)
DR> is more likely to run out of file descriptors until the next
DR> gc. [...]
I may be missing the point here, but what is to prevent an implementation
from triggering a gc (an possibly a full gc) when it is informed by the OS
that it has run out of fd's?
cheers,
BM
Bulent Murtezaoglu <··@acm.org> writes:
>
>
> DR> [...] Using
> DR> finalizations is dangerous, because if you rely on the gc to
> DR> trigger a finalization on a stream to close its file
> DR> descriptor, then any application which has a high stream
> DR> creation/destruction rate (such as a web-server, for example)
> DR> is more likely to run out of file descriptors until the next
> DR> gc. [...]
>
> I may be missing the point here, but what is to prevent an implementation
> from triggering a gc (an possibly a full gc) when it is informed by the OS
> that it has run out of fd's?
If nothing else, there are lots of different kinds of gc's.
Some implementations use what you might the "null" gc. They don't gc.
They have enough memory to run for a credible time and then they die.
Some implementations use conservative gc. This means that you might not
get back the fd if it were held by something being conservatively pinned.
Some implementations take a great while to gc. The Lisp Machine had what
I imagine are both the fastest and slowest gc. The fastest gc was the
multi-tiered ephemeral gc, capable of reclaiming recently consed/lost memory
in tiny periods of time so that the long gc's were very rare. But those
gc's can't be relied upon to free any PARTICULAR memory (e.g., the ones
with the fd's in them). The longer gc's took sometimes hours. Making
your program hinge for its correctness on one of these is not a good idea.
People used to often ask why we didn't put a (gc) command in the language.
The answer is that it varied so widely that users would be surprised and
irritated by the non-portable variation of effects. Maybe if we'd had the
time to do the "CLIM treatment" on gc, so you'd do
(gc :if-generational t :if-estimated-time-less-than-n-seconds 0.01
:if-multiprocessing-can-continue-concurrently t
:if-no-paging-will-be-forced t
...)
But as just (gc), no. There used to be a (gc) in Maclisp when address
space was 256K words (usually less than the amount of physical memory
in the machine) and when mark/sweep gc was reliably easy, but those days
are gone.
Duane Rettig <·····@franz.com> writes:
> > CMUCL uses finalizations to close the Unix file descriptor associated
> > with a stream. Perhaps not quite the same as what you're saying here.
>
> I think it probably is the same, or similar. Using finalizations is
> dangerous, because if you rely on the gc to trigger a finalization on a
> stream to close its file descriptor, then any application which has a
> high stream creation/destruction rate (such as a web-server, for example)
> is more likely to run out of file descriptors until the next gc.
I assume you don't mean this statement ("using finalizations is dangerous")
to apply out of context.
I think the problem people are narrowing in on is that "expecting program
correctness to hinge on an object being gc'd and therefore a finalization
to run is dangerous". But surely if you're going to gc an object, giving
the object a chance to clean house before it dies isn't dangerous--in fact,
it would be more dangerous to just gc it without this. Weak pointers or
not, finalization seems to me to have its place. It just has to be a
protection/preparation mechanism for the dying, not something where the
living are sitting around salivating about their expected inheritance.
Kent M Pitman <······@world.std.com> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > > CMUCL uses finalizations to close the Unix file descriptor associated
> > > with a stream. Perhaps not quite the same as what you're saying here.
> >
> > I think it probably is the same, or similar. Using finalizations is
> > dangerous, because if you rely on the gc to trigger a finalization on a
> > stream to close its file descriptor, then any application which has a
> > high stream creation/destruction rate (such as a web-server, for example)
> > is more likely to run out of file descriptors until the next gc.
>
> I assume you don't mean this statement ("using finalizations is dangerous")
> to apply out of context.
Correct. I apparently did a bad job of tying in the last sentence of my above
quoted paragraph, since it has now been left off of three replies. So I
will try again: Using finalizations is dangerous whenever relying on such
finalizations to free up limited system resources. Of course, such danger
can be managed, by knowing what resources are likely to be consumed and
by knowing how the application is expected to scale. But misjudging system
resources is often the source of bugs in any language. And since Lisp tends
to reduce the need to think about resources, it tends to come as even more
of a shock to the poor web-server programmer whose lisp program suddenly
barfs with an out-of-file-descriptors error.
> I think the problem people are narrowing in on is that "expecting program
> correctness to hinge on an object being gc'd and therefore a finalization
> to run is dangerous". But surely if you're going to gc an object, giving
> the object a chance to clean house before it dies isn't dangerous--in fact,
> it would be more dangerous to just gc it without this. Weak pointers or
> not, finalization seems to me to have its place. It just has to be a
> protection/preparation mechanism for the dying, not something where the
> living are sitting around salivating about their expected inheritance.
Inheritance! When did OO enter into this discussion? :-) :-)
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
In article <···············@world.std.com>,
Kent M Pitman <······@world.std.com> wrote:
>I think the problem people are narrowing in on is that "expecting program
>correctness to hinge on an object being gc'd and therefore a finalization
>to run is dangerous". But surely if you're going to gc an object, giving
>the object a chance to clean house before it dies isn't dangerous--in fact,
>it would be more dangerous to just gc it without this. Weak pointers or
>not, finalization seems to me to have its place. It just has to be a
>protection/preparation mechanism for the dying, not something where the
>living are sitting around salivating about their expected inheritance.
Good point. Finalization is a fine method to CYA, in case you lose track
of an object without properly cleaning up the extra-lingual things that it
references.
What this suggests is that programmers who use finalizations should
probably have them display some kind of warning. If a finalization is
invoked, it suggests a bug somewhere else in the code (the object should
have been cleaned up explicitly before all references were dropped). The
nice thing about this is that consequences are relatively innocuous;
compare it to memory management in languages like C, where a screwup can
easily result in overwriting data, segmentation faults, etc.
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> What this suggests is that programmers who use finalizations should
> probably have them display some kind of warning. If a finalization is
> invoked, it suggests a bug somewhere else in the code (the object should
> have been cleaned up explicitly before all references were dropped).
Not always. It may be as simple as that there's a monitor window or
buffer for every object of a certain kind and that this resource needs
to be returned when the object is no longer alive. Although this brings
up the issue that if there is such a window, that window needs to itself
have some way of noting that the *human user* holds a pointer into the
object considered for GC and perhaps should be considered as reason not
to collect it.
I have this filed in my brain under the title "the munkam problem".
I'll leave it to Barry or some other history/trivia buff to explain
that paradigmatic problem.
In article <···············@world.std.com>,
Kent M Pitman <······@world.std.com> wrote:
>Barry Margolin <······@genuity.net> writes:
>
>> What this suggests is that programmers who use finalizations should
>> probably have them display some kind of warning. If a finalization is
>> invoked, it suggests a bug somewhere else in the code (the object should
>> have been cleaned up explicitly before all references were dropped).
>
>Not always. It may be as simple as that there's a monitor window or
>buffer for every object of a certain kind and that this resource needs
>to be returned when the object is no longer alive.
Well, obviously if finalization is *not* being used to catch errors, the
warning makes no sense. My suggestion was only intended for that
particular use of finalization.
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Kent M Pitman <······@world.std.com> writes:
> I think the problem people are narrowing in on is that "expecting program
> correctness to hinge on an object being gc'd and therefore a finalization
> to run is dangerous". But surely if you're going to gc an object, giving
> the object a chance to clean house before it dies isn't dangerous--in fact,
> it would be more dangerous to just gc it without this. Weak pointers or
> not, finalization seems to me to have its place. It just has to be a
> protection/preparation mechanism for the dying, not something where the
> living are sitting around salivating about their expected inheritance.
That's exactly what I think. However I would worry slightly about
putting user-level finalisations into the standard language because it
seems to me that they place a much larger burden on implementors than
weak pointers. A minimal conforming implementation of (say) weak
hashtables consists of altering make-hashtable to allow but ignorep
some extra keyword arguments. A minimal conforming implementation of
finalisations requires GC support for them, on the assumption that if
an object with a finalisation becomes garbage then that finalisation
should run. I guess an implementation would be free to not run them,
since there's really no promise that they ever will be run since
there's no promise that an object ever will become detectible garbage,
so I suppose an implementation could just ignore them the way it can
ignore weakness, but somehow that seems a lot worse than ignoring
weakness.
Then there's the whole question of how easy it is to support in a GC
implementation, or whether it constrains future GC implementations.
Perhaps that's not really a problem, but I can imagine, say, a vendor
who had a mostly-hardware GC scheme, where it might be hugely
expensive to support them.
This isn't to say that I think they aren't useful - I'm just not sure
they should be mandated (unlike weak somethings, which I think
probably could be).
Maybe I'm wrong though.
--tim
In article <···············@tfeb.org>, Tim Bradshaw <···@tfeb.org> wrote:
>Then there's the whole question of how easy it is to support
>[finalization] in a GC
>implementation, or whether it constrains future GC implementations.
I don't think they're too difficult or constraining; I'm sure you can find
details in L&FP and OOPSLA proceedings.
My concern about including finalization in the standard would be that many
programmers might take this as an indication that it's OK to depend on it
for proper application semantics, i.e. they wouldn't know all the caveats
and warnings we've been discussing in this thread. Since it can only be
used correctly in fairly limited ways, I'm not sure it's necessary to
include it in the standard.
On the other hand, since so many implementations already provide it, it
would be nice to provide a standard syntax rather than requiring
programmers to use implementation-dependent functions and #+ to
conditionalize.
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> My concern about including finalization in the standard [...]
Vaguely related, does the standard actually say anything about GC at
all? I can't find anything in there that explicitly says that
unreferenced memory should be automatically reclaimed.
Admittedly, a conforming implementation with no GC would generally
need infinite storage capacity if it were to _stay_ conforming for any
length of time ...
-dan
--
http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources
In article <··············@noetbook.telent.net>,
Daniel Barlow <···@noetbook.telent.net> wrote:
>Barry Margolin <······@genuity.net> writes:
>
>> My concern about including finalization in the standard [...]
>
>Vaguely related, does the standard actually say anything about GC at
>all? I can't find anything in there that explicitly says that
>unreferenced memory should be automatically reclaimed.
Nope, there's no mention of GC.
If it did mention GC, what would you expect it to say? A language standard
specifies how a program behaves when run by an implementation, but GC is
pretty much invisible. The best I can think of is something like a
requirement that the following must not fail due to memory exhaustion:
(defun conser ()
(cons nil nil))
(defun test-gc ()
(loop (conser)))
(test-gc)
We'd probably also have to specify that the two definitions be in separate
compilation units, otherwise a smart compiler could recognize that the loop
doesn't actually return anything and CONSER has no side effects, so
optimize away the consing.
>Admittedly, a conforming implementation with no GC would generally
>need infinite storage capacity if it were to _stay_ conforming for any
>length of time ...
Right. Since there's nothing analogous to C++'s delete operator, a garbage
collector is needed implicitly. Garbage collection is a quality of
implementation issue.
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Raymond Toy <···@rtp.ericsson.se> writes:
> CMUCL uses finalizations to close the Unix file descriptor associated
> with a stream. Perhaps not quite the same as what you're saying here.
>
I think it's fine for a system to do something like this, since it can
make things better - what I don't like is its adoption as a
programming technique, because it's a bad one I think. In fact, I
think it would be fine to write systems which had a finalisation like
this *as well* as the normal mechanism, but not *instead* of it.
A specific example with filehandles - imagine you close all
filehandles via the GC in an application like a web server. It's
easily possible to see cases where there you hit some system limit on
filehandles but have lost your references to them because they're all
waiting to be GCd. This is not a good approach to the problem.
(I really mean streams by filehandles above, I think).
--tim
In article <···············@tfeb.org>, Tim Bradshaw <···@tfeb.org> wrote:
>Raymond Toy <···@rtp.ericsson.se> writes:
>
>> CMUCL uses finalizations to close the Unix file descriptor associated
>> with a stream. Perhaps not quite the same as what you're saying here.
>>
>
>I think it's fine for a system to do something like this, since it can
>make things better - what I don't like is its adoption as a
>programming technique, because it's a bad one I think. In fact, I
>think it would be fine to write systems which had a finalisation like
>this *as well* as the normal mechanism, but not *instead* of it.
I agree. The problem with finalization is that you can't reliably depend
on it portably, since you don't know if GC will happen in any timely
fashion. As memory becomes cheaper, it's likely that GC's will happen less
frequently. If a stream lives long enough to get out of the youngest
generations, it could easily take hours between dropping all references to
the stream and the associated file descriptor being closed.
On the other hand, I think that weak references (or at least weak hash
tables) *would* be useful as part of the standard. As I mentioned earlier,
weak tables are a good way to use tables to store object attributes; they
solve the problem that symbol property lists were originally invented for
(by attaching properties to the symbol, rather than storing the
relationship in a table, the property relationship doesn't prevent the
symbol from being GC'ed). Both this and the DYNAMIC-EXTENT declaration
exist to give hints to the implementation that can affect garbage
collection. And they both could be ignored by implementations that don't
wish to make use of the hint. In fact, I don't think garbage collection is
ever mentioned in the standard, so you can't complain that something isn't
collected when there's no requirement to collect anything! Making
something a weak reference just means that the reference *could* go away,
but isn't required to, just as the destructive sequence functions are
allowed, but generally not required, to reuse the original sequence (yes, I
know there are some that have specific destruction specified, like NCONC).
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
>
> On the other hand, I think that weak references (or at least weak hash
> tables) *would* be useful as part of the standard.
Yes, I agree with this completely. Actually, I think I'd like
weakness but not like finalisations to be standard, since I think that
finalisations imply too much about the GC. Also I think they're kind
of less useful and more dangerous.
--tim
>>>>> "bm" == Barry Margolin <······@genuity.net> writes:
bm> As I mentioned earlier, weak tables are a good way to use tables
bm> to store object attributes; they solve the problem that symbol
bm> property lists were originally invented for (by attaching
bm> properties to the symbol, rather than storing the relationship in
bm> a table, the property relationship doesn't prevent the symbol
bm> from being GC'ed).
Based on this, I'm now sure that I don't understand what's going on
here. If a symbol's plist is non-nil, or if it's nil, what does this
have to do with the symbol itself being collected? What you're saying
almost implies that there's a back-reference from the plist to the
symbol, and I don't think that this is the case.
Alternatively, if you're referring to the fact that the plist keys
and/or values could be collected, then I still disagree.
Can you be more specific about your point?
dave
David Bakhash <·····@alum.mit.edu> writes:
> Based on this, I'm now sure that I don't understand what's going on
> here. If a symbol's plist is non-nil, or if it's nil, what does this
> have to do with the symbol itself being collected? What you're saying
> almost implies that there's a back-reference from the plist to the
> symbol, and I don't think that this is the case.
>
The point is, I think that a plist is pointed to by a symbol, so if
the symbol becomes garbage then the plist does too. However if you
implement `plists' by a hashtable with the symbol as a key and the
plist as the value, then unless the hashtable is weak the symbol
never becomes garbage and thus neither does the plist.
--tim
>>>>> "tb" == Tim Bradshaw <···@tfeb.org> writes:
tb> David Bakhash <·····@alum.mit.edu> writes:
>> Based on this, I'm now sure that I don't understand what's going
>> on here. If a symbol's plist is non-nil, or if it's nil, what
>> does this have to do with the symbol itself being collected? What
>> you're saying almost implies that there's a back-reference from
>> the plist to the symbol, and I don't think that this is the case.
>>
tb> The point is, I think that a plist is pointed to by a symbol, so
tb> if the symbol becomes garbage then the plist does too. However
tb> if you implement `plists' by a hashtable with the symbol as a key
tb> and the plist as the value, then unless the hashtable is weak the
tb> symbol never becomes garbage and thus neither does the plist.
The standard simply says that symbols have a plist, and that they
operate in certain ways. My current understanding is that until
package symbols are uninterned, or their package is deleted, the
symbols persist, along with their plists. If an implementation wanted
to internally implement symbol plists using an EQ hashtable, then I'm
sure it could, and this would probably make symbols smaller, since
they no longer need the plist slot.
This would mean that, for example, the implementation of SYMBOL-PLIST
might change to a GETHASH, etc, but this would all take place in the
implementation. The weakness of the hashtable with respect to keys
and values would just have to make sense so that the implementation
was in accordance to the ANSI specification. If implementors can do
it this way (and I'm sure they have already), then all the power to
them; it definitely seems like an improvement.
So you make a symbol, you give it an entry in the plist (key-weak)
hashtable, and, for example, push it only a list of symbols of its
package. A particular implementation might have DELETE-PACKAGE run
through the symbols and clean them up. That's an operation where, if
slow, I personally wouldn't care.
Bottom line is that if implementors can use weak references
(specifically, weak hashtables) to improve performance, size, etc.,
then great. But this doesn't mean that weak references should find
their way into ANSI CL. Any time I've wanted them, though, they have
been via weak hashtables.
dave
In article <··············@alum.mit.edu>,
David Bakhash <·····@alum.mit.edu> wrote:
>The standard simply says that symbols have a plist, and that they
>operate in certain ways. My current understanding is that until
>package symbols are uninterned, or their package is deleted, the
>symbols persist, along with their plists. If an implementation wanted
>to internally implement symbol plists using an EQ hashtable, then I'm
>sure it could, and this would probably make symbols smaller, since
>they no longer need the plist slot.
I wasn't talking about using a hash table to implement plists internally, I
was talking about using hash tables to do things that were traditionally
done using plists. E.g. instead of
(setf (get symbol 'propname) value)
you might do:
(setf (gethash symbol *propname-table*) value)
With the first form, if the symbol is uninterned and all other references
to it go away, the symbol can be GC'ed. In the second form, uninterning
the symbol and dropping all other references won't allow the symbol to be
GC'ed unless the hash table is weak.
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
>>>>> "bm" == Barry Margolin <······@genuity.net> writes:
bm> I wasn't talking about using a hash table to implement plists
bm> internally, I was talking about using hash tables to do things
bm> that were traditionally done using plists. E.g. instead of
bm> (setf (get symbol 'propname) value)
bm> you might do:
bm> (setf (gethash symbol *propname-table*) value)
bm> With the first form, if the symbol is uninterned and all other
bm> references to it go away, the symbol can be GC'ed. In the second
bm> form, uninterning the symbol and dropping all other references
bm> won't allow the symbol to be GC'ed unless the hash table is weak.
I now understand what you were originally talking about, and I'm
*glad* I misunderstood the original intent, since it opened up this
discussion to a possible optimization in the implementation of symbol
plists. I have a rule that is to never use symbol plists except for
macroexpansion purposes and other compile-time hacks. Otherwise, I
too use EQ hashtables, and occasionally wish for weak hashtables.
ACL supports quite a few extensions in this way. LispWorks, as far as
I know, does not have a documented interface to weak hashtables. But
even XEmacs Lisp supports weak hashtables, and yes, they are very
useful.
Question: Does anyone know of a Common Lisp implementation that uses
(key-weak) hashtables to implement symbol plists? I've
heard recently of at least one consideration for this.
dave
* David Bakhash wrote:
> I now understand what you were originally talking about, and I'm
> *glad* I misunderstood the original intent, since it opened up this
> discussion to a possible optimization in the implementation of symbol
> plists.
I'm sorry I pushed the discussion that way - I didn't mean to. I
meant, rather, what Barry described, where you want to have some
general bit of information associated with a symbol.
--tim
Tim Bradshaw <···@tfeb.org> writes:
> Raymond Toy <···@rtp.ericsson.se> writes:
>
> > CMUCL uses finalizations to close the Unix file descriptor associated
> > with a stream. Perhaps not quite the same as what you're saying here.
> >
>
> I think it's fine for a system to do something like this, since it can
> make things better - what I don't like is its adoption as a
> programming technique, because it's a bad one I think. In fact, I
It was a life-safer for me when I was debugging a program that had a
lot of files open, none of which with with-open-file (for strange
reasons). When I had to nuke the call because of yet-another bug of
mine, there was no easy way to find all the open files (yes I could
have used a heap of open files) so adding a finalisation to the
objects containing the file was a quick and easy debugging aide.
Yes, of course later on the system got 'proper' error-handling with
automatic closing of files and such, but in the beginning error with
files were a lot easier to debug if the file was still open :-).
Also in CMUCL it is used as a method of last resort and issues a stern
warning about the 'lost' stream :-)
Groetjes, Peter
--
It's logic Jim, but not as we know it. | ········@debian.org
"God, root, what is difference?" - Pitr|
"God is more forgiving." - Dave Aronson| http://cvs2.cons.org/~pvaneynd/
> However I also think there's a flip side to weak pointers and related
> things like finalisations - they're quite vulnerable to abuse by
> people who don't understand the difference between storage management
> and object management.
So then do tell, what *is* the difference between storage management
and object management, as it relates to finalizations?
More generally, what is a valid use for finalizations?
I once had a program that had a capability for user code to provide
the user with a window. Shortly after that, the window (and its
underlying object) would only have references squirreled away in
closures and continuations. (Apologies if Scheme references are a
faux pas here.)
The code which would cause these continuations to become unreferenced
was completely unrelated to the code which would brought them into
existence, so it was very complex to determine when that window would
no longer be needed.
To address this, I put a finalization on the object underlying the
window, which would delete the window. Then, at particular convenient
points in the UI, I would explicitly gc, expecting the window to go
away. (It turned out that it took two or three gc's in a row to do
so.)
After reading this discussion, it's pretty clear that I was misusing
finalizations, but it's still unclear to me: when would they be
appropriate, other than error checking?
Thanks,
joelh
Joel Ray Holveck <·····@juniper.net> writes:
> So then do tell, what *is* the difference between storage management
> and object management, as it relates to finalizations?
Well, for me, storage management is what the system does, somehow, to
make sure that you don't run out of memory. This happens mostly
beneath the level of user programs, which don't deal with memory
directly, and should be automatic. Programs deal with objects, which
are built out of bits of memory. Object management is looking after
these things - making sure they get created correctly and that
suitable things happen when they are no longer needed. Object
management can't be fully automatic because it depends on what the
program is trying to do.
An example (not related to finalisations) of where object management
can not be automatic is structure-sharing. It can matter that two
objects share structure, but it can also matter that they don't, so
you may have to make copies of (parts of) objects. And what is meant
by a `copy' is partly program-defined.
> More generally, what is a valid use for finalizations?
Finalisations are a way to have bits of the object management get
triggered by things that the storage management system is doing. This
is fine as a way of ensuring that things get cleaned up as a last
resort, but you don't want to make the object management *rely* on the
storage management: because it's automatic, so you have really no
control over when things happen, and because of the characteristics of
storage management systems that we know about mean that things may
effectively never happen.
>
> The code which would cause these continuations to become unreferenced
> was completely unrelated to the code which would brought them into
> existence, so it was very complex to determine when that window would
> no longer be needed.
>
> To address this, I put a finalization on the object underlying the
> window, which would delete the window. Then, at particular convenient
> points in the UI, I would explicitly gc, expecting the window to go
> away. (It turned out that it took two or three gc's in a row to do
> so.)
I think the problem with this is that you're depending on various
properties of the GC - that it exists, that it actually does look at
everything, and that it doesn't take hours to do so if it does. For
many high-performance GCs the latter two are not really true - the
fast, ephemeral, GC does not look at everything, and the full GC may
take from many seconds to many hours to run (if the system is much
bigger than physical memory).
>
> After reading this discussion, it's pretty clear that I was misusing
> finalizations, but it's still unclear to me: when would they be
> appropriate, other than error checking?
>
I think that they're a mechanism of last resort - you have them so
that you have some hope that cleanups that you really need to happen
will happen, eventually - so if your program is leaking objects (as
opposed to storage) they will get cleaned up eventually.
I haven't described this very well, sorry (and I may be wrong).
--tim
Tim Bradshaw <···@tfeb.org> writes:
> Finalisations are a way to have bits of the object management get
> triggered by things that the storage management system is doing. This
> is fine as a way of ensuring that things get cleaned up as a last
> resort, but you don't want to make the object management *rely* on the
> storage management: because it's automatic, so you have really no
> control over when things happen, and because of the characteristics of
> storage management systems that we know about mean that things may
> effectively never happen.
It would be nice if one could have some automatic support, though. A
while back I played around with ideas at the coffee table with some
colleagues. We more or less came to a conclusion that we would settle
for a system that could track objects that involve external resources
(windows, file handles etc.) and finalize them as soon as they are not
reachable from some other similar object. If one is willing to live
with some limitations, this could be done e.g. with reference
counting. Regular gc would then clean up cyclic garbage and run
finalizers, and maybe issue warnings.
This would seem to be relatively straightforward to do at least in a
statically typed language (sorry), so I'm sure we are not the first
ones to dream this up. Does anyone know if it has been done?
--
Pertti Kellom\"aki, Tampere Univ. of Technology, Software Systems Lab
Kellom{ki Pertti <··@arokyyhky.cs.tut.fi> writes:
> It would be nice if one could have some automatic support, though. A
> while back I played around with ideas at the coffee table with some
> colleagues. We more or less came to a conclusion that we would settle
> for a system that could track objects that involve external resources
> (windows, file handles etc.) and finalize them as soon as they are not
> reachable from some other similar object. If one is willing to live
> with some limitations, this could be done e.g. with reference
> counting. Regular gc would then clean up cyclic garbage and run
> finalizers, and maybe issue warnings.
I think the problem is with what you mean by `similar'. It's probably
perfecly doable to have pools of objects which have some special
memory management behaviour, so you get this prompt finalising, but
what counts as a reference? If you want any reference to count, then
the system has to find out promptly when anything is garbage, which is
pretty bad. So you probably want to have only some special kinds of
references. But do you count binding the object to a variable as a
reference? If you do then I think you have problems.
It seems to me that to resolve these problems in any kind of general
way you'd need to make things be such that you took some action to
garbagify something, in which case a reference-counting scheme for
these objects is not too hard to implement by hand I think (though
slow, if course) by things like AFTER methods on SETF of accessors.
but maybe I don't understand what you are after.
--tim
David Bakhash <·····@alum.mit.edu> writes:
> >>>>> "marco" == Marco Antoniotti <·······@cs.nyu.edu> writes:
>
> >> for a discussion of weak pointers and finalizations for one
> >> implementation. Note that weak-ness is not a standard CommonLisp
> >> concept.
>
> marco> Unfortunately. However most implementations do offer it.
>
> Actually, I don't think that this is unfortunate. I actually prefer
> it not to be part of the standard. I've never seen it used (and never
> used it) for anything but cache optimization.
It's useful for interning as well. Consider an implementation of pathnames,
like on the lisp machine, where pathnames are immutable and all pathnames
are interned so you can compare them with EQ instead of the very expensive
EQUALP. If you don't think this is important, try doing remove-duplicates
oon two large results of (directory "...some wildcard...") and notice the
difference in speed if EQ can be used instead of EQUALP... The interning
process really only needs to cache pointers as long as there are pointers
to those pathnames that might matter. If the uses have gone, the table
should empty out. Another case is class structure connectivity. It happens
from time to time that a class would like to get gc'd, but I don't see how
it can be if every pointer in the class-system is strong. Same with the
structure consulted to do list-all-packages; you want a list of all packages,
perhaps even including deleted packages, but you'd like packages pointed to
by no one to go away. I think there are myriad good uses for weak pointers,
and it's only an artifact of the time at which they came into good use (right
around feature freeze time for ansi cl) that they are not in the standard.
Note that there WERE weak pointer implementations earlier than this, but
they were badly done--not that they didn't work, just that their interface
sucked--and that didn't up the sense of "warm fuzzies" in people's mind when
it came to standardizing things. Maclisp, for example, had a type NIL arrays
(it didn't have the meaning of CL's NIL type, so this naming wasn't as
incongruous then as it would be now) which were like type T except they were
weakly marked through. You might use them to hold populations in case you
wanted to map over all instances of something, but you didn't want it to
be the case that the holding of a pointer to all instances meant that no
instance could be gc'd. So if the system removed an element from the array,
it left NIL in its place. It was clumsy, but it had some uses.
> I'm not even sure how
> big of a win it is when it's there, and you're using it correctly.
> The point is that it's messy business,
why? it's quite abstract and explainable, as much as the gc is anyway.
> and it's gotta do with
> something that's very unspecified: the GC mechanism. I think it was
> not unfortunate, but wise that it's not part of the standard.
the gc isn't messy.
Kent M Pitman <······@world.std.com> writes:
> David Bakhash <·····@alum.mit.edu> writes:
>
> > >>>>> "marco" == Marco Antoniotti <·······@cs.nyu.edu> writes:
> >
> > >> for a discussion of weak pointers and finalizations for one
> > >> implementation. Note that weak-ness is not a standard CommonLisp
> > >> concept.
> >
> > marco> Unfortunately. However most implementations do offer it.
> >
> > Actually, I don't think that this is unfortunate. I actually prefer
> > it not to be part of the standard. I've never seen it used (and never
> > used it) for anything but cache optimization.
>
> It's useful for interning as well. Consider an implementation of pathnames,
> like on the lisp machine, where pathnames are immutable and all pathnames
> are interned so you can compare them with EQ instead of the very expensive
> EQUALP.
Another example of using this technique for interning is for certain
uses of constant lists, such as arglists and function-specs (not a
standard term, function-specs are those function names which are lists,
such as (setf foo). Allegro CL extends the available kinds of
function-specs to include names like (flet foo bar) and
(method print-object (t t)). These specs are converted to an internal
form which increases sharing per cons cell, and are stored into a
weak-keyed hash-table sans values (i.e. the keys are themselves the
values, and each time the garbage collector collects a key as garbage,
it reduces the size of the hash-table by 1). This provides for
internment and space savings.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)