From: Pierre THIERRY
Subject: Garbage collection tweaking
Date: 
Message-ID: <pan.2006.08.17.13.54.57.176516@levallois.eu.org>
I'm wondering: do some Lisp implementations allow the programmer to
interact with how garbage is collected? For example, tell the system to
use reference counting to respect some real time constraints?

Another use I would see would be to help the GC by using some free()
call that would not really free the memory but tell the GC to do so.
That would have the benefit, compared to a real manual memory
management, that an incorrect call would not free the memory at all and
then avoid creating dangling pointers...

Are these viable options for a Lisp implementation? Does this already
exist?

Semi-automatically,
Nowhere man
-- 
···········@levallois.eu.org
OpenPGP 0xD9D50D8A

From: Rob Thorpe
Subject: Re: Garbage collection tweaking
Date: 
Message-ID: <1155827370.943204.201470@m73g2000cwd.googlegroups.com>
Pierre THIERRY wrote:
> I'm wondering: do some Lisp implementations allow the programmer to
> interact with how garbage is collected?

Many lisp implementations allow the programmer to activate the GC.
This can be useful, it means you can find a place in the execution path
where the system isn't busy and GC then.  There are all kinds of other
tweaking various implementations allow.

> For example, tell the system to
> use reference counting to respect some real time constraints?

Ref counting is not done for any lisp I know of.  It is not really
realtime safe anyway, when the refcount on a large object holding many
others goes to zero then effect ripples through it's children causing a
pause like GC.  Its just better than normal GC.

> Another use I would see would be to help the GC by using some free()
> call that would not really free the memory but tell the GC to do so.
> That would have the benefit, compared to a real manual memory
> management, that an incorrect call would not free the memory at all and
> then avoid creating dangling pointers...

The question is: how would the GC respond?
It can't safely take the "free" request at face value, it must check
it.  To check it it must perform at least the mark portion of the
process.
From: Pierre THIERRY
Subject: Re: Garbage collection tweaking
Date: 
Message-ID: <pan.2006.08.17.17.28.06.727387@levallois.eu.org>
Le Thu, 17 Aug 2006 08:09:30 -0700, Rob Thorpe a écrit :
> Many lisp implementations allow the programmer to activate the GC.

How is it done in general?

The problem is, in any way, no object passed to the GC as
reclaimable could be safely reclaimed in the lexical scope of the
variable used to pass it. Obviously, within that scope, the object is
reachable, and thus reclaiming would result in a dangling reference.

So the problem boils down to passing an object to the GC in a way that let
the GC know when it could reclaim. If it was by way of a normal Lisp
form, one of it's upper form could be expanded in a way that breaks the
whole purpose, by creating a too narrow lexical scope where the
reclaiming would be asked for.

For example this form:

(foobar container element
  (do-something-useful element)
  (do-something-else container)
  (remove-from container element)
  (please-reclaim element))

Which could be expanded in:

(let ((#:1 (proxy container))
      (#:2 (proxy element)))
  (let ((container #:1)
        (element #:2))
    (do-something-useful element)
    (do-something-else container)
    (remove-from container element)
    (please-reclaim element)))

In this expanded form, the GC would just notice that element (because of
the outer let) is still reachable, and would not reclaim it immediately.

So the only safe way to do this, as I see it, was to use a declaration,
because they are not forms that are subject to this kind of expansion
arbitrarily, IIUC. Something like:

(foobar container element
  (declare (reclaimable element))
  (do-something-useful element)
  (do-something-else container)
  (remove-from container element))

> The question is: how would the GC respond?
> It can't safely take the "free" request at face value, it must check
> it.  To check it it must perform at least the mark portion of the
> process.

Yes, but it would not need do anything else, like reclaiming anything
else than the specified objects, which could help much in some cases, I
suspect.

Doubtfully,
Nowhere man
-- 
···········@levallois.eu.org
OpenPGP 0xD9D50D8A
From: Pascal Bourguignon
Subject: Re: Garbage collection tweaking
Date: 
Message-ID: <87y7tnl9av.fsf@thalassa.informatimago.com>
Pierre THIERRY <···········@levallois.eu.org> writes:

> I'm wondering: do some Lisp implementations allow the programmer to
> interact with how garbage is collected? For example, tell the system to
> use reference counting to respect some real time constraints?
>
> Another use I would see would be to help the GC by using some free()
> call that would not really free the memory but tell the GC to do so.
> That would have the benefit, compared to a real manual memory
> management, that an incorrect call would not free the memory at all and
> then avoid creating dangling pointers...
>
> Are these viable options for a Lisp implementation? Does this already
> exist?

The Common Lisp Standard doesn't specify a garbage collector.

IMU, would be perfectly conforming to propose an implementation of
Common Lisp with a SYSTEM:FREE function and a
SYSTEM:DANGLING-REFERENCE-P predicate, and a
SYSTEM:DANGING-REFERENCE-ERROR condition.

(let ((p (list 1 2 3)))
    (SYSTEM:FREE p)
    (print (SYSTEM:DANGLING-REFERENCE-P p))
    (print p))

prints:  T
signals: SYSTEM:DANGING-REFERENCE-ERROR
and leaks three conses and three fixnums.

Indeed, SYSTEM:FREE can only free the CONS pointed to by p.
You'd have to write:

(defun free-list (list)
   (warn "This function is buggy: it doesn't work with circular lists")
   (cond ((null list))
         ((atom list) (system:free list))
         (t (if (consp (car list))
                (free-list (car list))
                (system:free (car list)))
            (free-list (cdr list)))))

and this is only the tip of the iceberg.


Very soon, it looks like heavy memory leaking would be a great solution...

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

READ THIS BEFORE OPENING PACKAGE: According to certain suggested
versions of the Grand Unified Theory, the primary particles
constituting this product may decay to nothingness within the next
four hundred million years.
From: Pierre THIERRY
Subject: Re: Garbage collection tweaking
Date: 
Message-ID: <pan.2006.08.17.15.01.13.759576@levallois.eu.org>
Le Thu, 17 Aug 2006 16:35:20 +0200, Pascal Bourguignon a écrit :
> IMU, would be perfectly conforming to propose an implementation of
> Common Lisp with a SYSTEM:FREE function and a
> SYSTEM:DANGLING-REFERENCE-P predicate, and a
> SYSTEM:DANGING-REFERENCE-ERROR condition.

It should not be possible to create a dangling reference. Though you
made me realize one could implement a Lisp without GC...

> (let ((p (list 1 2 3)))
>     (SYSTEM:FREE p)
>     (print (SYSTEM:DANGLING-REFERENCE-P p))
>     (print p))
> 
> prints:  T
> signals: SYSTEM:DANGING-REFERENCE-ERROR
> and leaks three conses and three fixnums.

The goal would precisely be that (free foo) would not really free it but
tell the GC to see if it can be safely reclaimed. The problem is, if you
can refer to it in the (free) call, it is still in the lexical scope.

Shit. I'm sure there is a way to achieve this, but it seems trickier
than I thought.

Dangingly,
Nowhere man
-- 
···········@levallois.eu.org
OpenPGP 0xD9D50D8A
From: Tim Bradshaw
Subject: Re: Garbage collection tweaking
Date: 
Message-ID: <ec2c2h$isi$3$830fa7a5@news.demon.co.uk>
On 2006-08-17 14:55:07 +0100, Pierre THIERRY 
<···········@levallois.eu.org> said:

> I'm wondering: do some Lisp implementations allow the programmer to
> interact with how garbage is collected? For example, tell the system to
> use reference counting to respect some real time constraints?
> 
> Are these viable options for a Lisp implementation? Does this already
> exist?

Yes, many Lisp implementations allow extensive interaction with the GC.
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: Garbage collection tweaking
Date: 
Message-ID: <87ac63rzds.fsf@qrnik.zagroda>
Pierre THIERRY <···········@levallois.eu.org> writes:

> Another use I would see would be to help the GC by using some free()
> call that would not really free the memory but tell the GC to do so.

I haven't heard of GC algorithms which would benefit from some hints
about freeing a given object.

-- 
   __("<         Marcin Kowalczyk
   \__/       ······@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/