From: Dmitrii Manin
Subject: reduce garbage collection in a program: how?
Date: 
Message-ID: <MANIN.94Jun1104425@sticky.rockefeller.edu>
(Sorry if comp.lang.lisp is inappropriate for elisp questions :)

I wrote an elisp program which works fine, but collects garbage too
frequently. Are there any tips as to how I should write a program to
reduce GC? For example, should I use more local variables, or on the
contrary, nest as much expressions as possible? To be specific,
consider this example: what is better (from the point of view of
garbage collection)

;------ example 1 ------
(while not-done
	(do-something (regexp-quote my-string)))

or

;------ example 2 ------
(setq my-string-req (regexp-quote my-string))
(while not-done
	(do-something my-string-req))

What (else) affects garbage accumulation?

Thanks

--
- M

This signature was automatically generated specifically for gnu.emacs.help

From: Michael Callahan
Subject: Re: reduce garbage collection in a program: how?
Date: 
Message-ID: <2sjj0q$8ue@magus.cs.utah.edu>
Dmitrii Manin (·····@sticky.rockefeller.edu) wrote:

: (Sorry if comp.lang.lisp is inappropriate for elisp questions :)

: I wrote an elisp program which works fine, but collects garbage too
: frequently. Are there any tips as to how I should write a program to
: reduce GC? For example, should I use more local variables, or on the
: contrary, nest as much expressions as possible? To be specific,
: consider this example: what is better (from the point of view of
: garbage collection)

: ;------ example 1 ------
: (while not-done
: 	(do-something (regexp-quote my-string)))

: or

: ;------ example 2 ------
: (setq my-string-req (regexp-quote my-string))
: (while not-done
: 	(do-something my-string-req))

: What (else) affects garbage accumulation?

: Thanks

: --
: - M

: This signature was automatically generated specifically for gnu.emacs.help

Sure, the first one is probably less efficient, because it keeps
recalculating the (regexp-quote my-string form.  It's probably
a good idea to not do lots of recalculations.

If you want to reduce Garbabe Collection, reduce memory allocations
as much as possible.

  mike
From: Dmitrii Manin
Subject: Re: reduce garbage accumulation in a program: how?
Date: 
Message-ID: <MANIN.94Jun2110209@sticky.rockefeller.edu>
In article <··········@magus.cs.utah.edu> ········@sunset.cs.utah.edu (Michael Callahan) writes:

o Dmitrii Manin (·····@sticky.rockefeller.edu) wrote:
o 
o : consider this example: what is better (from the point of view of
o : garbage collection)
o 
o : ;------ example 1 ------
o : (while not-done
o : 	(do-something (regexp-quote my-string)))
o 
o : or
o 
o : ;------ example 2 ------
o : (setq my-string-req (regexp-quote my-string))
o : (while not-done
o : 	(do-something my-string-req))

o Sure, the first one is probably less efficient, because it keeps
o recalculating the (regexp-quote my-string form.  It's probably
o a good idea to not do lots of recalculations.

From the point of view of recalculation, yes. But if this sequence is
itself within a frequently-called function, it seems to be more
efficient to avoid introducing an extra local variable, thus reducing
garbage accumulation. Am I right? 

o If you want to reduce Garbabe Collection, reduce memory allocations
o as much as possible.

Thanks, but what precisely are the cases when memory is being
allocated? Setq-ing new local variables? I guess so. What else? What
about nesting functions calls within function calls? Suppose nested
functions do not use local variables. Does returning result from them
involve memory allocation? 

*******************************************************************
Again, suppose I have a frequently-called function. What techniques
should I use within it to reduce accumulation of garbage?
*******************************************************************
--
- M

This signature was automatically generated specifically for comp.lang.lisp
From: Barry Margolin
Subject: Re: reduce garbage accumulation in a program: how?
Date: 
Message-ID: <2snv8rINNbgk@early-bird.think.com>
In article <··················@sticky.rockefeller.edu> ·····@sticky.rockefeller.edu (Dmitrii Manin) writes:
>o Sure, the first one is probably less efficient, because it keeps
>o recalculating the (regexp-quote my-string form.  It's probably
>o a good idea to not do lots of recalculations.
>
>From the point of view of recalculation, yes. But if this sequence is
>itself within a frequently-called function, it seems to be more
>efficient to avoid introducing an extra local variable, thus reducing
>garbage accumulation. Am I right? 

No.  Variables aren't garbage, objects are.  Garbage is created each time
you let go of an object.  Each time you call regexp-quote, it creates an
object.  When you you do

(while not-done
  (do-something (regexp-quote my-string))

you allocate and let go of a string each time around the loop (I'm assuming
do-something doesn't save its argument in any permanent data structure),
and also regexp-quote may generate its own garbage while it's running.  So
garbage is created each time you iterate.  On the other hand, when you do

(let ((variable (regexp-quote my-string)))
  (while not-done
    (do-something variable)))

you only create one string of garbage.  You also create a symbol, but it
doesn't become garbage because it's interned; if you use the same variable
name in multiple places, they all refer to the same symbol, so the storage
is amortized.

>o If you want to reduce Garbabe Collection, reduce memory allocations
>o as much as possible.
>
>Thanks, but what precisely are the cases when memory is being
>allocated? Setq-ing new local variables? I guess so. What else? What
>about nesting functions calls within function calls? Suppose nested
>functions do not use local variables. Does returning result from them
>involve memory allocation? 

Memory is allocated when you create new objects.  Setting a variable just
puts a reference to the object in the variable, so it doesn't allocate
memory.  Function calls and returns are all done by passing around
references to objects, so they don't allocate memory.  Functions like cons,
make-string, and vector create new objects.  Functions like car and cdr
don't allocate memory, since they just return objects that already existed.
-- 
Barry Margolin
System Manager, Thinking Machines Corp.

······@think.com          {uunet,harvard}!think!barmar
From: Michael Callahan
Subject: Re: reduce garbage accumulation in a program: how?
Date: 
Message-ID: <2so0c6$r6b@magus.cs.utah.edu>
Dmitrii Manin (·····@sticky.rockefeller.edu) wrote:

: In article <··········@magus.cs.utah.edu> ········@sunset.cs.utah.edu (Michael Callahan) writes:

: From the point of view of recalculation, yes. But if this sequence is
: itself within a frequently-called function, it seems to be more
: efficient to avoid introducing an extra local variable, thus reducing
: garbage accumulation. Am I right? 

Briefly, no. 

The recalculation probably does some consing, which allocates memory.
Also, the result that the function regexp-quote returns is going
to require allocation as well.

The mistake you seem to be making is confusing variables and the
values that they point to.  The variables themselves aren't going
to take up much room, probably little more than a pointer (and a
symbol in emacs lisp, since it's dynamic).  It's the data objects
themselves that are going to take up most of the room.  Things like
lists, strings, vectors, etc.

: *******************************************************************
: Again, suppose I have a frequently-called function. What techniques
: should I use within it to reduce accumulation of garbage?
: *******************************************************************

Most of the things that you would normally do to speed it up.  Like
avoid recalculating results over and over, get rid of function calls
that you don't need, etc.  To some extent, doing these calculations
will involve memory allocations for results.

Avoid constructors.  Things like cons, or make-string, or other functions
that create new results.  For instance, regexp-quote probably 
returns a new regular expression object every time you run it.
Look into using the available destructive operators.  They are
harder to use, because they mutate the arguements that you pass in
to them, but they do less object creation.  Examples of
such functions are setcdr and setcar.

  mike
From: Larry Hunter
Subject: Re: reduce garbage collection in a program: how?
Date: 
Message-ID: <HUNTER.94Jun2114505@work.nlm.nih.gov>
Dmitrii Manin asks about reducing gc'ing in elisp:

   : I wrote an elisp program which works fine, but collects garbage too
   : frequently. Are there any tips as to how I should write a program to
   : reduce GC? 

What you want to reduce is the amount of CONSing (memory allocation) going
on.  Since memory allocation is often hidden from you in LISP (that's one of
its advantages) it can sometimes be hard to figure out what to fix.  To
start, read the chapter on garbage collection in the ELISP documentation.

The best thing to do in this case is use profiling tools to figure out where
you are consing, and focus on that code.  Unfortunately, I'm not aware of an
elisp profiler that tracks consing.  Barry Warsaw's ELP I think just does
timings.  You should ask on the gnu.emacs.help board if anyone knows of a
space profiler.  You can at least run the function "garbage-collect" before
and after your function(s), since it tells you how many cons cells are in
use, have been freed, etc.   This is the information you need to figure out
which parts of your program are the main garbage offenders.  [Maybe someone
could integrate this into ELP... :-)]

There are actually two issues to be concerned about: how many cons cells
your program generates, and how many of those conses are persistant (that
is, are NOT gc'd and just accumulate).   Both need to be controlled in a
large program (or one that just gc's too much).

   : For example, should I use more local variables, or on the
   : contrary, nest as much expressions as possible? To be specific,
   : consider this example: what is better (from the point of view of
   : garbage collection)

   : ;------ example 1 ------
   : (while not-done
   : 	(do-something (regexp-quote my-string)))

   : or

   : ;------ example 2 ------
   : (setq my-string-req (regexp-quote my-string))
   : (while not-done
   : 	(do-something my-string-req))


Well, neither of these quite get it right.  Regexp-quote conses, so you want
to apply it only once.  On the other hand, my-string-req persists forever,
and you only need it locally.  Do this instead:

  (let ((my-string-req (regexp-quote my-string)))
    (while not-done
       (do-something my-string-req)))	

Hope this gets you started...

Larry

--
Lawrence Hunter, PhD.
National Library of Medicine
Bldg. 38A, MS-54
Bethesda. MD 20894 USA
tel: +1 (301) 496-9300
fax: +1 (301) 496-0673 
internet: ······@nlm.nih.gov
encryption: public key via RIPEM server or "finger ······@work.nlm.nih.gov"
From: Barry A. Warsaw
Subject: Re: reduce garbage collection in a program: how?
Date: 
Message-ID: <WARSAW.94Jun2181547@anthem.nlm.nih.gov>
>>>>> "LH" == Larry Hunter <······@work.nlm.nih.gov> writes:

    LH> Barry Warsaw's ELP I think just does timings.

Right.
    
    LH> [Maybe someone could integrate this into ELP... :-)]

I don't have time to do this, but if anybody does, please send me the
results.  It would be really cool if elp.el did this!

-Barry