hi,
I'm interested to know if there are ways to inhibit the GC from taking over.
Suppose you have a fn that you don't want to be interrupted by the GC. I've
seen, in MP packages mp::without-scheduling, and (in ACL)
excl::without-interrupts, but I don't think these prevent GC. Of course, this
might not sound like a good idea, but the code I'm trying to block would be
short enough not to cause potential problems with memory.
is one of these the way to go? Or is there a better way?
dave
* David Bakhash <·····@bu.edu>
| I'm interested to know if there are ways to inhibit the GC from taking
| over. Suppose you have a fn that you don't want to be interrupted by the
| GC.
("taking over"? is that differing from "being performed"?)
there are two basic ways to avoid GC in a particular function: (1) don't
cons, and (2) GC before calling it. it is not particularly hard to avoid
consing, but it takes a special concern when writing the code, and you
will realize that it involves doing your own local memory management,
which basically means you either use all pre-allocated structures or you
have to reimplement the functions that do the allocation. a non-consing
CONS could re-use cons cells off of a pre-allocated list, but you would
have to free them when you are done with them.
the problem is basically this: when GC occurs because you're out of free
memory, what would you rather do? fail, somehow? that would easily cost
more than a GC would, especially if your worry is real-time response.
if you are only looking to avoid global gc's, that's controllable. see
the writeup in doc/cl/gc.htm in your Allegro CL installation directory.
if you are looking for ways to minimize the cost of a scavenge in ACL,
you can pretty much control that with the size of the newspace and the
size of the oldspace allocation size. if you don't ask for global gc
when oldspace is full (the default, by the way), you don't do a global gc
automatically (see *GLOBAL-GC-BEHAVIOR* and *TENURED-BYTES-LIMIT*), you
can do global gc when you feel like it. if your system is idle for long
periods of time, you can make global GC happen in the idle time if that
is more convenient.
in general, you have pretty good control over the garbage collection, but
I highly recommend reading the manual. this is generally tricky stuff.
#:Erik
--
@1999-07-22T00:37:33Z -- pi billion seconds since the turn of the century
Erik Naggum <····@naggum.no> writes:
> the problem is basically this: when GC occurs because you're out
> of free memory, what would you rather do? fail, somehow?
Ask more memory from the system?
I'm not very familiar with other GC's, but I know that Elisp memory
allocator does not GC when there is no more free memory -- in that
case, it allocates more from the heap. The GC happens after a certain
number of bytes have been allocated. In Emacs, doing what David asks
would be achieved with:
(let ((gc-cons-threshold most-positive-fixnum))
... code ...)
All of this would be, of course, very implementation-dependent.
* Hrvoje Niksic <·······@srce.hr>
| Ask more memory from the system?
you seem to have taken "no free memory" a bit differently than it was
intended, but let's assume for the sake of argument that the system
denies your request. since this no different from asking more memory
from a bounded set, the same argument applies: if GC occurs when there is
no free memory available, what do you do?
| I'm not very familiar with other GC's, but I know that Elisp memory
| allocator does not GC when there is no more free memory
since he was asking about the Allegro CL scavenging garbage collector, I
thought it would be more instructive to answer in that context, but there
are of course other GC implementations around. Emacs', for instance, is
horribly, horribly in efficient, and Allegro CL's is very fast and also
very wasteful. I'm not sure the reason is solely a question of when GC
occurs, but after I modified my Emacs to let GC free pages that it had
emptied of objects, it no longer grew to be quite so big. go figure...
#:Erik
--
@1999-07-22T00:37:33Z -- pi billion seconds since the turn of the century
In article <················@naggum.no>, Erik Naggum <····@naggum.no> wrote:
>* Hrvoje Niksic <·······@srce.hr>
>| Ask more memory from the system?
>
> you seem to have taken "no free memory" a bit differently than it was
> intended, but let's assume for the sake of argument that the system
> denies your request. since this no different from asking more memory
> from a bounded set, the same argument applies: if GC occurs when there is
> no free memory available, what do you do?
I would recommend that whatever mechanism is provided by the implementation
be in the form of a "hint". Many GC's are proactive or use heuristics to
try to determine when a good time for GC is (e.g. waiting until the
application is idle), but the application should be able to indicate "this
isn't a good time for a GC". If there's an emergency need for a GC, it
should probably ignore the hint.
There could be a stronger version that completely prohibits GC. If it runs
out of available memory during that time, it should crash the application,
just as if it had performed a GC and it didn't free up any memory.
--
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin wrote:
> There could be a stronger version that completely prohibits GC. If it runs
> out of available memory during that time, it should crash the application,
> just as if it had performed a GC and it didn't free up any memory.
Even better would be the ability to dinamically specify the size of
a preallocated virtual memory reserve to use if you run out of normal blocks
when GC is turned off. It would be good if the implementation could ensure
that a virgin page only needed to get a physical frame allocated (or it could
even be already locked in) instead of having to swap in a boring block from disk.
* Erik Naggum <····@naggum.no>
| since he was asking about the Allegro CL scavenging garbage collector, I
| thought it would be more instructive to answer in that context, but there
| are of course other GC implementations around. Emacs', for instance, is
| horribly, horribly in efficient, and Allegro CL's is very fast and also
| very wasteful.
cursed be editing without proper substructure navigation. that last
sentence was intended to read:
Emacs', for instance, is horribly, horribly inefficient and also very
wasteful, and Allegro CL's is very fast.
sigh.
#:Erik
--
@1999-07-22T00:37:33Z -- pi billion seconds since the turn of the century
* Erik Naggum wrote:
[About GCs]
> Emacs', for instance, is horribly, horribly inefficient and also very
> wasteful, and Allegro CL's is very fast.
There's an interesting story about GC and Emacs. Back before Java,
the standard mythology was that languages with GC spent all their time
in GC which was why real programmers used sbrk instead. At some
point, an Emacs version I use (Lemacs as was) came with a profiler
which worked at a pretty low level (below elisp, although it was
controlled from elisp), and would tell you a lot of interesting things
about where the system spent its time.
I once turned this thing on and left it turned on for several days in
the Emacs I was using to do all my work, including several largish
elisp packages like VM and gnus and probably psgml &c. The results
were that it spent about 1-2% of its time in GC, and about 60-80% in
updating the screen. Like Erik says, Emacs' GC is rudimentary, but it
was so far from the critical path in terms of performance as to be
totally irrelevant. So much for mythology.
--tim
* Tim Bradshaw <···@tfeb.org>
| I once turned this thing on and left it turned on for several days in the
| Emacs I was using to do all my work, including several largish elisp
| packages like VM and gnus and probably psgml &c. The results were that
| it spent about 1-2% of its time in GC, and about 60-80% in updating the
| screen. Like Erik says, Emacs' GC is rudimentary, but it was so far from
| the critical path in terms of performance as to be totally irrelevant.
| So much for mythology.
there are several interesting aspects to this: (1) people see that
something is happening when the screen updates, (2) people see a pause
when the "Garbage collecting..." message lights up, and (3) they thought
Emacs got a lot faster when that message was removed (at my request).
if it moves slowly, humans can deal well with it. if it moves in jerks
and spasms, humans get nervous. the overall impression of "smoothness"
is very important to user perception of speed; any break in smoothness is
then taken as a serious flaw. we see the same thing in many areas:
people had no problems dealing with 110 baud ttys 20 years ago, and if
the information trickles in, people can still deal with very slow lines
(witness the World Wide Wait, and how people happily wait for images of
text to crawl across the line at the rate of 1 character a second), but
if you give somebody a 56kbps line and it goes from 56 kbps to 110 bps
and back at random intervals, you can drive people literally insane, and
that's precisely what the WWW does to congested links. still, it's the
greatest thing since sliced bread. all we have to do now is make the WWW
better than sex for the dumb people who click on the advertising, so they
won't procreate! ahem, I digress.
#:Erik
--
@1999-07-22T00:37:33Z -- pi billion seconds since the turn of the century
Erik Naggum <····@naggum.no> writes:
> very wasteful. I'm not sure the reason is solely a question of when GC
> occurs, but after I modified my Emacs to let GC free pages that it had
> emptied of objects, it no longer grew to be quite so big. go figure...
Did you report this modification to the Emacs community? I would like to know
how you did this (esp. if it can speed up XEmacs's GC).
dave
David Bakhash <·····@bu.edu> writes:
> > very wasteful. I'm not sure the reason is solely a question of when GC
> > occurs, but after I modified my Emacs to let GC free pages that it had
> > emptied of objects, it no longer grew to be quite so big. go figure...
>
> Did you report this modification to the Emacs community?
Given that Erik certainly is part of that community the question is
rather useless..
> I would like to know how you did this (esp. if it can speed up
> XEmacs's GC).
XEmacs's GC already does this.
Jan
* Erik Naggum wrote:
> there are two basic ways to avoid GC in a particular function: (1) don't
> cons, and (2) GC before calling it.
I'm not sure if the original poster was interested just in ACL (where
Erik knows more than me!), but in general this isn't a safe way of
doing it because the GC may well be living in a different thread than
your code, and could be making all sorts of complex decisions about
whether know was a good time to GC. Even non-multithreaded
implementations may have GCs that live off the back of timer
interrupts or something.
So I think in general you need some kind of special WITHOUT-GC type
form which the implementation might provide (and of course for
multithreaded implementations you also need WITH-SCHEDULING-INHIBITED
or something at least as much as GC inhibition).
A good example of the kind of hairiness this can incolve is Genera
which has a GC which can run in its own thread and a large number of
GC, process & memory controlling switches (which fortunately you
almost never need to adjust) up to and including things to lock bits
of virtual memory in core (very useful for doing audio on a 36xx).
--tim