I have a question about memory utilization in Lisp. Using CLISP, I created a
simple array using the following form:
(setq a1 (make-array '(1000 1000)))
The RAM usage (under Windows NT4) jumped from 94MB to just under 200MB as
the array was being allocated. That equates to around 100MB for a 1,000,000
item array. Something must be going on behind the scenes as there is no way
100KB can be allocated per variable. Does anyone have any insight to this ?
-Luke
My apologies, I failed to mention that I created the array referenced in my
previous post within the CLISP interactive environment (GNU readline).
I ran the same form using the compiler.. RAM utilization increased from 94MB
to 102MB, a total of 8MB. The 8MB includes the memory used by the LISP
environment itself. So I think that the RAM allocated per variable is more
in the 1KB to 1.5KB range.
-Luke
"Luke Crook" <······@wsbnet.com> wrote in message
···············@dispatch.concentric.net...
> I have a question about memory utilization in Lisp. Using CLISP, I created
a
> simple array using the following form:
>
> (setq a1 (make-array '(1000 1000)))
>
> The RAM usage (under Windows NT4) jumped from 94MB to just under 200MB as
> the array was being allocated. That equates to around 100MB for a
1,000,000
> item array. Something must be going on behind the scenes as there is no
way
> 100KB can be allocated per variable. Does anyone have any insight to this
?
>
> -Luke
>
>
"Luke Crook" <······@wsbnet.com> writes:
> My apologies, I failed to mention that I created the array referenced in my
> previous post within the CLISP interactive environment (GNU readline).
>
> I ran the same form using the compiler.. RAM utilization increased from 94MB
> to 102MB, a total of 8MB. The 8MB includes the memory used by the LISP
> environment itself. So I think that the RAM allocated per variable is more
> in the 1KB to 1.5KB range.
Unless I'm very much mistaken, if allocating a 1000*1000 array takes
8MB of RAM, then that comes out to 8*1024*1024 bytes divided by
1000*1000 cells, or in Lisp:
(float (/ (* 8 1024 1024) (* 1000 1000)))
=> 8.388608
i.e. a little over 8 bytes or two 32bit words per cell. I'd have
expected something on the order of 4 bytes or 1 32bit word per cell
for an unspecialized array on a 32bit machine. OTOH your figure
includes the CLISP environment itself (why?). Furthermore your figure
might include memory usage due to a copying GC. You might want to use
(time ...) to measure such things from inside CLISP, which is going to
be more accurate. On Linux I get:
[5]> (time (prog1 nil (make-array '(1000 1000))))
Real time: 0.168675 sec.
Run time: 0.16 sec.
Space: 4000032 Bytes
GC: 2, GC time: 0.1 sec.
NIL
Which is completely in line with my expectations...
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
"Pierre R. Mai" <····@acm.org> wrote in message
···················@orion.bln.pmsf.de...
> "Luke Crook" <······@wsbnet.com> writes:
>
> Unless I'm very much mistaken, if allocating a 1000*1000 array takes
> 8MB of RAM, then that comes out to 8*1024*1024 bytes divided by
> 1000*1000 cells, or in Lisp:
You are correct. Not sure what I was doing to come up with 1KB per variable.
>You might want to use
> (time ...) to measure such things from inside CLISP, which is going to
> be more accurate. On Linux I get:
>
> [5]> (time (prog1 nil (make-array '(1000 1000))))
>
> Real time: 0.168675 sec.
> Run time: 0.16 sec.
> Space: 4000032 Bytes
> GC: 2, GC time: 0.1 sec.
> NIL
>
Apart from the execution time, my results (dual PPro 200) compare with
yours:
[1]> (time (prog1 nil (make-array '(1000 1000))))
Real time: 0.875 sec.
Run time: 0.84375 sec.
Space: 4000032 Bytes
GC: 2, GC time: 0.71875 sec.
NIL
However executing the following form from within CLISP takes... well, put it
this way, it still hadn't finished after two minutes.
[2]> (setq a1 (make-array '(1000 1000)))
Assigning the array to a variable seems to be the cause of the rampant
memory usage. The previous form swallows almost 100Megs before it has
completed.
-Luke
From: Roland Kaufmann
Subject: Re: UDATE Lisp and arrays, memory utilization
Date:
Message-ID: <tl2zo2xt2ou.fsf@space.at>
>>>>> "Luke" == Luke Crook <······@wsbnet.com> writes:
[snip]
Luke> However executing the following form from within CLISP
Luke> takes... well, put it this way, it still hadn't finished
Luke> after two minutes.
Luke> [2]> (setq a1 (make-array '(1000 1000)))
Luke> Assigning the array to a variable seems to be the cause of
Luke> the rampant memory usage. The previous form swallows almost
Luke> 100Megs before it has completed.
I guess this is a side-effect of returning the new value of a1 to the
REPL. Since I do not expect that you want to see a million NIL's
printed, I suggest something like
> (time (setq a1 (make-array '(1000 1000))
dummy nil))
Real time: 0.266209 sec.
Run time: 0.27 sec.
Space: 4000040 Bytes
GC: 2, GC time: 0.18 sec.
NIL
--
best regards
Roland Kaufmann
Roland Kaufmann schrieb:
> > (time (setq a1 (make-array '(1000 1000))
> dummy nil))
> Real time: 0.266209 sec. 0.6 sec.
> Run time: 0.27 sec. 0.6 sec.
> Space: 4000040 Bytes 4000032 Bytes
> GC: 2, GC time: 0.18 sec. GC:2 GC time: 0.49 sec.
> NIL NIL
:(( seems something different
stefan
In article <··········@dispatch.concentric.net>,
Luke Crook <······@wsbnet.com> wrote:
>However executing the following form from within CLISP takes... well, put it
>this way, it still hadn't finished after two minutes.
>
>[2]> (setq a1 (make-array '(1000 1000)))
>
>Assigning the array to a variable seems to be the cause of the rampant
>memory usage. The previous form swallows almost 100Megs before it has
>completed.
Do you have *PRINT-CIRCLE* enabled? If so, the problem is all the
bookkeeping the printer has to do to detect if the same object is in
multiple entries in the array, so that it will use #n= and #n# notation for
the duplicate entries. Or if you have *PRINT-PRETTY* enabled, this might
cause quite a bit of overhead.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
"Luke Crook" <······@wsbnet.com> writes:
> However executing the following form from within CLISP takes... well, put it
> this way, it still hadn't finished after two minutes.
>
> [2]> (setq a1 (make-array '(1000 1000)))
>
> Assigning the array to a variable seems to be the cause of the rampant
> memory usage. The previous form swallows almost 100Megs before it has
> completed.
It isn't the assignment to the variable that is causing you problems,
it is the printing of the result of your form: Remember that setq
returns the last assigned value as its value, and that the
read-eval-print loop will print the result value of each form you
enter. That means that CLISP will try to print the 1000x1000 array.
Since both *print-array* and *print-pretty* are true by default, this
will try to pretty-print the array, which will take a huge amount of
time.
When experimenting with large arrays, you might want to set
*print-array* to nil, so that arrays aren't printed in re-readable
form (i.e. including all element values), but in an opaque form.
Otherwise you'll have to ensure not to return the array as a result
value, e.g. if you do
(defparameter *a1* (make-array '(1000 1000)))
or
(progn (setq *a1* (make-array '(1000 1000))) nil)
instead, the array will not be returned, and hence not printed.
Either of those approaches should solve your memory- and
time-consumption issues.
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
"Pierre R. Mai" wrote:
> When experimenting with large arrays, you might want to set
> *print-array* to nil, so that arrays aren't printed in re-readable
> form (i.e. including all element values), but in an opaque form.
>
> Otherwise you'll have to ensure not to return the array as a result
> value, e.g. if you do
>
> (defparameter *a1* (make-array '(1000 1000)))
Thank you man. This has been also a problem in LispWorks (and bothered me). Your
method suppresses printing out the array during the course of making it.
AllegroCommonLisp prints only some part of the array (even when making it via
setq):
(setq array (make-array '(60 60))
S. Gonzi
Siegfried Gonzi <···············@kfunigraz.ac.at> writes:
> "Pierre R. Mai" wrote:
>
> > When experimenting with large arrays, you might want to set
> > *print-array* to nil, so that arrays aren't printed in re-readable
> > form (i.e. including all element values), but in an opaque form.
> >
> > Otherwise you'll have to ensure not to return the array as a result
> > value, e.g. if you do
> >
> > (defparameter *a1* (make-array '(1000 1000)))
>
> Thank you man. This has been also a problem in LispWorks (and
> bothered me). Your method suppresses printing out the array during
> the course of making it.
>
> AllegroCommonLisp prints only some part of the array (even when
> making it via setq):
>
> (setq array (make-array '(60 60))
You can control this behaviour in any conforming ANSI CL
implementation by diddling with *print-length* and *print-level*. See
the HyperSpec for details.
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
From: Steven M. Haflich
Subject: Re: UDATE Lisp and arrays, memory utilization
Date:
Message-ID: <3C5A0F19.5F775C7B@pacbell.net>
Siegfried Gonzi wrote:
> AllegroCommonLisp prints only some part of the array (even when making it via
> setq):
>
> (setq array (make-array '(60 60))
setq has nothing whatever to do with anything, and the phrase "making it
via setq" indicates you haven't yet figured out what's going on.
(progn (identity (let () (locally (make-array '(60 60))))))
would be just the same.
Clearly, normal *print-length* truncation is happening. You should
read the documentation about the top-level listener loop to find
out the details, but basically, the ACL top-level loop is something
like
(loop (prompt)
(write (eval (read))
:length tpl:*print-length*
:level tpl:*print-level*)
(terpri))
ignoring prompting and multiple values and other details.
The reason for these print control variables specific for the top
level is that in normal interactive use it is very easy to encounter
a huge or even circular value, and having one of these printed into
your session can be disruptive. If you decide you _really_ want to
see all of some large object, you can always (pprint *). If you
don't like the toplevel truncation at all, you can set these
variables to nil.