From: Victor Kryukov
Subject: Handling large amount of data with SBCL
Date: 
Message-ID: <1172957960.499577.89420@z35g2000cwz.googlegroups.com>
Hi group

I've noticed a couple of bugs (or should I say features?) in the way
SBCL handles large amount of data.

1. In (room t) statistics, some number should be BIGNUM, not FIXNUM

<... using a lot of memory...>
* (room t)

Dynamic space usage is:   1,260,898,064 bytes.
Read-only space usage is:      1,920 bytes.
Static space usage is:         1,936 bytes.
Control stack usage is:        1,340 bytes.
Binding stack usage is:          328 bytes.
Control and binding stack usage is for the current thread only.
Garbage collection is currently enabled.
^X^F
debugger invoked on a TYPE-ERROR in thread #<THREAD "initial
thread" {117307E9}>:
  The value 536873216 is not of type FIXNUM.

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

((LAMBDA (SB-VM::OBJ TYPE SB-VM::SIZE))
 #<unavailable argument>
 #<unavailable argument>
 #<unavailable argument>)
0]

2. Currently, SBCL just dies silently (and goes into LDB prompt) if
heap memory is exhausted. Is there any workaround for that - i.e. I
would like to check periodically if heap memory is almost exhausted. I
also would like to know what are the instrinsic limits of SBCL
depending on available RAM and swap size  and if there any way to
increase them - e.g. on a machine with 1GB RAM and 2GB swap heap is
exhausted after SBCL uses approx. 500MB of it (sbcl 0.9.14 on ubuntu
server).

Any suggestions would be much appreciated,
Victor.

From: Zach Beane
Subject: Re: Handling large amount of data with SBCL
Date: 
Message-ID: <m37ityj6si.fsf@unnamed.xach.com>
"Victor Kryukov" <··············@gmail.com> writes:

> Hi group
> 
> I've noticed a couple of bugs (or should I say features?) in the way
> SBCL handles large amount of data.

The SBCL development mailing list is a better forum for such reports.

https://lists.sourceforge.net/lists/listinfo/sbcl-devel has some info
about joining.

Zach
From: André Thieme
Subject: Re: Handling large amount of data with SBCL
Date: 
Message-ID: <escrfp$dgl$1@registered.motzarella.org>
Victor Kryukov schrieb:
> Hi group
> 
> I've noticed a couple of bugs (or should I say features?) in the way
> SBCL handles large amount of data.
> 
> 1. In (room t) statistics, some number should be BIGNUM, not FIXNUM
> 
> <... using a lot of memory...>
> * (room t)
> 
> Dynamic space usage is:   1,260,898,064 bytes.
> Read-only space usage is:      1,920 bytes.
> Static space usage is:         1,936 bytes.
> Control stack usage is:        1,340 bytes.
> Binding stack usage is:          328 bytes.
> Control and binding stack usage is for the current thread only.
> Garbage collection is currently enabled.
> ^X^F
> debugger invoked on a TYPE-ERROR in thread #<THREAD "initial
> thread" {117307E9}>:
>   The value 536873216 is not of type FIXNUM.
> 
> Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
> 
> restarts (invokable by number or by possibly-abbreviated name):
>   0: [ABORT] Exit debugger, returning to top level.
> 
> ((LAMBDA (SB-VM::OBJ TYPE SB-VM::SIZE))
>  #<unavailable argument>
>  #<unavailable argument>
>  #<unavailable argument>)
> 0]
> 
> 2. Currently, SBCL just dies silently (and goes into LDB prompt) if
> heap memory is exhausted. Is there any workaround for that - i.e. I
> would like to check periodically if heap memory is almost exhausted. I
> also would like to know what are the instrinsic limits of SBCL
> depending on available RAM and swap size  and if there any way to
> increase them - e.g. on a machine with 1GB RAM and 2GB swap heap is
> exhausted after SBCL uses approx. 500MB of it (sbcl 0.9.14 on ubuntu
> server).

Yes, 512 MB is the default limit on a 32-Bit system.
The manual explains how you can change the dynamic space size:
http://www.sbcl.org/manual/Runtime-Options.html#Runtime-Options

On a 64-Bit system the default limit is 8 GB I think.


Andr�
-- 
From: Victor Kryukov
Subject: Re: Handling large amount of data with SBCL
Date: 
Message-ID: <1172961137.069893.128680@8g2000cwh.googlegroups.com>
On Mar 3, 4:05 pm, André Thieme <address.good.until.
···········@justmail.de> wrote:

> Yes, 512 MB is the default limit on a 32-Bit system.
> The manual explains how you can change the dynamic space size:http://www.sbcl.org/manual/Runtime-Options.html#Runtime-Options
>
> On a 64-Bit system the default limit is 8 GB I think.
>
> André
> --

Thanks for your suggetion, André - but that doesn't work for SBCL
0.9.14: it dies at the same point (total bytes allocated=525141264)
even when called 'sbcl --dynamic-space-size 2000'.

Interestingly enough, 'sbcl --dynamic-space-size XXX' works for any
value of XXX (even XXX=30000), while 'lisp -dynamic-space-size=XXX'
works for XXX=1500 but dumps core for XXX=1600 - which lead me to
believe that CMUCL is more honest about memory it's trying to allocate
than SBCL.

And on my other computer (MacBook with 2GB RAM) sbcl 1.0.1.30
allocates much bigger heap size (around 1.5GB I guess) without any
switches.