On Mar 15, 10:31 pm, ············@gmail.com" <···········@gmail.com>
wrote:
> Hi,
>
> Using SBCL 1.0.3.
>
> I'm experimenting with "large" (100MB) data and have tried the
> following:
>
> (defparameter *large-data*
> (make-sequence '(vector (unsigned-byte 8)) 100000000
> :initial-element 200))
>
> SBCL confirms this is created by printing *LARGE-DATA*.
>
> OK, now I want to write this to disk like so:
>
> (with-open-file
> (out "/mega-store/temp/large2.data" :direction :output
> :if-exists :supersede :element-type '(unsigned-byte 8))
> (write-sequence *large-data* out))
>
> After some time (quite a while), I get a "Heap exhausted during
> garbage collection" message like so:
>
> Heap exhausted during garbage collection: 0 bytes available, 24
> requested.
> Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB LUB !move Alloc
> Waste Trig WP GCs Mem-age
> 0: 0 0 0 0 0 0 0 0 0 0
> 0 2000000 0 0 0.0000
> 1: 0 0 0 0 0 0 0 0 0 0
> 0 2000000 0 0 0.0000
> 2: 0 0 0 0 0 0 0 0 0 0
> 0 2000000 0 0 0.0000
> 3: 248838 107096 0 0 108611 2 0 0 0
> 444538912 339936 2000000 0 0 1.0000
> 4: 104027 103482 0 0 577 3 97657 0 97671
> 402359488 19264 2000000 563 0 0.0000
> 5: 0 0 0 0 0 0 0 0 0 0
> 0 2000000 0 0 0.0000
> 6: 0 0 0 0 5775 0 0 0 0 23654400
> 0 2000000 5706 0 0.0000
> Total bytes allocated=1048146872
> fatal error encountered in SBCL pid 5016:
> Heap exhausted, game over.
> LDB monitor
> ldb>
>
> Are my expressions valid or am I doing something wrong?
>
> Joubert
Juho helped me figure this one out.
write-sequence returns the value of the sequence, which meant the
100MB sequence was being pretty-printed and this caused the heap
exhaustion.
(see the write-sequence documentation at
http://www.lispworks.com/documentation/HyperSpec/Body/f_wr_seq.htm#write-sequence)
Anyway, the solution is to add NIL as the return value of the
expression, like so:
(with-open-file
(out "/mega-store/temp/large2.data" :direction :output
:if-exists :supersede :element-type '(unsigned-byte 8))
(write-sequence *large-data* out)
NIL)
Now it works and the 100MB file gets created very fast.
Joubert
On Fri, 16 Mar 2007 04:02:06 +0100, ···········@gmail.com
<···········@gmail.com> wrote:
>
> Anyway, the solution is to add NIL as the return value of the
> expression, like so:
>
> (with-open-file
> (out "/mega-store/temp/large2.data" :direction :output
> :if-exists :supersede :element-type '(unsigned-byte 8))
> (write-sequence *large-data* out)
> NIL)
>
> Now it works and the 100MB file gets created very fast.
>
> Joubert
>
A even better solution is to return (values) which returns nothing.
(I recognise this problem.. It seems to crop up rather a lot in Lisp.)
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
On 15 Mar 2007 20:02:06 -0700, ············@gmail.com" <···········@gmail.com> said:
| ...
| 100MB sequence was being pretty-printed and this caused the heap
| exhaustion.
| ...
| Anyway, the solution is to add NIL as the return value of the
| expression, like so:
| (with-open-file
| (out "/mega-store/temp/large2.data" :direction :output
| :if-exists :supersede :element-type '(unsigned-byte 8))
| (write-sequence *large-data* out)
| NIL)
You cannot be forced to change the return value of your functions
because the REPL's P has a stupid problem like this one. If you must
seek a workaround, at least try one along the lines of setting
*PRINT-LENGTH* or *PRINT-LINES* (the "or" is inclusive, of course) to
some reasonable small value (not a bad idea anyway, since you don't
normally want reaps of output at the REPL).
---Vassil.
--
Definitely worth seeing: "Das Leben der Anderen" ("The Lives of Others").
(message (Hello ············@gmail.com)
(you :wrote :on '(15 Mar 2007 19:31:46 -0700))
(
jn> After some time (quite a while), I get a "Heap exhausted during
jn> garbage collection" message like so:
this means SBCL sucks :)
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"?? ???? ??????? ?????")