Hi. I'm running the newest SBCL, and it's been really great ( thanks
for the great compiler guys!)
But i seem to have a problem. I'm writing a multithreaded server, and
i want to make a data stream that other threads can tap into without
mutexes. So my strategy is to have a global *stream* which is a cons
cell, set (cdr *stream*) to a cons cell containing the new data, then
setf *stream* (cdr stream)
Makes sense to me, and i hacked it up in java before. But this
eventually causes a memory overflow in SBCL. Here's some sample code
that eventually exhausts the heap:
(defparameter *stream* nil)
(defun stream-adder( row )
(let ((new-cell (cons row nil)))
(setf (cdr *stream*) new-cell)
(setf *stream* (cdr *stream*))))
(defun test()
(setf *stream* (cons nil nil))
(loop for a from 0 to 100000000 do
(stream-adder a)))
This seems to be fine in clisp, for example.
I'm guessing either there is an int pointing to my stream, and the
conservative collector keeps everything, or maybe there is a register
left somewhere pointing to the head of the stream? Any hints how to
make this work, compiler gurus?
···········@gmail.com writes:
> Hi. I'm running the newest SBCL, and it's been really great ( thanks
> for the great compiler guys!)
>
> But i seem to have a problem. I'm writing a multithreaded server, and
> i want to make a data stream that other threads can tap into without
> mutexes. So my strategy is to have a global *stream* which is a cons
> cell, set (cdr *stream*) to a cons cell containing the new data, then
> setf *stream* (cdr stream)
>
> Makes sense to me, and i hacked it up in java before. But this
> eventually causes a memory overflow in SBCL. Here's some sample code
> that eventually exhausts the heap:
>
> (defparameter *stream* nil)
>
> (defun stream-adder( row )
> (let ((new-cell (cons row nil)))
> (setf (cdr *stream*) new-cell)
> (setf *stream* (cdr *stream*))))
>
> (defun test()
> (setf *stream* (cons nil nil))
> (loop for a from 0 to 100000000 do
> (stream-adder a)))
>
> This seems to be fine in clisp, for example.
>
> I'm guessing either there is an int pointing to my stream, and the
> conservative collector keeps everything, or maybe there is a register
> left somewhere pointing to the head of the stream? Any hints how to
> make this work, compiler gurus?
You may have some other code that is still keeping a reference to the
head of the stream.
Note that when you use slime, slime itself keeps references to all the
results displayed in its windows.
When you use a bare REPL, you have *, ** and *** that keep references
to the last three results.
;; So if you write at the REPL:
(stream-adder 1)
;; then:
(loop repeat 10000 do (stream-adder 2))
;; and finally:
(stream-adder 3)
;; and (implementation-dependant):
(ext:gc)
you will still have *** pointing to (1 2 2 2 .... 2 3),
** pointing to (2 3) and * pointing to (3).
--
__Pascal Bourguignon__ http://www.informatimago.com/
COMPONENT EQUIVALENCY NOTICE: The subatomic particles (electrons,
protons, etc.) comprising this product are exactly the same in every
measurable respect as those used in the products of other
manufacturers, and no claim to the contrary may legitimately be
expressed or implied.
Well, the code sample i supplied exhausts the heap in SBCL, no
external pointers to the data, and no making big lists at the prompt.
Also, as i said, my code works fine in clisp, no leaks.
I'm pretty sure that the problem is the conservative collector not
freeing some element that some bytes happens to point to. Once a
single cell points to one of the cons cells all the others after it
end up being retained.
Drat, the code was soooo elegant!
···········@gmail.com writes:
> Well, the code sample i supplied exhausts the heap in SBCL, no
> external pointers to the data, and no making big lists at the prompt.
> Also, as i said, my code works fine in clisp, no leaks.
>
> I'm pretty sure that the problem is the conservative collector not
> freeing some element that some bytes happens to point to. Once a
> single cell points to one of the cons cells all the others after it
> end up being retained.
>
> Drat, the code was soooo elegant!
Probably you should report it to sbcl-devel. It might be connected to a
problem I reported there yesterday.
Nicolas