From: james anderson
Subject: Re: reducing number consing
Date: 
Message-ID: <3FDC5236.52B028C4@setf.de>
Cliff Crawford wrote:
> 
> I'm trying to write a simple neural network simulation in Lisp, but
> I'm having trouble with number consing in the inner loop in the
> training function.  According to SBCL's profiler, the function
> feed-forward (see below) conses 32 bytes every time it's called, but
> I'd like it to not cons at all if possible.  Does anyone see what's
> wrong with it?  TIA...

the mystery for this reader is why it should cons only 32 bytes each time it
is called, where the arrays are of unspecified bounds. this would imply that
the external calls on input are already inlined and do not cons, and that the
array references also do not cons, and suggests that it is consing
temporaries. 32 bytes at 16 per double would imply two values. perhaps input
and sum. should that be true, perhaps despite the compiler's evident
perspicacity, one needs to inform it that these two variables have dynamic
extent. in whatever manner it needs to be told such things. in some
implementations it would look like:

(defun feed-forward-o (layer1 layer2 weights biases activation-function)
  (declare (optimize (speed 3) (safety 0))
           (net-vector layer1 layer2 biases)
           (net-matrix weights)
           (net-activation-function activation-function))
  (let ((input 0.0d0)
        (sum 0.0d0))
    (declare (type double-float input sum)
             (dynamic-extent input sum))
    (loop :for i :below (length layer2) :do
          (setf sum 0.0d0)
          (dotimes (j (length layer1))
            (setf sum (+ sum (* (aref weights i j) (aref layer1 j)))))
          (setf input (+ (aref biases i) sum))
          (setf (aref layer2 i) (case activation-function
                                  (:linear input)
                                  (:logistic (logistic input))
                                  (:exp (exp input))
                                  (:tanh (tanh input)))))))

in mcl these declarations eliminate the consing for the temporaries. sbcl must
have somthing similar.

nb. i'm loop impared and can't suggest the formulation for such expressions

...