From: mac
Subject: How to efficiently deep copy a fixnum / unsigned byte array?
Date: 
Message-ID: <1160081254.188593.170070@m7g2000cwm.googlegroups.com>
Hi,

I'm using a few very large array of element type fixnum for
calculation.

Very often I need to copy from one array to another and then do some
destructive operations on the later.

I found that the idiom

(dotimes (n (array-total-size copy))
  (setf (aref copy n) (aref array n)))

to be rather slow compared to memcpy operation in c.

Another idiom

(let *(grid1 (make-array n ...))
      (grid2 (make-array n ... :initial-contents grid1))
   ...)

does not work because as I said I have already preallocated a list of
array, and since they are huge I can't afford to create one from the
heap everytime I want a copy.

Also I wonder if the implementation uses memcpy under the hood in this
case.

On the same note, is there a fast way to reinitialize a
fixnum/bit/unsigned byte array?

After some destructive operations, sometime I need to reset all the
elements to 0.

Again the dotimes idiom is quite slow compare to memset in c.

If this cannot be done in cl using array, is there a way to create an
alternative data structure in cl to facilitate fast copy /
initialization of a block of memory with random access to individual
element?

Thanks in advance.

From: Carl Shapiro
Subject: Re: How to efficiently deep copy a fixnum / unsigned byte array?
Date: 
Message-ID: <ouyiriyjxi7.fsf@panix3.panix.com>
"mac" <········@gmail.com> writes:

> I found that the idiom
>
> (dotimes (n (array-total-size copy))
>   (setf (aref copy n) (aref array n)))
>
> to be rather slow compared to memcpy operation in c.

You should not write such loops by hand.  Use REPLACE.  It subsumes
the functionality of memmove and memcpy.

(replace dst-array src-array)

http://www.lisp.org/HyperSpec/Body/fun_replace.html
From: Pascal Costanza
Subject: Re: How to efficiently deep copy a fixnum / unsigned byte array?
Date: 
Message-ID: <4olb07Ff1banU1@individual.net>
mac wrote:
> Hi,
> 
> I'm using a few very large array of element type fixnum for
> calculation.
> 
> Very often I need to copy from one array to another and then do some
> destructive operations on the later.
> 
> I found that the idiom
> 
> (dotimes (n (array-total-size copy))
>   (setf (aref copy n) (aref array n)))
> 
> to be rather slow compared to memcpy operation in c.

replace

> On the same note, is there a fast way to reinitialize a
> fixnum/bit/unsigned byte array?
> 
> After some destructive operations, sometime I need to reset all the
> elements to 0.
> 
> Again the dotimes idiom is quite slow compare to memset in c.

fill


In general, when it comes to one-dimensional arrays, i.e. vectors, you 
should always also consult the section about sequences in the HyperSpec. 
(Sequence is the common supertype of both lists and vectors.)


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: ·········@gmail.com
Subject: Re: How to efficiently deep copy a fixnum / unsigned byte array?
Date: 
Message-ID: <1160087259.037250.132290@m73g2000cwd.googlegroups.com>
Pascal Costanza wrote:
[...]
> In general, when it comes to one-dimensional arrays, i.e. vectors, you
> should always also consult the section about sequences in the HyperSpec.
> (Sequence is the common supertype of both lists and vectors.)

While the subject is up....

Is there a preferred idiom for filling a vector with different
instances of the same type (or class) of object? Right now, I use
something based on dotimes, but looking at the CLHS I see one could
also do (for an already-created *array*)

(map-into *array* #'(lambda (e) (declare (ignore e)) (make-instance
'foo)) *array*)

which looks a little clunky to my eye (having to ignore the argument in
the lambda and needing that second reference to *array* there).
From: Pascal Costanza
Subject: Re: How to efficiently deep copy a fixnum / unsigned byte array?
Date: 
Message-ID: <4omis8Ffd4r9U1@individual.net>
·········@gmail.com wrote:
> Pascal Costanza wrote:
> [...]
>> In general, when it comes to one-dimensional arrays, i.e. vectors, you
>> should always also consult the section about sequences in the HyperSpec.
>> (Sequence is the common supertype of both lists and vectors.)
> 
> While the subject is up....
> 
> Is there a preferred idiom for filling a vector with different
> instances of the same type (or class) of object? Right now, I use
> something based on dotimes, but looking at the CLHS I see one could
> also do (for an already-created *array*)
> 
> (map-into *array* #'(lambda (e) (declare (ignore e)) (make-instance
> 'foo)) *array*)
> 
> which looks a little clunky to my eye (having to ignore the argument in
> the lambda and needing that second reference to *array* there).

(loop for i below (length vector)
       do (setf (svref vector i)
                (make-instance 'foo)))


...or build your own abstraction:

(defun set-each-in-vector (vector thunk)
   (loop for i below (length vector)
         do (setf (svref vector i) (funcall thunk))))

(defmacro setf/each (vector &body body)
   `(set-each-in-vector ,vector (lambda () ,@body)))

Then you can say this:

(setf/each vector (make-instance 'foo))


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/