I'm writing some numeric code, and needed to decide if it would be worth
to sacrifice some precision and use double-floats instead of rationals.
So I ran some simple micro-benchmarks (in this case this makes some
sense, because most of what the application will do is number-crunching).
I thought I'd share the results (I tested ECL, CMUCL, SBCL, Clisp and
Allegro). Here are the numbers, and the code is at the bottom of the
message.
(SBCL seems to be 1700 faster when you switch to double-float!)
J.
|| ECL
--- RATIONAL ---
run time : 10.683 secs
gc count : 7 times
consed : 788612160 bytes
--- DOUBLE-FLOAT ---
run time : 0.015 secs
gc count : 1 times
consed : 14745984 bytes
Rational time 712x slower and conses 53x more
|| CMUCL
--- RATIONAL ---
1.870716 seconds of user run time
0.120982 seconds of system run time
113,285,560 bytes consed.
--- DOUBLE-FLOAT ---
0.059991 seconds of user run time
0.003999 seconds of system run time
2,164,016 bytes consed.
Rational time 31x slower and conses 52x more
|| SBCL
--- RATIONAL ---
1.705740 user
274,248,352 bytes consed
--- DOUBLE-FLOAT ---
0.001000 user
719,504 bytes consed
Rational time 1705x slower (!) and conses 381x more
|| Clisp
--- RATIONAL ---
Run time: 2.663594
Space: 124616352 Bytes
--- DOUBLE-FLOAT ---
Run time: 0.047993
Space: 1276984 Bytes
Rational time 55x slower, conses 97x more
|| Allegro
--- RATIONAL ---
3,900 msec user
792,953 cons cells, 93,489,248 other bytes
--- DOUBLE-FLOAT ---
280 msec user
702,867 cons cells, 6,030,992 other bytes
Rational 13x slower, conses almost the same 1.1x,
but uses 15x more memory
The code:
(defun mean (x)
(declare (optimize (speed 3))
(type (simple-array rational *) x)
(type rational))
(/ (the rational (reduce #'+ x)) (length x)))
(defun float-mean (x)
(declare (optimize (speed 3))
(type (simple-array double-float *) x)
(type double-float))
(the double-float (/ (the double-float (reduce #'+ x)) (length x))))
(defun random-array (n)
(declare (optimize (speed 3))
(type integer n))
(let ((a (make-array n :element-type 'rational)))
(loop for i from 0 to (1- n) do
;; Multiplying by 13 and 7 in order to force rational
;; arithmetic:
(setf (aref a i) (/ (* 13 (coerce (random most-positive-fixnum)
'rational))
(* 7 (coerce (random most-positive-fixnum)
'rational)))))
a))
(defun float-random-array (n)
(declare (optimize (speed 3))
(type integer n))
(let ((a (make-array n :element-type 'double-float)))
(loop for i from 0 to (1- n) do
(setf (aref a i) (/ (random most-positive-double-float) n)))
a))
(defun benchmark (m n)
(declare (optimize (speed 3)))
(let ((x 0))
(declare (type rational x))
(time
(dotimes (i m)
(let ((A (random-array n)))
(incf x (mean A)))))
NIL))
(defun float-benchmark (m n)
(declare (optimize (speed 3)))
(let ((x 0d0))
(declare (type double-float x))
(time
(dotimes (i m)
(let ((A (float-random-array n)))
(incf x (float-mean A)))))
NIL))
(pprint '----------------------)
(benchmark 10 1000)
(pprint '----------------------)
(float-benchmark 10 1000)
On 12 maio, 19:56, Jeronimo Pellegrini <····@aleph0.info> wrote:
> I'm writing some numeric code, and needed to decide if it would be worth
> to sacrifice some precision and use double-floats instead of rationals.
> So I ran some simple micro-benchmarks (in this case this makes some
> sense, because most of what the application will do is number-crunching).
>
> I thought I'd share the results (I tested ECL, CMUCL, SBCL, Clisp and
> Allegro). Here are the numbers, and the code is at the bottom of the
> message.
It looks like from your code that what is taking most of the time is
finding the random array. It is not fair since the random arrays are
calculated differently, so I believe the result is inaccurate. I would
say to you cons up a list of m random vectors and call the time
function as you iterate throw the list. Something like this:
(defun float-benchmark (m n)
(declare (optimize (speed 3)))
(let ((x 0d0)
(arrays (loop repeat m collect (float-random-array n)))
(declare (type double-float x))
(time
(dolist (A arrays)
(incf x (float-mean A)))))
NIL))
This way, the calls to random wouldn't count in the benchmark. By the
way
(coerce (random most-positive-fixnum) 'rational)
does nothing - coercing an integer to a rational gives you the integer
back.
One last thing: a rational composed by two fixnums should be faster.
This is most likely not the case when you call
(/ (* 13 (coerce (random most-positive-fixnum)
'rational))
(* 7 (coerce (random most-positive-fixnum)
'rational)))
So, I would say that what is slowing down your benchmark is bignum
arithmetic.
There are couple of declaration mistakes
> (defun float-mean (x)
> (declare (optimize (speed 3))
> (type (simple-array double-float *) x)
> (type double-float))
> (the double-float (/ (the double-float (reduce #'+ x)) (length x))))
For example:
(type (simple-array double-float *) x)
Here you are indicating an array of multiple-dimensions - *
You really want this:
(type (simple-array double-float (*)) x)
e.g. an array of one dimension with unlimited size
Also (type double-float) just byitself is doing nothing.
Then instaead of adding (optimize (speed 3)) to each function do it once
for all:
(declaim (optimize (speed 3) (safety 0) (space 0) (debug 0)
(compilation-speed 0)))
Putting safety to 0 would speed up your stuff (but be careful)
Making debug to 0 might also speed up on certain implementations.
But in most of the implementations (SBCL, ACL, LispWorks, and probably
others) - you won't have inlined RATIONAL + - at the best with Lispworks
instead of + you can do system::+$RATIONAL - but that's LW specific, and
internally that function is doing a lot of hairy stuff - it checks if
the numbers are integers to proceed to +INTEGER, if not - do the GCD,
etc, etc.
Which simply means the data you would be using would directly affect the
benchmark (be careful there).
>
> (defun random-array (n)
> (declare (optimize (speed 3))
> (type integer n))
> (let ((a (make-array n :element-type 'rational)))
> (loop for i from 0 to (1- n) do
> ;; Multiplying by 13 and 7 in order to force rational
> ;; arithmetic:
> (setf (aref a i) (/ (* 13 (coerce (random most-positive-fixnum)
> 'rational))
> (* 7 (coerce (random most-positive-fixnum)
> 'rational)))))
> a))
Instead of
(type integer n)
Do
(type fixnum n)
Always do (disassemble 'your-function) and see the results iteratively,
learn X86 / PPC assembly if you really want to gain speed results.
From: Frank GOENNINGER
Subject: Re: Micro-benchmarks (rational x double-float speed)
Date:
Message-ID: <m2ab5g4ww1.fsf@goenninger.net>
Jeronimo Pellegrini <···@aleph0.info> writes:
> || Allegro
> --- RATIONAL ---
> 3,900 msec user
> 792,953 cons cells, 93,489,248 other bytes
>
> --- DOUBLE-FLOAT ---
> 280 msec user
> 702,867 cons cells, 6,030,992 other bytes
>
> Rational 13x slower, conses almost the same 1.1x,
> but uses 15x more memory
> The code:
as modified by myself: Removed the optimize speed declarations and
replaced them by a global
(eval-when (:compile-toplevel :load-toplevel :execute)
(proclaim '(optimize (speed 3) (safety 0) (space 0) (debug 0))))
I also put the whole code into a file and compiled that one - oh, I am
using AllegroCL on Mac OS X (MacBook Pro, latest model, 4 GB RAM)...
Results:
BENCHMARK::----------------------
; cpu time (non-gc) 1,900 msec user, 10 msec system
; cpu time (gc) 110 msec user, 0 msec system
; cpu time (total) 2,010 msec user, 10 msec system
; real time 2,018 msec
; space allocation:
; 90 cons cells, 89,318,288 other bytes, 0 static bytes
BENCHMARK::----------------------
; cpu time (non-gc) 0 msec user, 0 msec system
; cpu time (gc) 0 msec user, 0 msec system
; cpu time (total) 0 msec user, 0 msec system
; real time 6 msec
; space allocation:
; 10 cons cells, 1,760,400 other bytes, 0 static bytes
... shows that it needs some more definition of conditions in order to
really say how fast or slow a partical implementation is ...
Frank