Hello guys,
Which is the best compiler of CL code (which produces the fastest
standalone application)? Is there such a thing as *best* CL compiler
or is it a matter of holy war? Is execution of compiled code any
faster then interpretation of the same code?
--
"Now the storm has passed over me
I'm left to drift on a dead calm sea
And watch her forever through the cracks in the beams
Nailed across the doorways of the bedrooms of my dreams"
From: Kent M Pitman
Subject: Re: CL compiler (no, not interpreter)
Date:
Message-ID: <ubqbush7j.fsf@nhplace.com>
Nikola Skoric <·········@net4u.hr> writes:
Most serious Common Lisps have a compiler, and if you run across one
that doesn't, it will probably advertise the fact strongly as somehow
a feature for special-purpose use. Talking about a "CL interpreter"
as if that describes the product form of CL you should expect to
find is catering to scare talk that should have died decades ago.
Most CL languages are constructed around a development model that
permits both an interpreter and a compiler, such that code is not all
of one or all of another. Calling a CL product a "CL compiler" or a
"CL interpreter" will confuse all conversations you have with people,
since most products contain both and it's only at application-creation
time that one usually compiles most or all of their code. The
language itself does not presuppose either model, but rather speaks in
terms of what the language is required to do, whether interpreted or
compiled. It is theoretically possible to have a CL that implements
EVAL using a compiler or COMPILE using an interpreter. The former happens
more often than the latter; the latter would normally only happen
for some niche market reason (e.g., severe lack of address space)
or perhaps bootstrapping reasons.
> Hello guys,
>
> Which is the best compiler of CL code (which produces the fastest
> standalone application)?
"fastest" assumes a one-dimensional nature of speed. The question is
approximately equivalent to having a company and telling your HR department
to hire only the "fastest" employees. (Fast typists? Fast thinkers? Fast
runners? People who eat quickly? ...)
CL is a language defined by a specification, not defined by an
implementation. Systems implement the language differently in order
to achieve different goals, sometimes in terms of platform or
environment or extensions, and sometimes in terms of speed. Even
where speed is the goal, there are wide-ranging techniques, some based
on user declarations and some based on code inferencing. And price
and support is a factor when addressing speed.
> Is there such a thing as *best* CL compiler
No.
> or is it a matter of holy war?
It's not a holy war. It's just a truth of the Universe that there can
be no single answer to this. If you say your preferences for
platform, price, extension capabilities, etc. and the approximate type
of applications you plan and why speed and not any other factor is
your criterion, people may help to narrow down what you're looking
for. Frankly, if you care about speed and absolutely nothing else,
I'm not 100% sure why Lisp is your first choice. Lisp can be quite
competitively fast for a great many things, but most people don't come
to Lisp if speed is their only concern. They often come for dynamic
flexibility, robustness of system-wide architecture, evolutionary
nature, etc. which often involve details where you have to trade
flexibility for speed. Anyone who tells you you can be super-flexible
without sacrificing speed should have his claim examined VERY carefully.
> Is execution of compiled code any faster then interpretation of the
> same code?
Generally, yes, though it is possible to pick particular interpreters that
run very fast and to construct pessimal compilers that run very slow such
that an absolute answer is tough. (There is also another axis of this,
which is that some Lisps are byte-coded, and the question of what is fast
and slow in those is often non-intuitive even if you know what's fast and
slow in most other Lisps.)
In general, worrying about speed is a terrible way to begin to use Lisp.
First learn how to program. Then learn how to optimize. Optimization is
a specialized task that should be done with respect to a particular program
and a particular platform once you know where the bottlenecks are. Early on,
designing a well-stuctured, flexible program should be your concern.
Just one person's opinion.
From: Klaus Schilling
Subject: Re: CL compiler (no, not interpreter)
Date:
Message-ID: <87myvcp8ai.fsf@web.de>
Kent M Pitman <······@nhplace.com> writes:
> compiled. It is theoretically possible to have a CL that implements
> EVAL using a compiler or COMPILE using an interpreter. The former happens
> more often than the latter; the latter would normally only happen
> for some niche market reason (e.g., severe lack of address space)
> or perhaps bootstrapping reasons.
where do those implementations fall in who translate lisp source
to C source, and invoke the C- or Java compiler externally ?
Klaus Schilling
P� Sat, 22 Sep 2007 17:34:11 +0200, skrev Nikola Skoric
<·········@net4u.hr>:
> Hello guys,
>
> Which is the best compiler of CL code (which produces the fastest
> standalone application)? Is there such a thing as *best* CL compiler
> or is it a matter of holy war? Is execution of compiled code any
> faster then interpretation of the same code?
>
No, there is no ONE compiler. For numerical compilation CMUCL was created.
SBCL will perform better on the arcitectures it support. Allego CL is
better on OO and has better performing libraries using this. If you don't
care much about ANSI support GCL (mostly CLTL2) gives fast code.
(message (Hello 'Nikola)
(you :wrote :on '(Sat, 22 Sep 2007 15:34:11 +0000 (UTC)))
(
NS> Which is the best compiler of CL code (which produces the fastest
NS> standalone application)? Is there such a thing as *best* CL compiler
NS> or is it a matter of holy war?
no, there's no single best compiler.
in my experience commercial implementation -- Allegro CL and Lispworks, for
example -- produce fast code with ease and have nice utilities to do
profiling/optimizations etc. but they cost quite a lot..
open-source implementations like CMUCL and SBCL also have pretty good
compilers, but they might need to have more type declarations etc. not as
polished.
another beasts are GCL and ECL -- they use C compiler to compile code, first
they generate C file and then compile them. so, armed with enough type
declarations, it's quite likely they can produce code just as you'd write in
C, and it'll have same performance as C code, certainly.
so choice of compiler depends on goals and abilities..
NS> Is execution of compiled code any faster then interpretation of the
NS> same code?
yes, often difference is HUGE.
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"Hanging In The Balance Of Deceit And Blasphemy")
Alex Mizrahi <········@users.sourceforge.net> wrote:
> in my experience commercial implementation -- Allegro CL and Lispworks, for
> example -- produce fast code with ease and have nice utilities to do
> profiling/optimizations etc. but they cost quite a lot..
>
> open-source implementations like CMUCL and SBCL also have pretty good
> compilers, but they might need to have more type declarations etc. not as
> polished.
That's a surprising claim. What do you base it on?
As far as I know, the type inference done by sbcl and cmucl is much
more sophisticated than what Allegro or Lispworks do. In fact, if the
few tests I just made are representative, the commercial Lisps can't
type inference their way out of a wet paperbag. It's possible that I
just magically managed to tests things that they don't support, and
they actually do some type inference somewhere. But certainly I see no
reason to believe that they would get by with fewer declarations than
cmucl and descendants.
Nor do I see any reason imply that Allegro and Lispworks are faster
than cmucl and sbcl. One reason is that it's hard to measure
performance, due to the difficulty of writing fair
cross-implementation benchmarks for CL. Real-life applications tend to
be tuned for the implementation that they were first written for, so
they're hardly suitable. An on the other hand, very small changes to
Lisp microbenchmarks can have big effects.
But if you for example run cl-bench with the current versions of sbcl,
Allegro and Lispworks you should find that sbcl is more than
competitive, and Lispworks is err.. less than competitive. (At least
that happened when I just ran the tests on x86. I won't post results,
since I used the crippled free versions of Allegro and Lispworks. This
could conceivably have an effect on the exact numbers, so it wouldn't
really be fair).
> another beasts are GCL and ECL -- they use C compiler to compile code, first
> they generate C file and then compile them. so, armed with enough type
> declarations, it's quite likely they can produce code just as you'd write in
> C, and it'll have same performance as C code, certainly.
You haven't tried that, have you? :-)
--
Juho Snellman
From: Kent M Pitman
Subject: Re: CL compiler (no, not interpreter)
Date:
Message-ID: <ufy16jd11.fsf@nhplace.com>
Juho Snellman <······@iki.fi> writes:
> > open-source implementations like CMUCL and SBCL also have pretty
> > good compilers, but they might need to have more type declarations
> > etc. not as polished.
>
> That's a surprising claim. What do you base it on?
>
> As far as I know, the type inference done by sbcl and cmucl is much
> more sophisticated than what Allegro or Lispworks do.
Well, it's possible that in some cases "need to have" may mean "are
able to make better use of them if present", where in other
compilers there is no extra advantage to such. So some of the issue
may be "what do you need to do to get a particular absolute level of
speed" vs "what do you need to do before you know the compiler has
given you all it has to give".
But it's also probably true that different compilers focus on getting
different levels of speed out of different things. And there are
differences in representation that lead to other incomparable effects.
So side-by-side comparison can be very hard and different people may
legitimately reach different conclusions based on their needs, their
standards, and their particular choice of tests and/or applications.
As such, hearing "I've found x to be better than y" is not a
refutation of someone saying the opposite. These are statements about
incidents, not universal quantifications. Consider statements like
"I've found Metropolis to be a very safe city." and think about
whether they refute a statement by someone else like "Metropolis
doesn't seem safe to me." Both could be correct personal perceptions
without conflict.
(message (Hello 'Juho)
(you :wrote :on '(23 Sep 2007 00:23:05 GMT))
(
??>> open-source implementations like CMUCL and SBCL also have pretty good
??>> compilers, but they might need to have more type declarations etc. not
??>> as polished.
JS> That's a surprising claim. What do you base it on?
just a personal experience..
i was testing some SVD (numeric matrix computations, singular vector
decomposition) code -- open source cl-mathstats -- on various
implementations. this code has some type declarations, but clearly not
enough.. and by the way it's just one single function, so optimization
across function boundaries is not in effect.
open-source implementatations i've tested it on -- ABCL, CLISP, SBCL -- gave
aprox same result -- 10 seconds.
while Allegro CL managed to do decomposition in some 3-4 seconds.
i've tried to add some more type declarations, and with them SBCL managed to
get more efficient -- to 5 seconds, still slower than ACL.
i didn't continue optimizations, maybe if i do everything what SBCL wants it
would even beat ACL.. but i'm not really interested in running my code on
SBCL or ACL..
JS> Real-life applications tend to
JS> be tuned for the implementation that they were first written for, so
JS> they're hardly suitable.
but if application was originally written with some MCL or ABCL, it would be
fair to use it to compare ACL and SBCL, won't it? :)
JS> But if you for example run cl-bench with the current versions of sbcl,
JS> Allegro and Lispworks you should find that sbcl is more than
JS> competitive, and Lispworks is err.. less than competitive.
well, quite possible..
??>> another beasts are GCL and ECL -- they use C compiler to compile code,
??>> first they generate C file and then compile them. so, armed with
??>> enough type declarations, it's quite likely they can produce code just
??>> as you'd write in C, and it'll have same performance as C code,
??>> certainly.
JS> You haven't tried that, have you? :-)
no, i tried on very simplistic floating point arithmetic -- and it was
really faster than anything else.
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"Hanging In The Balance Of Deceit And Blasphemy")
Alex Mizrahi <········@users.sourceforge.net> wrote:
> (message (Hello 'Juho)
> (you :wrote :on '(23 Sep 2007 00:23:05 GMT))
> (
>
> ??>> open-source implementations like CMUCL and SBCL also have pretty good
> ??>> compilers, but they might need to have more type declarations etc. not
> ??>> as polished.
>
> JS> That's a surprising claim. What do you base it on?
>
> just a personal experience..
The kinds of definite statements like yours should probably be backed
by something more than anecdotal evidence from a single benchmark
program :-)
> i was testing some SVD (numeric matrix computations, singular vector
> decomposition) code -- open source cl-mathstats -- on various
> implementations. this code has some type declarations, but clearly not
> enough.. and by the way it's just one single function, so optimization
> across function boundaries is not in effect.
>
> open-source implementatations i've tested it on -- ABCL, CLISP, SBCL -- gave
> aprox same result -- 10 seconds.
> while Allegro CL managed to do decomposition in some 3-4 seconds.
I assume the correct functions are svdcmp-sf and svdcmp-df? There's
not much that any compiler could do in the way of meaningful type
inference for those functions. Since so many of the input parameters
are undeclared, there's not really anything for the inferrer to work
with. Exactly what types do you think Allegro is inferring there?
Besides that completely bogus "arithmetic operations on two fixnums
always result in a fixnum" thing they do.
That said, running svdcmp-sf with no modifications on a 128x128 array
takes 2.05s +- 0.05s on sbcl 1.0.9, 4.85s +- 0.05s on Allegro 8.0, and
6.9s on CLisp for me.
Adding some obvious type declarations (obvious == suggested by sbcl)
speeds up sbcl to 0.7s and Allegro to 3.92s.
Is this roughly the kind of test you were running? The results
certainly seem very different from the ones you got.
> i didn't continue optimizations, maybe if i do everything what SBCL wants it
> would even beat ACL.. but i'm not really interested in running my code on
> SBCL or ACL..
Fair enough.
> JS> Real-life applications tend to
> JS> be tuned for the implementation that they were first written for, so
> JS> they're hardly suitable.
>
> but if application was originally written with some MCL or ABCL, it would be
> fair to use it to compare ACL and SBCL, won't it? :)
It's a fair test of how different implementations run code originally
written for MCL or ABCL :-) But for example I doubt that anybody who
cares at all about performance would actually use that svd code as-is,
so it's not obvious that it'd be a meaningful benchmark.
--
Juho Snellman
(message (Hello 'Juho)
(you :wrote :on '(23 Sep 2007 13:39:29 GMT))
(
JS> That said, running svdcmp-sf with no modifications on a 128x128 array
JS> takes 2.05s +- 0.05s on sbcl 1.0.9, 4.85s +- 0.05s on Allegro 8.0, and
JS> 6.9s on CLisp for me.
to my defence, it seems SBCL progresses quite fast :)
SBCL 1.0.9 does svdcmp-df of 128x128 array in 1.89 seconds.
while SBCL 0.9.16 in 3.871 seconds.
so quite possible i got that bad results where it was on par with CLISP for
some old version (although i'm pretty sure it was some 0.9, so not very
old..), and got wrong conclusions..
JS> written for MCL or ABCL :-) But for example I doubt that anybody who
JS> cares at all about performance would actually use that svd code as-is,
yeah, certainly -- i've finally switched to a FORTRAN code doing sparse SVD
:)
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"Hanging In The Balance Of Deceit And Blasphemy")
P� Sun, 23 Sep 2007 12:48:48 +0200, skrev Alex Mizrahi
<········@users.sourceforge.net>:
>
> i didn't continue optimizations, maybe if i do everything what SBCL
> wants it
> would even beat ACL.. but i'm not really interested in running my code on
> SBCL or ACL..
>
Last time I saw that kind of difference it was due to loop.
Did you unroll the loop and and remove temporaries out?
It seems ACL does a better job on loop.
(message (Hello 'John)
(you :wrote :on '(Sun, 23 Sep 2007 14:29:38 +0200))
(
??>> i didn't continue optimizations, maybe if i do everything what SBCL
??>> wants it
??>> would even beat ACL.. but i'm not really interested in running my code
??>> on SBCL or ACL..
??>>
JT> Last time I saw that kind of difference it was due to loop.
JT> Did you unroll the loop and and remove temporaries out?
JT> It seems ACL does a better job on loop.
unroll loop of SVD?
just a fragment (i think it's a port of some quite "famous" C or Fortran
code):
(do ((i 1 (1+ i)))((> i n))
(setf l (+ i 1))
(setf (aref1 rv1 i) (* scale g))
(setf g 0.0d0 s 0.0d0 scale 0.0d0)
(when (<= i m)
(do ((k i (1+ k)))((> k m)) (incf scale (abs (aref11 a k i))))
(when (not (= scale 0.0d0))
(do ((k i (1+ k)))((> k m))
(setf (aref11 a k i) (/ (aref11 a k i) scale))
(incf s (* (aref11 a k i) (aref11 a k i))))
(setf f (aref11 a i i))
(setf g (- (sign-df (sqrt s) f)))
(setf h (- (* f g) s))
(setf (aref11 a i i) (- f g))
(when (not (= i n))
(do ((j l (1+ j)))((> j n))
(setf s 0.0d0)
(do ((k i (1+ k)))((> k m)) (incf s (* (aref11 a k i)
(aref11 a k j))))
(setf f (/ s h))
(do ((k i (1+ k)))((> k m)) (incf (aref11 a k j) (* f
(aref11 a k i))))))
(do ((k i (1+ k)))((> k m)) (setf (aref11 a k i) (* (aref11 a
k i) scale)))))
(setf (aref1 w i) (* scale g))
(setf g 0.0d0 s 0.0d0 scale 0.0d0)
(when (and (<= i m) (not (= i n)))
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"Hanging In The Balance Of Deceit And Blasphemy")
On 9971 day of my life Alex Mizrahi wrote:
> open-source implementatations i've tested it on -- ABCL, CLISP, SBCL -- gave
> aprox same result -- 10 seconds.
> while Allegro CL managed to do decomposition in some 3-4 seconds.
>
> i've tried to add some more type declarations, and with them SBCL managed to
> get more efficient -- to 5 seconds, still slower than ACL.
I suspect that unoptimized code runs faster on ACL because of better
garbage collector. Not compiler.
If you insert type declaration, consing will be greatly reduced, and garbage
collector will 'suddenly' become irrelevant.
--
Ivan Boldyrev
XML -- new language of ML family.
(message (Hello 'Ivan)
(you :wrote :to '("Alex Mizrahi") :on '(Tue, 25 Sep 2007 21:38:09 +0700))
(
??>> open-source implementatations i've tested it on -- ABCL, CLISP, SBCL
??>> -- gave aprox same result -- 10 seconds. while Allegro CL managed to
??>> do decomposition in some 3-4 seconds.
??>>
??>> i've tried to add some more type declarations, and with them SBCL
??>> managed to get more efficient -- to 5 seconds, still slower than ACL.
IB> I suspect that unoptimized code runs faster on ACL because of better
IB> garbage collector. Not compiler.
i'm not sure which version of SBCL i was originally testing with, but now it
seems ACL suffers much more from GC:
;;SBCL 0.9.16
* (test-svd 128 128)
Evaluation took:
3.865 seconds of real time
3.604225 seconds of user run time
0.256016 seconds of system run time
[Run times include 0.14 seconds GC run time.]
0 page faults and
863,549,808 bytes consed.
;;SBCL 1.0.9
* (test-svd 128 128)
Evaluation took:
1.95 seconds of real time
1.740108 seconds of user run time
0.208013 seconds of system run time
[Run times include 0.2 seconds GC run time.]
0 calls to %EVAL
0 page faults and
863,557,784 bytes consed.
;;Allegro CL 8.0
CG-USER(3): (test-svd 128 128)
; cpu time (non-gc) 2,891 msec user, 16 msec system
; cpu time (gc) 2,375 msec user, 0 msec system
; cpu time (total) 5,266 msec user, 16 msec system
; real time 5,313 msec
; space allocation:
; 59 cons cells, 1,549,695,064 other bytes, 0 static bytes
so if we compare different versions of SBCL, same amount of garbage consed,
but user time differs almost 2x -- so it seems like codegeneration improved.
)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"Hanging In The Balance Of Deceit And Blasphemy")
On 26 sep, 09:40, "Alex Mizrahi" <········@users.sourceforge.net>
wrote:
> (message (Hello 'Ivan)
> (you :wrote :to '("Alex Mizrahi") :on '(Tue, 25 Sep 2007 21:38:09 +0700))
> (
>
> ??>> open-source implementatations i've tested it on -- ABCL, CLISP, SBCL
> ??>> -- gave aprox same result -- 10 seconds. while Allegro CL managed to
> ??>> do decomposition in some 3-4 seconds.
> ??>>
> ??>> i've tried to add some more type declarations, and with them SBCL
> ??>> managed to get more efficient -- to 5 seconds, still slower than ACL.
>
> IB> I suspect that unoptimized code runs faster on ACL because of better
> IB> garbage collector. Not compiler.
>
> i'm not sure which version of SBCL i was originally testing with, but now it
> seems ACL suffers much more from GC:
>
> ;;SBCL 0.9.16
> * (test-svd 128 128)
>
> Evaluation took:
> 3.865 seconds of real time
> 3.604225 seconds of user run time
> 0.256016 seconds of system run time
> [Run times include 0.14 seconds GC run time.]
> 0 page faults and
> 863,549,808 bytes consed.
>
> ;;SBCL 1.0.9
> * (test-svd 128 128)
>
> Evaluation took:
> 1.95 seconds of real time
> 1.740108 seconds of user run time
> 0.208013 seconds of system run time
> [Run times include 0.2 seconds GC run time.]
> 0 calls to %EVAL
> 0 page faults and
> 863,557,784 bytes consed.
>
> ;;Allegro CL 8.0
> CG-USER(3): (test-svd 128 128)
> ; cpu time (non-gc) 2,891 msec user, 16 msec system
> ; cpu time (gc) 2,375 msec user, 0 msec system
> ; cpu time (total) 5,266 msec user, 16 msec system
> ; real time 5,313 msec
> ; space allocation:
> ; 59 cons cells, 1,549,695,064 other bytes, 0 static bytes
>
> so if we compare different versions of SBCL, same amount of garbage consed,
> but user time differs almost 2x -- so it seems like codegeneration improved.
>
> )
> (With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
> "Hanging In The Balance Of Deceit And Blasphemy")
Hello,
I use, every days, Allegro and SBCL, SBCL essentially for testing
portability of code.
Acl offers in almost all cases better performance compared to SBCL.
I agree with you, SBCL are equal results in mathematic operations, but
for I/O, database access, objects management,... ACL is clearly
better.
Just for information, Acl 8.1 progressed a lot compared to Acl 8.0.
Performance of Acl is one of the reasons that I have acquired Acl.
Best Regards.
Nikola Skoric wrote:
> Hello guys,
>
> Which is the best compiler of CL code (which produces the fastest
> standalone application)? Is there such a thing as *best* CL compiler
> or is it a matter of holy war? Is execution of compiled code any
> faster then interpretation of the same code?
Fast execution doesn't primarily come from compilation. Fast execution
comes from not executing what's not necessary. Most of the time, this
amounts to compiling code, because then certain interpretation steps
don't have to be done over and over again.
However, there are cases when it's better not to compile but to
interpret, for example, when you are running scripts. As a concrete
example, I create some of my websites with LML. Here, using an
interpreted CL implementation allows you to build a website much faster
than in a compiled CL implementation. The reason is that in this case,
compiling the code and then executing it takes much longer than just
interpreting it, but with a compiler-based execution model, you have to
compile the scripts each time you run them. You cannot simply compile
them just once. So here, using an interpreter gives you much better
execution times. (That's why I use clisp for processing LML.)
That may be an extreme case, but still the message here is this: Don't
look at one aspect only if you're interested in fast execution. You need
a good understanding of the whole architecture of your systems. In other
words, the question which CL implementation is the best for getting fast
applications is ill-stated.
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
You can build ordinary binaries, .o object files, and .so and .a
libraries with ECL (it uses GCC in the background).
I think you can make .EXEs .DLLs on W#? or OSX Univesal Binaries too.
http://ecls.sourceforge.net
Carlo