Hi Lispers,
A while back, Apress had a fractal contest. The idea was to write a
program that would generate a bunch of m-set regions in JPEG format as
specified from an input file. The Apress URL for the contest is here:
http://www.apress.com/promo/fractal/result.html
I had tried to participate in the contest, but Murphy was against me.
The winning code (and other entries) has been published as well as the
input file used to test the entries (see above link). I've made some
minor modifications to my entry to see just how it would compare to
the winning entry which was written in C:
http://www.david-steuber.com/snippets/When_Speed_Is_King_Redux/
This is actually a followup to an earlier snippet which in turn links
to yet another page I have on my contest experience.
For the lazy, the winning C program is roughly 1.76 times faster than
my Lisp code. My Lisp code is pure Lisp. That is, the JPEG encoder
is also ANSI Common Lisp. No FFI is involved. The C program uses the
free libjpeg library. Details are in the README file for the C
program.
My Lisp code is far from ideal and manages to (probably) be slower
than the fastest C# and Java entries. However, it is faster than most
of the other entries, save the ones written in C or C++.
One thing that is noteable is that my Lisp code totally kills the code
submissions in the language category that Common Lisp was relegated
to. Even if the code were run in CLisp (which will run it just fine),
the Lisp code is faster than PHP and Python. There were no Perl
entries.
--
http://www.david-steuber.com/
The UnBlog: An island of conformity in a sea of quirks.
The lowest click through rate in Google's AdSense program.
----------------------------------------------------------------------
David Steuber wrote:
> For the lazy, the winning C program is roughly 1.76 times faster than
> my Lisp code. My Lisp code is pure Lisp. That is, the JPEG encoder
> is also ANSI Common Lisp. No FFI is involved. The C program uses the
> free libjpeg library. Details are in the README file for the C
> program.
I think that is very impressive. Were you expecting to beat C?
> My Lisp code is far from ideal and manages to (probably) be slower
> than the fastest C# and Java entries. However, it is faster than most
> of the other entries, save the ones written in C or C++.
Judging by the results posted on the APress site, your Lisp code is roughly
the same speed as the fastest from the C#/Java/VB category.
> One thing that is noteable is that my Lisp code totally kills the code
> submissions in the language category that Common Lisp was relegated
> to. Even if the code were run in CLisp (which will run it just fine),
> the Lisp code is faster than PHP and Python. There were no Perl
> entries.
Yes. I don't think Lisp belongs in that category. I'd have swapped Lisp with
VB.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
Jon Harrop <······@jdh30.plus.com> writes:
> David Steuber wrote:
> > For the lazy, the winning C program is roughly 1.76 times faster than
> > my Lisp code. My Lisp code is pure Lisp. That is, the JPEG encoder
> > is also ANSI Common Lisp. No FFI is involved. The C program uses the
> > free libjpeg library. Details are in the README file for the C
> > program.
>
> I think that is very impressive. Were you expecting to beat C?
Not with the specifications given. My fractal algorithm is geared
towards deeper zooms. I simply adapted my code to produce JPEG images
rather than TGA images and did some optimizations after profiling the
code. It was a relatively quick and dirty job that I hoped would
approach the performance of C. If the minimum iterations had been
mandated at 128 then I probably would have been competative with the C
entry as written. Actually I think I would have beaten it. If 255
was specified as the minimum number of iterations, I certainly would
have trumped the winning C entry. Although if that were the case, I
expect the winning C entry would use a different algorithm to avoid
being trapped in the set.
If I was feeling up to extending the color palette for the C program,
I would do that and raise the iteration maximum to 255 to prove it.
My Lisp code computes a palette at compile time so that I don't have
to go picking colors.
Almost all my time is spent in the JPEG encoder. If I knew more about
how a DCT worked and JPEG in general, I probably could have improved
the JPEG encoder. As it is, the fact that I had a Lisp version to
work with did allow me to avoid color space conversion and some
copying.
> > My Lisp code is far from ideal and manages to (probably) be slower
> > than the fastest C# and Java entries. However, it is faster than most
> > of the other entries, save the ones written in C or C++.
>
> Judging by the results posted on the APress site, your Lisp code is roughly
> the same speed as the fastest from the C#/Java/VB category.
Just about, yes. If I went to the added trouble to yank out CLOS (I
had a good reason for using it in more general code where dispatch
overhead would be quite negligible) I think my code would have a fair
chance at beating the fastest C# implementation.
> > One thing that is noteable is that my Lisp code totally kills the code
> > submissions in the language category that Common Lisp was relegated
> > to. Even if the code were run in CLisp (which will run it just fine),
> > the Lisp code is faster than PHP and Python. There were no Perl
> > entries.
>
> Yes. I don't think Lisp belongs in that category. I'd have swapped Lisp with
> VB.
That's a tricky call. VB has evolved over time. First it was byte
code. Then it produced real PE format executables. Now it does .NET
byte code. .NET presumably has a JIT that is just as good as Java's.
CLisp is byte code interpreted. For my code, it shows. However, I
still score a faster time than the winning PHP entry. If CLisp had a
good JIT, it would be a force to reckon with.
Apress was also supporting a CL implementation I had never heard of.
However, I did confirm ahead of time that SBCL would be accepted.
Unfortunately, my e-mail submission got lost in the ether and I didn't
know that happened until well after the deadline had passed. Someone
else that I CC'd my entry to did get it. So I thought things were
fine.
Murphy can be a mean SOB.
--
http://www.david-steuber.com/
The UnBlog: An island of conformity in a sea of quirks.
The lowest click through rate in Google's AdSense program.
----------------------------------------------------------------------
On Sun, 23 Oct 2005 00:12:03 -0400, David Steuber wrote:
>> Yes. I don't think Lisp belongs in that category. I'd have swapped Lisp
>> with VB.
>
> That's a tricky call. VB has evolved over time. First it was byte
> code. Then it produced real PE format executables. Now it does .NET
> byte code. .NET presumably has a JIT that is just as good as Java's.
Responding to your last sentence: Do NOT presume this. Sun and IBM have
mature and extremely advanced Virtual Machine designs. There is lots of
rivalry, public benchmarking and ongoing advances (e.g.
<http://www-128.ibm.com/developerworks/java/library/j-jtp10185/>)
Some Microsoft EULAs prohibits the disclosure of any .Net benchmarks
without prior authorisation by Microsoft. For example Windows Server 2003
contains this clause:
<http://www.gripe2ed.com/scoop/story/2004/10/29/84837/416>
"Benchmark Testing. The 32-bit version of the Software contains the
Microsoft .NET Framework. Disclosure of the results of any benchmark
test of the .NET Framework component of the Software to any third party
without Microsoft's prior written approval is prohibited."
The author of the article, Ed Foster, does note that while this
prohibition is probably not legally enforceable, "a competitor would
probably have a great deal of difficulty getting any of the major
independent labs to conduct the test without Microsoft's approval."
The prohibition is at least a threat to open public discourse.
Is this the likely action of a vendor with JIT compiler technology "just
as good as Java's"?
Regards,
Adam
Adam Warner <······@consulting.net.nz> writes:
> "Benchmark Testing. The 32-bit version of the Software contains the
> Microsoft .NET Framework. Disclosure of the results of any benchmark
> test of the .NET Framework component of the Software to any third party
> without Microsoft's prior written approval is prohibited."
Well if it was better than any one of the competition, they would brag
about it instead of forbid publication. Therefore they just confirm
they're the worst game in town, as usual and unsurprizingly. Why do
you even bother?
--
__Pascal Bourguignon__ http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
Adam Warner <······@consulting.net.nz> writes:
> "Benchmark Testing. The 32-bit version of the Software contains the
> Microsoft .NET Framework. Disclosure of the results of any benchmark
> test of the .NET Framework component of the Software to any third party
> without Microsoft's prior written approval is prohibited."
>
> The author of the article, Ed Foster, does note that while this
> prohibition is probably not legally enforceable, "a competitor would
> probably have a great deal of difficulty getting any of the major
> independent labs to conduct the test without Microsoft's approval."
> The prohibition is at least a threat to open public discourse.
>
> Is this the likely action of a vendor with JIT compiler technology "just
> as good as Java's"?
Just to amplify what Edi said, it is actually very common for
proprietary compiler vendors to include benchmarking prohibitions in
their EULAs. Borland did it for their C++ line since at least 2.0.
Microsoft has been doing it since at least that time frame.
Don't forget that Microsoft had full access to JIT technology when
they were licensing Java. Also, Microsoft is fully capable of reverse
engineering Symantic's JIT technology. And Microsoft's Java compiler
was so fast that people were filing bug reports that the compiler
wouldn't run because they were so used to the sluggish performance of
javac (which hasn't improved much). If you blinked, you missed it.
My presumption may not be wholly justified, but it is not unjustified
either. I had in fact hoped to totally thrash the C# and Java
submissions (the fastest ones) which is part of the reason why I was
comparing against the fastest C submission.
Heck, if I lacked integrity I would have found a way to claim victory
over the C program. Of course, to get away with that I would have to
have not published my complete methodology and source so that others
could reproduce the results (or find errors in my methodology).
All that said, Sun does have some impressive tech. I'm hoping that
any GC advances they make will have a chance to filter back to the
Lisp community. It would also be very, very nice if JIT tech could
find its way to CLisp. As I said in my previous post, that would make
CLisp a force to contend with. CLisp is already a very good
implementation (very fast bignums, good FFI, able to be started with
#! Unix conventions, etc).
--
http://www.david-steuber.com/
The UnBlog: An island of conformity in a sea of quirks.
The lowest click through rate in Google's AdSense program.
----------------------------------------------------------------------
David Steuber wrote:
> Just to amplify what Edi said, it is actually very common for
> proprietary compiler vendors to include benchmarking prohibitions in
> their EULAs. Borland did it for their C++ line since at least 2.0.
> Microsoft has been doing it since at least that time frame.
This fact makes it a bit ridiculous to buy commercial compilers for
performance. Why buy it when they won't even allow fair comparisons of
performance?
--
Blessed are the young for they shall inherit the national debt.
Herbert Hoover
From: Edi Weitz
Subject: Re: Yet Another Lisp Performance Thread
Date:
Message-ID: <ubr1g3uyp.fsf@agharta.de>
On Sun, 23 Oct 2005 20:19:34 +0200, Ulrich Hobelmann <···········@web.de> wrote:
> This fact makes it a bit ridiculous to buy commercial compilers for
> performance. Why buy it when they won't even allow fair comparisons
> of performance?
Who said that? It's about publishing your results not about
comparisons. There are tons of bogus "benchmarks" out there ("Lisp is
slow" and so on) - it is quite understandable that vendors try to
prevent Joe Random User from adding more.
Cheers,
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Edi Weitz wrote:
> On Sun, 23 Oct 2005 20:19:34 +0200, Ulrich Hobelmann <···········@web.de> wrote:
>
>> This fact makes it a bit ridiculous to buy commercial compilers for
>> performance. Why buy it when they won't even allow fair comparisons
>> of performance?
>
> Who said that? It's about publishing your results not about
> comparisons. There are tons of bogus "benchmarks" out there ("Lisp is
> slow" and so on) - it is quite understandable that vendors try to
> prevent Joe Random User from adding more.
But what's wrong with a user (of Borland whatever, or of Whatever Lisp)
publishing benchmarks?
If I can't find any benchmark results because some vendor has a
no-publish policy I assume his compiler is as slow as slow can be, or he
would encourage open comparisons instead of cowardly hiding from the
public.
--
Blessed are the young for they shall inherit the national debt.
Herbert Hoover
Ulrich Hobelmann <···········@web.de> writes:
> If I can't find any benchmark results because some vendor has a
> no-publish policy I assume his compiler is as slow as slow can be, or
> he would encourage open comparisons instead of cowardly hiding from
> the public.
That may not be a valid assumption. Another restriction in a lot of
commercial compilers is using them to compile a competitive compiler.
Borland had this restriction.
As far as benchmarks go, there are lies, damn lies, statistics, and
benchmarks. It would probably not be too difficult to modify the
input.txt file for the Apress contest so that my program beats the C
program. The nature of the fractal makes the contest a crap shoot.
My original post was actually intended to show that Lisp can be
performance competitive even with trivial applications. I didn't do
as good a job of that as I had hoped, but the C program coming out
ahead by a factor of 1.76 is less than a binary order of magnitude.
Furthermore, someone with decent Lisp chops can easily make my code a
bit faster. If that person understands JPEG and has the patience to
work the JPEG encoder, the Lisp program can pick up even more speed.
The C program, elegant as it is, would need a complete rewrite to
speed it up if that is at all possible for the given input.txt file.
--
http://www.david-steuber.com/
The UnBlog: An island of conformity in a sea of quirks.
The lowest click through rate in Google's AdSense program.
----------------------------------------------------------------------
David Steuber wrote:
> Don't forget that Microsoft had full access to JIT technology when
> they were licensing Java.
If they want .NET to be fast then they could do better than to copy Java:
http://www.ffconsultancy.com/free/ray_tracer/languages.html
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
David Steuber <·····@david-steuber.com> wrote:
> Just to amplify what Edi said, it is actually very common for
> proprietary compiler vendors to include benchmarking prohibitions in
> their EULAs. Borland did it for their C++ line since at least 2.0.
> Microsoft has been doing it since at least that time frame.
>
What you write nicely illustates the point: Borland C++ used to generate
_very_ slow code -- that I know first hand since I used it. The rumors
I heard were: Microsoft C while faster then Borland's was still slow.
--
Waldek Hebisch
·······@math.uni.wroc.pl
On Fri, 28 Oct 2005 16:03:38 +0200, Waldek Hebisch
<·······@math.uni.wroc.pl> wrote:
> David Steuber <·····@david-steuber.com> wrote:
>> Just to amplify what Edi said, it is actually very common for
>> proprietary compiler vendors to include benchmarking prohibitions in
>> their EULAs. Borland did it for their C++ line since at least 2.0.
>> Microsoft has been doing it since at least that time frame.
>>
>
> What you write nicely illustates the point: Borland C++ used to generate
> _very_ slow code -- that I know first hand since I used it. The rumors
> I heard were: Microsoft C while faster then Borland's was still slow.
>
>
That is irrelevant with todays computers.
And -no- it was never -very slow-.
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
John Thingstad wrote:
> On Fri, 28 Oct 2005 16:03:38 +0200, Waldek Hebisch
>> What you write nicely illustates the point: Borland C++ used to generate
>> _very_ slow code -- that I know first hand since I used it. The rumors
>> I heard were: Microsoft C while faster then Borland's was still slow.
>
> That is irrelevant with todays computers.
This kind of attitude us undoubtedly a major contributor to
the damnable slowness of much modern software.
Paul
From: Edi Weitz
Subject: Re: Yet Another Lisp Performance Thread
Date:
Message-ID: <uek6c5qh9.fsf@agharta.de>
On Sun, 23 Oct 2005 18:10:06 +1300, Adam Warner <······@consulting.net.nz> wrote:
> Some Microsoft EULAs prohibits the disclosure of any .Net benchmarks
> without prior authorisation by Microsoft.
FWIW, I have seen similar clauses in products distributed by Common
Lisp vendors.
> Is this the likely action of a vendor with JIT compiler technology
> "just as good as Java's"?
So we have to infer from this that the CLR JIT compiler is slower than
Java's? And from my remark above that Lisp is slower than its
competition? I don't think so. Just be more careful before you
confuse technical arguments with a political agenda.
Cheers,
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
On Sun, 23 Oct 2005 15:10:10 +0200, Edi Weitz wrote:
> On Sun, 23 Oct 2005 18:10:06 +1300, Adam Warner
> <······@consulting.net.nz> wrote:
>
>> Some Microsoft EULAs prohibits the disclosure of any .Net benchmarks
>> without prior authorisation by Microsoft.
>
> FWIW, I have seen similar clauses in products distributed by Common Lisp
> vendors.
>
>> Is this the likely action of a vendor with JIT compiler technology
>> "just as good as Java's"?
>
> So we have to infer from this that the CLR JIT compiler is slower than
> Java's?
I was careful to leave that inference to the reader. I'm attempting to
impart information without forcing an argument.
> And from my remark above that Lisp is slower than its competition?
That information would factor into any inference I make. In the Lisp world
I know that CMUCL and SBCL produce relatively fast code as native code
compilers. I would be extra suspicious of the performance of a
competitor's product if the vendor prohibited publication of benchmarks
against CMUCL and SBCL. I would perhaps say, "Is this the likely action of
a vendor with compiler technology 'just as good as' CMUCL and SBCL?"
> I don't think so. Just be more careful before you confuse technical
> arguments with a political agenda.
If I'm feeling lucky at Google and check out Microsoft .Net Benchmarks*
and find many graphs showing Sun and IBM products being annihilated, I
hope I don't confuse technical arguments with a political agenda.
Regards,
Adam
*<http://www.theserverside.net/articles/showarticle.tss?id=NET2Benchmarks>
From: Edi Weitz
Subject: Re: Yet Another Lisp Performance Thread
Date:
Message-ID: <u7jc36e6z.fsf@agharta.de>
On Mon, 24 Oct 2005 11:23:08 +1300, Adam Warner <······@consulting.net.nz> wrote:
> If I'm feeling lucky at Google and check out Microsoft .Net
> Benchmarks* and find many graphs showing Sun and IBM products being
> annihilated, I hope I don't confuse technical arguments with a
> political agenda.
MS wins, so the benchmarks /must/ be flawed, right?
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Edi Weitz <········@agharta.de> writes:
> On Mon, 24 Oct 2005 11:23:08 +1300, Adam Warner <······@consulting.net.nz> wrote:
>
>> If I'm feeling lucky at Google and check out Microsoft .Net
>> Benchmarks* and find many graphs showing Sun and IBM products being
>> annihilated, I hope I don't confuse technical arguments with a
>> political agenda.
>
> MS wins, so the benchmarks /must/ be flawed, right?
Definitively. Otherwise there would have been published other
benchmarks, some better, some worse, and you could conclude otherwise.
Imagine Einstein saying: "Ok, you can experiment as much as you want
with Relativity Theory, but you cannot publish results unless I
approve them.".
--
__Pascal Bourguignon__ http://www.informatimago.com/
Wanna go outside.
Oh, no! Help! I got outside!
Let me back inside!
On Mon, 24 Oct 2005 00:50:12 +0200, Edi Weitz wrote:
> On Mon, 24 Oct 2005 11:23:08 +1300, Adam Warner
> <······@consulting.net.nz> wrote:
>
>> If I'm feeling lucky at Google and check out Microsoft .Net Benchmarks*
>> and find many graphs showing Sun and IBM products being annihilated, I
>> hope I don't confuse technical arguments with a political agenda.
>
> MS wins, so the benchmarks /must/ be flawed, right?
How did you manage to twist the comments 'I would be extra suspicious of
the performance of a competitor's product if the vendor prohibited
publication of benchmarks' [without written permission] and 'Is this the
likely action of a vendor with JIT compiler technology "just as good as
Java's"?' into this characterisation: "MS wins, so the benchmarks /must/
be flawed, right?"
[Note I also employed your phrase 'don't confuse technical arguments with
a political agenda.']
It is healthy to be sceptical of benchmarks. The majority are flawed even
when conducted by a disinterested party of pure intention.
If the question we want to resolve is whether the Microsoft .NET VM is
just as fast as leading Java VMs then the linked benchmarks are
inappropriate. Microsoft is benchmarking entire software stacks which only
partially run upon each competitors virtual machines.
We may however be able to conclude that according to Microsoft's own
benchmarks the Microsoft .NET 2.0 Beta2 VM is substantially faster than
the Microsoft .NET 1.1 VM, their current production VM. If the Microsoft
.NET 1.1 VM is already as fast as leading Java VMs then the Microsoft .NET
2.0 Beta2 VM has extraordinary and unmatched performance. Or perhaps
Microsoft's current offering is immature and under-performing and the
Microsoft .NET 2.0 Beta2 VM just looks great relative to the .NET 1.1 VM.
Again the reader can come to his or her own conclusions.
Regards,
Adam