--On Friday, May 23, 2003 7:10 PM -0700 Michael Park
<···········@whoever.com> wrote:
> Why did CMUCL do so poorly in the "Shootout" ? It is a few times
> slower than C for matrix multiplication. All types were declared. On
> my machine, when I test it from REPL, so there are no start-up costs,
> Lisp is 4-5 times slower than C. Why?
>
> http://www.bagley.org/~doug/shootout/bench/matrix/
Why did C do so poorly in the "Shootout"? It is a few times slower that
CMUCL for regular expression matching. Why?
http://www.bagley.org/~doug/shootout/bench/regexmatch/
I'd hardly call 6th overall "doing so poorly"!
I guess when Doug states in his "Best of Show":
"I've dabbled with Lisp (e.g. CMUCL) on and off for many years, but the
shootout gave me an opportunity to look more at Common Lisp and at Scheme
(Bigloo) too, which I hadn't done before. This family of languages has so
much to offer, it's a shame that they are not more popular."
he simply didn't know what he was talking about.
Quit trying to get a rise out of the group...
"Scott A Douglass" <····@andrew.cmu.edu> wrote in message
························@pepermint...
>
> --On Friday, May 23, 2003 7:10 PM -0700 Michael Park
> <···········@whoever.com> wrote:
>
> > Why did CMUCL do so poorly in the "Shootout" ? It is a few times
> > slower than C for matrix multiplication. All types were declared. On
> > my machine, when I test it from REPL, so there are no start-up costs,
> > Lisp is 4-5 times slower than C. Why?
> >
> > http://www.bagley.org/~doug/shootout/bench/matrix/
>
> Why did C do so poorly in the "Shootout"? It is a few times slower that
> CMUCL for regular expression matching. Why?
>
> http://www.bagley.org/~doug/shootout/bench/regexmatch/
>
>
Doesn't this graph really show that we should all be using OCaml?
;-)
Scott McKay <···@attbi.com> wrote:
> Doesn't this graph really show that we should all be using OCaml?
Btw -- there is somewhere an interesting variation on the shootout, with
the *bytes* of code metric, using both plain and bzip2'edded
sources. Can't seem to find it right now, though.
IIRC Ocaml did a lot worse in those if any significant weight was given to
the source size.
What I'd like to see (or do, if I find the time) would be a version of the
shootout using various CL implementations -- both commercial and free --
plus maybe g++ and Sun javac to give a non-CL baseline. Also, a FP
intensive test would be interesting as well.
Cheers,
-- Nikodemus
In article <·······················@rwcrnsc52.ops.asp.att.net>,
"Scott McKay" <···@attbi.com> wrote:
> Doesn't this graph really show that we should all be using OCaml?
I've played with OCaml and it certainly seems to be a very attractive
language if you already know what you want before you start, but I never
seem to. The OCaml guys have had some spectacular sucesses in the ICFP
contest (e.g. the Ray Tracer in '00), but also some big failures (e.g.
last year).
Four weeks to the next one!
-- Bruce
sticking with Dylan, but hoping to see more CL entries
Bruce Hoult <·····@hoult.org> wrote:
> The OCaml guys have had some spectacular sucesses in the ICFP
> contest (e.g. the Ray Tracer in '00), but also some big
> failures (e.g. last year).
I think you mean two years ago. Last year first place was taken
by a team using OCaml.
Neil
In article <····················@nwrddc03.gnilink.net>,
Neil Schemenauer <··········@arctrix.com> wrote:
> Bruce Hoult <·····@hoult.org> wrote:
> > The OCaml guys have had some spectacular sucesses in the ICFP
> > contest (e.g. the Ray Tracer in '00), but also some big
> > failures (e.g. last year).
>
> I think you mean two years ago. Last year first place was taken
> by a team using OCaml.
Hmm ... you're correct.
I was thinking of "the OCaml guys" at INRIA (Camls 'R Us), who came in
81st last year.
There were a number of other teams using OCaml who did better. Which is
of course a good reflection on the language.
-- Bruce
From: Dan Schmidt
Subject: Re: CMUCL performance in the Shootout
Date:
Message-ID: <uof1qv1jn.fsf@dfan.org>
Bruce Hoult <·····@hoult.org> writes:
| In article <····················@nwrddc03.gnilink.net>,
| Neil Schemenauer <··········@arctrix.com> wrote:
|
|| Bruce Hoult <·····@hoult.org> wrote:
|| | The OCaml guys have had some spectacular sucesses in the ICFP
|| | contest (e.g. the Ray Tracer in '00), but also some big
|| | failures (e.g. last year).
||
|| I think you mean two years ago. Last year first place was taken
|| by a team using OCaml.
|
| Hmm ... you're correct.
|
| I was thinking of "the OCaml guys" at INRIA (Camls 'R Us), who came in
| 81st last year.
|
| There were a number of other teams using OCaml who did better. Which is
| of course a good reflection on the language.
My team came in 24th out of 168, and the three of us probably had
around 10 man-hours of experience in OCaml before the contest started.
That slowed us down, but there was a lot about OCaml that sped us up
even more. (I do like Lisp too.)
--
http://www.dfan.org
Bruce Hoult <·····@hoult.org> wrote in message news:<···························@copper.ipg.tsnz.net>...
> In article <·······················@rwcrnsc52.ops.asp.att.net>,
> "Scott McKay" <···@attbi.com> wrote:
>
> > Doesn't this graph really show that we should all be using OCaml?
>
> I've played with OCaml and it certainly seems to be a very attractive
> language if you already know what you want before you start, but I never
> seem to. The OCaml guys have had some spectacular sucesses in the ICFP
> contest (e.g. the Ray Tracer in '00), but also some big failures (e.g.
> last year).
>
> Four weeks to the next one!
Four weeks or four months?
> sticking with Dylan, but hoping to see more CL entries
How does one submit a CL entry? As code only? As an image? As fasl?
What if their version of CMUCL is different?
>>>>> "Michael" == Michael Park <···········@whoever.com> writes:
Michael> How does one submit a CL entry? As code only? As an image? As fasl?
Michael> What if their version of CMUCL is different?
There will be all the info you need on how to submit an entry. My
recollection is that it (at least has been that) you get some general
info on the environment in which you run, how the application is
started and how data is fed to the application and you basically just
submit a binary that can fit into that framework (I cannot remember
specifics but I would be surprised if the rules would exclude
something like CMUCL from participating).
Check out links from some of the old contests to learn more.
------------------------+-----------------------------------------------------
Christian Lynbech | christian ··@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
- ·······@hal.com (Michael A. Petonic)
In article <····························@posting.google.com>,
···········@whoever.com (Michael Park) wrote:
> Bruce Hoult <·····@hoult.org> wrote in message
> news:<···························@copper.ipg.tsnz.net>...
> > In article <·······················@rwcrnsc52.ops.asp.att.net>,
> > "Scott McKay" <···@attbi.com> wrote:
> >
> > > Doesn't this graph really show that we should all be using OCaml?
> >
> > I've played with OCaml and it certainly seems to be a very attractive
> > language if you already know what you want before you start, but I never
> > seem to. The OCaml guys have had some spectacular sucesses in the ICFP
> > contest (e.g. the Ray Tracer in '00), but also some big failures (e.g.
> > last year).
> >
> > Four weeks to the next one!
>
> Four weeks or four months?
27-29 June. OK, that's five weeks.
> > sticking with Dylan, but hoping to see more CL entries
>
> How does one submit a CL entry? As code only? As an image? As fasl?
> What if their version of CMUCL is different?
In the past the judge's machine has always been x86 Linux.
If your language allows you to submit a statically-linked binary then
that is the safest option.
They also put quite a number of interpreters and other language
implementations on the machine. If you ask for something -- e.g. the
latest version of CMUCL -- then going on past record they will be happy
to have that pre-installed.
Going from the results table, last year there were three entries using
Common Lisp, so clearly someone knows the best way to do it :-)
-- Bruce
Scott A Douglass <····@andrew.cmu.edu> writes:
> Why did C do so poorly in the "Shootout"? It is a few times slower that
> CMUCL for regular expression matching. Why?
In this case, Shootout does not measure C performance, but PCRE
performance, 8-)
Scott A Douglass <····@andrew.cmu.edu> wrote in message news:<···················@pepermint>...
> --On Friday, May 23, 2003 7:10 PM -0700 Michael Park
> <···········@whoever.com> wrote:
>
> > Why did CMUCL do so poorly in the "Shootout" ? It is a few times
> > slower than C for matrix multiplication. All types were declared. On
> > my machine, when I test it from REPL, so there are no start-up costs,
> > Lisp is 4-5 times slower than C. Why?
> >
> > http://www.bagley.org/~doug/shootout/bench/matrix/
>
> Why did C do so poorly in the "Shootout"? It is a few times slower that
> CMUCL for regular expression matching. Why?
>
> http://www.bagley.org/~doug/shootout/bench/regexmatch/
Probably because the C program was using a regular expression library?
The C program uses something called ``pcre'' and the C++ program uses
some library called ``zopyra''. So these are effectively benchmarks
for these libraries compiled with these compilers.
Regular expression libraries for C tend to be poorly optimized,
because the language provides no built in compiler. In effect, the
writer of a regular expression library is subject to Greenspun's Tenth
Rule: he has to not only write the parser and translator for regular
expressions from scratch, but choose an intermediate representation,
and write the virtual machine that interprets that representation.
I've never seen a regex library for C which dynamically compiled
regexes to native machine code; the reason is that it's probably too
much work to be worth the effort.
A Lisp-based regex compiler could, on the other hand, quite easily
take advantage of dynamic compilation. Just translate the regex to the
source code of a Lisp function, and compile it.
But the CMUCL code doesn't actually use regular expressions. It uses
the META package: META:WITH-STRING-META macro and explicit calls of
META:MATCH. It's not entirely clear that this should be categorized as
a regular expression benchmark.
It would make more sense to choose some actual regex implementation
for the benchmark, like Edi Weitz's CLPPCRE:
http://www.weitz.de/cl-ppcre/
From the CLPPCRE abstract:
``It is fast. If compiled with CMUCL it outperforms Perl's highly
optimized regex engine (written in C) which to my knowledge is faster
than most other regex engines around. If compiled with CLISP it is
still comparable to CLISP's own regex implementation which is also
written in C.''
···@ashi.footprints.net (Kaz Kylheku) writes:
> Scott A Douglass <····@andrew.cmu.edu> wrote in message
> news:<···················@pepermint>...
> > --On Friday, May 23, 2003 7:10 PM -0700 Michael Park
> > <···········@whoever.com> wrote:
> >
> > > Why did CMUCL do so poorly in the "Shootout" ? It is a few times
> > > slower than C for matrix multiplication. All types were
> > > declared. On my machine, when I test it from REPL, so there are
> > > no start-up costs, Lisp is 4-5 times slower than C. Why?
> > >
> > > http://www.bagley.org/~doug/shootout/bench/matrix/
> >
> > Why did C do so poorly in the "Shootout"? It is a few times
> > slower that CMUCL for regular expression matching. Why?
> >
> > http://www.bagley.org/~doug/shootout/bench/regexmatch/
>
> Probably because the C program was using a regular expression
> library? The C program uses something called ``pcre'' and the C++
> program uses some library called ``zopyra''. So these are
> effectively benchmarks for these libraries compiled with these
> compilers.
>
> Regular expression libraries for C tend to be poorly optimized,
> because the language provides no built in compiler. In effect, the
> writer of a regular expression library is subject to Greenspun's
> Tenth Rule: he has to not only write the parser and translator for
> regular expressions from scratch, but choose an intermediate
> representation, and write the virtual machine that interprets that
> representation. I've never seen a regex library for C which
> dynamically compiled regexes to native machine code; the reason is
> that it's probably too much work to be worth the effort.
>
> A Lisp-based regex compiler could, on the other hand, quite easily
> take advantage of dynamic compilation. Just translate the regex to
> the source code of a Lisp function, and compile it.
>
> But the CMUCL code doesn't actually use regular expressions. It uses
> the META package: META:WITH-STRING-META macro and explicit calls of
> META:MATCH. It's not entirely clear that this should be categorized
> as a regular expression benchmark.
>
> It would make more sense to choose some actual regex implementation
> for the benchmark, like Edi Weitz's CLPPCRE:
> http://www.weitz.de/cl-ppcre/
FWIW, I actually did that yesterday out of curiosity. It turned out
that my code (which basically just combined Jochen Schmidt's CMUCL
code with the Perl regex found at the "Shootout" site) was almost
identical in speed on my machine:
Perl 5.8: 27.0 sec
Perl 5.6.1: 19.7 sec
O'Caml 3.04: 19.3 sec
CMUCL 18e with META: 14.5 sec
CMUCL 18e with CL-PPCRE 0.5.4: 12.5 sec
[Average values for three consecutive runs with n = 100000 excluding
one initial run to fill the disk cache.]
BTW, CL-PPCRE does not do any compilation at run-time. It rather
translates the regex into a chain of closures which were compiled when
the library was built.
Edi.
(proclaim '(optimize (speed 3) (safety 0) (space 0) (debug 0) (compilation-speed 0)))
(setf ext:*bytes-consed-between-gcs* 5000000)
(use-package :cl-ppcre)
(defparameter *scanner* (load-time-value
(create-scanner
"(?: ^ | [^\\d\\(]) # must be preceeded by non-digit
( \\( )? # match 1: possible initial left paren
(\\d\\d\\d) # match 2: area code is 3 digits
(?(1) \\) ) # if match1 then match right paren
[ ] # area code followed by one space
(\\d\\d\\d) # match 3: prefix of 3 digits
[ -] # separator is either space or dash
(\\d\\d\\d\\d) # match 4: last 4 digits
(?: \\D | $ ) # must be followed by a non-digit"
:extended-mode t)))
(defun main ()
(let ((n (parse-integer (or (car (last extensions:*command-line-strings*)) "1")))
(input (loop for line = (read-line *standard-input* nil 'eof)
until (eq line 'eof) collect line)))
(loop for i of-type fixnum from 1 below n do
(loop for line of-type simple-base-string in input
do (scan *scanner* line)))
(loop with i of-type fixnum = 0
for line in input
do (multiple-value-bind (match-start match-end reg-starts reg-ends)
(scan *scanner* line)
(declare (ignore match-start match-end))
(when reg-starts
(locally
(declare (simple-base-string line))
(format t "~A: (~A) ~A-~A~%"
(incf i)
(subseq line (svref reg-starts 1) (svref reg-ends 1))
(subseq line (svref reg-starts 2) (svref reg-ends 2))
(subseq line (svref reg-starts 3) (svref reg-ends 3)))))))))
Kaz Kylheku writes, among other things:
> I've never seen a regex library for C which dynamically compiled
> regexes to native machine code;
actually, ken thompson's original (well known, rarely read) paper on
regular expressions shows how his implementation dynamically compiles
them to native machine code.
cheers... oz
---
[1] Thompson, K. "Regular expression search algorithm,"
Comm. ACM 11:6, 419-422, 1968.
---
logic's hell! - russell