On Fri, 21 Nov 1997, Steven D. Majewski wrote:
>
> No Python content, but all of you language benchmarking fans
> out there might want to look at:
>
> Timing Trials, or, the Trials of Timing:
> Experiments with Scripting and User-Interface Languages
> (10/13/97)
>
> Brian W. Kernighan
> Christopher J. Van Wyk
>
>
> http://cm.bell-labs.com/cm/cs/who/bwk/interps/pap.html
>
>
> Tcl, Perl, Awk, Limbo, Java, MIT Scheme, VB, compared to each other and C.
>
See Also:
Project Rocky: The Architectural Performance of Interpreted Languages
<http://www.cs.washington.edu/homes/romer/rocky/>
---| Steven D. Majewski (804-982-0831) <·····@Virginia.EDU> |---
---| Department of Molecular Physiology and Biological Physics |---
---| University of Virginia Health Sciences Center |---
---| P.O. Box 10011 Charlottesville, VA 22906-0011 |---
All power corrupts and obsolete power corrupts obsoletely." - Ted Nelson
"Steven D. Majewski" <·····@Virginia.EDU> writes:
>
>
> On Fri, 21 Nov 1997, Steven D. Majewski wrote:
>
> >
> > No Python content, but all of you language benchmarking fans
> > out there might want to look at:
> >
> > Timing Trials, or, the Trials of Timing:
> > Experiments with Scripting and User-Interface Languages
> > (10/13/97)
> >
> > Brian W. Kernighan
> > Christopher J. Van Wyk
> >
> >
> > http://cm.bell-labs.com/cm/cs/who/bwk/interps/pap.html
> >
> >
> > Tcl, Perl, Awk, Limbo, Java, MIT Scheme, VB, compared to each other and C.
> >
I found this paper not too accurate as far as Scheme is concerned.
Apparently they didn't use a good implentation.
I tried a couple of their tests (Array and Ackerman function) with
Scheme48 and found that it performs as good as Limbo, or even better.
As well, their discussion sometimes mixes up definition of the language
and implementational issues (e.g they claim that absense of buffered I/O
make Scheme I/O-performance suffer. But isn't this the feature of an
implementation rather than the language?)
D.Pasechnik at twi dot tudelft.nl
------------------
PS. My runs (on a 100MHz Pentium under Linux) of the array test:
bash-2.00$ scheme48 -h 6000000
Welcome to Scheme 48 0.48 (made by dima on Sat Nov 15 00:18:33 GMT 1997).
Copyright (c) 1993, 1994 by Richard Kelsey and Jonathan Rees.
Copyright (c) 1996 by NEC Research Institute, Inc.
Please report bugs to ··············@martigny.ai.mit.edu.
Type ,? (comma question-mark) for help.
> ,bench
Will compile some calls in line
> (define n 200000)
(define x (make-vector n))
(define y (make-vector n))> >
> (define (test1)
(do ((i 0 (+ 1 i)))
((= i n) #t)
(vector-set! x i i))
(do ((j (- n 1) (- j 1)))
((< j 0) #t)
(vector-set! y j (vector-ref x j)))
)
> ,time (test1)
Run time: 2.65 seconds; Elapsed time: 2.71 seconds
#t
> ,time (test1)
Run time: 2.64 seconds; Elapsed time: 2.64 seconds
#t
----------------------------------------------------------------
The Ackerman function test:
bash-2.00$ scheme48 -h 6000000
Welcome to Scheme 48 0.48 (made by dima on Sat Nov 15 00:18:33 GMT 1997).
Copyright (c) 1993, 1994 by Richard Kelsey and Jonathan Rees.
Copyright (c) 1996 by NEC Research Institute, Inc.
Please report bugs to ··············@martigny.ai.mit.edu.
Type ,? (comma question-mark) for help.
> ,bench
Will compile some calls in line
> (define (ack m n)
(cond ((= m 0) (+ n 1))
((= n 0) (ack (- m 1) 1))
(else (ack (- m 1) (ack m (- n 1))))))
>
> ,time (ack 3 7)
Run time: 4.76 seconds; Elapsed time: 4.78 seconds
1021
> ,time (ack 3 7)
Run time: 4.74 seconds; Elapsed time: 4.74 seconds
1021
------------------------------------------------------------------
I don't have enough RAM to do (ack 3 8) without the computer swapping :(
Dima Pasechnik <····@duti515a.twi.tudelft.nl> writes:
| I found this paper not too accurate as far as Scheme is concerned.
| Apparently they didn't use a good implentation.
They used MIT scheme.
| bash-2.00$ scheme48 -h 6000000
You need a 6M heap just to walk over two 200K vectors? And why do you
have to statically allocate it?
--
··········@[127.0.0.1]
··········@localhost.
schwartz+%!usenet! @ finch.cse.psu. ((no.)spam) edu (Scott Schwartz) wrote
in message <··············@finch.cse.psu.edu>...
>Dima Pasechnik <····@duti515a.twi.tudelft.nl> writes:
>| I found this paper not too accurate as far as Scheme is concerned.
>| Apparently they didn't use a good implentation.
>
>They used MIT scheme.
But they forgot to either ``SF'' or compile it.
They weren't benchmarking
compilers, so you might argue that they shouldn't have used the
compiler (but why did they include compiled C?), but neglecting
to run SF on the ``script'' causes a performance hit because
MIT Scheme doesn't assume that any built-in procedures are not
redefined.
(For those unfamiliar with MIT Scheme, there is a program called
SF that is essentially a source->source translator that can be used
to perform certain optimizations. The most used optimization is to
replace certain variables with their actual values -- in effect,
declaring common built-in procedures as constant.)
"JRM" <········@cape.com> writes:
>They weren't benchmarking
>compilers,
Actually, they _did_ (in section 7 of the paper), but my reading is that
they used the VSCM compiler, *not* the MIT Scheme compiler, and *not*
Gambit, or Bigloo, or anything that compiles to native code.
(Certainly not Stalin, which does as well as C on the file copy test.)
Since their main benchmarks were done with
*compiled* C
*byte-compiled* Java
*byte-compiled* Limbo
then it would have been fair to use
*byte-compiled* Scheme in the same tests.
It isn't clear that they understood Scheme.
To start with, they say "Scheme 7.3 [is a] pure interpreted that
repeatedly parse[s] the source code as [it] run[s]", which I don't
believe to be the case. Surely it works from list structures?
--
John �neas Byron O'Keefe; 1921/02/04-1997/09/27; TLG,TLTA,BBTNOTL.
Richard A. O'Keefe; RMIT Comp.Sci; http://www.cs.rmit.edu.au/%7Eok
>It isn't clear that they understood Scheme.
>To start with, they say "Scheme 7.3 [is a] pure interpreted that
>repeatedly parse[s] the source code as [it] run[s]", which I don't
>believe to be the case. Surely it works from list structures?
MIT Scheme interprets ``S-Code'' which is a tagged list structure.
Each primitive syntactic form corresponds to an S-Code type. For
instance, an scode-conditional has three elements: the predicate,
the consequence, and the alternative. When a form is passed to
the evaluator, it is first ``syntaxed'' (translated to S-Code), then
interpreted.
MIT Scheme by default makes no assumptions about what the user
might do with the code. You can re-define ``built-in'' procedures,
change procedure values on the fly, etc. This means that each and every
variable must be looked up each and every time it is used. While this is
useful for pedagogic reasons, experienced programmers rarely use this
flexibility.
To give an example of what SF does, consider the form (eq? x 'foo). When
syntaxed, this turns into a combination of 3 elements. Element 1 is the
variable
EQ?, element 2 the variable X, element 3 the constant 'FOO. Each time the
form is evaluated, EQ? is looked up and found to be the EQ? procedure.
SF, when encountering the same form (with USUAL-INTEGRATIONS turned on),
will create an scode primitive combination with the first element being the
EQ? procedure, the other elements being the same. Now the variable lookup
for EQ? will no longer be done. SF is also capable of doing beta reductions
(inline coding), and other source->source translations.
Code run through SF will perform faster than code that is not. A typical
result is a factor of about 2.5 (depends on the application, of course).
I think that running code through SF on MIT Scheme (with the default setting
of USUAL-INTEGRATIONS) is reasonable because:
1. no experienced MIT Scheme programmer would neglect to do so, and
2. most lisp implementations by default cache ``built-in'' procedures. (For
instance,
Common Lisp explicitly allows the system to cache ``built-in'' procedures
in this way).
I would argue that not using the native compiler is kind of silly, but to
each his own.
However, I think that not using SF when using MIT Scheme borders on misusing
the
interpreter.
Donal Fellows asks:
>Is SF supplied with MIT Scheme, or at least mentioned in the
>documentation that comes with it?
The SF program comes with MIT Scheme.
On Tue, 25 Nov 1997 21:39:12 -0500, Jrm <········@cape.com> wrote:
>Donal Fellows asks:
>>Is SF supplied with MIT Scheme, or at least mentioned in the
>>documentation that comes with it?
>
>The SF program comes with MIT Scheme.
Just to clarify a little, it's not a standalone program: it's invoked as a
procedure within MIT Scheme as
(sf FILENAME [DESTINATION])
It's described in the User's Guide under "Compiling Programs".
--------------------------------------------------------------------------
Steve Austin Powered by Linux since 1995
············@bigfoot.com RedHat 2.0.18
Fight Spam - Join CAUCE - http://www.cauce.org
In article <············@newsie2.cent.net>, JRM <········@cape.com> wrote:
[ Talking about MIT Scheme ]
> But they forgot to either ``SF'' or compile it.
Is SF supplied with MIT Scheme, or at least mentioned in the
documentation that comes with it? If not, how were they supposed to
figure out how to get it past _your_ stringent tests for "validity"
> They weren't benchmarking compilers, so you might argue that they
> shouldn't have used the compiler (but why did they include compiled
> C?), but neglecting to run SF on the ``script'' causes a performance
> hit because MIT Scheme doesn't assume that any built-in procedures
> are not redefined.
They probably included compiled C because it was easy to do, providing
something against which to compare the rest of the figures. Not
everyone has a C interpreter installed conveniently, but C compilers
are ubiquitous in the programming community...
Donal.
--
Donal K. Fellows http://r8h.cs.man.ac.uk:8000/ ········@cs.man.ac.uk
Department of Computer Science, University of Manchester, U.K. +44-161-275-6137
--
Have you hugged a software engineer today?
Donal K. Fellows wheezed these wise words:
> They probably included compiled C because it was easy to do, providing
> something against which to compare the rest of the figures. Not
> everyone has a C interpreter installed conveniently, but C compilers
> are ubiquitous in the programming community...
Most certainly. VC++ can compile to interpreted bytecodes. I've never
tried this myself, but apparently that feature is there. I've not yet
seen any benchmarks for such code, either.
--
Please note: my email address is munged; You can never browse enough
"There are no limits." -- ad copy for Hellraiser
According to JRM <········@cape.com>:
:
:schwartz+%!usenet! @ finch.cse.psu. ((no.)spam) edu (Scott Schwartz) wrote
:in message <··············@finch.cse.psu.edu>...
:>Dima Pasechnik <····@duti515a.twi.tudelft.nl> writes:
:>| I found this paper not too accurate as far as Scheme is concerned.
:>| Apparently they didn't use a good implentation.
:>
:>They used MIT scheme.
:
:
:But they forgot to either ``SF'' or compile it.
:They weren't benchmarking
:compilers, so you might argue that they shouldn't have used the
:compiler (but why did they include compiled C?), but neglecting
compiled C is typically used in these types of comparisons to provide a
'baseline' of what the best one might expect to reach is.
:(For those unfamiliar with MIT Scheme, there is a program called
:SF that is essentially a source->source translator that can be used
:to perform certain optimizations. The most used optimization is to
:replace certain variables with their actual values -- in effect,
:declaring common built-in procedures as constant.)
Curious that the interpreter just doesn't 'run SF' itself when ever the
code is run - or does it modify the original source?
--
Larry W. Virden INET: ·······@cas.org
<URL:http://www.teraform.com/%7Elvirden/> <*> O- "We are all Kosh."
Unless explicitly stated to the contrary, nothing in this posting should
be construed as representing my employer's opinions.
·······@cas.org writes:
[snipped discussion of sf in MIT Scheme]
>
> Curious that the interpreter just doesn't 'run SF' itself when ever the
> code is run - or does it modify the original source?
>
I imagine for the same reasons you don't invoke optimisers in other compilers
by default - it makes debugging harder. It's a matter of opinion whether
compilers should generate debug versions or optimised versions of code by
default (with the alternatives as options), but me, I go with this
quote from the perl manpage ( just making the crosspost relevant ;o) )
"BUGS
" The -w switch is not mandatory.
-Baz
--
Followups trimmed...
In article <············@srv38s4u.cas.org>, ·······@cas.org writes
>Curious that the interpreter just doesn't 'run SF' itself when ever the
>code is run - or does it modify the original source?
The problem is probably that SF makes some assumptions that are not true
in general of Scheme; just true in those 99.9% of cases that occur in
real life :^) The key assumption is probably[*] that none of the
built-in operators/functions are redefined. As long as this is true,
you can make things a lot faster by inlining the code for those ops, but
it is a change from the defined semantics of the language.
Curiously, in Tcl you actually have similar problems, but redefinition
of commands/procedures is far more common, so the people at Sun who
developed the compiler system had to resolve this issue (at some
cost to performance/memory usage too)
Donal.
[* Speaking with a knowledge of functional language systems and
compilers. Don't ask me if this is really the case.
--
Donal K. Fellows