From: Valery A.Khamenya
Subject: why I/O is so slow?
Date:
Message-ID: <3f1fd180@shknews01>
Hi All
Below there are two close implementations with the same output.
One is in C another is in CL.
CL version compiled under last CMUCL is about 10 times (!)
slower then C version compiled with "gcc -O2" (last version)
Could anyone suggest any improvements to CL code?
Or maybe it is just a problem of CMUCL?
Thank you in advance.
// -------- C version
#include <stdio.h>
#define M 100000
#define N 2
float flo[M][N];
void write_matrix() {
FILE *f;
int i,j;
f = fopen("tmp3.dat", "w");
fprintf(f, "%d %d\n", M, N);
for(i=0; i<M; i++) {
for(j=0; j<N; j++)
fprintf(f, "%f ", flo[i][j]);
fprintf(f, "\n");
}
fclose(f);
}
main() {
int i, j;
for(i=0; i<M; i++)
for(j=0; j<N; j++)
flo[i][j] = 12.13;
write_matrix();
}
;; -------------- CL version
(defun write-matrix-file (mat mat-file)
#| "write a matrix file with following format
<M> <N>
<content>
where M is the number of rows, N is the number of columns
and <content> contains MxN elements of the matrix listed
by rows"
|#
(let
((matfile (open mat-file :direction :output))
(M (car (array-dimensions mat))) ;; # of rows
(N (cadr (array-dimensions mat))) ;; # of columns
)
(format matfile "~A ~A~%" M N)
(do ((i 0 (1+ i)))
((eql i M))
(do ((j 0 (1+ j)))
((eql j N))
(format matfile "~A " (aref mat i j)))
(format matfile "~%"))
(close matfile)))
(defun go!()
(write-matrix-file
(make-array '(100000 2) :initial-element 12.13 ) "mat2.txt"))
;; and then:
(compile 'write-matrix-file )
(compile 'go! )
(go!)
--
Valery
"Valery A.Khamenya" <··········@biovision-discovery.de> writes:
> Below there are two close implementations with the same output.
I don't think these are close implementations.
> One is in C another is in CL.
> CL version compiled under last CMUCL is about 10 times (!)
> slower then C version compiled with "gcc -O2" (last version)
>
> Could anyone suggest any improvements to CL code?
> Or maybe it is just a problem of CMUCL?
> Thank you in advance.
I have a horrible feeling of deja vu... perhaps in future you could
check google before posting?
> #define M 100000
> #define N 2
M and N here are compile-time constants.
> (M (car (array-dimensions mat))) ;; # of rows
> (N (cadr (array-dimensions mat))) ;; # of columns
M and N here are runtime variables.
> fprintf(f, "%d %d\n", M, N);
> fprintf(f, "%f ", flo[i][j]);
%d and %f are specific printing controls.
> (format matfile "~A ~A~%" M N)
> (format matfile "~A " (aref mat i j)))
~A is a general printing control; it must dispatch on the type of its
argument.
> for(i=0; i<M; i++) {
> for(j=0; j<N; j++)
> for(i=0; i<M; i++)
> for(j=0; j<N; j++)
These comparisons and additions are known to be on machine-sized data.
> (do ((i 0 (1+ i)))
> ((eql i M))
> (do ((j 0 (1+ j)))
> ((eql j N))
These additions and comparisons involve generic arithmetic (at least
in the absence of a fairly smart compiler).
This leaves me with four lines of C
> fprintf(f, "\n");
> fclose(f);
> flo[i][j] = 12.13;
> write_matrix();
that are roughly equivalent to
> (format matfile "~%"))
> (close matfile)))
> (write-matrix-file
> (make-array '(100000 2) :initial-element 12.13 ) "mat2.txt"))
So we won't argue about those four lines. Just about everything else
in your two implementations is different between the C version and the
CL version.
Christophe
--
http://www-jcsu.jesus.cam.ac.uk/~csr21/ +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%") (pprint #36rJesusCollegeCambridge)
Christophe Rhodes wrote:
> I have a horrible feeling of deja vu... perhaps in future you could
> check google before posting?
is there any more unfriendly community then from comp.lang.lisp?
btw, any good query for google in this case from you?
> M and N here are compile-time constants.
> [...]
> M and N here are runtime variables.
i knew.
> %d and %f are specific printing controls.
i knew.
> ~A is a general printing control; it must dispatch on the type of its
> argument.
i knew.
>> for(i=0; i<M; i++) {
>> for(j=0; j<N; j++)
>> for(i=0; i<M; i++)
>> for(j=0; j<N; j++)
> These comparisons and additions are known to be on machine-sized data.
>> (do ((i 0 (1+ i)))
>> ((eql i M))
>> (do ((j 0 (1+ j)))
>> ((eql j N))
> These additions and comparisons involve generic arithmetic (at least
> in the absence of a fairly smart compiler).
not clear.
> So we won't argue about those four lines. Just about everything else
> in your two implementations is different between the C version and the
> CL version.
now, could you agree that only the following line (maybe together
with GC) eats the time:
(format matfile "~A " (aref mat i j))
Could we skip other weak influencing things?
It is not a court anyway, and you shouldn't define CL in formal
form. Are you able to bring this:
(format matfile "~A " (aref mat i j))
closer to
fprintf(f, "%f ", flo[i][j]);
or even make it quicker?
Thank you, Christophe.
--
Valery
"Valery A.Khamenya" <··········@biovision-discovery.de> writes:
> Christophe Rhodes wrote:
>
> > I have a horrible feeling of deja vu... perhaps in future you could
> > check google before posting?
>
> is there any more unfriendly community then from comp.lang.lisp?
>
> btw, any good query for google in this case from you?
>
> > M and N here are compile-time constants.
> > [...]
> > M and N here are runtime variables.
>
> i knew.
So, rigged demo.
> > %d and %f are specific printing controls.
>
> i knew.
So, rigged demo.
> > ~A is a general printing control; it must dispatch on the type of its
> > argument.
>
> i knew.
So, rigged demo.
> >> for(i=0; i<M; i++) {
> >> for(j=0; j<N; j++)
> >> for(i=0; i<M; i++)
> >> for(j=0; j<N; j++)
> > These comparisons and additions are known to be on machine-sized data.
> >> (do ((i 0 (1+ i)))
> >> ((eql i M))
> >> (do ((j 0 (1+ j)))
> >> ((eql j N))
> > These additions and comparisons involve generic arithmetic (at least
> > in the absence of a fairly smart compiler).
>
> not clear.
"Declare the types and the compiler has half a chance of optimising
the code"
> > So we won't argue about those four lines. Just about everything else
> > in your two implementations is different between the C version and the
> > CL version.
>
> now, could you agree that only the following line (maybe together
> with GC) eats the time:
> (format matfile "~A " (aref mat i j))
No, I will agree to nothing before testing.
> Could we skip other weak influencing things?
> It is not a court anyway, and you shouldn't define CL in formal
> form. Are you able to bring this:
> (format matfile "~A " (aref mat i j))
> closer to
> fprintf(f, "%f ", flo[i][j]);
>
> or even make it quicker?
A full 15 seconds clicking in the hyperspec (Front matter / Printer /
Formatted Output / Floating-point formatters) seems to indicate that
you want "~F" instead of "~A".
//Ingvar (doctor, it hurts when I do <thing>)
--
When in doubt, debug-on-entry the function you least suspect have
anything to do with something.
From: Valery A.Khamenya
Subject: Re: why I/O is so slow?
Date:
Message-ID: <3f1ff36b$1@shknews01>
Ingvar Mattsson wrote:
> No, I will agree to nothing before testing.
rigged demo?
could you pay attention to the post concerning changes towards:
(setq *q* (+ *q* (aref mat i j )))
?
It runs nice.
concerning "~A" to "~F" -- it doesn't improve the situation :-/
concerning A full 15 seconds clicking in the hyperspec:
one should be quite lucky to click non-glossary link.
--
Valery
"Valery A.Khamenya" <··········@biovision-discovery.de> wrote in message news:<················@biovision-discovery.de>...
> Christophe Rhodes wrote:
>
> > I have a horrible feeling of deja vu... perhaps in future you could
> > check google before posting?
>
> is there any more unfriendly community then from comp.lang.lisp?
In my experience, all of the comp.lang.* newsgroups I have read are
about the same.
Chris's response is practically boilerplate material that is
unsurprising in any of them.
Hi Valery,
I cannot see you original message, except on google. Benchmarking is
black art. The following Scheme/Bigloo code is twice as fast as your C
version:
old Pentium II 450 MHz, SuSE Linux 8
your C version: gcc -O3 valery.c takes 1.4 sec
the Bigloo version: bigloo -Obench valery.scm takes 0.7 sec
I think I can remember you already have installed the Bigloo compiler,
hence testing it by yourself should be no problem.
A side note. The following does not rely on any format directives. I
think there exists somewhere a format Scheme library for numbers. I am
not sure how much such a format library would impose on an overhead. One
should not forget that CommonLisp has to offer an overhelmingly ocean of
format directives where Scheme has nothing if you do not account on
add-on libraries (maybe SRFI-n):
==
(module valery)
(define (make-array rows cols #!key (val 0.0))
(let ((mx (make-vector rows val)))
(do ((i 0 (+ 1 i)))
((= i rows))
(let ((row (make-vector cols val)))
(do ((j 0 (+ 1 j)))
((= j cols))
(vector-set! row j val))
(vector-set! mx i row)))
mx))
(define (writeFloats file-name array)
(let* ((rows (vector-length array))
(cols (vector-length (vector-ref array 0))))
(call-with-output-file file-name
(lambda (port)
(do ((i 0 (+ i 1)))
((= i rows))
(do ((j 0 (+ j 1)))
((= j cols))
(write (vector-ref (vector-ref array i) j)
port)
(write-char #\tab port))
(write-char #\newline port))))))
(define do-main
(let* ((M 100000) (N 2)
(flo (make-array M N)))
(do ((i 0 (+ i 1)))
((= i M))
(do ((j 0 (+ j 1)))
((= j N))
(vector-set! (vector-ref flo i) j
12.13)))
(writeFloats "tm3.txt" flo)))
(print do-main)
==
Under Bigloo use it:
bigloo -Obench valery.scm
and then:
time ./a.out
Not only is Bigloo a fast compiler for numerical computations it is also
very fast in handling text input/output.
Additionaly one could give type information in the above Scheme code,
but I do not think this would improve things much. My experience with
type informations under Bigloo is as follows: they only pay off when one
deals with large arrays and floating point intense calculations.
S. Gonzi
"Valery A.Khamenya" <··········@biovision-discovery.de> wrote in message news:<········@shknews01>...
[snip]
> Could anyone suggest any improvements to CL code?
Yes, if you publish your test data. Upload a tarball of your matrix (and if it
is sensitive material, you can always generate a similar matrix with the same
types and ranges but not identical values.) somewhere and I will take a shot at it.
Also, don't forget to mention your memory requirements (can you afford to have
it all in-core, or is it too large.) How often do you want to read/write to it.
Is the matrix static or does it grow and shrink.
There is more to algorithm design than just _read_ time.
On Thu, 24 Jul 2003 14:29:22 +0200, "Valery A.Khamenya"
<··········@biovision-discovery.de> wrote:
>Hi All
>
>Below there are two close implementations with the same output.
>One is in C another is in CL.
>CL version compiled under last CMUCL is about 10 times (!)
>slower then C version compiled with "gcc -O2" (last version)
>
>Could anyone suggest any improvements to CL code?
>Or maybe it is just a problem of CMUCL?
>Thank you in advance.
>
Take a look at FORMATTER to produce a 'compiled' format string.
Use ~F, not ~A.
Declare all your variables.
Declare (optimize (safety 0) (speed 3))
See if writing it out as a string in-memory first
(using WITH-OUTPUT-TO-STRING)
and then using WRITE-SEQUENCE is faster.
>
> (format matfile "~A ~A~%" M N)
> (do ((i 0 (1+ i)))
> ((eql i M))
> (do ((j 0 (1+ j)))
> ((eql j N))
> (format matfile "~A " (aref mat i j)))
> (format matfile "~%"))
> (close matfile)))
>>>>> "Valery" == Valery A Khamenya <··········@biovision-discovery.de> writes:
Valery> Hi All Below there are two close implementations with the
Valery> same output. One is in C another is in CL. CL version
Valery> compiled under last CMUCL is about 10 times (!) slower
Valery> then C version compiled with "gcc -O2" (last version)
Valery> Could anyone suggest any improvements to CL code? Or
Valery> maybe it is just a problem of CMUCL? Thank you in
Valery> advance.
How do you know that the I/O is the slowest part? Maybe you should
try the cmucl profiler, to check where most time is spent.
I have don a very similar exercise and I got CMUCL within a factor
1,2 of the C(++) runtime, by inlining and type declarations. I tried
the same on allegro and as far as I could telll the problem was that
inlining was not done there make the code indeed substantially
slower.
Immanuel
From: Valery A.Khamenya
Subject: Re: why I/O is so slow?
Date:
Message-ID: <3f1fef83$1@shknews01>
Immanuel Litzroth wrote:
> How do you know that the I/O is the slowest part?
very easy. One could comment out "(format ..." lines
and in the very loop put something like:
(setq *q* (+ *q* (aref mat i j )))
and obtain result in time very close to zero.
With I/O it is however about 6 sec on my machine.
(Christophe, I hope you read this as well)
> Maybe you should try the cmucl profiler, to check where most time is spent.
I am not that fluent in CL and CMUCL yet ;)
But the note above shows the picture well enough for a moment.
Well, any soggestions about "(format ..."?
--
Valery
I am having the same problem. I came to CL because someone convinced
me that it was much faster than some of the other languages that I
use, Ruby, Python and Perl. Well, my first little benchmark for fun,
tower of hanoi, didn't go to well.
The CL version takes 45 seconds, the Python version takes 12 seconds,
the Perl version takes 9 seconds and the Ruby version takes just 6
seconds. Just for fun, I did it in C also, which runs in less than 1
second. These are not small differences.
I, too, would like to know what I am missing. Why is this version so
slow compared to all of the other languages? I am using CMUCL,
compiling to x86f under linux. The problem is with format. If I
remove the call to format then the speed goes to less than 1 second.
Which is still 8 times slower than the C version without printf.
Here is the CL code:
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(format ofs "~%Move ~a to ~a." from to))
(tower-of-hanoi t 20 "A" "B" "C")
(quit)
Here is the Python code:
def tower_of_hanoi(n, afrom, to, spare):
if n == 1:
print_move(afrom, to)
else:
tower_of_hanoi(n - 1, afrom, spare, to)
print_move(afrom, to)
tower_of_hanoi(n - 1, spare, to, afrom)
def print_move(afrom, to):
print("Move " + afrom + " to " + to + ".")
tower_of_hanoi(20, "A", "B", "C")
Here is the Perl code:
sub tower_of_hanoi
{
my ($n, $from, $to, $spare) = @_;
if ($n == 1) {
print_move($from, $to);
}
else {
tower_of_hanoi($n - 1, $from, $spare, $to);
print_move($from, $to);
tower_of_hanoi($n - 1, $spare, $to, $from);
}
}
sub print_move
{
my ($from, $to) = @_;
printf("Move %s to %s.\n", $from, $to);
}
tower_of_hanoi(20, "A", "B", "C");
Here is the Ruby code:
def tower_of_hanoi(n, from, to, spare)
if (n == 1)
print_move(from, to);
else
tower_of_hanoi(n - 1, from, spare, to);
print_move(from, to);
tower_of_hanoi(n - 1, spare, to, from);
end
end
def print_move(from, to)
printf("Move %s to %s.\n", from, to);
end
tower_of_hanoi(20, "A", "B", "C");
Here is the C code:
#include <stdio.h>
void tower_of_hanoi(int n, char *from, char *to, char *spare);
void print_move(char *from, char *to);
void tower_of_hanoi(int n, char *from, char *to, char *spare)
{
if (n == 1)
print_move(from, to);
else
{
tower_of_hanoi(n - 1, from, spare, to);
print_move(from, to);
tower_of_hanoi(n - 1, spare, to, from);
}
}
void print_move(char *from, char *to)
{
printf("Move %s to %s.\n", from, to);
}
int main(int argc, char *argv[])
{
tower_of_hanoi(20, "A", "B", "C");
}
·········@mindspring.com (John Proctor) writes:
> I, too, would like to know what I am missing. Why is this version so
> slow compared to all of the other languages? I am using CMUCL,
> compiling to x86f under linux. The problem is with format. If I
> remove the call to format then the speed goes to less than 1 second.
> Which is still 8 times slower than the C version without printf.
>
> Here is the CL code:
>
> (defun tower-of-hanoi (ofs n from to spare)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (cond ((= n 1) (print-move ofs from to))
> (t (tower-of-hanoi ofs (- n 1) from spare to)
> (print-move ofs from to)
> (tower-of-hanoi ofs (- n 1) spare to from))))
>
> (defun print-move (ofs from to)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (format ofs "~%Move ~a to ~a." from to))
>
> (tower-of-hanoi t 20 "A" "B" "C")
> (quit)
Thank you for producing a more reasoned example. :-)
I agree that this version superficially looks reasonable, and that
given that your tests show that the runtime of the logic is fine, the
problem lies in the call to FORMAT.
To understand the problem, we need to look a little carefully at what
the ~A directive to format means.
An arg, any object, is printed without escape characters (as by
princ).
So your call to format must be prepared to handle any object at all,
not just a one-character string. Your Python, Ruby, and C versions
all print just strings, and presuambly either throw an exception or
print garbage if they are not handed a string. (I don't speak Python
or Ruby with any fluency, I'm afraid). In other words, ~A is a little
too general to be optimizeable by lisp environments; it's designed for
general printing, when speed is not likely to be the issue.
So, how can we improve this? There are a number of ways; the most
aggressive is to "unroll" the call to format:
(defun print-move (ofs from to)
(declare (type (vector character) from to))
(declare (optimize (speed 3) (safety 0) (debug 0)))
(terpri ofs) ; I'd use FRESH-LINE, but ~% is strictly TERPRI
(write-string "Move " ofs)
(write-string from ofs)
(write-string " to " ofs)
(write-string to ofs)
(write-string "." ofs))
This version decreases run times on my slow sparc for 10000 iterations
(outputting to /dev/null) from 1.85 seconds to 0.45. So, this is an
improvement, but it's not yet perfect. Since there's probably some
overhead (a memory indirection, for instance), what about changing the
representation of FROM and TO to be characters, not strings?
(defun print-move (ofs from to)
(declare (type character from to))
(declare (optimize (speed 3) (safety 0) (debug 0)))
(terpri ofs)
(write-string "Move " ofs)
(write-char from ofs)
(write-string " to " ofs)
(write-char from ofs)
(write-char #\. ofs))
Now we're down to 0.27 seconds. This is a eight- or nine-fold speed
increase, so we're now in the Ruby/Perl ballpark, right? Is that
enough, or should we continue?
Cheers,
Christophe
--
http://www-jcsu.jesus.cam.ac.uk/~csr21/ +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%") (pprint #36rJesusCollegeCambridge)
> (defun print-move (ofs from to)
> (declare (type character from to))
> (declare (optimize (speed 3) (safety 0) (debug 0)))
> (terpri ofs)
> (write-string "Move " ofs)
> (write-char from ofs)
> (write-string " to " ofs)
> (write-char from ofs)
> (write-char #\. ofs))
> Cheers,
> Christophe
Thanks for the reply. I tried your suggestions, but it only cut the
time in half. I am running the ruby version via
bash# ./toh.rb > toh-rb.out
and I am running the cmucl version via
bash# time lisp-start ./toh.x86f > toh-cmucl.lisp
where lisp-start is a shell script that does
#/bin/sh
export CMUCLLIB=/usr/lib/cmucl/lib
exec /usr/bin/lisp -quiet -noinit -batch -load ······@"}
Here is the modified lisp code:
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (safety 0) (debug 0)))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (type character from to))
(declare (optimize (speed 3) (safety 0) (debug 0)))
(terpri ofs)
(write-string "Move " ofs)
(write-char from ofs)
(write-string " to " ofs)
(write-char to ofs)
(write-char #\. ofs))
(tower-of-hanoi t 20 #\A #\B #\C)
(quit)
This version now runs in 22 seconds compared to the original 45
seconds. However, the ruby version runs in 6 seconds, and I didn't
have to compile it, declare optimizations or give up formatted I/O to
get the performance.
I am very much interested in exploring this further because cmucl is
much faster than ruby/python/perl at operations that do not involve
formatting strings or using I/O. However, strings and I/O are a
hugely important aspect of most applications. All business related
apps are heavy on these 2 ops. Just think of a Web Server. Formatted
output is about all it does.
Any other angles to approach this from?
·········@mindspring.com (John Proctor) writes:
> This version now runs in 22 seconds compared to the original 45
> seconds. However, the ruby version runs in 6 seconds, and I didn't
> have to compile it, declare optimizations or give up formatted I/O to
> get the performance.
>
> I am very much interested in exploring this further because cmucl is
> much faster than ruby/python/perl at operations that do not involve
> formatting strings or using I/O. However, strings and I/O are a
> hugely important aspect of most applications. All business related
> apps are heavy on these 2 ops. Just think of a Web Server. Formatted
> output is about all it does.
>
> Any other angles to approach this from?
I've stayed out of this until now, because I have little time to offer
much in the way of experimentation or profiling. However, I have a
suggestion for someone else to try, which I note that nobody has yet
talked about: Sometimes the presence of some of the *print-?*
variables, most notably *print-pretty*, will cause multiple passes
through output, and will slow output formatting down by several times.
Whatever the default setting is, differences in personal preferences
and the current value might affect the result, and in fact could
explain differing results that different people are seeing. In the case
of *print-pretty*, I recommend setting it to nil explicitly.
Of course, setting *print-pretty* to nil is no guarantee that it will
stay nil - some outputting facilities bind it to true to do some
things. I think that the test is simple enough, but it may be the case
that the top-level might do some things with this variable. You might
want to do something like
(let ((*print-pretty* nil))
(time (form-to-test)))
so as to remove the top-level from the equation.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Duane Rettig <·····@franz.com> wrote in message news:<·············@beta.franz.com>...
> ·········@mindspring.com (John Proctor) writes:
> (let ((*print-pretty* nil))
> (time (form-to-test)))
This has been the single best option so far. With this alone, the
time is now 21 seconds. Using write-string, etc helped, but once I
added this, then there was no noticable difference between using
format vs write-*. It also nullified the benefits of declaring the
var types and changing the string vars to just char.
So, this is the current version:
(setf *print-pretty* nil)
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(format ofs "Move ~a to ~a~%." from to))
(tower-of-hanoi t 20 "A" "B" "C")
(quit)
This version is currently onl 3.5 times as slow as ruby. However,
here is another piece of valuable information. When I change the
above to write directly to a file instead of stdout and redirecting to
a file then the time goes to just over 3 seconds. Of course, the ruby
version just prints to stdout and is being redirected too.
So, we can determine that *print-pretty* has a huge impact on the
performance of format and that cmucl must be detecting that output is
bound to a tty and is doing something special. I bet something can be
done to fix this issue to.
Anyone know why there is such a huge difference in cmucl when writing
to stdout and redirecting to a file vs writing directly to a file? I
have not tried changing ruby, python, etc to use files directly, but I
don't think it would make much difference to them.
Here is the version that writes directly to file:
(setf *print-pretty* nil)
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(format ofs "Move ~a to ~a.~%" from to))
(with-open-file (ofs "toh-cmucl.out" :direction :output :if-exists
:supersede)
(tower-of-hanoi ofs 20 "A" "B" "C"))
(quit)
·········@mindspring.com (John Proctor) writes:
> So, this is the current version:
>
> (setf *print-pretty* nil)
>
>...
>
> This version is currently onl 3.5 times as slow as ruby. However,
> here is another piece of valuable information. When I change the
> above to write directly to a file instead of stdout and redirecting to
> a file then the time goes to just over 3 seconds. Of course, the ruby
> version just prints to stdout and is being redirected too.
Benchmarks notoriously dodgy. With that caveat, here are the numbers
I am getting:
CL (using ACL):
(defun towers (out-stream n from to spare)
(let* ((fromch (if (stringp from) (char from 0) from))
(toch (if (stringp to) (char to 0) to))
(sparech (if (stringp spare) (char spare 0) spare))
(move (format nil "~%Move X to Y."))
(fromloc (position #\X move))
(toloc (position #\Y move)))
(labels
((print-move (from to)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type base-char from to))
(setf (char move fromloc) from (char move toloc) to)
(write-string move out-stream))
(tower-of-hanoi (n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type fixnum n)
(type base-char from to spare))
(cond ((= n 1) (print-move from to))
(t (tower-of-hanoi (- n 1) from spare to)
(print-move from to)
(tower-of-hanoi (- n 1) spare to from)))))
(tower-of-hanoi n fromch toch sparech))
t))
(with-open-file (out "/tmp/junk.txt" :direction :output :if-exists :supersede)
(let ((*standard-output* out)
(*print-pretty* nil))
(time (towers *standard-output* 20 "A" "B" "C"))))
JOBS(218): ;;;Compiling with-open-file
; cpu time (non-gc) 580 msec user, 60 msec system
; cpu time (gc) 0 msec user, 0 msec system
; cpu time (total) 580 msec user, 60 msec system
; real time 665 msec
; space allocation:
; 1 cons cell, 152 other bytes, 0 static bytes
T
JOBS(219):
This is sort of in the middle. Some runs were .550 some were .600.
Who knows.
To make sure we have something like apples to apples here, the
following is what I get for your ruby and python code:
$ time ruby ruby.rb > /tmp/junk.txt
3.843u 0.087s 0:03.96 98.9% 0+0k 0+0io 221pf+0w
$
$ time python python.py > /tmp/junk.txt
time python python.py > /tmp/junk.txt
7.710u 0.044s 0:08.63 89.8% 0+0k 0+0io 329pf+0w
$
So, ACL is ~6.6 times _faster_ than ruby and ~13.3 times _faster_ than
python.
·········@mindspring.com (John Proctor) writes:
> So, this is the current version:
>
> (setf *print-pretty* nil)
This is not so important but I hope you realize that this is in
fact a very rude and inconsiderate measure. If you want
*PRINT-PRETTY* to be NIL in your code, the polite way of ensuring
this is
(defun my-function (arg)
(let ((*print-pretty* nil))
(do-the-rest arg)))
Long live special variables :-)
Regards,
--
Nils G�sche
Ask not for whom the <CONTROL-G> tolls.
PGP key ID #xD26EF2A0
·········@mindspring.com (John Proctor) writes:
> This version now runs in 22 seconds compared to the original 45
> seconds. However, the ruby version runs in 6 seconds, and I didn't
> have to compile it, declare optimizations or give up formatted I/O to
> get the performance.
>
> I am very much interested in exploring this further because cmucl is
> much faster than ruby/python/perl at operations that do not involve
> formatting strings or using I/O. However, strings and I/O are a
> hugely important aspect of most applications. All business related
> apps are heavy on these 2 ops. Just think of a Web Server. Formatted
> output is about all it does.
>
> Any other angles to approach this from?
Foreign function call directly to the system IO routines.
·········@mindspring.com (John Proctor) writes:
>> (defun print-move (ofs from to)
>> (declare (type character from to))
>> (declare (optimize (speed 3) (safety 0) (debug 0)))
>> (terpri ofs)
>> (write-string "Move " ofs)
>> (write-char from ofs)
>> (write-string " to " ofs)
>> (write-char from ofs)
>> (write-char #\. ofs))
>
>> Cheers,
>> Christophe
>
> Thanks for the reply. I tried your suggestions, but it only cut the
> time in half. I am running the ruby version via
> bash# ./toh.rb > toh-rb.out
>
> and I am running the cmucl version via
> bash# time lisp-start ./toh.x86f > toh-cmucl.lisp
> where lisp-start is a shell script that does
> #/bin/sh
> export CMUCLLIB=/usr/lib/cmucl/lib
> exec /usr/bin/lisp -quiet -noinit -batch -load ······@"}
>
> Here is the modified lisp code:
> [snip]
>
> This version now runs in 22 seconds compared to the original 45
> seconds. However, the ruby version runs in 6 seconds, and I didn't
> have to compile it, declare optimizations or give up formatted I/O to
> get the performance.
Alright. Here's another factor of three for you, then (again, as
measured on my slow sparc), that also restores (kind of) formatted
output:
(let ((template (copy-seq "
Move to .")))
(defun print-move (ofs from to)
(declare (type character from to))
;; I'm not convinced (DEBUG 0) is necessary.
(declare (optimize (speed 3) (safety 0) (debug 0)))
(setf (char template 6) from)
(setf (char template 11) to)
(write-string template ofs)))
Why do you consider compilation a disadvantage, out of interest?
> I am very much interested in exploring this further because cmucl is
> much faster than ruby/python/perl at operations that do not involve
> formatting strings or using I/O. However, strings and I/O are a
> hugely important aspect of most applications. All business related
> apps are heavy on these 2 ops. Just think of a Web Server. Formatted
> output is about all it does.
Really? I'd have thought that the time consumed in formatted output
is dwarfed by the time in serving static files (e.g. images). I'm
willing to be corrected by those with experience in the field on this,
though; it's just a conjecture.
Christophe
--
http://www-jcsu.jesus.cam.ac.uk/~csr21/ +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%") (pprint #36rJesusCollegeCambridge)
Christophe Rhodes <·····@cam.ac.uk> wrote in message news:<··············@lambda.jcn.srcf.net>...
> Alright. Here's another factor of three for you, then (again, as
> measured on my slow sparc), that also restores (kind of) formatted
> output:
>
> (let ((template (copy-seq "
> Move to .")))
> (defun print-move (ofs from to)
> (declare (type character from to))
> ;; I'm not convinced (DEBUG 0) is necessary.
> (declare (optimize (speed 3) (safety 0) (debug 0)))
> (setf (char template 6) from)
> (setf (char template 11) to)
> (write-string template ofs)))
I will try this tonight. Thanks.
> Why do you consider compilation a disadvantage, out of interest?
Not a disadvantage, but I should get something in exchange for having
to set this process up. Obviously, compiling requires an extra step
and maintenance of the build process. I don't mind this as long as it
has benefits. What I mean, is that, I don't even have to maintain a
build process for ruby/python. Just chmod u+x and execute. I think
that compiling CL code makes sense since it provides better
performance than uncompiled CL code. However, even compiled CL code
for this example does not compare with uncompiled ruby/python/perl
code.
> > I am very much interested in exploring this further because cmucl is
> > much faster than ruby/python/perl at operations that do not involve
> > formatting strings or using I/O. However, strings and I/O are a
> > hugely important aspect of most applications. All business related
> > apps are heavy on these 2 ops. Just think of a Web Server. Formatted
> > output is about all it does.
>
> Really? I'd have thought that the time consumed in formatted output
> is dwarfed by the time in serving static files (e.g. images). I'm
> willing to be corrected by those with experience in the field on this,
> though; it's just a conjecture.
Few business apps serve up images. Other than the login screen, none
of our apps have any images. Business websites do, but that is
generally not the application that runs the business. But, for most
business that build web apps, the goal is to have an application that
runs the business, so most of it is IO bound. Usually databases for
read and write, along with templates and xml for generating documents,
etc. Also, alot of backend work is done purely with files and
databases, loading, unloading, exports, imports, batch reports, etc.
>>>>> "John" == John Proctor <·········@mindspring.com> writes:
John> Not a disadvantage, but I should get something in exchange for having
John> to set this process up. Obviously, compiling requires an extra step
John> and maintenance of the build process. I don't mind this as long as it
John> has benefits. What I mean, is that, I don't even have to maintain a
John> build process for ruby/python. Just chmod u+x and execute. I think
John> that compiling CL code makes sense since it provides better
John> performance than uncompiled CL code. However, even compiled CL code
John> for this example does not compare with uncompiled ruby/python/perl
John> code.
So compiled CL is slower than uncompiled ruby/python/perl in this one
example. What happens if compiled CL is much faster than uncompiled
ruby/python/perl in other examples? How would you feel then? I'm
pretty sure I could come up with such examples, if I actually knew
ruby/python/perl.
Just curious,
Ray
Raymond Toy <···@rtp.ericsson.se> wrote in message news:<··············@edgedsp4.rtp.ericsson.se>...
> So compiled CL is slower than uncompiled ruby/python/perl in this one
> example. What happens if compiled CL is much faster than uncompiled
> ruby/python/perl in other examples? How would you feel then?
I would feel fine. No language is my religion. I use about 10 of
them regularly.
And yes, I was referring to this example. I did not try to present
this as a total benchmark of CL or CMUCL. As it turns out, the
problem is with the default setting of *print-pretty*. Which is
great, that seems to be an easy fix.
Hi,
·········@mindspring.com (John Proctor) writes:
> Christophe Rhodes <·····@cam.ac.uk> wrote in message news:<··············@lambda.jcn.srcf.net>...
> > Why do you consider compilation a disadvantage, out of interest?
> Not a disadvantage, but I should get something in exchange for having
> to set this process up. Obviously, compiling requires an extra step
> and maintenance of the build process.
Obviously, you haven't looked close enough at what each implementation
has to offer. Your statement/assumption is incorrect.
E.g. it seems you don't know about CLISP's -C option.
You're talking about ease of use and default behaviour of some
package/framework, not at all about compilation.
On my machine
time clisp hanoi.cl >/dev/null -> real 3m7.161s
time clisp -C hanoi.cl >/dev/null -> real 0m19.962s
Quite a tremendous effect, isn't it?
No extra process or maintenance step is needed.
So how about suggesting (setq *load-compiling* t) in your .clisprc?
The fact is that both Python and Perl compile your source file to some
intermediate representation (bytecodes in Python, much like bytecodes
in CLISP). It's like they always use -C. The difference is that they
do it even without you telling them to do so. Good or bad?
> What I mean, is that, I don't even have to maintain a
> build process for ruby/python. Just chmod u+x and execute.
To be precise, you added a #! /bin/env/python header line or some
os/shell hack/configuration to detect .py files. Please do the same
with your CL implementation of choice (see their documentation).
Thanks for your posting - I mean it. The maintainers of the different
packages will have to decide whether they will change some default
settings so as to shine more in simple benchmarks.
Regards,
Joerg Hoehle
TSI ITC-Security Technologiezentrum
·········@mindspring.com (John Proctor) writes:
> > (defun print-move (ofs from to)
> > (declare (type character from to))
> > (declare (optimize (speed 3) (safety 0) (debug 0)))
> > (terpri ofs)
> > (write-string "Move " ofs)
> > (write-char from ofs)
> > (write-string " to " ofs)
> > (write-char from ofs)
> > (write-char #\. ofs))
>
> > Cheers,
> > Christophe
>
> Thanks for the reply. I tried your suggestions, but it only cut the
> time in half. I am running the ruby version via
> bash# ./toh.rb > toh-rb.out
>
> and I am running the cmucl version via
> bash# time lisp-start ./toh.x86f > toh-cmucl.lisp
> where lisp-start is a shell script that does
> #/bin/sh
> export CMUCLLIB=/usr/lib/cmucl/lib
> exec /usr/bin/lisp -quiet -noinit -batch -load ······@"}
>
[snip]
I bet that what you are seeing are the load times of the rather big
CMUCL run-time.
Mario S. Mommer <········@yahoo.com> writes:
> ·········@mindspring.com (John Proctor) writes:
>> and I am running the cmucl version via
>> bash# time lisp-start ./toh.x86f > toh-cmucl.lisp
>> where lisp-start is a shell script that does
>> #/bin/sh
>> export CMUCLLIB=/usr/lib/cmucl/lib
>> exec /usr/bin/lisp -quiet -noinit -batch -load ······@"}
>>
> [snip]
>
> I bet that what you are seeing are the load times of the rather big
> CMUCL run-time.
Over 22 seconds? I don't think so. The overhead should be a couple
of seconds at the absolute maximum, surely?
Christophe
--
http://www-jcsu.jesus.cam.ac.uk/~csr21/ +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%") (pprint #36rJesusCollegeCambridge)
Christophe Rhodes <·····@cam.ac.uk> wrote in message news:<··············@lambda.jcn.srcf.net>...
> Mario S. Mommer <········@yahoo.com> writes:
> > I bet that what you are seeing are the load times of the rather big
> > CMUCL run-time.
>
> Over 22 seconds? I don't think so. The overhead should be a couple
> of seconds at the absolute maximum, surely?
The load time is nothing. If I comment out the call to format
completely then the exectuion time is about .5 seconds.
·········@mindspring.com (John Proctor) writes:
> This version now runs in 22 seconds compared to the original 45
> seconds. However, the ruby version runs in 6 seconds, and I didn't
> have to compile it, declare optimizations or give up formatted I/O to
> get the performance.
I don't get your results at all. Here are my tests (average of 3):
clisp 73 s
python 8.7 s
ruby 6.3 s
cmucl 1.4 s
C 0.7 s
The CL version used was the one below:
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type fixnum n)
(type base-char from to spare))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type stream ofs)
(type base-char from to))
(format ofs "~%Move ~C to ~C." from to))
(tower-of-hanoi *standard-output* 20 #\A #\B #\C)
Invocation for CMUCL was (both functions where compiled first):
(time (with-open-file (out "/dev/null" :direction :output)
(tower-of-hanoi out 20 #\A #\B #\C)))
For the others:
time clisp hanoi.cl > /dev/null
time python hanoi.py > /dev/null
time ruby hanoi.rb > /dev/null
time ./a.out > /dev/null
Bj�rn
Just so that I don't misrepresent clisp to badly...
I wrote:
> I don't get your results at all. Here are my tests (average of 3):
>
> clisp 73 s
Inserting compilation forms in the source file makes clisp complete it
in 8.2 s, slightly faster than python and a little slower than
ruby.
> python 8.7 s
> ruby 6.3 s
> cmucl 1.4 s
> C 0.7 s
Bj�rn
·······@nada.kth.se (Bj�rn Lindberg) wrote in message news:<···············@fnatte.nada.kth.se>...
clisp is indeed fast with *print-pretty* set to nil. My execution
time for the code below is 35 seconds with the default for
*print-pretty* and 7 seconds when set to nil. clisp is just as fast
writing directly to file as it is to stdout and redirecting to a file.
#!/usr/bin/clisp
(setf *print-pretty* nil)
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(format ofs "~%Move ~a to ~a." from to))
(compile 'tower-of-hanoi)
(compile 'print-move)
(tower-of-hanoi t 20 "A" "B" "C")
(quit)
·········@mindspring.com (John Proctor) wrote in message news:<····························@posting.google.com>...
> ·······@nada.kth.se (Bj�rn Lindberg) wrote in message news:<···············@fnatte.nada.kth.se>...
> (setf *print-pretty* nil)
>
> (defun tower-of-hanoi (ofs n from to spare)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (cond ((= n 1) (print-move ofs from to))
> (t (tower-of-hanoi ofs (- n 1) from spare to)
> (print-move ofs from to)
> (tower-of-hanoi ofs (- n 1) spare to from))))
>
> (defun print-move (ofs from to)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (format ofs "~%Move ~a to ~a." from to))
>
[snip snip]
I note that Bjorn's version declared types of local variables,
while yours did not. Might this account for some of the speed
difference, as well?
Avi Blackmore
········@cableone.net (Avi Blackmore) writes:
> ·········@mindspring.com (John Proctor) wrote in message news:<····························@posting.google.com>...
> > ·······@nada.kth.se (Bj�rn Lindberg) wrote in message news:<···············@fnatte.nada.kth.se>...
>
> > (setf *print-pretty* nil)
> >
> > (defun tower-of-hanoi (ofs n from to spare)
> > (declare (optimize (speed 3) (debug 0) (safety 0)))
> > (cond ((= n 1) (print-move ofs from to))
> > (t (tower-of-hanoi ofs (- n 1) from spare to)
> > (print-move ofs from to)
> > (tower-of-hanoi ofs (- n 1) spare to from))))
> >
> > (defun print-move (ofs from to)
> > (declare (optimize (speed 3) (debug 0) (safety 0)))
> > (format ofs "~%Move ~a to ~a." from to))
> >
> [snip snip]
>
> I note that Bjorn's version declared types of local variables,
> while yours did not. Might this account for some of the speed
> difference, as well?
My current version differs from the one above in that it has type
declarations, and uses ~C instead of ~A to format:
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type fixnum n)
(type base-char from to spare))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type stream ofs)
(type base-char from to))
(let ((*print-pretty* nil))
(format ofs "~%Move ~C to ~C." from to)))
;(tower-of-hanoi *standard-output* 20 #\A #\B #\C)
(with-open-file (out "/dev/null" :direction :output)
(tower-of-hanoi out 20 #\A #\B #\C))
(quit)
Running this at the REPL takes 2.0 seconds, compared to the C version
which takes 1.0 seconds. If I run it from the prompt, with
time lisp -quiet -noinit -batch -load hanoi.x86f
I get 2.0 seconds for the with-open-file version, and 3.0 seconds for
the *standard-output* version. Writing to a real file instead of
/dev/null, I get the same large 10x difference as John did, with 2.2
seconds for the with-open-file version. The C version writing to a
file instead of /dev/null takes 1.2 seconds.
Setting *print-pretty* to nil or not doesn't make any difference if ~C
is used instead of ~A. Using either of these makes a large 10-15x
difference.
Bj�rn
Hi,
·······@nada.kth.se (=?iso-8859-1?q?Bj=F6rn?= Lindberg) writes:
> I don't get your results at all. Here are my tests (average of 3):
> clisp 73 s
> python 8.7 s
> ruby 6.3 s
Another instance of apples and oranges.
Please use clisp -C
-C makes CLISP compile while loading, which is exactly what Python and
Perl do in your back.
You may wish to add (setq *load-compiling t) to your .clisprc
> For the others:
> time clisp hanoi.cl > /dev/null
> time python hanoi.py > /dev/null
> time ruby hanoi.rb > /dev/null
Now, trying to be even more constructive, here's what I get with
CLISP. The maintainers may wish to think about possible local code
improvements (e.g. make /dev/null buffered for the sake of simple
benchmarks, or set *load-compiling* so beginners get warnings from
the code).
(MAKE-BROADCAST-STREAM) is the CL idiom for unneeded output.
o (time (tower-of-hanoi (make-broadcast-stream) 20 #\A #\B #\C))
Real time: 10.054053 sec.
Run time: 10.05 sec.
GC: 32, GC time: 0.3 sec.
o (time (with-open-file(out "/dev/null" :direction :output :BUFFERED T)
(tower-of-hanoi out 20 #\A #\B #\C)))
Real time: 14.460926 sec.
Run time: 14.46 sec.
GC: 32, GC time: 0.31 sec.
o (time (with-open-file(out "/dev/null" :direction :output)
(tower-of-hanoi out 20 #\A #\B #\C)))
Real time: 18.663239 sec.
Run time: 18.67 sec.
GC: 32, GC time: 0.29 sec.
One can indeed observe that 8 out of 18 seconds are spent doing i/o in
the latter case. The path to improvement is trivial. Buffering has
been known for decades, and it's trivial to add as a user of CLISP
(witness :buffered option to OPEN).
Regards,
Joerg Hoehle
TSI ITC-Security Technologiezentrum
Joerg-Cyril Hoehle <······@spam.no> writes:
> Hi,
>
> ·······@nada.kth.se (=?iso-8859-1?q?Bj=F6rn?= Lindberg) writes:
> > I don't get your results at all. Here are my tests (average of 3):
>
> > clisp 73 s
> > python 8.7 s
> > ruby 6.3 s
> Another instance of apples and oranges.
> Please use clisp -C
If you would have read my own reply to that message, you would have
seen that I corrected myself shortly after making that post.
<snip>
> (MAKE-BROADCAST-STREAM) is the CL idiom for unneeded output.
This I didn't know. Thanks.
> One can indeed observe that 8 out of 18 seconds are spent doing i/o in
> the latter case. The path to improvement is trivial. Buffering has
> been known for decades, and it's trivial to add as a user of CLISP
> (witness :buffered option to OPEN).
CMUCL doesn't seem to have the :buffered keyword to open. Perhaps it
would be a good idea?
Bj�rn
·········@mindspring.com (John Proctor) writes:
> This version now runs in 22 seconds compared to the original 45
> seconds. However, the ruby version runs in 6 seconds, and I didn't
> have to compile it, declare optimizations or give up formatted I/O
> to get the performance.
>
> I am very much interested in exploring this further because cmucl is
> much faster than ruby/python/perl at operations that do not involve
> formatting strings or using I/O. However, strings and I/O are a
> hugely important aspect of most applications. All business related
> apps are heavy on these 2 ops. Just think of a Web Server.
> Formatted output is about all it does.
That's a pretty silly argument. On my laptop the Perl version takes
about 8.3 seconds while the Lisp version with CMUCL 18e is at 2.8
seconds. That's _exactly_ your code with just *PRINT-PRETTY* set to
NIL[1]. Even if I take the OPTIMIZE declarations away it's still at
8.2 seconds.[2]
Now, your program generates 13,631,475 Bytes of output. I have a 1.5
MBit/s DSL line at home and downloading this amount of data from a
server with a dedicated 2 MBit/s line which had nothing to do but
serve this one file took more than 90 seconds.
So, you think it is important to save some tenths of a second under
these circumstances? Hmm, tell that to someone who has to use a
modem... :)
Edi.
[1] I think that Duane Rettig's remark was actually the most important
contribution in this thread. I had almost forgotten that about the
same thing happened to me when I created a very big and
complicated HTML table with a predecessor of my CL-WHO
library. The code took about 12 seconds and I was really
desperate. Douglas Crosher advised me to set *PRINT-PRETTY* to NIL
and, boom, I was at 0.8 seconds... :)
[2] Granted, the C version only takes 0.6 seconds but one type
declaration and using characters ("%C") instead of strings will
bring CMUCL down to 1.7 seconds here. Still one second more, but
you'll have a vastly superior development environment - or do you
argue in favor of writing web apps in C?
In article <··············@bird.agharta.de>, ···@agharta.de wrote:
> [2] Granted, the C version only takes 0.6 seconds but one type
> declaration and using characters ("%C") instead of strings will
> bring CMUCL down to 1.7 seconds here. Still one second more, but
> you'll have a vastly superior development environment - or do you
> argue in favor of writing web apps in C?
If you're in a large-volume production environment it's not 1.0 seconds
difference, it's 200% more resources required. That can be a significant
difference.
Mind you, I'm a Lisp fan, and there are many situations where people worry
about a second or two where it's really not appropriate. But a blanket
statement that a good development environment trumps a second faster every
time is not true.
E.
···@jpl.nasa.gov (Erann Gat) writes:
> In article <··············@bird.agharta.de>, ···@agharta.de wrote:
>
> > [2] Granted, the C version only takes 0.6 seconds but one type
> > declaration and using characters ("%C") instead of strings will
> > bring CMUCL down to 1.7 seconds here. Still one second more, but
> > you'll have a vastly superior development environment - or do you
> > argue in favor of writing web apps in C?
>
> If you're in a large-volume production environment it's not 1.0
> seconds difference, it's 200% more resources required. That can be
> a significant difference.
>
> Mind you, I'm a Lisp fan, and there are many situations where people
> worry about a second or two where it's really not appropriate. But
> a blanket statement that a good development environment trumps a
> second faster every time is not true.
You made this "a blanket statement that a good development environment
trumps a second faster every time" by quoting it out of context. I
haven't said anything like that and I can't see why you're trying to
make it look like I did.
Edi.
In article <··············@bird.agharta.de>, ···@agharta.de wrote:
> ···@jpl.nasa.gov (Erann Gat) writes:
>
> > In article <··············@bird.agharta.de>, ···@agharta.de wrote:
> >
> > > [2] Granted, the C version only takes 0.6 seconds but one type
> > > declaration and using characters ("%C") instead of strings will
> > > bring CMUCL down to 1.7 seconds here. Still one second more, but
> > > you'll have a vastly superior development environment - or do you
> > > argue in favor of writing web apps in C?
> >
> > If you're in a large-volume production environment it's not 1.0
> > seconds difference, it's 200% more resources required. That can be
> > a significant difference.
> >
> > Mind you, I'm a Lisp fan, and there are many situations where people
> > worry about a second or two where it's really not appropriate. But
> > a blanket statement that a good development environment trumps a
> > second faster every time is not true.
>
> You made this "a blanket statement that a good development environment
> trumps a second faster every time" by quoting it out of context. I
> haven't said anything like that and I can't see why you're trying to
> make it look like I did.
Well, you asked "... do you argue in favor of writing web apps in C?" It
seemed to me that you posed this question rhetorically and you intended
the "correct" answer to be "no". My point was just that under some
circumstances a "yes" answer is defensible.
E.
Edi Weitz <···@agharta.de> wrote in message news:<··············@bird.agharta.de>...
> That's a pretty silly argument. On my laptop the Perl version takes
> about 8.3 seconds while the Lisp version with CMUCL 18e is at 2.8
> seconds. That's _exactly_ your code with just *PRINT-PRETTY* set to
> NIL[1]. Even if I take the OPTIMIZE declarations away it's still at
> 8.2 seconds.[2]
Is that redirecting stdout to a file or /dev/null? I cannot get
anywhere near that unless I use with-open-file instead of printing to
stdout and redirecting. CMUCL seems to know it is bound to a tty and
is doing something different.
> So, you think it is important to save some tenths of a second under
> these circumstances? Hmm, tell that to someone who has to use a
> modem... :)
It is not tenths of a second. It is still over 3.5 times slower than
the C version. However, before the discussion of *print-pretty* came
up, it was 20 times slower than the C version.
> [1] I think that Duane Rettig's remark was actually the most important
> contribution in this thread. I had almost forgotten that about the
> same thing happened to me when I created a very big and
> complicated HTML table with a predecessor of my CL-WHO
> library. The code took about 12 seconds and I was really
> desperate. Douglas Crosher advised me to set *PRINT-PRETTY* to NIL
> and, boom, I was at 0.8 seconds... :)
I question that number. I do not believe that writing to stdout and
redirecting to a file for the posted code only takes 0.8 seconds. I
don't even believe that using with-open-file only takes .8 seconds.
2.8 maybe. How exactly are you running this? What OS? What CPU?
> [2] Granted, the C version only takes 0.6 seconds but one type
> declaration and using characters ("%C") instead of strings will
> bring CMUCL down to 1.7 seconds here. Still one second more, but
> you'll have a vastly superior development environment - or do you
> argue in favor of writing web apps in C?
The purpose here was not to compare C to CL as an environment. C
executes fast under *nix. If it was as productive as CL then I would
not be interested in this problem. I think CL could be a great
environment. However, I also think that many people believe that
their code is as fast as it could be and as we can see from this
discussion many people are probably running CL apps that do not
perform as well as ruby and python out of the box.
Bottom line. For this example, CMUCL is about 3.5 times slower than C
on both of my boxes and faster than all of the other languages except
bigloo. That is very acceptable. I would trade a 3.5X performance
any day to keep from using C for high level apps.
I would still like to know why format under CMUCL is much slower when
bound to a tty and redirected versus writing directly to a file.
clisp does not have this problem as do none of the other languages.
In article <····························@posting.google.com>, John Proctor wrote:
> Edi Weitz <···@agharta.de> wrote in message news:<··············@bird.agharta.de>...
> > That's a pretty silly argument. On my laptop the Perl version takes
> > about 8.3 seconds while the Lisp version with CMUCL 18e is at 2.8
> > seconds. That's _exactly_ your code with just *PRINT-PRETTY* set to
> > NIL[1]. Even if I take the OPTIMIZE declarations away it's still at
> > 8.2 seconds.[2]
>
> Is that redirecting stdout to a file or /dev/null? I cannot get
> anywhere near that unless I use with-open-file instead of printing to
> stdout and redirecting. CMUCL seems to know it is bound to a tty and
> is doing something different.
I too like silly benchmarks :-)
Here are some more observations:
1) when printing to stdout, CMUCL flushes its output buffer
at every end of line which would be every 13 characters
in this case. Output to a stream uses buffers of 4096
bytes (in the version I use) so obvioulsy there is a gain.
2) CMUCL seems to be able to do some other optimizations when
it knows that it's printing to a file. This version of print-move
(defun print-move1 (ofs from to)
(declare (type base-char from to)
(type (or stream (member t)) ofs)
(optimize (speed 3) (safety 0) (debug 0)))
(if (eq ofs t)
(format t "Move ~C to ~C.~%" from to)
(format ofs "Move ~C to ~C.~%" from to)))
is 1.7 times faster than this one:
(defun print-move2 (ofs from to)
(declare (type base-char from to)
(type (or stream (member t)) ofs)
(optimize (speed 3) (safety 0) (debug 0)))
(format ofs "Move ~C to ~C.~%" from to))
...which conses a lot (the version that knows it's printing
to a stream doesn't cons noticeably)
By the way *print-pretty* has no effect once the format
string contains no ~A
3) The version with the "unrolled" format:
(defun print-move (ofs from to)
(declare (type base-char from to)
(type (or stream (member t)) ofs)
(optimize (speed 3) (safety 0) (debug 0)))
(if (eq ofs t)
(format t "Move ~C to ~C.~%" from to)
(progn (write-string "Move " ofs)
(write-char from ofs)
(write-string " to " ofs)
(write-char to ofs)
(write-string "." ofs)
(write-char #\Linefeed ofs))))
Isn't any faster. So once format knows it's going to write
strings and chars, the dominant factor becomes write-string.
Indeed, removing all the write-string:
(defun print-move4 (ofs from to)
(declare (type base-char from to)
(type (or stream (member t)) ofs)
(optimize (speed 3) (safety 0) (debug 0)))
(if (eq ofs t)
(format t "Move ~C to ~C.~%" from to)
(progn ;(write-string "Move " ofs)
(write-char from ofs)
;(write-string " to " ofs)
(write-char to ofs)
;(write-string "." ofs)
(write-char #\Linefeed ofs))))
makes the performance of print-move4 comparable to an equivalent
C function (2 times slower or better). My hunch is that write-string has
to work for al types of strings, and isn't optimized for the
simple-base-string case and ends up writing characters one by one in a
loop. ... yep it seems to be the case, see the internal function
STREAM-WRITE-STRING.
> > So, you think it is important to save some tenths of a second under
> > these circumstances? Hmm, tell that to someone who has to use a
> > modem... :)
>
> It is not tenths of a second. It is still over 3.5 times slower than
> the C version. However, before the discussion of *print-pretty* came
> up, it was 20 times slower than the C version.
>
I think it's a good thing that people do these mini benchmarks of various
Lisp implementations from time to time. Don't worry about those who tell
you that you don't need I/O to be fast. I'm sure you're not the only one
for whom I/O performance , and if you don't compare it to other languages,
well, people will never know whether .
As to whether write-string can be improved... CMUCL is free and the
implementors must have lost to do, and you may need to tackle the problem
yourself. Fortunately I've found that the source code of CMUCL is easy to
understand in general. Many library functions don't use any magic, they're
just plain old lisp, and you can make your own copy of an individual
function and play with it, even sometimes redefine an internal function
when lisp is running.
--
Eric Daniel
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
Eric Daniel <···········@barberic.vancouver.wa.us> wrote in message news:<············@corp.newsgroups.com>...
> I think it's a good thing that people do these mini benchmarks of various
> Lisp implementations from time to time. Don't worry about those who tell
> you that you don't need I/O to be fast. I'm sure you're not the only one
> for whom I/O performance , and if you don't compare it to other languages,
> well, people will never know whether .
Yeah, I agree, I think it is good to do this sometimes. I get the
feeling that alot of CMUCL users are quietly checking their code
because of this. While, CMUCL is quite fast in this little test, it
was not until *print-pretty* was set to nil and buffering was enabled
for *standard-output*. I suspect alot of people may not have known
this or forgotten and were happy thinking that their lisp code was
fast, when in reality it was running very slow.
> As to whether write-string can be improved... CMUCL is free and the
> implementors must have lost to do, and you may need to tackle the problem
> yourself. Fortunately I've found that the source code of CMUCL is easy to
> understand in general. Many library functions don't use any magic, they're
> just plain old lisp, and you can make your own copy of an individual
> function and play with it, even sometimes redefine an internal function
> when lisp is running.
I am sure it could be improved, but at this point, it seems quite
adequate to me. I am just starting to learn CL, but, my main
interest in CL was performance. There are many languages at this
point that offer good high level development (Ruby and Python being
noted in this discussion), but their performance is not always
adequate. So, I was looking for a language that had good performance
like C, but quick development cycles like Ruby.
·········@mindspring.com (John Proctor) writes:
> Edi Weitz <···@agharta.de> wrote in message news:<··············@bird.agharta.de>...
> > That's a pretty silly argument. On my laptop the Perl version
> > takes about 8.3 seconds while the Lisp version with CMUCL 18e is
> > at 2.8 seconds. That's _exactly_ your code with just
> > *PRINT-PRETTY* set to NIL[1]. Even if I take the OPTIMIZE
> > declarations away it's still at 8.2 seconds.[2]
>
> Is that redirecting stdout to a file or /dev/null? I cannot get
> anywhere near that unless I use with-open-file instead of printing
> to stdout and redirecting. CMUCL seems to know it is bound to a tty
> and is doing something different.
WITH-OPEN-FILE, see below.
> > So, you think it is important to save some tenths of a second
> > under these circumstances? Hmm, tell that to someone who has to
> > use a modem... :)
>
> It is not tenths of a second.
It's tenths of a second compared to Perl on my machine. Please read
what I wrote. I consider Perl to be a viable alternative for web
development. As opposed to, say, Erann Gat I wouldn't even think about
writing web apps in C.[1]
> It is still over 3.5 times slower than the C version.
The whole point of my article was that all this doesn't matter as long
as you are sending 12 MB of data to a web browser. It was you who
brought up web apps, not me.
> However, before the discussion of *print-pretty* came up, it was 20
> times slower than the C version.
>
> > [1] I think that Duane Rettig's remark was actually the most important
> > contribution in this thread. I had almost forgotten that about the
> > same thing happened to me when I created a very big and
> > complicated HTML table with a predecessor of my CL-WHO
> > library. The code took about 12 seconds and I was really
> > desperate. Douglas Crosher advised me to set *PRINT-PRETTY* to NIL
> > and, boom, I was at 0.8 seconds... :)
>
> I question that number. I do not believe that writing to stdout and
> redirecting to a file for the posted code only takes 0.8 seconds. I
> don't even believe that using with-open-file only takes .8 seconds.
> 2.8 maybe.
Either you can't read or you are _extremely_ rude. Please read the
above paragraph again. I was talking about code you have never seen
writing data you have never seen. What on earth makes you think you
can "question" these numbers? What on earth makes you think you are in
a position to talk about a program you have never seen? How can you
expect others to believe and discuss what you've posted if you in turn
are only willing to accept things that fit with your assumptions?
Sigh...
Had you been a bit more polite and open-minded I would have offered
you to log into my machine and try it yourself but now I don't see why
I should do this.
> How exactly are you running this? What OS? What CPU?
See below for the code. OS is Gentoo Linux 1.1a. IBM Thinkpad T23
laptop (Pentium III 1.2 GHz, 768 MB RAM).
> > [2] Granted, the C version only takes 0.6 seconds but one type
> > declaration and using characters ("%C") instead of strings will
> > bring CMUCL down to 1.7 seconds here. Still one second more, but
> > you'll have a vastly superior development environment - or do you
> > argue in favor of writing web apps in C?
>
> The purpose here was not to compare C to CL as an environment.
For me it was. I think it is silly not to look at the whole picture
but to base your decision on some micro-benchmarks.
> C executes fast under *nix. If it was as productive as CL then I
> would not be interested in this problem. I think CL could be a
> great environment. However, I also think that many people believe
> that their code is as fast as it could be and as we can see from
> this discussion many people are probably running CL apps that do not
> perform as well as ruby and python out of the box.
Thanks for making the world a better place - the whole CL community
owes you.
> Bottom line. For this example, CMUCL is about 3.5 times slower than
> C on both of my boxes and faster than all of the other languages
> except bigloo. That is very acceptable. I would trade a 3.5X
> performance any day to keep from using C for high level apps.
We definitely agree here. FWIW, the factor is usually much better than
3.5.
> I would still like to know why format under CMUCL is much slower
> when bound to a tty and redirected versus writing directly to a
> file. clisp does not have this problem as do none of the other
> languages.
I think others have explained this already.
Edi.
[1] Yes, sure, C will be a bit (or more than a bit) faster in many
cases. But for any moderately complex application this can usually
be reduced to two or three bottlenecks. If, _after_ you have
managed to generate code that does what you want, you still can't
accept your timings, you always have the chance to call into C at
these places. And, again, we're talking about web applications
here. Let me re-iterate that these miniscule differences almost
never matter compared to the overhead of sending the data to the
client. I guess that more than 90% of the world's dynamic web
applications are written in Perl, PHP, Python, Java, or some
strange MS language all of which are a lot slower than C. Why
don't they use C then?
···@bird:/tmp > cat hanoi.pl
#!/usr/bin/perl
sub tower_of_hanoi
{
my ($n, $from, $to, $spare) = @_;
if ($n == 1) {
print_move($from, $to);
}
else {
tower_of_hanoi($n - 1, $from, $spare, $to);
print_move($from, $to);
tower_of_hanoi($n - 1, $spare, $to, $from);
}
}
sub print_move
{
my ($from, $to) = @_;
printf("Move %s to %s.\n", $from, $to);
}
tower_of_hanoi(20, "A", "B", "C");
···@bird:/tmp > time perl hanoi.pl > /tmp/foo
real 0m8.849s
user 0m8.197s
sys 0m0.086s
···@bird:/tmp > time perl hanoi.pl > /tmp/foo
real 0m8.462s
user 0m8.198s
sys 0m0.088s
···@bird:/tmp > time perl hanoi.pl > /tmp/foo
real 0m8.323s
user 0m8.197s
sys 0m0.084s
···@bird:/tmp > cat hanoi.c
#include <stdio.h>
void tower_of_hanoi(int n, char from, char to, char spare);
void print_move(char from, char to);
void tower_of_hanoi(int n, char from, char to, char spare)
{
if (n == 1)
print_move(from, to);
else
{
tower_of_hanoi(n - 1, from, spare, to);
print_move(from, to);
tower_of_hanoi(n - 1, spare, to, from);
}
}
void print_move(char from, char to)
{
printf("Move %c to %c.\n", from, to);
}
int main(int argc, char *argv[])
{
tower_of_hanoi(20, 'A', 'B', 'C');
}
···@bird:/tmp > gcc -O3 -o hanoi hanoi.c
···@bird:/tmp > time ./hanoi > /tmp/foo
real 0m0.645s
user 0m0.544s
sys 0m0.089s
···@bird:/tmp > time ./hanoi > /tmp/foo
real 0m0.645s
user 0m0.552s
sys 0m0.083s
···@bird:/tmp > time ./hanoi > /tmp/foo
real 0m0.643s
user 0m0.566s
sys 0m0.067s
···@bird:/tmp > cat hanoi.lisp
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(format ofs "~%Move ~a to ~a." from to))
(defun foo ()
(let ((*print-pretty* nil))
(with-open-file (s "/tmp/foo" :direction :output :if-exists :supersede)
(tower-of-hanoi s 20 "A" "B" "C"))))
(defun tower-of-hanoi-2 (n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(declare (fixnum n))
(cond ((= n 1)
(print-move-2 from to))
(t
(tower-of-hanoi-2 (1- n) from spare to)
(print-move-2 from to)
(tower-of-hanoi-2 (1- n) spare to from))))
(defun print-move-2 (from to)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(format t "~%Move ~C to ~C." from to))
(defun foo-2 ()
(let ((*print-pretty* nil))
(with-open-file (*standard-output* "/tmp/foo" :direction :output :if-exists :supersede)
(tower-of-hanoi-2 20 #\A #\B #\C))))
···@bird:/tmp > cmucl
; Loading #p"/home/edi/.cmucl-init".
CMU Common Lisp 18e, running on bird.agharta.de
With core: /usr/local/lib/cmucl/lib/lisp.core
Dumped on: Thu, 2003-04-03 15:47:12+02:00 on orion
See <http://www.cons.org/cmucl/> for support information.
Loaded subsystems:
Python 1.1, target Intel x86
CLOS 18e (based on PCL September 16 92 PCL (f))
* (compile-file "hanoi.lisp" :load t)
;;; [Lots of compiler messages snipped]
; hanoi.x86f written.
; Compilation finished in 0:00:00.
; Loading #p"/tmp/hanoi.x86f".
#p"/tmp/hanoi.x86f"
T
NIL
* (gc :full t)
NIL
* (time (foo))
; Compiling LAMBDA NIL:
; Compiling Top-Level Form:
; Evaluation took:
; 2.76 seconds of real time
; 2.594727 seconds of user run time
; 0.144531 seconds of system run time
; 3,308,956,542 CPU cycles
; 0 page faults and
; 16,778,088 bytes consed.
;
NIL
* (gc :full t)
NIL
* (time (foo))
; Compiling LAMBDA NIL:
; Compiling Top-Level Form:
; Evaluation took:
; 2.76 seconds of real time
; 2.604492 seconds of user run time
; 0.136719 seconds of system run time
; 3,311,256,196 CPU cycles
; 0 page faults and
; 16,778,088 bytes consed.
;
NIL
* (gc :full t)
NIL
* (time (foo))
; Compiling LAMBDA NIL:
; Compiling Top-Level Form:
; Evaluation took:
; 2.76 seconds of real time
; 2.603516 seconds of user run time
; 0.136718 seconds of system run time
; 3,307,588,254 CPU cycles
; 0 page faults and
; 16,778,088 bytes consed.
;
NIL
* (gc :full t)
NIL
* (time (foo-2))
; Compiling LAMBDA NIL:
; Compiling Top-Level Form:
; Evaluation took:
; 1.69 seconds of real time
; 1.572266 seconds of user run time
; 0.106445 seconds of system run time
; 2,030,741,460 CPU cycles
; 0 page faults and
; 888 bytes consed.
;
NIL
* (gc :full t)
NIL
* (time (foo-2))
; Compiling LAMBDA NIL:
; Compiling Top-Level Form:
; Evaluation took:
; 1.7 seconds of real time
; 1.566406 seconds of user run time
; 0.112305 seconds of system run time
; 2,031,299,640 CPU cycles
; 0 page faults and
; 888 bytes consed.
;
NIL
* (gc :full t)
NIL
* (time (foo-2))
; Compiling LAMBDA NIL:
; Compiling Top-Level Form:
; Evaluation took:
; 1.7 seconds of real time
; 1.586914 seconds of user run time
; 0.09375 seconds of system run time
; 2,030,774,661 CPU cycles
; 0 page faults and
; 888 bytes consed.
;
NIL
In article <··············@bird.agharta.de>, ···@agharta.de wrote:
> As opposed to, say, Erann Gat I wouldn't even think about
> writing web apps in C.[1]
Actually, I would never write a web app in C. If I had an application
that was high-volume enuogh to justify being written in C I would hire
someone to write it in C for me.
> I guess that more than 90% of the world's dynamic web
> applications are written in Perl, PHP, Python, Java, or some
> strange MS language all of which are a lot slower than C. Why
> don't they use C then?
Perhaps because 90% of the world's dynamic web applications are not
high-volume enough to recoup the extra cost of developing in C.
E.
Erann Gat wrote:
> In article <··············@bird.agharta.de>, ···@agharta.de wrote:
>
>
>>As opposed to, say, Erann Gat I wouldn't even think about
>>writing web apps in C.[1]
>
>
> Actually, I would never write a web app in C. If I had an application
> that was high-volume enuogh to justify being written in C I would hire
> someone to write it in C for me.
>
>
>> I guess that more than 90% of the world's dynamic web
>> applications are written in Perl, PHP, Python, Java, or some
>> strange MS language all of which are a lot slower than C. Why
>> don't they use C then?
>
>
> Perhaps because 90% of the world's dynamic web applications are not
> high-volume enough to recoup the extra cost of developing in C.
Is "high-volume" even the right description of the quality that would
repay development in C? As others have pointed out, in many typical
applications bandwidth is your limiting factor: fairly ordinary machines
can easily saturate a 1-Mb/s connection. If you're up in T3 territory or
above (or on a LAN), replicating the hardware will still likely be
cheaper than programmer time. (Note that this ties nicely back to the
newbie discussion about the Lisp philosophy of minimizing total cost by
minimizing spending on programmer time.)
Where development in C might repay the cost is in applications that both
perform excessive amounts of computation per byte served and manage not
to be I/O-bound on the disk side. How many of those are there?
paul
Paul Wallich <··@panix.com> wrote in message news:<············@reader1.panix.com>...
> Is "high-volume" even the right description of the quality that would
> repay development in C? As others have pointed out, in many typical
> applications bandwidth is your limiting factor: fairly ordinary machines
> can easily saturate a 1-Mb/s connection.
It would probably almost never pay to write a web app in C at this
point. But, I have to disagree that saturating the pipe is not the
problem of most web based enterprise applications. Server performance
is the issue. That is why, despite the fact that an Intel 486 could
probably saturate a T-1, that it is not a good web server. It is too
slow to process real requests for a large number of users. And, slow
code can make a Sun E10K seem like a 486 even when attached to an OC-3
with alot of users. Of course with only a few users even bad code on
a big box can still saturate the pipe.
Take a call center trouble ticket system for a large company. They
may have 300 concurrent users. Any request from the user is going to
result in a relatively small amount of data compared to the burst
capabilities of most internet connections. Even for this many users,
it is unlikely to ever even come close to saturating a T-1.
So, what matters? The amount of time it takes the server to process
each request. What affects this? The amount of time that it takes to
get the requested information from the data source, format it and
write it to the server side of the pipe. The faster this operation
is, the quicker the user gets the requested information and/or the
number of users that any given server can service appropriately.
In article <····························@posting.google.com>,
·········@mindspring.com (John Proctor) wrote:
> Paul Wallich <··@panix.com> wrote in message
news:<············@reader1.panix.com>...
>
> > Is "high-volume" even the right description of the quality that would
> > repay development in C? As others have pointed out, in many typical
> > applications bandwidth is your limiting factor: fairly ordinary machines
> > can easily saturate a 1-Mb/s connection.
>
> It would probably almost never pay to write a web app in C at this
> point.
But when it does the payback can be substantial. Google, for example, has
tens of thousands of servers in production. Let's say it's 10,000, at
$1000 each (both low estimates). A factor of 2 in performance immediately
pays back $10 million. And that does not take into account that the
actual cost of maintaining a production server is much higher than the
acquisition cost.
E.
···@jpl.nasa.gov (Erann Gat) writes:
> > It would probably almost never pay to write a web app in C at this
> > point.
>
> But when it does the payback can be substantial. Google, for example, has
> tens of thousands of servers in production. Let's say it's 10,000, at
> $1000 each (both low estimates). A factor of 2 in performance immediately
> pays back $10 million. And that does not take into account that the
> actual cost of maintaining a production server is much higher than the
> acquisition cost.
That's true, but Google is the single biggest web application in the
world, probably by a large factor. And developer time is very
expensive. So your numbers say to me that for the vast majority of
cases, the tradeoffs favor developer efficiency over run time
efficiency. And every year the tradeoff becomes more compelling
because of Moore's law!!
-russ
* ···@jpl.nasa.gov (Erann Gat)
| In article <····························@posting.google.com>,
| ·········@mindspring.com (John Proctor) wrote:
|
| > Paul Wallich <··@panix.com> wrote in message
| news:<············@reader1.panix.com>...
| >
| > > Is "high-volume" even the right description of the quality that would
| > > repay development in C? As others have pointed out, in many typical
| > > applications bandwidth is your limiting factor: fairly ordinary machines
| > > can easily saturate a 1-Mb/s connection.
| >
| > It would probably almost never pay to write a web app in C at this
| > point.
|
| But when it does the payback can be substantial. Google, for example, has
| tens of thousands of servers in production. Let's say it's 10,000, at
| $1000 each (both low estimates). A factor of 2 in performance immediately
| pays back $10 million. And that does not take into account that the
| actual cost of maintaining a production server is much higher than the
| acquisition cost.
Let's do the numbers the other way around. If you can get by
using 20 developers intead of 30 (at $50,000 each) by using CL
instead of C then you can buy 500 more servers.
Anyway, I don't think that 10,000 servers running the exact
same code is a typical setup for almost any company.
--
Eduardo Mu�oz | (prog () 10 (print "Hello world!")
http://213.97.131.125/ | 20 (go 10))
···@jpl.nasa.gov (Erann Gat) writes:
> In article <··············@bird.agharta.de>, ···@agharta.de wrote:
>
> > As opposed to, say, Erann Gat I wouldn't even think about
> > writing web apps in C.[1]
>
> Actually, I would never write a web app in C. If I had an application
> that was high-volume enuogh to justify being written in C I would hire
> someone to write it in C for me.
I write web apps in C, from time to time. Mostly because I prefer C to
perl (I still haven't managed to make perl fit my mind or my mind to
fit perl) and especially when I'm interfacing to C libraries (perl FFI
looks scary). The one problem I had, I solved by writing a
sufficiently-general input/output layer, in C.
//Ingvar (not a programmer)
--
(defun m (f)
(let ((db (make-hash-table :key #'equal)))
#'(lambda (&rest a)
(or (gethash a db) (setf (gethash a db) (apply f a))))))
Edi Weitz <···@agharta.de> wrote in message news:<··············@bird.agharta.de>...
> The whole point of my article was that all this doesn't matter as long
> as you are sending 12 MB of data to a web browser.
That is not true at all. For a web based enterprise app a user can
move 12 MB in a few hours easily. Now imagine 200 concurrent users
doing that on that webserver. If language A spends 7 times more cpu
formatting that data than language B then that could become a problem.
And at the time I made this analogy, Ruby was outperforming CMUCL for
me by 7X. However, I assumed that this was my problem, hence I posted
the question.
However, my original intent was not to start a benchmark war. I
simply took this little algorithm from scheme and tried in under CMUCL
only to find it very slow. Then, after exhausting what little
knowledge I have of CL, I posted the question.
> Either you can't read or you are _extremely_ rude. Please read the
> above paragraph again. I was talking about code you have never seen
> writing data you have never seen. What on earth makes you think you
> can "question" these numbers? What on earth makes you think you are in
> a position to talk about a program you have never seen? How can you
> expect others to believe and discuss what you've posted if you in turn
> are only willing to accept things that fit with your assumptions?
> Sigh...
I apologize. I did misread your post. I will be more careful next
time.
> [1] Yes, sure, C will be a bit (or more than a bit) faster in many
> cases. But for any moderately complex application this can usually
> be reduced to two or three bottlenecks. If, _after_ you have
> managed to generate code that does what you want, you still can't
> accept your timings, you always have the chance to call into C at
> these places. And, again, we're talking about web applications
> here. Let me re-iterate that these miniscule differences almost
> never matter compared to the overhead of sending the data to the
> client. I guess that more than 90% of the world's dynamic web
> applications are written in Perl, PHP, Python, Java, or some
> strange MS language all of which are a lot slower than C. Why
> don't they use C then?
I never once advocated using C to write web apps. Hardware is
generally cheaper than developer time.
John Proctor <·········@mindspring.com> wrote:
+---------------
| Edi Weitz <···@agharta.de> wrote in message
|
| > The whole point of my article was that all this doesn't matter
| > as long as you are sending 12 MB of data to a web browser.
|
| That is not true at all. For a web based enterprise app a user can
| move 12 MB in a few hours easily. Now imagine 200 concurrent users
| doing that on that webserver...
+---------------
Sheesh! You're consistently missing Edi's rather obvious point!!
As long as your web based enterprise app can saturate your Internet
connection using *anything* less than 100% of the CPU, it doesn't
*matter* how many users you pile on it -- they're all going to
bottleneck on the outgoing network pipe, leaving some of the CPU
idle anyway!!
"Good enough" means being able to fill whatever speed pipe you can
afford, and once you can do that, well, then it's good enough.
It only takes a modest amount of tuning to saturate, say, a 10 Mb/s pipe.
Running Edi's last version on my Athlon 1600+ (1.4GHz), it wrote a
13.6 MB file in 1.27 seconds of real time (1.19 sec. CPU time).
That's over 10 MB/s -- over 85 Mb/s! -- with is a lot more Internet
bandwidth than I'll ever see.
-Rob
p.s. My current web server chugs along very happily with only
768 kb/s (SDSL), which is why (for example) I don't really worry
about the performance differences between HTOUT & CL-WHO [see
the recent "html generation API" thread].
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) wrote in message news:<······················@speakeasy.net>...
> Sheesh! You're consistently missing Edi's rather obvious point!!
> As long as your web based enterprise app can saturate your Internet
> connection using *anything* less than 100% of the CPU, it doesn't
> *matter* how many users you pile on it -- they're all going to
> bottleneck on the outgoing network pipe, leaving some of the CPU
> idle anyway!!
That is not a completely accurate assessment. The original problem
was with format, at it was extremely slow for me. Check the original
post by me for times. Creating formatted strings in a web server is a
very common operation. Creating html, xml, sql queries, reports,
logs, etc. There is alot more going on than just pumping data to the
users. In my original post, CMUCL was 7 times slower than Ruby for
me. Are you saying that a 7X performance loss of formatted IO in a
heavily loaded app server would have absolutely no difference on the
performance off the app relative to the users? I think that it would.
Right now, our app server at work runs about 50 percent saturated with
200 users. If I increased the time it takes to do formatted string IO
by 7X then I suspect that the server would come to a crawl. Do you
think not?
However, the original question had nothing to do with web apps. It
was merely a performance question on IO with format. I think someone
else asked why IO performance even mattered. I replied with the web
server analogy because of the amound of formatted IO web based app
servers do.
·········@mindspring.com (John Proctor) writes:
> The original problem was with format, at it was extremely slow for
> me. Check the original post by me for times. Creating formatted
> strings in a web server is a very common operation. Creating html,
> xml, sql queries, reports, logs, etc. There is alot more going on
> than just pumping data to the users. In my original post, CMUCL was
> 7 times slower than Ruby for me. Are you saying that a 7X
> performance loss of formatted IO in a heavily loaded app server
> would have absolutely no difference on the performance off the app
> relative to the users? I think that it would.
Your example is still flawed. Yes, the CL implementations tested so
far seem to be quite slow in formatted I/O if you use the default
settings and compare them to languages where "formatted I/O" means
something entirely different and less capable, but your
micro-benchmark was obviously tuned to measure I/O performance and
nothing else. Your little Hanoi program virtually does _nothing_
except printing to a stream.
A "web application" that does nothing but printing to a socket is
called a static file - use Apache to serve it... :)
Real web apps will do a lot more than that and in my experience the
difference in I/O speed is almost always negligible. Not to forget
that Lisp-y setups like Apache/mod_lisp, AllegroServe, CL-HHTP, or
Araneida have other advantages as well like the ability to keep state
between request in RAM which is much more difficult to achieve in
mod_perl, PHP, or other more "standard" web app toolkits.
Edi.
* Edi Weitz <···@agharta.de>
| [1] I think that Duane Rettig's remark was actually the most important
| contribution in this thread. I had almost forgotten that about the
| same thing happened to me when I created a very big and
| complicated HTML table with a predecessor of my CL-WHO
| library. The code took about 12 seconds and I was really
| desperate. Douglas Crosher advised me to set *PRINT-PRETTY* to NIL
| and, boom, I was at 0.8 seconds... :)
And I want to thank you and Duane by bringing up the issue
of html generation + *print-pretty*. I was a bit worried
about my server consing too much and now look at the
numbers:
* (time (test-apache-address "/lisp/weblog"))
Compiling LAMBDA NIL:
Compiling Top-Level Form:
Evaluation took:
0.13 seconds of real time
0.12 seconds of user run time
0.0 seconds of system run time
0 page faults and
410992 bytes consed.
"<!DOCTYPE HTML PUBLIC \"-//W3C//...."
* (time (let (*print-pretty*) (test-apache-address "/lisp/weblog")))
Compiling LAMBDA NIL:
Compiling Top-Level Form:
Evaluation took:
0.04 seconds of real time
0.03 seconds of user run time
0.01 seconds of system run time
0 page faults and
66880 bytes consed.
"<!DOCTYPE HTML PUBLIC \"-//W3C//...."
Playing with ApacheBench I see that the number of requests
per second went from 9 to 50 :)
--
Eduardo Mu�oz | (prog () 10 (print "Hello world!")
http://213.97.131.125/ | 20 (go 10))
On 24 Jul 2003 21:02:48 -0700, ·········@mindspring.com (John Proctor)
wrote:
>I am having the same problem. I came to CL because someone convinced
>me that it was much faster than some of the other languages that I
>use, Ruby, Python and Perl. Well, my first little benchmark for fun,
>tower of hanoi, didn't go to well.
It is possible to write CL programs that are efficient.
At http://www.norvig.com/Lisp-retro.html Peter Norvig writes "Lisp is
about 1.5 to 4 times faster than Java, and about 10 to 50 times faster
than Python. Lisp is probably within 20% to 60% of C/C++ in efficiency
on most tasks, which is close enough that the differences depend more
on the programmers involved than the language, and close enough that
for most applications that speed is not an issue for Lisp. ... Python
is a different story: there is a large class of problems for which
Python is too slow.".
Furthermore, the commercial Lisp applications listed at
http://www.alu.org/table/commercial-use.htm include a number of
examples where I would expect performance to be a critical factor.
As starting points, read up on what optimization criteria are offered
by your development environment of choice, also consider use of type
declarations.
Mark
Hi,
I ran your CL (with some modifications) and C-version (now C++) but
writing to a file instead of standard-output. My results are below.
I think CL compares quite well with C++.
In my (still limited) experience CMUCL is indeed slower than C/C++. But
in many cases it is only a factor of about 2. If I need ultimate
performance I have to use C/C++, but if I can I use CL (CMUCL) because
it's much faster to program in and it is more fun. Maybe it's because I
prefer interactive programming and testing over compile/link/run/bug.
This is of course a matter of taste.
Rolf Wester
---------------------------------------------------------------------
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(declare (type base-char from to))
(declare (type stream ofs))
;(format ofs "~%Move ~C to ~C." from to)
)
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(declare (type fixnum n))
(declare (type base-char from to spare))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from))))
(time (with-open-file (out "thn_lisp.txt" :direction :output)
(tower-of-hanoi out 20 #\A #\B #\C)))
With output to file
Evaluation took:
1.89f0 seconds of real time
1.61f0 seconds of user run time
0.25f0 seconds of system run time
1 page fault and
0 bytes consed.
Without output to file
Evaluation took:
0.06f0 seconds of real time
0.05f0 seconds of user run time
0.01f0 seconds of system run time
0 page faults and
0 bytes consed.
--------------------------------------
#include <fstream>
using namespace std;
void tower_of_hanoi(int n, char *from, char *to, char *spare);
void print_move(char *from, char *to);
ofstream out("toh_c.txt");
void tower_of_hanoi(int n, char *from, char *to, char *spare)
{
if (n == 1)
print_move(from, to);
else
{
tower_of_hanoi(n - 1, from, spare, to);
print_move(from, to);
tower_of_hanoi(n - 1, spare, to, from);
}
}
void print_move(char *from, char *to)
{
//out << "Move " << from << " to " << to << endl;
}
int main(int argc, char *argv[])
{
tower_of_hanoi(20, "A", "B", "C");
}
/*
with output in file:
real 0m11.302s
user 0m4.180s
sys 0m6.960s
without output in file:
real 0m0.041s
user 0m0.030s
sys 0m0.010s
*/
John Proctor wrote:
> I am having the same problem. I came to CL because someone convinced
> me that it was much faster than some of the other languages that I
> use, Ruby, Python and Perl. Well, my first little benchmark for fun,
> tower of hanoi, didn't go to well.
>
> The CL version takes 45 seconds, the Python version takes 12 seconds,
> the Perl version takes 9 seconds and the Ruby version takes just 6
> seconds. Just for fun, I did it in C also, which runs in less than 1
> second. These are not small differences.
>
> I, too, would like to know what I am missing. Why is this version so
> slow compared to all of the other languages? I am using CMUCL,
> compiling to x86f under linux. The problem is with format. If I
> remove the call to format then the speed goes to less than 1 second.
> Which is still 8 times slower than the C version without printf.
>
> Here is the CL code:
>
> (defun tower-of-hanoi (ofs n from to spare)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (cond ((= n 1) (print-move ofs from to))
> (t (tower-of-hanoi ofs (- n 1) from spare to)
> (print-move ofs from to)
> (tower-of-hanoi ofs (- n 1) spare to from))))
>
> (defun print-move (ofs from to)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (format ofs "~%Move ~a to ~a." from to))
>
> (tower-of-hanoi t 20 "A" "B" "C")
> (quit)
>
>
> Here is the Python code:
>
> def tower_of_hanoi(n, afrom, to, spare):
> if n == 1:
> print_move(afrom, to)
> else:
> tower_of_hanoi(n - 1, afrom, spare, to)
> print_move(afrom, to)
> tower_of_hanoi(n - 1, spare, to, afrom)
>
> def print_move(afrom, to):
> print("Move " + afrom + " to " + to + ".")
>
> tower_of_hanoi(20, "A", "B", "C")
>
> Here is the Perl code:
>
> sub tower_of_hanoi
> {
> my ($n, $from, $to, $spare) = @_;
> if ($n == 1) {
> print_move($from, $to);
> }
> else {
> tower_of_hanoi($n - 1, $from, $spare, $to);
> print_move($from, $to);
> tower_of_hanoi($n - 1, $spare, $to, $from);
> }
> }
>
> sub print_move
> {
> my ($from, $to) = @_;
> printf("Move %s to %s.\n", $from, $to);
> }
>
> tower_of_hanoi(20, "A", "B", "C");
>
>
> Here is the Ruby code:
>
> def tower_of_hanoi(n, from, to, spare)
> if (n == 1)
> print_move(from, to);
> else
> tower_of_hanoi(n - 1, from, spare, to);
> print_move(from, to);
> tower_of_hanoi(n - 1, spare, to, from);
> end
> end
>
> def print_move(from, to)
> printf("Move %s to %s.\n", from, to);
> end
>
> tower_of_hanoi(20, "A", "B", "C");
>
> Here is the C code:
>
> #include <stdio.h>
>
> void tower_of_hanoi(int n, char *from, char *to, char *spare);
> void print_move(char *from, char *to);
>
> void tower_of_hanoi(int n, char *from, char *to, char *spare)
> {
> if (n == 1)
> print_move(from, to);
> else
> {
> tower_of_hanoi(n - 1, from, spare, to);
> print_move(from, to);
> tower_of_hanoi(n - 1, spare, to, from);
> }
> }
>
> void print_move(char *from, char *to)
> {
> printf("Move %s to %s.\n", from, to);
> }
>
> int main(int argc, char *argv[])
> {
> tower_of_hanoi(20, "A", "B", "C");
> }
Rolf Wester <······@ilt.fraunhofer.de> wrote in message news:<············@nets3.rz.RWTH-Aachen.DE>...
> I ran your CL (with some modifications) and C-version (now C++) but
> writing to a file instead of standard-output. My results are below.
> I think CL compares quite well with C++.
Try printing directly to stdout and redirecting to a file. I think
you will find that the CL version is MUCH slower that the C++ version.
Currently the CL version is even MUCH slower than python/ruby/perl.
Also, FYI, using C++ I/O as a benchmark is not nearly the same as C.
I have had contracts in the past converting C++ apps back to using
standard C I/O because the C++ I/O model is extremely slow. Try
bencharking printf vs cout for large data on complex formatting.
John Proctor wrote:
> I am having the same problem. I came to CL because someone convinced
> me that it was much faster than some of the other languages that I
> use, Ruby, Python and Perl. Well, my first little benchmark for fun,
> tower of hanoi, didn't go to well.
>
> The CL version takes 45 seconds, the Python version takes 12 seconds,
> the Perl version takes 9 seconds and the Ruby version takes just 6
> seconds. Just for fun, I did it in C also, which runs in less than 1
> second. These are not small differences.
>
> I, too, would like to know what I am missing. Why is this version so
> slow compared to all of the other languages? I am using CMUCL,
> compiling to x86f under linux. The problem is with format. If I
> remove the call to format then the speed goes to less than 1 second.
> Which is still 8 times slower than the C version without printf.
Your version literally translated to Scheme/Bigloo is as fast as the C
version. Note: as I wrote elsewhere "Literally" except for the printing
directives:
==
(module tower)
(define (tower-of-hanoi ofs n from to spare)
(cond ((= n 1) (print-move ofs from to))
(#t (begin
(tower-of-hanoi ofs (- n 1) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (- n 1) spare to from)))))
(define (print-move ofs from to)
(print "Move " from " to " to))
(tower-of-hanoi #t 20 "A" "B" "C")
==
Use it under Bigloo:
bigloo -Obench tower.scm
time ./a.out
produces (timings in parenthes are the results from your C version and
timings in [] denote the timings of your Python version):
real: 0m8sec (0m7sec) [0m44sec]
user: 0m1.8sec (0m2.8sec) [0m36sec]
sys: 0m0sec (0m0sec) [0m0sec]
S. Gonzi
Siegfried Gonzi <···············@kfunigraz.ac.at> wrote in message news:<················@kfunigraz.ac.at>...
> Your version literally translated to Scheme/Bigloo is as fast as the C
> version. Note: as I wrote elsewhere "Literally" except for the printing
> directives:
>
> [snip]
> Use it under Bigloo:
>
> bigloo -Obench tower.scm
>
> time ./a.out
Wow. I just compiled and tested bigloo. My C version takes about .9
seconds and the bigloo version takes about 1.6 seconds. So,
technically, the C version is not quite twice as fast. That is very
impressive. Bigloo is about twice as fast as my CMUCL version writing
directly to a file with *print-pretty* set to nil.
John Proctor wrote:
> Wow. I just compiled and tested bigloo. My C version takes about .9
> seconds and the bigloo version takes about 1.6 seconds. So,
> technically, the C version is not quite twice as fast. That is very
> impressive. Bigloo is about twice as fast as my CMUCL version writing
> directly to a file with *print-pretty* set to nil.
>
Some note to Bigloo for numerical computations. The following ist just a
hint and there are no figures available how such inference would scale
up to larger problems. In reality, especially for numerical intense
calculations you will have to make a demarcation line between:
a) Are there any arrays involved
b) Do we actually have to deal with floating point calculations only
If you have arrays then Bigloo can become 2.5 times as slow as C.
Whereas OCaml can become 2 times as slow as C. See a discussion on
fa.caml list for matrix-matrix multiplications.
There is one note: Type declarations in Bigloo are way easier than under
CommonLisp with the additional goody that if you give types the compiler
really expects this type at compilation. Which means if you type your
array as floating point all the operators and accompanied expresions
must conform to this type declaration, wheres in CommonLisp you could
give somewhere type declarations but it will not improve the
performance, though, the compiler will not rebel.
And type declarations in Bigloo do not clutter up code, see the
following code snippet for the matrix-matrix multiplication. When you
give types you only annotate the operator, for example +fx means
integer. This is also transportable to another Scheme compiler even it
does not have this type mechnism:
==
(define (mmult rows cols m1 m2)
(let ((m3 (make-vector rows 0.0)))
(do ((i 0 (+fx 1 i)))
((=fx i rows))
(let ((m1i (vector-ref m1 i))
(row (make-vector cols 0.0)))
(do ((j 0 (+fx 1 j)))
((=fx j cols))
(let ((val 0.0))
(do ((k 0 (+fx 1 k)))
((=fx k cols))
(set! val (+fl val (*fl (vector-ref m1i k)
(vector-ref (vector-ref m2 k) j)))))
(vector-set! row j val)))
(vector-set! m3 i row)))
m3))
==
[Note in Bigloo you can also give all the C types, for example:
(define (fact:int x:int) ...)
This renders the compiler somehow in static type checker]
If you do not have large arrays then Bigloo without type declarations is
as fast as C. Maybe you noticed the Coyote Gulch benchmark thread on
slashdot. It is a benchmark for calculating ephemerides of the planets.
The following table is interesting, because in nowadays Java performs
very well and the produced Java code by Bigloo /is as fast as the C
code/. Bigloo has also a Java back-end. But most interestingly is the
fact that OCaml is very slow on this benchmark (see also fa.caml to get
confirmation that experts there agree on this). In ascedning order:
OCaml (by Oleg), Bigloo (by Gonzi and Manuel S.), Java JDK (original
code), Bigloo Java backend (by Gonzi and Manuel S.), gcc (original code):
==
I am wondering whether they did analyse the Bigloo (Scheme) results:
[according to Manuel based on code by S. Gonzi; see comp.lang.scheme]
Compiler usr+usr
-----------------------------------------------------------+---------------
ocamlopt -unsafe -noassert -inline 2: 95.01s
bigloo -Obench -jvm (jdk1.3.1): 55.73s
java (jdk1.3.1): 52.53s
bigloo -Obench -copt "-ffast-math -fomit-frame-pointer -O3": 40.57s
gcc -ffast-math -fomit-frame-pointer -O3: 38.37s
Btw: the Stalin compiler produces code (note: common Scheme operators)
which runs faster than the C++ version even.
==
[See also the thread:
http://groups.google.com/groups?q=Coyote+Gulch+benchmark+group:comp.lang.scheme.*&hl=en&lr=&ie=UTF-8&oe=UTF-8&group=comp.lang.scheme.*&selm=3E1BE6FA.2080901%40kfunigraz.ac.at&rnum=1
]
And a third point. As you noticed in the thread Bigloo is very fast at
writing to and reading from text files. I once wrote a C++ program for
reading large text files. The C++ program relied on the vector class
(templates), and the Bigloo version without type declarations was as
fast as the C version. I also have a Fortran 90 version where Bigloo is
as fast for reading text files as the Fortran version. See also the
thread on comp.lang scheme.
And last but not least. The days are over where Java is considered slow
for numerical computations. The following links points to a paper where
they show that a proper Java class can come close to Fortran 90:
http://www.research.ibm.com/journal/sj/391/moreira.html
/BUT/ there is one aspect not to forget. Nobody knows the behavior on
larger projects. There was once an article in the journal "Computing in
Science and Engineering" where they showed that a 20000 lines of Fortran
code translated to plain C was 5% faster on execution than the original
Fortran 77 version even. I only mention this because some people still
believe Fortran is that faster than C. Maybe on super scalar and vector
machines, but times are over at least for ordinary PCs. The
aformentioned had rised some discusission, see the rebuttal of Dan N.:
http://users.erols.com/dnagle/reply.html
Is it wise to use Scheme for scientific computing? It depends. Neither
of them have good facilities for dealing with binary data. Thanks to
Oleg for his small Scheme program for reading satellite binary data; I
use it very often.
Nobody or very few are using Scheme or CommonLisp for numerical
computations. The percentage is negligible.I use Scheme every day but I
am quite sure that Scheme (or even functional programming) will not much
improve your programming style nor is it a guarantee for better and more
reusable programs. I am 29 and plan to stay into academia and henceforth
I am way to old for better learning C++, but if I had the chance to
re-appear again I would invest all my time into C++. In reality it is
really all waht you need. On the other side: Scheme or such functional
style of programming is exactly for me (mediocore programmer and I am
proud of it), because I do not have much to fiddle around with details.
There are millions of user every day who use something like Lisp and it
is called: Mathematica, Maple, Matlab, IDL, Scilab,...
S. Gonzi
John Proctor wrote:
> Siegfried Gonzi <···············@kfunigraz.ac.at> wrote in message news:<················@kfunigraz.ac.at>...
>
>
>>Your version literally translated to Scheme/Bigloo is as fast as the C
>>version. Note: as I wrote elsewhere "Literally" except for the printing
>>directives:
>>
>>[snip]
>>Use it under Bigloo:
>>
>>bigloo -Obench tower.scm
>>
>>time ./a.out
>
>
> Wow. I just compiled and tested bigloo. My C version takes about .9
> seconds and the bigloo version takes about 1.6 seconds. So,
> technically, the C version is not quite twice as fast. That is very
> impressive. Bigloo is about twice as fast as my CMUCL version writing
> directly to a file with *print-pretty* set to nil.
Well. BigLoo is an implementation of .... BigLoo. That is another
one-implementation language (well yes, it is sort of a Scheme
implementation, but then, really, any pure Scheme implementation is not
all that useful :) BigLoo is no different: all the "extensions" to
BigLoo are not standard and are not portable to other Scheme
implementations). CMUCL, OTOH, is an implementation of Common Lisp.
So the question is for you. If you feel that CMUCL could be improved,
why not participate in the effort instead of turning to the next
language around the block?
Cheers
--
Marco Antoniotti
Marco Antoniotti wrote:
> Well. BigLoo is an implementation of .... BigLoo. That is another
> one-implementation language (well yes, it is sort of a Scheme
> implementation, but then, really, any pure Scheme implementation is not
> all that useful :) BigLoo is no different: all the "extensions" to
> BigLoo are not standard and are not portable to other Scheme
> implementations). CMUCL, OTOH, is an implementation of Common Lisp.
This is true. Often it is really annoying that every Scheme is a Scheme
for its own sake. Surely, some proponents on comp.lang.scheme say this
exactly speaks up for Scheme: every implementation is different.
In reality: every Scheme implemenation is as different as Mathematica
vs. Maple.
But lets get real: Every CommonLisp implementation tries to conform to
the huge standard, but everything beyond that is non-standard too. Lets
take Quail for example, it is only useable on Macintosh Lisp and Allegro
Common Lisp.
In the meatime even a port to Windows actually exists for Bigloo. I mean
we are now facing a situation where you likely will find a common Bigloo
implementation for: Mac OS X, Linux, Unix, Windows, (maybe some other
Unix derivates and machines too).
It is not unlikely that Bigloo will be as common as lets say gnu C,
eventually. And another important point: Bigloo is actually C! And
Bigloo can also produce Java back-ends.
Have you ever tried to incorporate your CommonLisp function into a C
program? BtW: I think the Scheme Chicken compiler has a seemingless easy
integration into C too.
I think CommonLisp needs better books. The Graham book for example is
really boring and only deals with macros. Okay, macros are key in
CommonLisp but the book becomes a boring dwarf if I compare it for
example to the Clean book. I mention this because I often try to
convince myslef to use CommonLisp exactly for the very non pleasent
situation that Scheme has many more flavors than CommonLisp but I was
never convinced this shift by myself would pay-off.
S. Gonzi
Siegfried Gonzi wrote:
> Marco Antoniotti wrote:
>
>> Well. BigLoo is an implementation of .... BigLoo. That is another
>> one-implementation language (well yes, it is sort of a Scheme
>> implementation, but then, really, any pure Scheme implementation is
>> not all that useful :) BigLoo is no different: all the "extensions"
>> to BigLoo are not standard and are not portable to other Scheme
>> implementations). CMUCL, OTOH, is an implementation of Common Lisp.
>
>
> This is true. Often it is really annoying that every Scheme is a Scheme
> for its own sake. Surely, some proponents on comp.lang.scheme say this
> exactly speaks up for Scheme: every implementation is different.
Exactly my point.
>
> In reality: every Scheme implemenation is as different as Mathematica
> vs. Maple.
I would find this a little stretching the envelope :) Or I would throw
Macsyma/Maxima in. :)
>
> But lets get real: Every CommonLisp implementation tries to conform to
> the huge standard, but everything beyond that is non-standard too.
The point is that Type Declarations are standard in CL. So are multi
dimensional arrays. I find much more interesting discussing what type
relationship should (ARRAY NIL) satisfy rather than discussing how
multidimensional arrays should be supported, if at all.
> Lets
> take Quail for example, it is only useable on Macintosh Lisp and Allegro
> Common Lisp.
I am not familiar with Quail. I do not know why it works only on MCL
and Allegro. That does not imply that the core of Quail may work on
CMUCL. My point is: why waste time on yet another Scheme implementation
when you could make Quail work under CMUCL?
> In the meatime even a port to Windows actually exists for Bigloo. I mean
> we are now facing a situation where you likely will find a common Bigloo
> implementation for: Mac OS X, Linux, Unix, Windows, (maybe some other
> Unix derivates and machines too).
Oooops. I have the same situation for CL. I fail to see your point.
> It is not unlikely that Bigloo will be as common as lets say gnu C,
> eventually. And another important point: Bigloo is actually C! And
> Bigloo can also produce Java back-ends.
>
> Have you ever tried to incorporate your CommonLisp function into a C
> program?
Like writing a COM IActiveScript interface completely in CL (LW)?
> BtW: I think the Scheme Chicken compiler has a seemingless easy
> integration into C too.
AFAIK, the Chicken compiler is a very good thing, but why wasn't it
written for CL?
> I think CommonLisp needs better books. The Graham book for example is
> really boring and only deals with macros. Okay, macros are key in
> CommonLisp but the book becomes a boring dwarf if I compare it for
> example to the Clean book. I mention this because I often try to
> convince myslef to use CommonLisp exactly for the very non pleasent
> situation that Scheme has many more flavors than CommonLisp but I was
> never convinced this shift by myself would pay-off.
I will grant you that better books for CL are sorely needed. However,
switching to CL will buy you a lot of mileage. You should definitively
try. And people here can be very helpful.
Cheers
--
Marco
From: Valery A.Khamenya
Subject: Re: why I/O is so slow?
Date:
Message-ID: <3f255745$1@shknews01>
Marco Antoniotti wrote:
> My point is: why waste time on yet another Scheme implementation
> when you could make Quail work under CMUCL?
for example, because CMUCL does not run under win32 and Bigloo does.
--
Valery
Valery A.Khamenya wrote:
> Marco Antoniotti wrote:
>
>> My point is: why waste time on yet another Scheme implementation when
>> you could make Quail work under CMUCL?
>
>
> for example, because CMUCL does not run under win32 and Bigloo does.
Your point is moot. CLisp runs under Win32 as ECL does. And this is
not counting the commercial implementations (also running under MacOS X).
Now, if you want CMUCL under Windows 32 then it is still better to work
on a port of it, than working on the godzillionth incompatible version
of Scheme. :)
Cheers
--
Marco Antoniotti
Marco Antoniotti wrote:
> The point is that Type Declarations are standard in CL. So are multi
> dimensional arrays.
What do you mean by multi dimensional arrays. One of the nasties fucking
things around Scheme is the following misbehavior (this did cost me a
whole day):
(define misbehavior (make-vector 2 (make-vector 3 0.0)))
One now thinks having created an array with 2 rows and 3 columns. But it
is not really obvious that (make-vector 3 0.0) actually is a /pointer/
to a memory adress and /every/ vector element points to this adress,
which means if you change things in the second row things in the first
row become changed too!:
(vector-set! (vector-ref misbehavior 1) 1 3.45)
results into:
#( #(0.0 3.45 0.0) #(0.0 3.45 0.0))
and not as expected:
#( #(0.0 0.0 0.0) #(0.0 3.45 0.0))
In order to avoid the aformentioned one has to explicitely create a new
object in a loop for every row. The standard for example does not
explicitely say that the first initialization of the vector elements
reads as: "If you create a vector and specifiy another vector/objetc as
filling element you should think of as filling the elements by a pointer
which points to the same storage location".
> I will grant you that better books for CL are sorely needed.
I know different strokes for different people, but I think it needs more
books which demonstrates/underlines programming at hand of math. Often
finding the root of numbers for example (as in the SICP book) is a lot
more fun. Honestly speaking I find the C book by K&R also very boring,
oh man reading this book makes you really think: what the hell went
wrong in my life that I have to read such a boring book.
> However,
> switching to CL will buy you a lot of mileage. You should definitively
> try. And people here can be very helpful.
I am not sure about ECL, but I think they go the right way down the
road, because isn't this project exactly what people always complain
about CommonLisp: hey show me a way to ship my application as a tight
stand-alone facility.
Bit of speculation and offense intented: People should kick ass CMUCL
and try to shave ECL.
I am not sure whether ECL will have problems, because it compiles to
ordinary C. In Bigloo for example one can only use the object system in
combination with the compiler. This is maybe due to the compiler and
introduces some hurdles if you want to use the object system in your
interpreter which is impossible. But on the other side I have often
wondered why people stretch the comand line interpeter (listener) in
CommonLisp that much. Think of a CommonLisp session where you have
created 10 functions and after 5 hours or so you are in command line 10
000 and have changed a lot of this functions and got messed up and have
to scroll back and down and forget what you did at function three 200
lines above. Isn't it better to have this 10 functions in a "static"
file and change things always in place?
Isn't it curious that numer crunching on CommonLisp stopped 10 years
ago. I think there were a lot of projects geared toward CommonLisp. I
have seen Fortran to CommonLisp convertes and things like that, such
convertes are not written by toy programmers and such projetcs arev
often writteny by people heavily involed into numerical analysis. I
cannot comment about Scheme, maybe it is even worse, but something must
have been happend to them that exactly that people now exclusively are
using C++.
Regards,
S. Gonzi
Siegfried Gonzi <···············@kfunigraz.ac.at> writes:
> I am not sure about ECL, but I think they go the right way down the
> road, because isn't this project exactly what people always complain
> about CommonLisp: hey show me a way to ship my application as a
> tight stand-alone facility.
No, it isn't. It's what our monthly trolls (I don't mean you) are
complaining about who wouldn't use CL anyway. ECL is a fine project
but telling people to dump CMUCL in favor of it is ridiculous to say
the least. (Not no mention that the CVS CMUCL _can_ create
'stand-alone' executables if that's the holy grail.)
Edi.
Edi Weitz wrote:
> No, it isn't. It's what our monthly trolls (I don't mean you) are
> complaining about who wouldn't use CL anyway. ECL is a fine project
> but telling people to dump CMUCL in favor of it is ridiculous to say
> the least. (Not no mention that the CVS CMUCL _can_ create
> 'stand-alone' executables if that's the holy grail.)
It is not my intention to deride the CMUCL project. I gather they are
working hard and are investing a lot of time to make the project
success, but I would be rather interested in a poll of the following:
Rationale: You are a scientist comming from Python, Perl, Ruby and
having read there on the discussion groups and lists: You guys all suck
choose CommonLisp and it can even surpass Fortran with proper declarations.
Result: All the people really trying and using CommonLisp then: are they
actually happy with the new situation? I would like to see figures of
people who thought CommonLisp is the paneace (in favor of Python for
example) due to the alledged speed with proper declarations.
Let speak them out what they had thought they might expect as someone
mentioned CommonLisp on the Python discussion group and what they then
really experienced after using it.
To make a long winded story short: How many scientist tried out
CommonLisp but didn't see any advantage of using it, because in reality
the alledged speed is not there.
I often mention Bigloo here for example. Someone now could infere Bigloo
is a very mature Scheme compiler. Indeed it is. But I have a program for
example which I have translated literally (as good as possible) to C++
(using constructors and a bit of object oriented programming). First I
thought oh man I am a very beginner in C++ and the translated program
will run very slow in C++ but to my great surprise it runs 20 times
faster than the Bigloo version. I haven't figured out yet what is going
on under the hood, maybe I used a bit too much recusrion in my Scheme
program (it searches for zeros of a Bessel function of first kind).
S. Gonzi
Siegfried Gonzi <···············@kfunigraz.ac.at> writes:
> Edi Weitz wrote:
>
> > No, it isn't. It's what our monthly trolls (I don't mean you) are
> > complaining about who wouldn't use CL anyway. ECL is a fine project
> > but telling people to dump CMUCL in favor of it is ridiculous to say
> > the least. (Not no mention that the CVS CMUCL _can_ create
> > 'stand-alone' executables if that's the holy grail.)
>
> It is not my intention to deride the CMUCL project. I gather they are
> working hard and are investing a lot of time to make the project
> success, but I would be rather interested in a poll of the following:
>
> Rationale: You are a scientist comming from Python, Perl, Ruby and
> having read there on the discussion groups and lists: You guys all
> suck choose CommonLisp and it can even surpass Fortran with proper
> declarations.
>
> Result: All the people really trying and using CommonLisp then: are
> they actually happy with the new situation? I would like to see
> figures of people who thought CommonLisp is the paneace (in favor of
> Python for example) due to the alledged speed with proper declarations.
>
> Let speak them out what they had thought they might expect as someone
> mentioned CommonLisp on the Python discussion group and what they then
> really experienced after using it.
I'm not a scientist but I came to CL after Python. I was looking for
a "lightweight" high level language to do quickie tasks; fiddle with a
bunch of records in a file, calculate something, or do a series of
calculations while I fiddled with the equation & parameters. Python
works OK for all those, but its slow and not very portable between
Windows/Unix once you get into files and sockets- that was 2 years ago
maybe its improved. Being unsatisfied by Python's speed and remaining
annoyed by the whitespace driven scoping, I gave CLISP a try. Two
days later I entirely abandoned Python.
I ultimately bought Lispworks to gain the compiler and convienent GUI
library. I add declarations where and when I need to maximize the
performance of something, otherwise I don't declare at all. I think a
big CL win is the vast standard library which allows complex programs
to be fairly portable between implementations. I have relied on that
feature a number of times when running code on Lispworks, then CLISP
and sometimes CMUCL on a varying mix of operating systems.
Gregm
Greg Menke wrote:
> I ultimately bought Lispworks to gain the compiler and convienent GUI
> library. I add declarations where and when I need to maximize the
> performance of something, otherwise I don't declare at all. I think a
> big CL win is the vast standard library which allows complex programs
> to be fairly portable between implementations. I have relied on that
> feature a number of times when running code on Lispworks, then CLISP
> and sometimes CMUCL on a varying mix of operating systems.
This is very interesting. I made a similar experience but ended up with
Scheme/Bigloo.
Personally I think Python gets too much attention especially for number
crunching.
I am still trying to figure out why Python attracts that much on people
and especially experts with a lot of experience in functional
programming. I mean it can't be that all of them are rather pragmatic
when it comes to programming especially with a functional programming
background, because then Python really wins hand down at least for a
whole bunch of every-day libraries/problems.
I have never looked back to Python but as I said: often lurk and watch
what CommonLisp has to offer.
It is not true that always companies push things. For example it is not
unlikely that Java gets pushed by Sun, but Python is one of the scarce
projects which is driven by manpower in all its glory. But it does not
explain why they succeed when at the same time CommonLisp and Scheme
does not. It cannot be all about parenthesis - or can it?
S. Gonzi
Siegfried Gonzi wrote:
> It is not true that always companies push things. For example it is not
> unlikely that Java gets pushed by Sun, but Python is one of the scarce
> projects which is driven by manpower in all its glory. But it does not
> explain why they succeed when at the same time CommonLisp and Scheme
> does not. It cannot be all about parenthesis - or can it?
The paretheses play a role but not the most important one.
My impression is as follows:
Scripting languages like Python and Ruby are primarily perceived as
advancements over Perl and PHP. The latter have gotten much attention
because they offer considerable advantages for web programming over C
and C++.
Scheme and Common Lisp are the next steps after Python and Ruby.
So Python and Ruby don't win although they are worse than Lisps, but
they win because they are better than the rest.
Let's just wait a few years until people get it... ;-)
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
Pascal Costanza wrote:
> Siegfried Gonzi wrote:
>
>> It is not true that always companies push things. For example it is
>> not unlikely that Java gets pushed by Sun, but Python is one of the
>> scarce projects which is driven by manpower in all its glory. But it
>> does not explain why they succeed when at the same time CommonLisp and
>> Scheme does not. It cannot be all about parenthesis - or can it?
>
>
> The paretheses play a role but not the most important one.
>
> My impression is as follows:
>
> Scripting languages like Python and Ruby are primarily perceived as
> advancements over Perl and PHP. The latter have gotten much attention
> because they offer considerable advantages for web programming over C
> and C++.
>
> Scheme and Common Lisp are the next steps after Python and Ruby.
Actually, Common Lisp is the next step after Scheme :)
Zealotly yours :)
--
Marco Antoniotti
Pascal Costanza <········@web.de> wrote in message news:<············@f1node01.rhrz.uni-bonn.de>...
> My impression is as follows:
>
> Scripting languages like Python and Ruby are primarily perceived as
> advancements over Perl and PHP. The latter have gotten much attention
> because they offer considerable advantages for web programming over C
> and C++.
>
> Scheme and Common Lisp are the next steps after Python and Ruby.
This is exactly the situation that I am looking at. For anything that
is performance sensitive, I just write it in C. This is rare enough
now that it is not a big deal. Most things I do in Ruby when I can.
However, I am looking at CL hoping that it is even better. Ruby seems
generally faster than the other scripting languages, but is still a
little slow. However, it prototypes very quickly and has a nice brief
sytax for common idioms.
So, what areas have you found CL to be better than Ruby? I know
performance is better for CMUCL and this can be important. But, are
there other areas that you find where CL is the next step forward over
Ruby?
"John Proctor" <·········@mindspring.com> wrote in message
·································@posting.google.com...
> This is exactly the situation that I am looking at. For anything that
> is performance sensitive, I just write it in C. This is rare enough
> now that it is not a big deal. Most things I do in Ruby when I can.
> However, I am looking at CL hoping that it is even better. Ruby seems
> generally faster than the other scripting languages, but is still a
> little slow. However, it prototypes very quickly and has a nice brief
> sytax for common idioms.
>
> So, what areas have you found CL to be better than Ruby? I know
> performance is better for CMUCL and this can be important. But, are
> there other areas that you find where CL is the next step forward over
> Ruby?
To clarify Pascals "macro programming" answer, consider your statement "it
prototypes very quickly and has a nice brief syntax for common idioms".
The beauty here is that while there are certainly truly common idioms for
languages, many applications also have common idioms within them. Through
keen macrology, you can get "brief syntax for common idioms" throughout your
entire application, both globally common idioms and application specific
idioms.
Many modern languages are expressive and handle abstraction well, but CL
takes it to the next level with the ability to not only abstract application
and language concepts, but to abstract the syntax used to represent them.
Consider, simplistically, the concept of Collections and Iterators within
Java. Java is more than capable of representing these structures and
abstractions to the programmer, but it required a core parser and compiler
change to add a "foreach" statement, something every coder had to "do by
hand" over and over and over again.
With CL, it is certainly trivial to add at least a basic "foreach"
statement, especially if you simply stick with the given S-Expr format of
Lisp programs. But observe how the Lisp LOOP facility pretty much punted on
that format for its own ends. Now its a source of discussion whether LOOPs
syntax is good or bad, but the important fact is that it's possible at all.
But taking macros and then stacking them on top of the Lisp Reader, and
things get even more interesting. The reader can do truly frightening things
to text. (Un)CommonSQL uses the reader to make SQL much easier to use within
Lisp programs, or Henry Bakers parser generator based on changes to the
reader.
I just saw the announcement of Python 2.3 being released. One of its new
facilities is a CSV, or comma seperated value, reading facility. It would be
an interesting exercise to tweak the reader to do that, leveraging the
facilities that already exist within the system to add some interesting
functionality.
Once tweaked, you could then "embed" CSV data within your Lisp source code.
The motivation is low simply because we use lists everywhere for similar
functionality. But who knows, perhaps it's easier to just take an Excel CSV
export and wrap it with #{ 1,2,"three",4 } than to convert it into a list
with a utility. Depends on the application.
But it's a simple example of what can be done to extend the language to suit
your application. Certainly, most don't go to this level.
All of the other languages skip this detail. They let you "do what you want"
but only up to the point where it fits their syntax. Lisp transcends that.
The only place Lisp really puts limits on you is pretty much at the
implementation level.
A simple example is that while you could certainly develop and use your own
Object System (for example, Garnet), many implementations "know" about CLOS,
and can therefore improve its performance within their implementation with
their compiler. It's pretty difficult to get a CLOS aware compiler to
optimize your Object System as well as it can CLOS.
Of course, perhaps you implemented you Object System on top of CLOS using
macros...
Regards,
Will Hartung
(·····@msoft.com)
Will Hartung wrote:
> Many modern languages are expressive and handle abstraction well, but CL
> takes it to the next level with the ability to not only abstract application
> and language concepts, but to abstract the syntax used to represent them.
>
> Consider, simplistically, the concept of Collections and Iterators within
> Java. Java is more than capable of representing these structures and
> abstractions to the programmer, but it required a core parser and compiler
> change to add a "foreach" statement, something every coder had to "do by
> hand" over and over and over again.
>
> With CL, it is certainly trivial to add at least a basic "foreach"
> statement, especially if you simply stick with the given S-Expr format of
> Lisp programs. But observe how the Lisp LOOP facility pretty much punted on
> that format for its own ends. Now its a source of discussion whether LOOPs
> syntax is good or bad, but the important fact is that it's possible at all.
Well, one of my gripes with LOOP is that the "loop extension facility"
is not defined. E.g. it is exceedingly easy to write in CL the
equivalent of Java Collections/Iterators/Enumerations/Whatever in CL.
But then you cannot extend LOOP in a portable way.
This is what MaiSQL and UncommonSQL hade to face when adding the
(loop for x being each tuple of ...) et similia.
> But taking macros and then stacking them on top of the Lisp Reader, and
> things get even more interesting. The reader can do truly frightening things
> to text. (Un)CommonSQL uses the reader to make SQL much easier to use within
> Lisp programs, or Henry Bakers parser generator based on changes to the
> reader.
Let's add here the INFIX package by Mark Kantrovitz. That is something
to make the CSV stuff you mention below look ridicolous.
> I just saw the announcement of Python 2.3 being released. One of its new
> facilities is a CSV, or comma seperated value, reading facility. It would be
> an interesting exercise to tweak the reader to do that, leveraging the
> facilities that already exist within the system to add some interesting
> functionality.
>
> Once tweaked, you could then "embed" CSV data within your Lisp source code.
> The motivation is low simply because we use lists everywhere for similar
> functionality. But who knows, perhaps it's easier to just take an Excel CSV
> export and wrap it with #{ 1,2,"three",4 } than to convert it into a list
> with a utility. Depends on the application.
That is a good idea. Of course it can get hairy.
Cheers
--
Marco
In article <····························@posting.google.com>,
·········@mindspring.com (John Proctor) wrote:
> Pascal Costanza <········@web.de> wrote in message
> news:<············@f1node01.rhrz.uni-bonn.de>...
>
> > My impression is as follows:
> >
> > Scripting languages like Python and Ruby are primarily perceived as
> > advancements over Perl and PHP. The latter have gotten much attention
> > because they offer considerable advantages for web programming over C
> > and C++.
> >
> > Scheme and Common Lisp are the next steps after Python and Ruby.
[...]
> So, what areas have you found CL to be better than Ruby? I know
> performance is better for CMUCL and this can be important. But, are
> there other areas that you find where CL is the next step forward over
> Ruby?
Two words: macro programming. ;)
Pascal
P.S.: I haven't actually used Ruby yet. But I don't see from a
conceptual point of view how the expressiveness of Lisp-style structural
macros can be beaten by other language constructs. See http://www.paulgraham.com/diff.html for a more detailed response.
From: Valery A.Khamenya
Subject: Re: why I/O is so slow?
Date:
Message-ID: <3f276eb5@shknews01>
Pascal Costanza wrote:
> Let's just wait a few years until people get it... ;-)
Let's just wait a few years until people get "good portable CL
implementations" ... ;-)
--
Valery
Siegfried Gonzi wrote:
> Rationale: You are a scientist comming from Python, Perl, Ruby and
> having read there on the discussion groups and lists: You guys all suck
> choose CommonLisp and it can even surpass Fortran with proper
> declarations.
>
> Result: All the people really trying and using CommonLisp then: are they
> actually happy with the new situation? I would like to see figures of
> people who thought CommonLisp is the paneace (in favor of Python for
> example) due to the alledged speed with proper declarations.
>
> Let speak them out what they had thought they might expect as someone
> mentioned CommonLisp on the Python discussion group and what they then
> really experienced after using it.
This is a bit my story: I'm doing image processing, machine learning,
computer vision and was looking for a nice prototyping environment where I
could work more productively than in C++. I tried Python and found it a
bit ugly here and there, but very easy to grasp and with the libs you can
do cute things quickly.
But for my work it was too slow. 100-1000 times too slow. The people in
the python newsgroup told me to code the speed-critical parts in C. I did
this for a while, but I didn't like the reference-counting scheme in
python's C interface and, worse, it felt _less_ productive than using raw
C++ in the first place.
So I tried cmucl which I had learned and almost forgotten some years ago.
The speed was ok. I even found some microbenchmarks which ran faster under
cmucl than in optimized C++ under gcc. Interesting.
But quickly three issues appeared: 1.) cmucls performance was unpredictable
to me. Sometimes adding declarations improved speed, sometimes it made
things worse. On non-trivial tasks it took me non-neglegtable time to get
acceptable performance using trial-and-error (although it always was
possible, and over time I would probably get better with adding just the
right amount of declarations). 2.) I didn't find libraries to easily read
and display an image. No plotting, no easy-to-use gui. I tought about
interfacing imagemagic myself, but retained from it as I expected it to be
non-trivial and my free time is limited. 3.) None of my colleagues was
interested in learning CL. No recent algorithms are published in CL. I
was on my own.
So I did what everyone does in my community: Switch to matlab. This is a
quite limited language. But the libs are great and you can do linear
algebra really easily. Performance is acceptable if you know how to code
properly. Important: performance is predictable most of the time. There
are excellent domain-specific libraries available.
I haven't fell in love with matlab. I see its uglyness and its limits (it's
hard to build new abstractions in matlab and it's almost impossible to
build new abstractions which run quickly). But in my domain the built-in
stuff is adequate and sometimes excellent. For numerics and images I
usually use matlab, while for other stuff (things which do not need
external libraries) I use CL. A compromise.
>>>>> "Matthias" == Matthias <····@yourself.pl> writes:
Matthias> But quickly three issues appeared: 1.) cmucls performance was unpredictable
Matthias> to me. Sometimes adding declarations improved speed, sometimes it made
Matthias> things worse. On non-trivial tasks it took me non-neglegtable time to get
I would be interested in knowing in what cases declarations made
things worse.
Matthias> acceptable performance using trial-and-error (although it always was
Matthias> possible, and over time I would probably get better with adding just the
Matthias> right amount of declarations).
I've always found the compiler notes from CMUCL extremely helpful in
selecting just the right declarations to speed things up.
Matthias> So I did what everyone does in my community: Switch to matlab. This is a
Matthias> quite limited language. But the libs are great and you can do linear
Matthias> algebra really easily. Performance is acceptable if you know how to code
There is matlisp. Includes some parts of LAPACK, numerical
integration, numerical optimization. May or may not be useful to you,
and since you've switched to matlab, perhaps matlisp is not useful at
all.
Ray
Matthias <····@yourself.pl> writes:
> Siegfried Gonzi wrote:
> > Rationale: You are a scientist comming from Python, Perl, Ruby and
> > having read there on the discussion groups and lists: You guys all suck
> > choose CommonLisp and it can even surpass Fortran with proper
> > declarations.
> >
> > Result: All the people really trying and using CommonLisp then: are they
> > actually happy with the new situation? I would like to see figures of
> > people who thought CommonLisp is the paneace (in favor of Python for
> > example) due to the alledged speed with proper declarations.
> >
> > Let speak them out what they had thought they might expect as someone
> > mentioned CommonLisp on the Python discussion group and what they then
> > really experienced after using it.
>
> This is a bit my story: I'm doing image processing, machine learning,
> computer vision and was looking for a nice prototyping environment where I
> could work more productively than in C++. I tried Python and found it a
> bit ugly here and there, but very easy to grasp and with the libs you can
> do cute things quickly.
>
> But for my work it was too slow. 100-1000 times too slow. The people in
> the python newsgroup told me to code the speed-critical parts in C. I did
> this for a while, but I didn't like the reference-counting scheme in
> python's C interface and, worse, it felt _less_ productive than using raw
> C++ in the first place.
>
> So I tried cmucl which I had learned and almost forgotten some years ago.
> The speed was ok. I even found some microbenchmarks which ran faster under
> cmucl than in optimized C++ under gcc. Interesting.
>
> But quickly three issues appeared: 1.) cmucls performance was unpredictable
> to me. Sometimes adding declarations improved speed, sometimes it made
> things worse. On non-trivial tasks it took me non-neglegtable time to get
> acceptable performance using trial-and-error (although it always was
> possible, and over time I would probably get better with adding just the
> right amount of declarations). 2.) I didn't find libraries to easily read
> and display an image. No plotting, no easy-to-use gui. I tought about
> interfacing imagemagic myself, but retained from it as I expected it to be
> non-trivial and my free time is limited. 3.) None of my colleagues was
> interested in learning CL. No recent algorithms are published in CL. I
> was on my own.
This roughly parallels my experience. But I see it now in a mostly
positive light. It took longer to learn than I wanted, the
documentation was less good than I wanted, there were lots of "stupid
gotchas", lots of low-level stuff that wasn't as easy I wanted (e.g.,
fast raw IO), it took lots longer than I wanted to figure out the
compiler declarations and messages, the debuggers are much tougher to
use than I want, but now, after all this time and effort sunk in, I
feel like I can write CL programs that are fast enough (within say a
factor of 2 of C++) in a small fraction of the time.
> So I did what everyone does in my community: Switch to matlab. This is a
> quite limited language. But the libs are great and you can do linear
> algebra really easily. Performance is acceptable if you know how to code
> properly. Important: performance is predictable most of the time. There
> are excellent domain-specific libraries available.
>
> I haven't fell in love with matlab. I see its uglyness and its limits (it's
> hard to build new abstractions in matlab and it's almost impossible to
> build new abstractions which run quickly). But in my domain the built-in
> stuff is adequate and sometimes excellent. For numerics and images I
> usually use matlab, while for other stuff (things which do not need
> external libraries) I use CL. A compromise.
Yeah, I'm tempted by this too. I find matlab so frustrating though.
If what you're doing is matrix manipulation, matlab's just the thing.
But try doing anything complex that can't easily be exprsesed as
multiplication, and you're in for a world of hurt. What if you need a
hash table? What if you need a tree? Also, I don't find Matlab
especially predicdtable, especially with large matrices --- I find it
hard to tell how many times a matrix will be copied, and I find the
"global" declarations troublesome. Of course, Matlab's great if
someone else has already written the tool you need, and often they
have.
Cheers,
rif
Matthias wrote:
> But quickly three issues appeared: 1.) cmucls performance was unpredictable
> to me. Sometimes adding declarations improved speed, sometimes it made
> things worse. On non-trivial tasks it took me non-neglegtable time to get
> acceptable performance using trial-and-error (although it always was
> possible, and over time I would probably get better with adding just the
> right amount of declarations). 2.) I didn't find libraries to easily read
> and display an image. No plotting, no easy-to-use gui. I tought about
> interfacing imagemagic myself, but retained from it as I expected it to be
> non-trivial and my free time is limited. 3.) None of my colleagues was
> interested in learning CL. No recent algorithms are published in CL. I
> was on my own.
That was the reason why I asked, because I do not buy the "type
declaration" argument in CommonLisp. I am not going to stress Bigloo
again here, but when one is using types in Bigloo then Bigloo will get
as picky as the OCaml compiler. I think Manuel S., the computer
scientist behind Bigloo, was on the Ocaml project during his PhD (when
you look carefully you will sometimes see his name in some of the OCaml
libraries).
The aformentioned means giving types in Bigloo has really an effect on
all involved operations and not like in CommonLisp where you can give
types but it does not make sure that it will affect the operators,
though, the CommonLisp compiler will get let you away with your program.
> I haven't fell in love with matlab. I see its uglyness and its limits (it's
> hard to build new abstractions in matlab and it's almost impossible to
> build new abstractions which run quickly). But in my domain the built-in
> stuff is adequate and sometimes excellent. For numerics and images I
> usually use matlab, while for other stuff (things which do not need
> external libraries) I use CL. A compromise.
I am alone in my Phd and can use what I want. Honestly speaking a great
help is my binding to the high quality plotting library DISLIN. I am due
to release my binding. Using DISLIN from Bigloo is exactly as
comfortable as using DISLIN from Python; one can even use the same
manual as the one for Python/DISLIN. I have to solve some minor glitches
prior to my release but all the demos (even the widget demos) work.
Maybe you are interested in Lush.
http://lush.sourceforge.net/index.html
It is a stripped down Lisp environment for numerical analysis but claims
"being faster than Matlab". It has at least the advantage not to hide
its Lisp heritage, because it uses this nasty paranthesis. For example
the R language claims it relies on some Scheme/Lisp heritage but they
hide it by an awful insane braindamaged syntax without paranthesis.
S. Gonzi
>
Siegfried Gonzi wrote:
[...]
> That was the reason why I asked, because I do not buy the "type
> declaration" argument in CommonLisp.
Well, the type declarations in cmucl mostly helped, although it always took
me a few iterations until I found the right ones.
[...]
> Maybe you are interested in Lush.
>
> http://lush.sourceforge.net/index.html
Two things made me not even want to try lush: 1.) Dynamic scope. 2.) Lush's
compiled language is different from its interpreted language and the
differences are significant (e.g., scoping rules change). I'm too old for
such adventures.
Siegfried Gonzi <···············@kfunigraz.ac.at> writes:
> What do you mean by multi dimensional arrays.
Arrays that have elements indexed in more than one dimension.
(defvar *array* (make-array '(2 3) :initial-element 0.0))
;; => *ARRAY*
*array*
;; => #2A((0.0 0.0 0.0) (0.0 0.0 0.0))
(setf (aref *array* 1 1) 3.45)
;; => 3.45
*array*
;; => #2A((0.0 0.0 0.0) (0.0 3.45 0.0))
> In order to avoid the aformentioned one has to explicitely create a
> new object in a loop for every row.
Right. In Scheme and CL alike, merely saving an object somewhere
does not implicitly make a copy of that object behind your back.
(CL makes an exception for numbers and characters, but those
aren't mutable anyway so it isn't a problem.) This applies to
all types of data structures, not only arrays.
I'm curious though -- how would you have designed it? I see two
ways to get the result you expected.
a) make-array could automatically copies of the second parameter.
The problem is then how deep those copies should be. If the
data structure refers to objects (such as symbols or closures)
that are also being used elsewhere, you might not want to copy
those. C++ disambiguates this with explicit pointer types.
b) make-array could evaluate its second argument as many times
as the array has elements. For example, if the function
next-value incremented a counter, (make-array 3 (next-value))
might return #(1 2 3). This is easy to implement with a
macro.
From: Siegfried Gonzi
Subject: Re: arrays and implicit copying (was: why I/O is so slow?)
Date:
Message-ID: <3F28C05E.3030203@kfunigraz.ac.at>
Kalle Olavi Niemitalo wrote:
> Right. In Scheme and CL alike, merely saving an object somewhere
> does not implicitly make a copy of that object behind your back.
> (CL makes an exception for numbers and characters, but those
> aren't mutable anyway so it isn't a problem.) This applies to
> all types of data structures, not only arrays.
>
> I'm curious though -- how would you have designed it? I see two
> ways to get the result you expected.
>
> a) make-array could automatically copies of the second parameter.
> The problem is then how deep those copies should be. If the
> data structure refers to objects (such as symbols or closures)
> that are also being used elsewhere, you might not want to copy
> those. C++ disambiguates this with explicit pointer types.
>
> b) make-array could evaluate its second argument as many times
> as the array has elements. For example, if the function
> next-value incremented a counter, (make-array 3 (next-value))
> might return #(1 2 3). This is easy to implement with a
> macro.
First I thought (make-vector 2 (make-vector 3 0.0)) has some
peculiarities and one has to live with, but then someone posted on the
Bigloo mailing list why this is so and since then I am more than ever
irritated, because I would never expect such a behavior especially when:
==
(let ((erg (make-vector 2 0.0))
(a (make-vector 3 0.0)))
(vector-set! erg 0 a)
(vector-set! erg 1 a)))
==
If I remember correctly the above is the same, but even here I would
expect that vector-set! actually stores something in a memory location
rather than storing a pointer to something.
I mean such a behavior is not really obvious in hindsight of the fact
that Scheme people are brought up by "without pointers". Surely,
pointers are like sliced bred and butter when considered internally in a
Scheme implementation.
Okay, one could argue and say "vector-set! stores an object (literally)
into a location". But this is not as I read it, I read it as follows:
"vector-set! stores the content of an object".
S. Gonzi
Siegfried Gonzi wrote:
> I mean such a behavior is not really obvious in hindsight of the fact
> that Scheme people are brought up by "without pointers". Surely,
> pointers are like sliced bred and butter when considered internally in a
> Scheme implementation.
>
> Okay, one could argue and say "vector-set! stores an object (literally)
> into a location".
Exactly. (make-vector 3 (make-vector 2 0.0)) stores the same (eq?) value
at all three indices. [No need to discuss pointers (duck)]
> But this is not as I read it, I read it as follows:
> "vector-set! stores the content of an object".
procedure: (make-vector k)
procedure: (make-vector k fill)
Returns a newly allocated vector of k elements. If a second argument
is given, then each element is initialized to fill. Otherwise the
initial contents of each element is unspecified.
It says that fill is stored, not a copy of fill.
You are right though, at first it is confusing -- but consider the
possibilities for confusion if copies of fill were automatically made.
Since fill can be anything, then what should "a copy of fill" mean
exactly? Deep copy? Shallow copy?
--
Jens Axel S�gaard
Siegfried Gonzi <···············@kfunigraz.ac.at> writes:
> your interpreter which is impossible. But on the other side I have
> often wondered why people stretch the comand line interpeter
> (listener) in CommonLisp that much. Think of a CommonLisp session
> where you have created 10 functions and after 5 hours or so you are in
> command line 10 000 and have changed a lot of this functions and got
> messed up and have to scroll back and down and forget what you did at
> function three 200 lines above. Isn't it better to have this 10
> functions in a "static" file and change things always in place?
This is not the way people use the prompt. You keep your definitions
in a file, compile them there and try them at the prompt. Check out
ilisp or eli. But wait! You might find you like it and never want to
go back and use Bigloo's module compilation. And then you're stuck
with library-starved Common Lisp. :-)
Andras
In article <················@cs.nyu.edu>,
Marco Antoniotti <·······@cs.nyu.edu> wrote:
> > BtW: I think the Scheme Chicken compiler has a seemingless easy
> > integration into C too.
>
> AFAIK, the Chicken compiler is a very good thing, but why wasn't it
> written for CL?
The primary reason for the way Chicken works is to enable efficient
support for tail call optimization and full continuations in a compiler
that generates C. CL doesn't require these things (though I suspect
most CL users wouldn't object to them if they were cheap and easy).
-- Bruce
Bruce Hoult wrote:
> In article <················@cs.nyu.edu>,
> Marco Antoniotti <·······@cs.nyu.edu> wrote:
>
>
>>>BtW: I think the Scheme Chicken compiler has a seemingless easy
>>>integration into C too.
>>
>>AFAIK, the Chicken compiler is a very good thing, but why wasn't it
>>written for CL?
>
>
> The primary reason for the way Chicken works is to enable efficient
> support for tail call optimization and full continuations in a compiler
> that generates C. CL doesn't require these things (though I suspect
> most CL users wouldn't object to them if they were cheap and easy).
Exactly my point :)
Cheers
--
Marco
In article <················@kfunigraz.ac.at>,
Siegfried Gonzi <···············@kfunigraz.ac.at> wrote:
> Have you ever tried to incorporate your CommonLisp function into a C
> program? BtW: I think the Scheme Chicken compiler has a seemingless easy
> integration into C too.
Yes, Chicken integrates into C programs easily. It compiles into C,
uses standard C calling conventions (including using STDARGS to pass
variable argument lists), and lets you do inline C code (including
inline assembler, if your C compiler lets you). The only strange things
are that you have to remember to add/remove tags bits (there are C
macros for this) and none of the C functions Chicken creates ever
return, so there is special handling for C code calling Scheme code in
that it needs to garbage collect the scheme stack frames created by the
callback before returning to the C caller.
-- Bruce
I have rewritten the lisp version to:
(defun print-move (from to)
(write-line (concatenate 'string "Move " from " to " to ".")))
(defun tower-of-hanoi (n from to spare)
(if (= n 1)
(print-move from to)
(progn
(tower-of-hanoi (- n 1) from spare to)
(print-move from to)
(tower-of-hanoi (- n 1) spare to from))))
(compile 'tower-of-hanoi)
(compile 'print-move)
(tower-of-hanoi 20 "A" "B" "C")
and compared it with this perl version.
sub tower_of_hanoi
{
my ($n, $from, $to, $spare) = @_;
if ($n == 1) {
print_move($from, $to);
}
else {
tower_of_hanoi($n - 1, $from, $spare, $to);
print_move($from, $to);
tower_of_hanoi($n - 1, $spare, $to, $from);
}
}
sub print_move
{
my ($from, $to) = @_;
print "Move $from to $to.\n";
}
tower_of_hanoi(20, "A", "B", "C");
Using clisp 2.29 and perl 5.6.0 on a MacOSX system:
time clisp TOH.lisp > out: 10.300u 0.620s 0:13.84 78.9% 0+0k 0+45io 0pf+0w
time perl TOH.PL > out: 11.530u 0.510s 0:19.22 62.6% 0+0k 0+43io 0pf+0w
Notice that perl and clisp both load noncompiled code and that the
lisp code has no declarations.
If I comment out the code that does the printing (make the print_move
function body empty) I get thefollowing result
clisp: 1.160u 0.040s 0:01.79 67.0% 0+0k 0+19io 0pf+0w
perl: 7.440u 0.010s 0:09.91 75.1% 0+0k 0+1io 0pf+0w
Can I count on you to create a nonprinting version for perl that is
as fast as the clisp version?
Immanuel
Immanuel Litzroth <·········@enfocus.be> wrote in message news:<··············@enfocus.be>...
> I have rewritten the lisp version to:
>
> If I comment out the code that does the printing (make the print_move
> function body empty) I get thefollowing result
> clisp: 1.160u 0.040s 0:01.79 67.0% 0+0k 0+19io 0pf+0w
> perl: 7.440u 0.010s 0:09.91 75.1% 0+0k 0+1io 0pf+0w
>
> Can I count on you to create a nonprinting version for perl that is
> as fast as the clisp version?
Nope. No doubt that lisp is faster when IO is not involved. That is
why the title of the subject is "Why is I/O so slow".
John Proctor wrote:
> I am having the same problem. I came to CL because someone convinced
> me that it was much faster than some of the other languages that I
> use, Ruby, Python and Perl. Well, my first little benchmark for fun,
> tower of hanoi, didn't go to well.
>
> The CL version takes 45 seconds, the Python version takes 12 seconds,
> the Perl version takes 9 seconds and the Ruby version takes just 6
> seconds. Just for fun, I did it in C also, which runs in less than 1
> second. These are not small differences.
>
> I, too, would like to know what I am missing. Why is this version so
> slow compared to all of the other languages? I am using CMUCL,
> compiling to x86f under linux. The problem is with format. If I
> remove the call to format then the speed goes to less than 1 second.
> Which is still 8 times slower than the C version without printf.
>
> Here is the CL code:
>
> (defun tower-of-hanoi (ofs n from to spare)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (cond ((= n 1) (print-move ofs from to))
> (t (tower-of-hanoi ofs (- n 1) from spare to)
> (print-move ofs from to)
> (tower-of-hanoi ofs (- n 1) spare to from))))
>
> (defun print-move (ofs from to)
> (declare (optimize (speed 3) (debug 0) (safety 0)))
> (format ofs "~%Move ~a to ~a." from to))
>
> (tower-of-hanoi t 20 "A" "B" "C")
> (quit)
>
Try
(defun tower-of-hanoi (fd n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0)))
(cond ((= n 1) (print-move fd from to))
(t (tower-of-hanoi fd (- n 1) from spare to)
(print-move fd from to)
(tower-of-hanoi fd (- n 1) spare to from))))
(defun print-move (fd from to)
(declare (optimize (speed 3) (debug 0) (safety 1)))
(unix:unix-write fd #.(format nil "~%Move ") 0 6)
(unix:unix-write fd from 0 1)
(unix:unix-write fd " to " 0 4)
(unix:unix-write fd to 0 1))
(tower-of-hanoi (system:fd-stream-fd system:*stdout*) 20 "A" "B" "C")
(quit)
All I have is a Pentium 200 running FreeBSD so I cannot really
tell what the final speed will be.
Wade
·········@mindspring.com (John Proctor) wrote in message news:<····························@posting.google.com>...
> Mucho snipped code...
Thanks for all of the replies and effort. I was a little hesitant to
post this originally since some people seem to get all worked up about
this kind of issue.
I am quite pleased with CMUCL. It seems to be very fast and the issue
with format is mostly resolved. My only remaining question involves
using format with standard output vs a file. It is still slow.
Using the command:
time lisp-start ./toh.x86f > toh-lisp.out
this takes about 20 seconds if format is printing to standard output.
But, if I change it to use with-open-file instead and run
time lisp-start ./toh.x86f
it only takes about 3.5 seconds.
clisp returns the same time either way (about 7 seconds).
Any idea why format with CMUCL is slow when bound to a tty even when
being redirected to a file?
On Fri, Jul 25, 2003 at 06:27:59PM -0700, John Proctor wrote:
> Using the command:
> time lisp-start ./toh.x86f > toh-lisp.out
> this takes about 20 seconds if format is printing to standard output.
>
> But, if I change it to use with-open-file instead and run
> time lisp-start ./toh.x86f
> it only takes about 3.5 seconds.
>
> clisp returns the same time either way (about 7 seconds).
>
> Any idea why format with CMUCL is slow when bound to a tty even when
> being redirected to a file?
I tried straceing the processes to see if there was anything
outstanding, but both clisp and cmucl seem to be making normal write()
calls to fd 1. Yet clisp performs much better with the redirected
stdout. I didn't spot anything in the initialization routines that
might lead to this, but perhaps some property is being set on the fd
which is slowing it down. Or something to do with CMUCL buffering; so I
checked out *standard-output* with DESCRIBE and INSPECT. Turns out it's
a SYNONYM-STREAM to SYSTEM:*STDOUT* which is an FD-STREAM with :LINE
buffering. On the other hand, a stream opened by OPEN is a FD-STREAM
with :FULL buffering. So, I created a FD-STREAM for fd 1 using:
(sys:make-fd-stream 1
:element-type 'character
:input nil :output t
:buffering :full)
And with this stream, I get a 10x speed-up over using *standard-output*.
For comparsion:
~0.78sec with GCC
~1.15sec with CMUCL
~4.6sec with CLISP
~10sec with Python2.2
Athlon XP 1800+ 512MB RAM
On a side note, if you intend to invoke CMUCL a great deal like this, I
highly advise rethinking the cost of loading the core file. I think,
that for small applications like this, it might be beneficial to look
into some way of keeping CMUCL continuously running. I generally use it
like that for web-applications (Allegroserve or mod_lisp) and that
avoids the (relatively) large start-up cost plus it gives a nice
interactive environment for working with the running web-app. I work on
projects where I am continuously changing the code, while a
web-developer is working on the HTML templates--on the same, live,
system that I am working on.
But whatever suits your application and taste... I hope this fd-stream
trick helps out. You can use #+cmu to conditionalize it; such as:
(defparameter *my-stdout*
#-cmu *standard-output*
#+cmu (sys:make-fd-stream ...))
and (let ((*standard-output* *my-stdout*)) (run-my-app)) or similar.
(and SBCL would be SB-SYS:MAKE-FD-STREAM)
--
; Matthew Danish <·······@andrew.cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian.org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."
Matthew Danish <·······@andrew.cmu.edu> wrote in message news:<······················@lain.mapcar.org>...
> Or something to do with CMUCL buffering; so I
> checked out *standard-output* with DESCRIBE and INSPECT. Turns out it's
> a SYNONYM-STREAM to SYSTEM:*STDOUT* which is an FD-STREAM with :LINE
> buffering. On the other hand, a stream opened by OPEN is a FD-STREAM
> with :FULL buffering. So, I created a FD-STREAM for fd 1 using:
>
> (sys:make-fd-stream 1
> :element-type 'character
> :input nil :output t
> :buffering :full)
>
That's it. Now it is as fast with redirected *standard-output* as
with with-open-file.
>
> For comparsion:
>
> ~0.78sec with GCC
> ~1.15sec with CMUCL
> ~4.6sec with CLISP
> ~10sec with Python2.2
How did you get CMUCL to be so close to gcc? My version with CMUCL is
still about 3.5 times slower tha gcc. I used
gcc -O2 toh.c
to compile the C version and
(compile-file "toh.lisp")
to compile the cmucl version.
Did you do anything else? Did you compile with O2 for gcc?
On Fri, Jul 25, 2003 at 10:18:35PM -0700, John Proctor wrote:
> Matthew Danish <·······@andrew.cmu.edu> wrote in message news:<······················@lain.mapcar.org>...
> > For comparsion:
> >
> > ~0.78sec with GCC
> > ~1.15sec with CMUCL
> > ~4.6sec with CLISP
> > ~10sec with Python2.2
>
> How did you get CMUCL to be so close to gcc? My version with CMUCL is
> still about 3.5 times slower tha gcc. I used
> gcc -O2 toh.c
> to compile the C version and
> (compile-file "toh.lisp")
> to compile the cmucl version.
(defun tower-of-hanoi (ofs n from to spare)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type fixnum n)
(type base-char from to spare))
(cond ((= n 1) (print-move ofs from to))
(t (tower-of-hanoi ofs (1- n) from spare to)
(print-move ofs from to)
(tower-of-hanoi ofs (1- n) spare to from))))
(defun print-move (ofs from to)
(declare (optimize (speed 3) (debug 0) (safety 0))
(type stream ofs)
(type base-char from to))
(format ofs "Move ~C to ~C.~%" from to))
Invoked as:
(tower-of-hanoi (sys:make-fd-stream 1
:element-type 'character
:output t
:input nil
:buffering :full)
20
#\a #\b #\c)
Notes: Adding optimize declarations resulted in about 700% speed-up. Adding
type declarations resulted in about 15-20% speed-up from there. Using ~C
directives to FORMAT doesn't invoke any fancy behavior when *PRINT-PRETTY* is
T. Using FORMATTER had no affect on speed, probably because of a smart
compiler-macro or deftransform for FORMAT in CMUCL (when the format-string is
static).
> Did you do anything else? Did you compile with O2 for gcc?
Yes, and it didn't have any difference from the default optimization level.
Thanks for your questions on this matter, btw; I am using ~A a lot in my code,
and didn't realize it was so grossly affected by *PRINT-PRETTY*.
--
; Matthew Danish <·······@andrew.cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian.org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."
John Proctor wrote:
> ·········@mindspring.com (John Proctor) wrote in message news:<····························@posting.google.com>...
>
> Any idea why format with CMUCL is slow when bound to a tty even when
> being redirected to a file?
It may have to do with the buffering method for *stdout*, :line, the
default seems to be :full.
From the CMUCL src tree.
;;; STREAM-REINIT -- internal interface
;;;
;;; Called whenever a saved core is restarted.
;;;
(defun stream-reinit ()
(setf *available-buffers* nil)
(setf *stdin*
(make-fd-stream 0 :name "Standard Input" :input t :buffering :line))
(setf *stdout*
(make-fd-stream 1 :name "Standard Output" :output t :buffering :line))
(setf *stderr*
(make-fd-stream 2 :name "Standard Error" :output t :buffering :line))
(let ((tty (and (not *batch-mode*)
(unix:unix-open "/dev/tty" unix:o_rdwr #o666))))
(setf *tty*
(if tty
(make-fd-stream tty :name "the Terminal" :input t :output t
:buffering :line :auto-close t)
(make-two-way-stream *stdin* *stdout*))))
nil)
;;; MAKE-FD-STREAM -- Public.
;;;
;;; Returns a FD-STREAM on the given file.
;;;
(defun make-fd-stream (fd
&key
(input nil input-p)
(output nil output-p)
(element-type 'base-char)
(buffering :full)
timeout
file
original
delete-original
pathname
input-buffer-p
(name (if file
(format nil "file ~S" file)
(format nil "descriptor ~D" fd)))
auto-close)
(declare (type index fd) (type (or index null) timeout)
(type (member :none :line :full) buffering))
"Create a stream for the given unix file descriptor.
If input is non-nil, allow input operations.
If output is non-nil, allow output operations.
If neither input nor output are specified, default to allowing input.
Element-type indicates the element type to use (as for open).
Buffering indicates the kind of buffering to use.
Timeout (if true) is the number of seconds to wait for input. If NIL
(the
default), then wait forever. When we time out, we signal IO-TIMEOUT.
File is the name of the file (will be returned by PATHNAME).
Name is used to identify the stream when printed."
(cond ((not (or input-p output-p))
(setf input t))
((not (or input output))
(error "File descriptor must be opened either for input or output.")))
(let ((stream (%make-fd-stream :fd fd
:name name
:file file
:original original
:delete-original delete-original
:pathname pathname
:buffering buffering
:timeout timeout)))
(set-routines stream element-type input output input-buffer-p)
(when (and auto-close (fboundp 'finalize))
(finalize stream
#'(lambda ()
(unix:unix-close fd)
(format *terminal-io* "** Closed file descriptor ~D~%"
fd))))
stream))
Wade
I compiled and ran your C version and it runs for *far* longer than 1
second when actually printing the numbers to a DOS window. Did you mean
1 second when the stdout is directed to a disk file?
I know there have been other interesting variations on the CL code. I
haven't gotten to the end of the posts so far, and so I don't know if
anyone else has taken the following approach, but I get nearly the same
run time in C and CL with this code. (I'm using Lispworks for Windows. I
'delivered' the code and, when I ran it, directed the output to a file
in exactly the same manner as when I ran the C version. The delivered
Lispworks ends up opening an extra DOS window, where the output normally
would appear. The performance graph in Task Manager shows nearly the
same peak of CPU usage both for the C and the Lisp runs. The Lisp peak
does have a wider base but it's hard to say how much wider. I don't
think Lisp's is double that of C.)
If I were to make a practice of this style of coding, I first would
spend some time on macrology to simplify the foreign calls. I'd have a
short, pretty program taking advantage of the best of both worlds.
(fli:define-foreign-function
(printf "printf" :calling-convention :stdcall)
((format-string (:unsigned :int))
(from (:unsigned :int))
(to (:unsigned :int))))
(defun run-toh ()
(let ((f-s "Move %s to %s\n")
(from "A")
(to "B")
(spare "C"))
(fli:with-foreign-string
(format-ptr ec1 bc1 :external-format :ascii)
f-s
(fli:with-foreign-string
(from-ptr ec2 bc2 :external-format :ascii)
from
(fli:with-foreign-string
(to-ptr ec3 bc3 :external-format :ascii)
to
(fli:with-foreign-string
(spare-ptr ec4 bc4 :external-format :ascii)
spare
(let ((format-ptr (fli:pointer-address format-ptr)))
(labels ((print-move (from-ptr to-ptr)
(printf format-ptr from-ptr to-ptr))
(tower-of-hanoi (n from to spare)
(declare (optimize (speed 3) (debug 0)
(safety 0)))
(cond
((= n 1) (print-move from to))
(t (tower-of-hanoi (- n 1) from spare to)
(print-move from to)
(tower-of-hanoi (- n 1) spare to from)))))
(tower-of-hanoi 20 (fli:pointer-address from-ptr)
(fli:pointer-address to-ptr)
(fli:pointer-address spare-ptr))))))))))
(compile 'run-toh)
(deliver 'run-toh "towerofhanoi" 1)
(quit)
John Proctor wrote:
> I am having the same problem. I came to CL because someone convinced
> me that it was much faster than some of the other languages that I
> use, Ruby, Python and Perl. Well, my first little benchmark for fun,
> tower of hanoi, didn't go to well.
>
> The CL version takes 45 seconds, the Python version takes 12 seconds,
> the Perl version takes 9 seconds and the Ruby version takes just 6
> seconds. Just for fun, I did it in C also, which runs in less than 1
> second. These are not small differences.
I thought I'd mention that when I built the 'delivery' application at
level 1, the exe file size was over 17Mb and time showed 3.448 seconds.
At level 5 the exe file was nearly 3Mb and time showed 2.845 seconds. C
showed 1.715 seconds. The C exe was 40Kb. (This was when redirecting to
a file, which ended up around 14Mb.)
It's hard to say how much of the 1.13 second difference is caused by the
difference in exe file size or by the fact that Lispworks opens up a
separate window for stdout, at least when things aren't redirected. In
other words, Lispworks has higher initiation overhead. If the app were
up and stayed up, instead of restarting, I wonder how large the
difference would be over time.
Jeff Caldwell wrote:
...
> The performance graph in Task Manager shows nearly the
> same peak of CPU usage both for the C and the Lisp runs. The Lisp peak
> does have a wider base but it's hard to say how much wider. I don't
> think Lisp's is double that of C.)
From: Valery A.Khamenya
Subject: Re: why I/O is so slow?
Date:
Message-ID: <3f20dfd6@shknews01>
Hi All
small detail:
if "%f" and "~F" change to dummy "12.13", then
1. CMUCL is 3 times slower then gcc.
2. Allegro CL (win32):
; cpu time (non-gc) 3,532 msec user, 15 msec system
; cpu time (gc) 15 msec user, 0 msec system
; cpu time (total) 3,547 msec user, 15 msec system
; real time 3,578 msec
; space allocation:
; 192 cons cells, 811,472 other bytes, 72 static bytes
and gcc (cygwin):
0.07user 0.12system 0:01.12elapsed 17%CPU
(0avgtext+0avgdata 13616maxresident)k
0inputs+0outputs (851major+0minor)pagefaults 0swaps
It's a pity :-(
--
Valery