From: Tim Bradshaw
Subject: Myth and lore -- I/O performance
Date:
Message-ID: <ey3vh2m1z0h.fsf@cley.com>
I've just got the program I was talking about in the CHARACTER thread
to the point where the low-level bits run. It reads and writes stuff
to other programs, and I'm quite interested in performance.
I haven't profiled it at all and there aren't many declarations,
although I've been reasonably careful about buffers (and in fact none
of the tests below cons more than 100k).
These tests are writing and reading to a memory filesystem on a 333MHz
Sun with enough memory that it really *is* a memory filesystem. The
number of records written/read were adjusted to keep the file size
about between 50 and 150Mb. I expect the average size in practice
will be somewhere between the medium and big records, but I'm not
sure what the scatter is.
The tests aren't really fair because in real life I will be talking
down a network, but I don't have the other end working yet, or fast
enough ethernet to test. Disk tests were likewise limited (the
machine has rather slow disks).
This isn't just slurping a huge file, it reads stuff, looks at it,
adjusts buffer sizes, reads more based on what it finds and so on.
The Lisp is acl5.0.1, I may try some others in due course.
The code is 100% portable CL (at least it compiles in everything I've
tried it on).
The figures are more approximate than the precision indicates, as they
depend on other stuff on the machine &c.
;;; for 37k records, to a memory filesystem, about 32Mb/sec write
;;; from 100Mb/sec read
;;; for 4112 byte records 24Mb/sec write
;;; 31Mb/sec read
;;; for 528 byte records ... 2.8Mb/sec write
;;; 7.9Mb/sec read
;;; for 32byte records ... 0.3Mb/sec write
;;; 0.5Mb/sec read
For the big record size, assuming 1 instruction per clock that's a
little over 3 instructions per byte read.
Clearly I can smash a 100Mb ethernet no problem here, and I seriously
doubt that the far end will be able to handle data at anything like
the rates concerned.
--tim