From: Peter Seibel
Subject: Buffering of streams?
Date: 
Message-ID: <m3r7o0j1zc.fsf@javamonkey.com>
Do implementations typically provided buffered streams for files and
sockets? I'm writing a Shoutcast server that needs to read MP3 files
from disk and shove them out a socket. My first version was "smart"
using READ-SEQUENCE and WRITE-SEQUENCE on the theory that one
READ-SEQUENCE call is much better than the equivalent number of
READ-BYTE calls. However for a variety of reasons I tried a simpler
version that essentially does

  (loop for byte = (read-byte mp3 nil nil)
     while byte do (write-byte socket))

and it works fine (i.e. plenty speedy to stream audio to at least two
clients simultaneously). Then it occured to me that in Allegro,
streams are by default simple streams which are (as I understand it)
by default buffered. Which means that the difference between N
READ-BYTE calls and 1 READ-SEQUENCE call is approximately (1- N)
(relatively inexpensive) function calls, not (1- N) (expensive) system
calls. Or maybe my computer is just so fast that it doesn't matter.

In other implementations do READ-BYTE/WRITE-BYTE normally result in a
system call?

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp

From: Duane Rettig
Subject: Re: Buffering of streams?
Date: 
Message-ID: <48ya8pym8.fsf@franz.com>
Peter Seibel <·····@javamonkey.com> writes:

> Do implementations typically provided buffered streams for files and
> sockets? I'm writing a Shoutcast server that needs to read MP3 files
> from disk and shove them out a socket. My first version was "smart"
> using READ-SEQUENCE and WRITE-SEQUENCE on the theory that one
> READ-SEQUENCE call is much better than the equivalent number of
> READ-BYTE calls. However for a variety of reasons I tried a simpler
> version that essentially does
> 
>   (loop for byte = (read-byte mp3 nil nil)
>      while byte do (write-byte socket))
> 
> and it works fine (i.e. plenty speedy to stream audio to at least two
> clients simultaneously). Then it occured to me that in Allegro,
> streams are by default simple streams which are (as I understand it)
> by default buffered. Which means that the difference between N
> READ-BYTE calls and 1 READ-SEQUENCE call is approximately (1- N)
> (relatively inexpensive) function calls, not (1- N) (expensive) system
> calls. Or maybe my computer is just so fast that it doesn't matter.
> 
> In other implementations do READ-BYTE/WRITE-BYTE normally result in a
> system call?

It depends on what you call a "system call".  If you are explicitly
talking about a call to the kernel read() function, then it is likely
that most implementations do mostly buffered reads, and you'll only get
one kernel call per several-Kbyte buffers-full.

However, one thing I've learned in the process of implementing and
developing simple-streams is that a "system call" may be the least of
your worries wrt time spent.  There may not even be an explicit system
call, as is the case where your file is memory-mapped (a system call to
sync the memory and the file is indeed involved, but several operating
systems have their own strategies for doing such synching, and one could
get away with filling most of physical memory without one kernel call).
Another thing to be concerned about exists at a higher level; if Gray
Streams are involved, then speed is limited by method dispatch speed.
Thus, your "relatively inexpensive" parenthetical becomes a mistaken
assumption, and even if all read-byte calls (which thus go through
stream-read-byte) go through a buffering layer, they might still be
much slower than a single read-sequence call (which thus dispatches
only once on the stream-read-sequence efective-method).

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Bulent Murtezaoglu
Subject: Re: Buffering of streams?
Date: 
Message-ID: <87r7o0n6qy.fsf@p4.internal>
>>>>> "PS" == Peter Seibel <·····@javamonkey.com> writes:
[...]
    PS> In other implementations do READ-BYTE/WRITE-BYTE normally
    PS> result in a system call?

You meant one syscall per byte, of course.  I tried reding from a file 
like so

(with-open-file (str #P"/etc/services" :direction :input 
                :element-type '(unsigned-byte 8)) 
  (dotimes (i 100 ) (read-byte str)))

cmucl, smcl, clisp do a single call to read with a buffer size of 
4096, Lispworks pro uses 8192.  All on uncompiled cut and paste code 
to an straced repl in an xterm under recent debian testing, with the 
free lisps from debian unstable + fully patched lispworks.  

This is from a file though, they might behave differently for sockets. 

cheers,

BM