From: Robert Maas, see http://tinyurl.com/uh3t
Subject: Re: Testing for the EOF on byte streams
Date: 
Message-ID: <rem-2008feb09-003@yahoo.com>
> From: Marco Antoniotti <·······@gmail.com>
> Suppose I have something like
>    (setf s (open "foo.dat" :element-type 'unsigned-byte :direction :input))
>    (if (am-I-at-the-eof-p s) (do-eof-stuff s) (do-some-other-stuff s))
> I am looking for a portable way to write AM-I-AT-THE-EOF-P.

You really want "am I just about to hit EOF if I read even one more byte".

Put a wrapper around your binary input stream such that it buffers
one byte of lookahead, setting the private EOF flag if the
lookahead has already ran off EOF. Something like this:
(defvar *lookahead*)
(defun lookahead (s)
  (setq *lookahead* (read-byte s nil :eofahead)))
(defun open-file-using-lookahead (name)
  (prog1
    (open name :element-type 'unsigned-byte :direction :input) ;hard-coded
    (lookahead)))
(defun am-I-at-the-eof-p (s)
  (eq :eofahead *lookahead*))
(defun read-byte-using-lookahead (s)
  (prog1
    *lookahead* ;Might want to signal error if this is already :eofahead
    (lookahead)))
(setf s (open-file-using-lookahead "foo.dat"))
(loop
  (if (am-I-at-the-eof-p s)
      (progn (do-eof-stuff s) (return :ALLDONE))
      (do-some-other-stuff s)))
;Everywhere inside do-some-other-stuff where you called read-byte,
; you need to change that to call read-byte-using-lookahead instead.

You might want to make open-file-using-lookahead a macro that
passes &rest to open instead of hard-coding the call to open to
always use unsigned-byte, so that you can vary the byte size and
have the code still work.

If this were MacLisp you could have produced a funny kind of stream
(a "filearray") that has the lookahead built in so you'd just call
read-byte and it'd do the right thing. In Java all streams are
fullfledged instances of a class satisfying an interface, so you
can make your own funny kind of stream that wraps around a regular
buffered input stream. But CL doesn't have that feature.

Warning: Your do-some-other-stuff had better not try to read more
than one byte!! You can see what that problem would be, right?

For newbies:
File bytes: 7  11  13
(defun do-some-other-stuff (s)
  (format t "Got data ~S~%" (list (read-byte s) (read-byte s))))
EOF not yet reached, so it tries to call do-some-other-stuff:
  First call returns (7 11)
EOF not yet reached, so it tries to call do-some-other-stuff:
  Second call reads 13 then bombs out.

From: Kent M Pitman
Subject: Re: Testing for the EOF on byte streams
Date: 
Message-ID: <usl02owo2.fsf@nhplace.com>
·······@yahoo.com (Robert Maas, see http://tinyurl.com/uh3t) writes:

> If this were MacLisp you could have produced a funny kind of stream
> (a "filearray") that has the lookahead built in so you'd just call
> read-byte and it'd do the right thing.

Actually, you're thinking of "software file array", or SFA.  The "S" in
the name is to distinguish it from the primitively implemented kind of
file, sometimes called a "file array", that you got back from OPEN.
Note the similarity in how the two printed:

  (*array nil 'fixnum 5)
  #FIXNUM-5-70776

  (open "test file")
  #FILE-IN-|DSK:KMP;TEST FILE|-70774

Those who, unlike REM, were not users of Maclisp should not look too hard
at this because they may worry that sharpsign is a reader macro.  It was 
not yet so when the printed representation was devised, and consequently
arrays and files did not print re-readably.

This is part of why Rainer was making the point in a companion thread
that CL was quite innovative.  things we take for granted now, like
system-wide print/read invertibility were new in Lisp, except in the
trivial sense that in the distant past, when there had been only
symbols, numbers, and lists things had worked that way.  There had
been a dark age in between where the importance of this had been lost,
and the whole notion of printing other objects re-readably had to be
re-evolved.... and SFAs were part of the climb back, because they gave
users the ability to enter a world in which a printer might be devised
that could descend an object and print it differently even if the
system printer was not inclined to.

Historical aside: An SFA was a programmable stream.  The facility for
it was designed by Howard Cannon.  He also designed the original
Flavors implementation for the Lisp Machine implementation, and it's
easy to see the evolutionary progression of those two seemingly
unrelated items if you know this fact.

For more info on SFAs, see documentation in
  http://www.maclisp.info/pitmanual/sfa.html
and a sample of coding one at the bottom of that file.

The modern "equivalent" of SFAs would be things like:
  * Gray streams 
    http://www.nhplace.com/kent/CL/Issues/stream-definition-by-user.html
  * Simple Streams
    http://www.franz.com/support/documentation/6.2/doc/streams.htm
Gray streams were a much earlier design and are pretty widely distributed.
Simple streams are more elaborated and more recent, but I haven't checked
lately to know how many vendors do and don't implement them.
From: Robert Maas, see http://tinyurl.com/uh3t
Subject: Re: Testing for the EOF on byte streams
Date: 
Message-ID: <rem-2008feb20-003@yahoo.com>
> > If this were MacLisp you could have produced a funny kind of stream
> > (a "filearray") that has the lookahead built in so you'd just call
> > read-byte and it'd do the right thing.
> From: Kent M Pitman <······@nhplace.com>
> Actually, you're thinking of "software file array", or SFA.

Ah, yes. It's been a lot of years since I used MacLisp remotely
from California over the ArpaNet, and I remembered only two thirds
of the name. Thanks for the correction.

> ... SFA was a programmable stream. ...
> The modern "equivalent" of SFAs would be things like:
>   * Gray streams
>     http://www.nhplace.com/kent/CL/Issues/stream-definition-by-user.html

I think I saw that before. It looked reasonable then, still does,
and it's somewhat sad it didn't pass.