From: ···········@gmail.com
Subject: Changing stream :element-type
Date: 
Message-ID: <1112433965.190671.152690@f14g2000cwb.googlegroups.com>
I understand there is no portable way to change element-type on an
already open stream but:

I have an input stream from a server that was opened as a character
stream to read the headers sent back by the server.   The rest of the
stream is binary data, hence the need to change the type to
'unsigned-byte.

So:

1) is there an easy way to change the type specific to SBCL,
2) what would be the best way to get around it in a portable fashion?
The data is probably on the 1-2MB side, so perhaps copy everything to a
string, and then open as a binary stream?

Thanks for everyones help!

From: Adam Warner
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <pan.2005.04.04.13.30.49.930373@consulting.net.nz>
On Sat, 02 Apr 2005 02:02:39 -0800, dade.watson wrote:
> I understand there is no portable way to change element-type on an
> already open stream but:
> 
> I have an input stream from a server that was opened as a character
> stream to read the headers sent back by the server.   The rest of the
> stream is binary data, hence the need to change the type to
> 'unsigned-byte.
> 
> So:
> 
> 1) is there an easy way to change the type specific to SBCL,

No. The last person to ask about bivalent streams on the SBCL mailing list
appears to be Kevin Rosenberg, who received no replies:
<http://www.caddr.com/macho/archives/sbcl-devel/2004-10/4182.html>

Note that the LANG=C hack to write octets to a character stream appears to
no longer work since SBCL got external format support:

$ LANG=C sbcl
This is SBCL 0.8.20.5, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (string (code-char 128))


debugger invoked on a SB-INT:STREAM-ENCODING-ERROR in thread 5284:
  encoding error on stream #<FILE-STREAM for "standard output" {90032E1}>
  (:EXTERNAL-FORMAT :ASCII):
    the character with code 128 cannot be encoded.

> 2) what would be the best way to get around it in a portable fashion?

Don't open a character stream in the first place! Read the header as an
octet stream and perform manual conversion to characters and strings.
There's elementary functionality missing from binary streams, e.g.
UNREAD-BYTE. If READ-BYTE and WRITE-BYTE are too slow for your purposes
consider READ-SEQUENCE, WRITE-SEQUENCE and explicit buffers.

If you want to serialise Lisp S-expressions and data over a binary stream
you're in for more trouble. Perhaps the greatest failing of portable
Common Lisp is that "code is data" and "data is code" only applies to
character data (reading and writing Lisp S-expressions and code to and
from binary streams is not supported).

If efficiency is not a primary concern just write character output to a
string and then convert the string to binary data. Otherwise you'll need
to write many thousands of lines of code to replace the printer and reader
functions (including FORMAT and READ) with equivalents that can write to
and read from binary streams.

> The data is probably on the 1-2MB side, so perhaps copy everything to a
> string, and then open as a binary stream?

Yes, consider opening the stream initially as a binary stream and perform
explicit conversions to and from binary data & strings. You may be able to
avoid some string conversions by operating upon octet arrays instead of
strings. This should be especially easy if the textual data is in ASCII.

You may want to write some macro characters that return octets or
octet sequences instead of character data. Otherwise it's going to get
rather tedious writing #.(char-code #\A), etc.

Regards,
Adam
From: Duane Rettig
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <4oecu8wt6.fsf@franz.com>
Adam Warner <······@consulting.net.nz> writes:

> On Sat, 02 Apr 2005 02:02:39 -0800, dade.watson wrote:
> > I understand there is no portable way to change element-type on an
> > already open stream but:
> > 
> > I have an input stream from a server that was opened as a character
> > stream to read the headers sent back by the server.   The rest of the
> > stream is binary data, hence the need to change the type to
> > 'unsigned-byte.
> > 
> > So:
> > 
> > 1) is there an easy way to change the type specific to SBCL,
> 
> No. The last person to ask about bivalent streams on the SBCL mailing list
> appears to be Kevin Rosenberg, who received no replies:
> <http://www.caddr.com/macho/archives/sbcl-devel/2004-10/4182.html>

I'm sure you already know this, but it may not be obvious to the
original poster: Bivalency is an added feature many CLs have that
the spec doesn't address, which allows a stream to be treated _as_ _if_
it had multiple element-types; in particular, read-char and read-byte can
be invoked on the same stream with no error.  There is actually no need
to change the element-type, in fact.  And usually, the width of the "byte"
doesn't matter; nowadays binary data is read and written in terms of
the number of octets (8-bit bytes) anyway, so bivalent streams sometimes
act as though they are ([un}signed-byte 8) rather than a particular width.
Simple-streams provides a second vector-reading function (besides
read-sequence) callled read-vector - it lays octets down side-by-side into
a specialized vector regardless of its element-type (i.e., width), and it
provides for byte-swapping for endianness efficiently so the binary data
need not be architecture-specific.

> Note that the LANG=C hack to write octets to a character stream appears to
> no longer work since SBCL got external format support:

That is unfortunate.  How about their simple-streams implementaton;
did that break as well?

Our own simple-streams implementation always allows bivalency, even in
the face of changes in external-formats.

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Dave Watson
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <d2rjhb$3ul$1@gnus01.u.washington.edu>
On 2005-04-04, Duane Rettig <·····@franz.com> wrote:
>> Note that the LANG=C hack to write octets to a character stream appears to
>> no longer work since SBCL got external format support:
>
> That is unfortunate.  How about their simple-streams implementaton;
> did that break as well?
>
> Our own simple-streams implementation always allows bivalency, even in
> the face of changes in external-formats.
>

Yes, someone pointed out simple-streams to me in an email, and I got
enough of it working in sbcl to do what I needed.   Thanks for the help!

-- 
-Dave Watson
········@docwatson.org
From: Christophe Rhodes
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <sqbr8u61xp.fsf@cam.ac.uk>
Duane Rettig <·····@franz.com> writes:

> Adam Warner <······@consulting.net.nz> writes:
>
>> Note that the LANG=C hack to write octets to a character stream appears to
>> no longer work since SBCL got external format support:
>
> That is unfortunate.  How about their simple-streams implementaton;
> did that break as well?

Erm.  "Did we stop beating our wives?"

The POSIX C locale, in character set terms, corresponds pretty closely
to the ANSI_X3.4-1968 (more popularly known as ASCII) repertoire.  As
such, using it to encode octets is a pretty poor idea, because, as
Adam has discovered, ASCII is a 7-bit character repertoire.  It is
probably true that this requirement was not rigorously enforced
previously, and high-bit characters were silently passed through;
however, this only _started_ to be an issue once stream
external-formats began to be supported; before, no hack was necessary
for a na�ve treatment, because the only external-format supported was,
in effect, the na�ve one.  I will respond to Adam's message directly
to provide a solution to his immediate problem, but I thought I should
also make clear the cause of it.

As for the brokenness or otherwise of our simple-streams
implementation: it's a little hard to judge when there are no known
users of that code; if a tree is felled in the forest, does it really
matter?  For what it's worth, all of the regression tests that have
been written by sbcl users and developers of the simple-streams
machinery continue to pass.

Christophe
From: Duane Rettig
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <4k6ni8888.fsf@franz.com>
Christophe Rhodes <·····@cam.ac.uk> writes:

> Duane Rettig <·····@franz.com> writes:
> 
> > Adam Warner <······@consulting.net.nz> writes:
> >
> >> Note that the LANG=C hack to write octets to a character stream appears to
> >> no longer work since SBCL got external format support:
> >
> > That is unfortunate.  How about their simple-streams implementaton;
> > did that break as well?
> 
> Erm.  "Did we stop beating our wives?"

I can certainly see how that terse question could look like that,
but I assure you that that was never my intention in asking it.
I apologize for the unintended implication.

> The POSIX C locale, in character set terms, corresponds pretty closely
> to the ANSI_X3.4-1968 (more popularly known as ASCII) repertoire.  As
> such, using it to encode octets is a pretty poor idea, because, as
> Adam has discovered, ASCII is a 7-bit character repertoire.  It is
> probably true that this requirement was not rigorously enforced
> previously, and high-bit characters were silently passed through;
> however, this only _started_ to be an issue once stream
> external-formats began to be supported; before, no hack was necessary
> for a na�ve treatment, because the only external-format supported was,
> in effect, the na�ve one.  I will respond to Adam's message directly
> to provide a solution to his immediate problem, but I thought I should
> also make clear the cause of it.

Yes.  My interpretation of this is that there is a gap between strict
conformance to the spec wrt streams, and the practical use of streams
in modern (i.e. not Vax) architectures.  The tension between compliance
and practicality is always a tricky one, since ANSI tends to view the
character as the fundamental unit of transfer, and bytes being variable,
whereas modern exchange technology view octets as the fundamental unit
and characters to be variable (to the extent that characters might span
multiple octets, let alone 8 bits vs 7 bits).  If you doubt this gap,
ask the spec how to listen for an octet...

> As for the brokenness or otherwise of our simple-streams
> implementation: it's a little hard to judge when there are no known
> users of that code; if a tree is felled in the forest, does it really
> matter?  For what it's worth, all of the regression tests that have
> been written by sbcl users and developers of the simple-streams
> machinery continue to pass.

Heh; well, from another article on this thread, you have at least one.
Did you hear _that_ tree fall?
:-)

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Steven E. Harris
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <jk4ekdqceph.fsf@W003275.na.alarismed.com>
Duane Rettig <·····@franz.com> writes:

> If you doubt this gap, ask the spec how to listen for an octet...

Similarly, why is there no apparent way to do a non-blocking
read-byte, even in simple-streams?

The documentation mentions a function excl::read-byte-no-hang� in the
context of read-char-no-hang, but even that non-exported function is
not visible in my Allegro implementation. The function
excl:read-vector is almost there, but it blocks on the first byte even
when called with a one-element buffer range.

Oh, wait: is this a job for (listen stream 1)?

,----[ listen documentation ]
| If 1 octet is being listened for, then if the buffer is not empty,
| true is returned. Otherwise device-read is called with a null blocking
| argument. If that returns 0, then nil is returned, otherwise true is
| returned.
`----


Footnotes: 
� http://www.franz.com/support/documentation/6.2/doc/streams.htm#cl-funs-2

-- 
Steven E. Harris
From: Duane Rettig
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <41x9pu7gw.fsf@franz.com>
"Steven E. Harris" <···@panix.com> writes:

> Duane Rettig <·····@franz.com> writes:
> 
> > If you doubt this gap, ask the spec how to listen for an octet...
> 
> Similarly, why is there no apparent way to do a non-blocking
> read-byte, even in simple-streams?
> 
> The documentation mentions a function excl::read-byte-no-hang� in the

[Note on the footnote: it might be advantageous to use the 7.0 docs for
discussions about what the leading edge is, since the 6.2 docs follow
the 6.2 version and not the current thinking.  I don't believe anything
we're discussing here is actually different, but there are a few noticible
additions and changes, such as the addition of j-read-byte and j-write-byte
strategy slots, and some differences in device-open, etc.  So I'll use
the 7.0 docs when I reference them - to see if they pertain to 6.2, you
might be able to just substitute 6.2 for 7.0 to get there]

> context of read-char-no-hang, but even that non-exported function is
> not visible in my Allegro implementation.

Correct; it was always a phantom function designed only to allow the
reader to understand it in terms of its action, which is in turn based
on the required action of read-char-no-hang by the spec.

I don't know if the others in the design team felt the same way, but
for me personally, the combining of two trivial functionalities into
one operation (and making it overlap with two other primitive operations)
becomes confusing.  read-char-no-hang is effectively a listen followed
conditionally by a read-char (with complexities for the idiosynchrosies of
listen).  In short, it is a combination of waiting function and reading
function.  So when it came time to enumerating the various reading functions
for simple-streams, as shown in
http://www.franz.com/support/documentation/7.0/doc/streams.htm#block-non-block-3,
we found read-char-no-hang to be pretty much by itself.  We did not want
to make things more confusing by adding such combinational fucntions, but
instead wanted to add the primitives so you could build them up yourself
to do precisely what you want with them.

> The function
> excl:read-vector is almost there, but it blocks on the first byte even
> when called with a one-element buffer range.

The idea behind read-vector (and the whole notion of B/NB shown in the
link above) is I/O progress.  If a vector is to be read, then progress
should be made in the reading and that progress reported; this tends
to be the way many op/sys reads operate.  Simple-streams tends to side
with this separation of wait and read functionalities, and shuns the
complexities of combining read and wait operations.

Note that at a lower level, you can use read-octets and write-octets, which
does receive a blocking argument.  *-octets is intended to be more of an
implementation function for device-write, but for that it gives you the
power to control the blocking style.  However, this would seem to be a bit
of overkill, and you already have the right idea, below, anyway:

> Oh, wait: is this a job for (listen stream 1)?
======^^^^  A pun, right? :-)

> ,----[ listen documentation ]
> | If 1 octet is being listened for, then if the buffer is not empty,
> | true is returned. Otherwise device-read is called with a null blocking
> | argument. If that returns 0, then nil is returned, otherwise true is
> | returned.
> `----

Yes, but note that even this extended version of listen is limited to input
streams (or, in simple-stream parlance, streams opened at least for input),
and it only affects the read direction.  I prefer the "*-no-hang-p" functions,
http://www.franz.com/support/documentation/7.0/doc/operators/excl/read-no-hang-p.htm
and
http://www.franz.com/support/documentation/7.0/doc/operators/excl/write-no-hang-p.htm

because they not only are precise in what they are after, but in the direction
that they are looking at.

> Footnotes: 
> � http://www.franz.com/support/documentation/6.2/doc/streams.htm#cl-funs-2
> 
> -- 
> Steven E. Harris

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Steven E. Harris
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <jk41x9pcisb.fsf@W003275.na.alarismed.com>
Duane Rettig <·····@franz.com> writes:

> Note on the footnote: it might be advantageous to use the 7.0 docs
> for discussions about what the leading edge is, since the 6.2 docs
> follow the 6.2 version and not the current thinking.

Yes, I'm looking at the 7.0 documentation as part of my Allegro
installation, but wanted to post a globally-accessible URL for others
to follow along. Unfortunately, I didn't notice that Google is
favoring the 6.2 document set in its top results.

> Correct; it was always a phantom function designed only to allow the
> reader to understand it in terms of its action, which is in turn
> based on the required action of read-char-no-hang by the spec.

It would be nice to add some verbiage in that section to clarify that
this function does not actually exist. Perhaps "as if using the
imaginary and unimplemented excl::..."

[...]

> we found read-char-no-hang to be pretty much by itself.  We did not
> want to make things more confusing by adding such combinational
> fucntions, but instead wanted to add the primitives so you could
> build them up yourself to do precisely what you want with them.

That's excellent reasoning.

> The idea behind read-vector (and the whole notion of B/NB shown in
> the link above) is I/O progress.  If a vector is to be read, then
> progress should be made in the reading and that progress reported;
> this tends to be the way many op/sys reads operate.  Simple-streams
> tends to side with this separation of wait and read functionalities,
> and shuns the complexities of combining read and wait operations.

I find the final sentence contradictory to the rest of the paragraph,
unless I'm misunderstanding what you mean by "wait operations." In
read-vector would expect to find a function that either blocks until
it can read some minimum number of octets (I've written such a
function several times in C++), or one that reads only as many as are
available, possibly /including zero/, EOF notwithstanding.

> Note that at a lower level, you can use read-octets and
> write-octets, which does receive a blocking argument.

Yes, I was using read-octets as part of my device-read implementation,
but then figured out that I could save a lot of trouble by using
call-next-method instead. That's a whole separate discussion.

My question here concerns free functions written to operate on streams
at the "user level", not necessarily those part of implementing the
simple-stream contract.

>> Oh, wait: is this a job for (listen stream 1)?
> ======^^^^  A pun, right? :-)

Not intentional, but I'll take credit where I can get it.

> Yes, but note that even this extended version of listen is limited
> to input streams (or, in simple-stream parlance, streams opened at
> least for input), and it only affects the read direction.

That would suffice for my needs.

> I prefer the "*-no-hang-p" functions,
> http://www.franz.com/support/documentation/7.0/doc/operators/excl/read-no-hang-p.htm
> and
> http://www.franz.com/support/documentation/7.0/doc/operators/excl/write-no-hang-p.htm
>
> because they not only are precise in what they are after, but in the
> direction that they are looking at.

Again, I find the documentation here misleading. These functions are
mentioned in Section 10.4, "Implementation Helpers for device-read and
device-write", but noted that they "can be called at any level." Well,
here, I'm looking at simple-streams as a user, looking at how to
perform my own pseudo-read-byte-no-hang, so I'm not likely to touch
functions marked as being intended for stream implementation.

With your recommendation, I'll use excl:read-no-hang-p, but that I
overlooked it several times already suggests that it warrants mention
in other parts of the stream documentation.

Thanks for your help.

-- 
Steven E. Harris
From: Duane Rettig
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <4psx9nntc.fsf@franz.com>
"Steven E. Harris" <···@panix.com> writes:

> Duane Rettig <·····@franz.com> writes:
> 
> > Note on the footnote: it might be advantageous to use the 7.0 docs
> > for discussions about what the leading edge is, since the 6.2 docs
> > follow the 6.2 version and not the current thinking.
> 
> Yes, I'm looking at the 7.0 documentation as part of my Allegro
> installation, but wanted to post a globally-accessible URL for others
> to follow along. Unfortunately, I didn't notice that Google is
> favoring the 6.2 document set in its top results.

Likely due to its popularity over the former, due to the age of each.

> > Correct; it was always a phantom function designed only to allow the
> > reader to understand it in terms of its action, which is in turn
> > based on the required action of read-char-no-hang by the spec.
> 
> It would be nice to add some verbiage in that section to clarify that
> this function does not actually exist. Perhaps "as if using the
> imaginary and unimplemented excl::..."

Yes, that may be a way to go.  Note: if you have issues about the
documentation, I suggest that you send to ·······@franz.com, where it
will be logged and tracked.  Even if you are only a trial user, and
even if you don't (though obviously, you do) use Allegro CL, using the
support channel is more likely to result in things being done.  We
never promise anything to non-customers, but we do tend to listen
at all times.

> > The idea behind read-vector (and the whole notion of B/NB shown in
> > the link above) is I/O progress.  If a vector is to be read, then
> > progress should be made in the reading and that progress reported;
> > this tends to be the way many op/sys reads operate.  Simple-streams
> > tends to side with this separation of wait and read functionalities,
> > and shuns the complexities of combining read and wait operations.
> 
> I find the final sentence contradictory to the rest of the paragraph,
> unless I'm misunderstanding what you mean by "wait operations."

Poor choice of words:

> In
> read-vector would expect to find a function that either blocks until
> it can read some minimum number of octets (I've written such a
> function several times in C++), or one that reads only as many as are
> available, possibly /including zero/, EOF notwithstanding.

Yes.  In unix, reads are sometimes performed after select() has cleared
the way.  And select has the option of either waiting or not waiting.
I would have used the term "select" operation, but that is too
unix-centric.  I could have used "decision-whether-to-block" or "listen",
but the first is too wordy, and the second is to ANSI-spec-centric
(and thus not generic, as I was intending).

> > Note that at a lower level, you can use read-octets and
> > write-octets, which does receive a blocking argument.
> 
> Yes, I was using read-octets as part of my device-read implementation,
> but then figured out that I could save a lot of trouble by using
> call-next-method instead. That's a whole separate discussion.
> 
> My question here concerns free functions written to operate on streams
> at the "user level", not necessarily those part of implementing the
> simple-stream contract.

Right.  As you know, the answer is below.

> >> Oh, wait: is this a job for (listen stream 1)?
> > ======^^^^  A pun, right? :-)
> 
> Not intentional, but I'll take credit where I can get it.

:-)

> > Yes, but note that even this extended version of listen is limited
> > to input streams (or, in simple-stream parlance, streams opened at
> > least for input), and it only affects the read direction.
> 
> That would suffice for my needs.
> 
> > I prefer the "*-no-hang-p" functions,
> > http://www.franz.com/support/documentation/7.0/doc/operators/excl/read-no-hang-p.htm
> > and
> > http://www.franz.com/support/documentation/7.0/doc/operators/excl/write-no-hang-p.htm
> >
> > because they not only are precise in what they are after, but in the
> > direction that they are looking at.
> 
> Again, I find the documentation here misleading. These functions are
> mentioned in Section 10.4, "Implementation Helpers for device-read and
> device-write", but noted that they "can be called at any level." Well,
> here, I'm looking at simple-streams as a user, looking at how to
> perform my own pseudo-read-byte-no-hang, so I'm not likely to touch
> functions marked as being intended for stream implementation.
> 
> With your recommendation, I'll use excl:read-no-hang-p, but that I
> overlooked it several times already suggests that it warrants mention
> in other parts of the stream documentation.

Yes, the documentation can use some sprucing up; it had started out
as a spec, and grew into a expositional document, sometimes without
necessary rewrites out of its spec-nature.  As for *-no-hang-p, I
think that they were always internal functions during original
implementation, but got cleaned up and exported when we started
thinking about async/io in the write direction.

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Christophe Rhodes
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <sqk6nh1biz.fsf@cam.ac.uk>
Duane Rettig <·····@franz.com> writes:

> Christophe Rhodes <·····@cam.ac.uk> writes:
>
>> Duane Rettig <·····@franz.com> writes:
>> 
>> > That is unfortunate.  How about their simple-streams implementaton;
>> > did that break as well?
>> 
>> Erm.  "Did we stop beating our wives?"
>
> I can certainly see how that terse question could look like that,
> but I assure you that that was never my intention in asking it.
> I apologize for the unintended implication.

Sorry, Duane, I know.  I blame my caffeine intake.

>> The POSIX C locale, in character set terms, corresponds pretty closely
>> to the ANSI_X3.4-1968 (more popularly known as ASCII) repertoire.  As
>> such, using it to encode octets is a pretty poor idea, because, as
>> Adam has discovered, ASCII is a 7-bit character repertoire.  It is
>> probably true that this requirement was not rigorously enforced
>> previously, and high-bit characters were silently passed through;
>> however, this only _started_ to be an issue once stream
>> external-formats began to be supported; before, no hack was necessary
>> for a na�ve treatment, because the only external-format supported was,
>> in effect, the na�ve one.  I will respond to Adam's message directly
>> to provide a solution to his immediate problem, but I thought I should
>> also make clear the cause of it.
>
> Yes.  My interpretation of this is that there is a gap between
> strict conformance to the spec wrt streams, and the practical use of
> streams in modern (i.e. not Vax) architectures.  The tension between
> compliance and practicality is always a tricky one, since ANSI tends
> to view the character as the fundamental unit of transfer, and bytes
> being variable, whereas modern exchange technology view octets as
> the fundamental unit and characters to be variable (to the extent
> that characters might span multiple octets, let alone 8 bits vs 7
> bits).  If you doubt this gap, ask the spec how to listen for an
> octet...

Quite.  It should be said that the sbcl side of all this technology is
under development (on which more in Amsterdam in a few weeks), and so
people shouldn't be terribly surprised if the interfaces aren't
maximally convenient.  On the other hand, I think that there is also
value in being able to ask to be told when the user has violated some
contract that they have made, implicitly or explicitly, with the
environment -- and printing a high-bit character to an ascii character
stream is in some sense one such contract violation.  I certainly
won't argue with you over the impedance mismatch between the CL
standardized stream operators and the rest of the world, though it is
at least better than the mismatch between CL pathnames and the rest of
the world.  :-)

>> As for the brokenness or otherwise of our simple-streams
>> implementation: it's a little hard to judge when there are no known
>> users of that code; if a tree is felled in the forest, does it really
>> matter?
>
> Heh; well, from another article on this thread, you have at least one.
> Did you hear _that_ tree fall?

Very encouraging -- I look forward to the bug reports :-)

Christophe
From: Adam Warner
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <pan.2005.04.04.14.51.22.312629@consulting.net.nz>
On Mon, 04 Apr 2005 07:02:13 -0700, Duane Rettig wrote:
>> Note that the LANG=C hack to write octets to a character stream appears to
>> no longer work since SBCL got external format support:
> 
> That is unfortunate.  How about their simple-streams implementaton;
> did that break as well?
> 
> Our own simple-streams implementation always allows bivalency, even in
> the face of changes in external-formats.

I've never used simple streams. While not portable/ANSI specified they
could provide a solution to reading and writing characters and binary to
and from the same stream.

Writing characters and bytes is implemented:

* (require 'sb-simple-streams)

; loading system definition from #P"/usr/lib/sbcl/systems/sb-posix.asd" into
; #<PACKAGE "ASDF4006">
; registering #<SYSTEM SB-POSIX {9B91C51}> as SB-POSIX
; registering #<SYSTEM SB-POSIX-TESTS {90D8799}> as SB-POSIX-TESTS
; loading system definition from #P"/usr/lib/sbcl/systems/sb-bsd-sockets.asd"
; into #<PACKAGE "ASDF4208">
; registering #<SYSTEM SB-BSD-SOCKETS {9532CB1}> as SB-BSD-SOCKETS
; registering #<SYSTEM SB-BSD-SOCKETS-TESTS {989D1F1}> as SB-BSD-SOCKETS-TESTS
("SB-GROVEL" "SB-BSD-SOCKETS" "SB-POSIX" "SB-SIMPLE-STREAMS")
* (defparameter *s* (sb-simple-streams:open "/tmp/s.txt" :direction :output :if-exists :supersede :element-type '(unsigned-byte 8)))

*S*
* (sb-simple-streams:write-char #\A *s*)

#\A

* (sb-simple-streams:write-byte 255 *s*)

255
* (sb-simple-streams:close *s*)

T

The TODO file includes these notes:
- Implement & test read-sequence, write-sequence for (un)signed-8 vectors
- Test write-octets / read-octets handling of encapsulated streams
- Make reader work with simple-streams

Dade should certainly check out the sb-simple-streams package.

Regards,
Adam
From: Christophe Rhodes
Subject: Re: Changing stream :element-type
Date: 
Message-ID: <sq7jji61rc.fsf@cam.ac.uk>
Adam Warner <······@consulting.net.nz> writes:

> On Sat, 02 Apr 2005 02:02:39 -0800, dade.watson wrote:
>> 1) is there an easy way to change the type specific to SBCL,
>
> No. The last person to ask about bivalent streams on the SBCL mailing list
> appears to be Kevin Rosenberg, who received no replies:
> <http://www.caddr.com/macho/archives/sbcl-devel/2004-10/4182.html>

Yes, it's a shame that no-one who needs this feature has yet
implemented it.  We're always happy to accept contributions,
particularly if they come with documentation and regression tests.

> Note that the LANG=C hack to write octets to a character stream appears to
> no longer work since SBCL got external format support:

The C (an alias for POSIX on Unix systems) locale has a seven-bit
character repertoire.  Try something which has an eight-bit
repertoire, such as IBM037 or ISO-8859-1/ISO-8859-15.

Christophe