Hello,
Due to a bout of influenza, I've had the occasion this week-end to
read the simple streams document. With plenty of tea.
To my unexercised eyes, it simple streams look like a rather nice
piece of work. I do not like Gray stream very much, and the simple
streams document clarified to me what it was that bothered me about
them. I feel that I do understand what problems Franz are trying to
solve, and it looks to me like they have pretty much succeeded.
In certain ways, the document still looks a little bit underspecified.
In particular, I couldn't find any information on the metaclass of
SIMPLE-STREAM (how do you use mixins with instances of
SIMPLE-STREAM?). (Checking the implementation shows that simple
streams are standard objects.)
I was a little bit surprised to see that the ``implementation
helpers'' were implemented as distinct methods rather than mixins.
There are probably good reasons for that, and I would like to see them
spelled out.
A minor nit: I couldn't find a way for the device layer to distinguish
between FORCE-OUTPUT and FINISH-OUTPUT (think e.g. of a TCP/IP layer
that does provide a handle to the PUSH flag, or an application-level
protocol that forces a round-trip on FINISH-OUTPUT but not necessarily
on FORCE-OUTPUT).
I didn't like the new behaviour of OPEN. I feel that OPEN should be
controlled by a special variable or a new keyword argument (or both),
not the current :ELEMENT-TYPE hack.
I was also a little disturbed by the fact that, apparently, the device
writer has to fall back to Gray streams for unbuffered I/O. Of
course, I am not suggesting that simple streams should be extended for
unbuffered functionality (that would be contrary to the very goal of
simple streams), but I would like to see a similarly elegant subclass
of STREAM designed to cater for unbuffered I/O, and Gray streams
deprecated.
Finally, I was frustrated by the complete lack of credits. Simple
streams were designed by human beings, and I would dearly like to know
who they are.
(I was also disappointed by the lack of pointers to related work,
other than Gray streams of course. Java streams, obviously, but I
also seem to recall something in relation to MacLisp. I have no idea
what streams were like in ZetaLisp; perhaps somebody can enlighten me?)
I'm wondering whether other people have thoughts to share on the
subject. Simple streams sure look like a nice abstraction, and I
suspect that some people at Franz would welcome feedback on the
subject.
Juliusz Chroboczek
The best person at Franz to answer your questions, Duane Rettig,
is on vacation and won't be back for a few weeks. If you could
bring this up again when he returns I'm sure he would
appreciate it.
In article <··············@pps.jussieu.fr>, "Juliusz Chroboczek"
<···@pps.jussieu.fr> wrote:
> Hello,
> Due to a bout of influenza, I've had the occasion this week-end to read
> the simple streams document. With plenty of tea. To my unexercised
> eyes, it simple streams look like a rather nice piece of work.
Hope you're feeling better...
>In certain ways, the document still looks a
> little bit underspecified.
No doubt. The strategy interface seems to be completely undocumented.
>In particular, I couldn't find any
> information on the metaclass of SIMPLE-STREAM (how do you use mixins
> with instances of SIMPLE-STREAM?). (Checking the implementation shows
> that simple streams are standard objects.)
It would be pretty weird for SIMPLE-STREAM to not have metaclass
STANDARD-CLASS, given the emphasis on performance.
> (I was also disappointed by the lack of pointers to related work, other
> than Gray streams of course. Java streams, obviously, but I also seem
> to recall something in relation to MacLisp. I have no idea what streams
> were like in ZetaLisp; perhaps somebody can enlighten me?) I'm
> wondering whether other people have thoughts to share on the subject.
> Simple streams sure look like a nice abstraction, and I suspect that
> some people at Franz would welcome feedback on the subject.
>
> Juliusz Chroboczek
I find it frustrating that there's no apparent way to derive a new type
of string stream; simple string streams apparently bypass the device
interface (which makes sense, but still...) Also, the number of hacks
specific to ACL's CLOS (instance flags? WTF?) is a bit distracting.
Disclaimer: These comments are based on the implementation and
documentation in ACL 6.0; I've only glanced at the new documentation in
6.1.
Tim
"Tim Moore" <·····@bricoworks.com> writes:
> In article <··············@pps.jussieu.fr>, "Juliusz Chroboczek"
> <···@pps.jussieu.fr> wrote:
> >In certain ways, the document still looks a
> > little bit underspecified.
>
> No doubt. The strategy interface seems to be completely undocumented.
Actually, the strategy interface is documented to be undocumented :-)
See:
http://www.franz.com/support/documentation/6.1/doc/streams.htm#ss-description-2
in the third bulleted item in the list describing the various kinds
of programmers. Until a few weeks ago, I was the only strategy programmer,
and the only useful strategy has been the one which implements the CL
spec, and I didn't feel that it would be good to confuse device-level
programmers with the more detailed concepts at this time. Criticisms
have already been levelled at the streams document that it is too
complex, and I want to relegate more detailed descriptions to later
chapters as time goes on.
Having said that, the newer version of this document:
http://www.franz.com/support/documentation/6.1/doc/streams.htm
(which should be out within a week or so) will describe some of the
strategy level, in enough detail that device-level programming can be
done for encapsulations.
> >In particular, I couldn't find any
> > information on the metaclass of SIMPLE-STREAM (how do you use mixins
> > with instances of SIMPLE-STREAM?). (Checking the implementation shows
> > that simple streams are standard objects.)
>
> It would be pretty weird for SIMPLE-STREAM to not have metaclass
> STANDARD-CLASS, given the emphasis on performance.
Correct.
> I find it frustrating that there's no apparent way to derive a new type
> of string stream; simple string streams apparently bypass the device
> interface (which makes sense, but still...) Also, the number of hacks
> specific to ACL's CLOS (instance flags? WTF?) is a bit distracting.
This is not true; string-simple-streams use the device layer, but they
do not use device-read/device-write. Instead they use device-extend,
which is a device-level method intended for modifying a workspace, rather
than reading or writing from a buffer. (device-extend is also useful for
working on mapped-files and on buffer-oriented streams).
> Disclaimer: These comments are based on the implementation and
> documentation in ACL 6.0; I've only glanced at the new documentation in
> 6.1.
Right. You should have a look at the new docs, especially in a week or so.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
In article <·············@beta.franz.com>, "Duane Rettig"
<·····@franz.com> wrote:
> "Tim Moore" <·····@bricoworks.com> writes:
>> In article <··············@pps.jussieu.fr>, "Juliusz Chroboczek"
>> <···@pps.jussieu.fr> wrote:
>
>> >In certain ways, the document still looks a
>> > little bit underspecified.
>> No doubt. The strategy interface seems to be completely undocumented.
> Actually, the strategy interface is documented to be undocumented :-)
And that's my problem :)
>
>> I find it frustrating that there's no apparent way to derive a new type
>> of string stream; simple string streams apparently bypass the device
>> interface (which makes sense, but still...) Also, the number of hacks
>> specific to ACL's CLOS (instance flags? WTF?) is a bit distracting.
> This is not true; string-simple-streams use the device layer, but they
> do not use device-read/device-write. Instead they use device-extend,
> which is a device-level method intended for modifying a workspace,
> rather than reading or writing from a buffer. (device-extend is also
> useful for working on mapped-files and on buffer-oriented streams).
Paul Foley pointed this out to me, but it seems that device-extend
doesn't get called in FORCE-OUTPUT, which was a problem in my
application. I'll wait to see your rot13 example, assuming it works on
string streams, before dissing simple string streams further :)
Tim
"Tim Moore" <·····@bricoworks.com> writes:
> In article <·············@beta.franz.com>, "Duane Rettig"
> <·····@franz.com> wrote:
>
>
> > "Tim Moore" <·····@bricoworks.com> writes:
> >> In article <··············@pps.jussieu.fr>, "Juliusz Chroboczek"
> >> <···@pps.jussieu.fr> wrote:
> >
> >> >In certain ways, the document still looks a
> >> > little bit underspecified.
> >> No doubt. The strategy interface seems to be completely undocumented.
> > Actually, the strategy interface is documented to be undocumented :-)
>
> And that's my problem :)
I see. Tell me if it remains a problem, once the revision is finalized.
> >> I find it frustrating that there's no apparent way to derive a new type
> >> of string stream; simple string streams apparently bypass the device
> >> interface (which makes sense, but still...) Also, the number of hacks
> >> specific to ACL's CLOS (instance flags? WTF?) is a bit distracting.
> > This is not true; string-simple-streams use the device layer, but they
> > do not use device-read/device-write. Instead they use device-extend,
> > which is a device-level method intended for modifying a workspace,
> > rather than reading or writing from a buffer. (device-extend is also
> > useful for working on mapped-files and on buffer-oriented streams).
>
> Paul Foley pointed this out to me, but it seems that device-extend
> doesn't get called in FORCE-OUTPUT, which was a problem in my
> application. I'll wait to see your rot13 example, assuming it works on
> string streams, before dissing simple string streams further :)
Ah, yes; what you are describing was a _bug_ in 6.0. I fixed it and
many others regarding encapsulation and string streams when I started
working up the examples. One example of this was the fact that we were
using the same buffer slot for string-input-simple-streams as for
string-output-simple-streams, and this disallowed any bidirectional-buffer
encapsulations in a bidirectional string stream inheriting from the two
classes (such streams aren't defined by CL, but are useful as extensions).
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
Juliusz Chroboczek <···@pps.jussieu.fr> writes:
> Hello,
>
> ...
> Finally, I was frustrated by the complete lack of credits. Simple
> streams were designed by human beings, and I would dearly like to know
> who they are.
Our style, at Franz is not to put forth our documentation as the
company's documentation, not as bits and pieces written by
individuals. That's just the way we do it. Please don't think I'm
saying any other way is wrong, either.
Duane Rettig, now on a well deserved vacation, is a primary author.
John Foderaro and Charley Cox contributed significantly. Others at
Franz contributed, too. Richard Barber, no longer at Franz, was the
person that light the initial flame for the project.
> (I was also disappointed by the lack of pointers to related work,
> other than Gray streams of course. Java streams, obviously, but I
> also seem to recall something in relation to MacLisp. I have no idea
> what streams were like in ZetaLisp; perhaps somebody can enlighten me?)
Other than gray streams, nothing came up in the design meetings that I
recall.
> I'm wondering whether other people have thoughts to share on the
> subject. Simple streams sure look like a nice abstraction, and I
> suspect that some people at Franz would welcome feedback on the
> subject.
Thank you. We do welcome feedback on it. I have been personally
pleased with how the theory was translated into practice, and how it
has held up over the last couple of releases.
Kevin
Kevin Layer <·····@--you-know-what-to-remove--.franz.com> writes:
> Our style, at Franz is not to put forth our documentation as the
That should have been (remove the first `not'):
Our style, at Franz is to put forth our documentation as the
company's documentation, not as bits and pieces written by
individuals.
On 12 Nov 2001 19:24:30 +0100, Juliusz Chroboczek <···@pps.jussieu.fr>
wrote:
> Finally, I was frustrated by the complete lack of credits. Simple
> streams were designed by human beings, and I would dearly like to know
> who they are.
I think one of the authors is Duane Rettig.
> subject. Simple streams sure look like a nice abstraction, and I
> suspect that some people at Franz would welcome feedback on the
> subject.
They did ask for feedback on simple streams here.
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://web.mclink.it/amoroso/ency/README
[http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/]
[sorry for the delay in responding (actually, no, I enjoyed my
vacation too much to be _very_ sorry)]
Juliusz Chroboczek <···@pps.jussieu.fr> writes:
> Hello,
>
> Due to a bout of influenza, I've had the occasion this week-end to
> read the simple streams document. With plenty of tea.
Hopefully by now you're feeling better. I'm glad though, that you had
the time to look at the document.
> To my unexercised eyes, it simple streams look like a rather nice
> piece of work. I do not like Gray stream very much, and the simple
> streams document clarified to me what it was that bothered me about
> them. I feel that I do understand what problems Franz are trying to
> solve, and it looks to me like they have pretty much succeeded.
Thank you, from all of us who worked on it, and for those external
to Franz who have been commenting on it, using it, and in at least one
other case, implementing it.
> In certain ways, the document still looks a little bit underspecified.
Agreed. And thanks for being kind by saying "a little bit"...
The document was a derivative of the design spec, which was
compiled from a long series of brainstorming sessions within
Franz Inc. As a design spec, it does not really qualify as
documentation, except perhaps as a reference guide and because
I was aware of that when I wrote the original document, I tried
to write it in a way that could then be converted to documentation.
Some of the holes in the documentation are due to its reference-like
quality; you have to understand the spec in order to understand the
spec :-) There are too few examples, and no easy way for the
programmer to see the whole picture, unless he has an idea what the
picture should be.
We're constantly working on this, and the current document is the
one you should be looking at:
http://www.franz.com/support/documentation/6.1/doc/streams.htm
In fact, this url should be consulted often, as it will be updated
soon. At the bottom of our 6.1 documentation pages, we have placed
the description "This is the initial (unrevised) page." As changes
are made, this description will change, and will include instructions
on how to download it into your distribution.
Some major changes to this document are forthcoming, probably in a week or
so, which include a couple of encapsulation examples: a rot13 translator,
and a base64 decoder.
> In particular, I couldn't find any information on the metaclass of
> SIMPLE-STREAM (how do you use mixins with instances of
> SIMPLE-STREAM?). (Checking the implementation shows that simple
> streams are standard objects.)
In fact, simple-streams are streams, which are then standard-objects.
You are correct that the simple-stream.htm page gives no description
of its cpl. I will file an enhancement request for those class pages to
include such info. However, in the new 6.1 documentation, there is a
new section, 11.0 "The simple-stream class heirarchy illustrated".
It shows the relationships of all streams, including the structure
of Gray streams (a developer here criticised simple-streams for not
being very simple, and I take his point, but this heirarchy does show
pictorially how simple simple-streams is in at least one dimension).
> I was a little bit surprised to see that the ``implementation
> helpers'' were implemented as distinct methods rather than mixins.
> There are probably good reasons for that, and I would like to see them
> spelled out.
I'm not sure what you mean here. The implementation helpers are not
methods, and not generic-functions at all; they are simple functions
to help users implement the device-layer methods. One can, however,
implement specialized methods either using these helpers, or by using
around methods and/or using call-next-method, or by writing the methods
from scratch.
> A minor nit: I couldn't find a way for the device layer to distinguish
> between FORCE-OUTPUT and FINISH-OUTPUT (think e.g. of a TCP/IP layer
> that does provide a handle to the PUSH flag, or an application-level
> protocol that forces a round-trip on FINISH-OUTPUT but not necessarily
> on FORCE-OUTPUT).
Yes. There is a fundamental problem that asynchronous partial writes
_should_ be available, but in reality we are used to a write always
completing. (Bear with me here, otherwise it is a stretch as to how
this applies to your question).
In general, we established a three-fold blocking concept, of which CL
uses two.
- Non-blocking mode (e.g. read-char-no-hang) does not fail
when input is not ready. And CL doesn't specify any non-blocking write
operations, but such operations are useful for such streams as sockets,
where low-level socket-library buffers may become full and not allow
further writes.
- Blocking mode (e.g. read-char, read-sequence, write-sequence) waits
until the complete sequence is transferred.
- bnb (blocking, then non-blocking) mode will block on the first
element, but will not block for subsequent elements, and thus operations
which allow bnb mode must always indicate how many elements are
successfully transferred. This is the way most operating systems
transfer data.
Blocking mode can be emulated from bnb mode by looping on the operation
until the transfer is complete (each iteration must either advance by
at least one element or block until more data becomes available). Thus,
in the simple-streams implementation, we stndardize on bnb mode and then
use loops to emulate blocking mode for CL functionality.
Finally, to bring the explanation around to the question, FORCE-OUTPUT
is simply a non-blocking device-write, and finish-output is a blocking
write (involving a looping call to device-write until it is finished).
The current _implementation_ of force-output in Allegro CL is the same
as that of force-output, because we do not lock the streams against
further actions after a force-output, which would be a requirement for
force-output to be truly asynchronous. But since most writes can
be initiated completely in one io operation, this is never a problem
in practice.
> I didn't like the new behaviour of OPEN. I feel that OPEN should be
> controlled by a special variable or a new keyword argument (or both),
> not the current :ELEMENT-TYPE hack.
I agree that it is a hack. But it's a good hack :-). Our goal in this
design was to increase the liklihood of acceptance of the new
implementation by reducing the amount of source-code changes required
by users to almost zero. And I think we succeeded. We also wanted the
ability to make Gray-streams autoloadable (since it is a huge system)
dand this required that simple-streams automatically be used in streams
where no source code changes were needed. On the other hand, we did not
want Gray-streams users to have to bend over backwards to re-establish
the use of Gray-streams - indeed, if possible this should also be
automatic. We then noted that most of the usages of Gray-streams in our
own code and in customers' code that we had seen had some element-type
specification, and that left as the only possible incompatible case the
situation where the user did not specify an element-type but was
expecting a Gray stream. In practice this has not turned out to be
a problem; adding ":element-type 'character" to the open call to make
the default specific is relaively easy to do. All other models that we
considered, including the ones that you suggested, would have required
much more extensive source-code changes, both by us and by our customers.
> I was also a little disturbed by the fact that, apparently, the device
> writer has to fall back to Gray streams for unbuffered I/O.
I take it that you are making that assumption based on the paragraph in
section 3.3 where it says that all simple-streams are buffered. And, in
fact, this statement is incorrect; due to encapsulations, not all
simple-streams are buffered. The new section in the upcoming document
revision will make that clearer. Also, I will make this paragraph more
precise, with an exception clause.
> Of
> course, I am not suggesting that simple streams should be extended for
> unbuffered functionality (that would be contrary to the very goal of
> simple streams), but I would like to see a similarly elegant subclass
> of STREAM designed to cater for unbuffered I/O, and Gray streams
> deprecated.
In fact, this is exactly what has happened. As it turns out,
string-simple-streams can be subclassed and used for unbuffered io.
This allows various kinds of encapsulation to be implemented.
Gray-streams has already been deprecated in Allegro CL, though they
will never really go away.
> Finally, I was frustrated by the complete lack of credits. Simple
> streams were designed by human beings, and I would dearly like to know
> who they are.
I think that this was answered by Kevin Layer. Richard Barber sparked
the idea, I wrote the document and implemented simple-streams, but it
was based on brainstorming from the team which included Charley Cox
(our internationalization expert), John Foderaro (our sockets and
advanced streams expert) and Kevin Layer (our manager). I also got
good inputs from several people, including Erik Naggum (who inspired
me to add mapped-files) and others who made general comments. Why
do I feel like I am standing at the podium at the Grammy's, running
out of time and afraid I'll miss someone? :-) Oh yes, thanks, Paul
Foley, for starting an open-source implementation for CMUCL.
> (I was also disappointed by the lack of pointers to related work,
> other than Gray streams of course. Java streams, obviously, but I
> also seem to recall something in relation to MacLisp. I have no idea
> what streams were like in ZetaLisp; perhaps somebody can enlighten me?)
I don't know ZetaLisp either. Any other resemblance to other related
work is purely coincidental, with two exceptions:
- The basic concept of simple-streams is similar to that of C
implementations of getc(), putc(), filbuf(), flsbuf(). To that
extent, and to the extent that I consider myself a very good C
programmer, related work might include C concepts.
- The encapsulation theory, sparked by an early critique of
simple-streams by John Mallery, was described to me by Steve Haflich,
who knows Java. I don't know Java, but saw one documentation page
describing some of the operations on encapsulated streams. To that
extent related work might include Java encapsulations.
> I'm wondering whether other people have thoughts to share on the
> subject. Simple streams sure look like a nice abstraction, and I
> suspect that some people at Franz would welcome feedback on the
> subject.
My ears are constantly tuned for comments on the subject, and I
would love to hear from anyone who has any such comments.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)