From: Liam Healy
Subject: A stream as a pipe
Date: 
Message-ID: <513f36bgcn.fsf@apogee.nrl.navy.mil>
I would like to create a stream in CL that plays the same role as a
pipe in unix.  Specifically, I would like to apply a filter to a
stream to transform the characters.  Let's call it with-filter-stream,
as this example:

(defun filter (char)
  (code-char (1+ (char-code char))))

(defun print-regular (string)
  (princ string *standard-output*)
  (values))

(defun print-shifted (string)
  (with-filter-stream (filter-stream *standard-output* #'filter)
      (princ string filter-stream))
  (values))

where I would get
 (print-regular "hal")  -> hal
 (print-shifted "hal")  -> ibm

I am essentially making a new (temporary) stream.  It exists solely to
accept characters, pass them to the filter, which then passes them to
the "real" stream.  The only functions I see in CL that create new
streams are getting or putting data from strings or files.  It is easy
to implement the macro using a temporary string or file:

(defmacro with-filter-stream ((var connected-stream filter) &body body)
  `(princ (map 'string ,filter
	       (with-output-to-string (,var) ,@body))
	  ,connected-stream))

but this is accumulating data unnecessarily.  I really only need (in
this example) one-character storage, enough to apply the function
before passing it on to the next stream.

Is this possible in Common Lisp?


-- 
Liam Healy
··········@nrl.navy.mil

From: Robert Munyer
Subject: Re: A stream as a pipe
Date: 
Message-ID: <4rmsh3$aek@Venus.mcs.com>
In article <··············@apogee.nrl.navy.mil>,
Liam Healy  <··········@nrl.navy.mil> wrote:

> I would like to create a stream in CL that plays the same role as
> a pipe in unix.  Specifically, I would like to apply a filter to
> a stream to transform the characters [but the] only functions I
> see in CL that create new streams are getting or putting data
> from strings or files.  [...]
>
> Is this possible in Common Lisp?

Sure.  When you mentioned the "only functions I see in CL that
create new streams," you were forgetting MAKE-INSTANCE.  Since
streams are objects, OPEN and other stream creation functions
conceptually call MAKE-INSTANCE eventually.

Here is a minimal implementation of your WITH-FILTER-STREAM which
will do what you want.  It could use some efficiency improvements
and some convenience features, but it will work.

  (defclass filter-output-stream (output-stream)
    ((sink :initarg :sink :reader filter-stream-sink)
     (filter :initarg :filter :reader filter-stream-filter)))

  (defmethod stream-tyo ((stream filter-output-stream) char)
    (stream-tyo (filter-stream-sink stream)
                (funcall (filter-stream-filter stream) char)))

  (defmacro with-filter-stream ((var sink filter) &body body)
    `(with-open-stream (,var (make-instance 'filter-output-stream
                               :sink ,sink :filter ,filter))
       ,@body))

You might want to extend this macro to handle input streams too.
If not, you could give it a name with "OUTPUT" in it, like
WITH-FILTERED-OUTPUT, to make its purpose more obvious.

-- Robert
From: Liam Healy
Subject: Re: A stream as a pipe
Date: 
Message-ID: <5120imx0bi.fsf@apogee.nrl.navy.mil>
······@MCS.COM (Robert Munyer) writes:
> 
> In article <··············@apogee.nrl.navy.mil>,
> Liam Healy  <··········@nrl.navy.mil> wrote:
> 
> > I would like to create a stream in CL that plays the same role as
> > a pipe in unix.  Specifically, I would like to apply a filter to
> > a stream to transform the characters [but the] only functions I
> > see in CL that create new streams are getting or putting data
> > from strings or files.  [...]
> >
> > Is this possible in Common Lisp?
> 
> Sure.  When you mentioned the "only functions I see in CL that
> create new streams," you were forgetting MAKE-INSTANCE.  Since
> streams are objects, OPEN and other stream creation functions
> conceptually call MAKE-INSTANCE eventually.
> 
> Here is a minimal implementation of your WITH-FILTER-STREAM which
> will do what you want.  It could use some efficiency improvements
> and some convenience features, but it will work.
> 
>   (defclass filter-output-stream (output-stream)
>     ((sink :initarg :sink :reader filter-stream-sink)
>      (filter :initarg :filter :reader filter-stream-filter)))
> 
>   (defmethod stream-tyo ((stream filter-output-stream) char)
>     (stream-tyo (filter-stream-sink stream)
>                 (funcall (filter-stream-filter stream) char)))
> 
>   (defmacro with-filter-stream ((var sink filter) &body body)
>     `(with-open-stream (,var (make-instance 'filter-output-stream
>                                :sink ,sink :filter ,filter))
>        ,@body))
> 
> You might want to extend this macro to handle input streams too.
> If not, you could give it a name with "OUTPUT" in it, like
> WITH-FILTERED-OUTPUT, to make its purpose more obvious.
> 
> -- Robert

Sorry, but this just doesn't fly on either LISP I have access to.
CMU Common Lisp 17f, running on eutaw

* (find-class 'output-stream)

Error in function FIND-CLASS:  Class not yet defined:
  OUTPUT-STREAM

I get a similar error in Symbolics LISP, and I find no documentation
of any such class in the CL references I have, which leads me to
believe that this must be a definition of your particular
implementation.  Closer investigation shows that a file stream is
implemented as a structure in CMUCL, and it's not clear (to me) how to
define a specialized stream.

But I am suprised that CL appears not to have something like this --
this is what I expected because it seems like a natural.

-- 
Liam Healy
··········@nrl.navy.mil
From: Barry Margolin
Subject: Re: A stream as a pipe
Date: 
Message-ID: <4rvjej$43v@tools.bbnplanet.com>
[Regarding a CLOS-based I/O system and standard way to extend it]

In article <··············@apogee.nrl.navy.mil>,
Liam Healy  <··········@nrl.navy.mil> wrote:
>But I am suprised that CL appears not to have something like this --
>this is what I expected because it seems like a natural.

Yes, it's a natural.  But designing a generic I/O system in a portable way
is extremely non-trivial.  Consider all the complexity in the Symbolics I/O
system in order to support rubout-processing.

By the time all the details of CLOS were worked out, there wasn't really
enough time left in the standardization process to develop something major
like this.  Also, most existing CL implementations don't use CLOS for their
I/O subsystem, so they all would have had to redo this in order to conform
(there's a similar reason why CL doesn't require the condition system to be
based on CLOS).
-- 
Barry Margolin
BBN Planet, Cambridge, MA
······@bbnplanet.com -  Phone (617) 873-3126 - Fax (617) 873-6351
(BBN customers, please call (800) 632-7638 option 1 for support)
From: Marco Antoniotti
Subject: Re: A stream as a pipe
Date: 
Message-ID: <s08n317zuos.fsf@salmon.ICSI.Berkeley.EDU>
In article <··············@apogee.nrl.navy.mil> Liam Healy <··········@nrl.navy.mil> writes:

   From: Liam Healy <··········@nrl.navy.mil>
   Newsgroups: comp.lang.lisp
   Cc: ······@MCS.COM (Robert Munyer) 
   Date: 08 Jul 1996 10:54:25 -0400
   Organization: Naval Research Laboratory
   Lines: 62
   Sender: ·····@apogee.nrl.navy.mil
   References: <··············@apogee.nrl.navy.mil> <··········@Venus.mcs.com>
   X-Newsreader: Gnus v5.2.25/XEmacs 19.14

   ······@MCS.COM (Robert Munyer) writes:
   > 
   > In article <··············@apogee.nrl.navy.mil>,
   > Liam Healy  <··········@nrl.navy.mil> wrote:
   > 
   > > I would like to create a stream in CL that plays the same role as
   > > a pipe in unix.  Specifically, I would like to apply a filter to
   > > a stream to transform the characters [but the] only functions I
   > > see in CL that create new streams are getting or putting data
   > > from strings or files.  [...]
   > >
   > > Is this possible in Common Lisp?
   > 
   > Sure.  When you mentioned the "only functions I see in CL that
   > create new streams," you were forgetting MAKE-INSTANCE.  Since
   > streams are objects, OPEN and other stream creation functions
   > conceptually call MAKE-INSTANCE eventually.
   > 
   > Here is a minimal implementation of your WITH-FILTER-STREAM which
   > will do what you want.  It could use some efficiency improvements
   > and some convenience features, but it will work.
   > 
   >   (defclass filter-output-stream (output-stream)
   >     ((sink :initarg :sink :reader filter-stream-sink)
   >      (filter :initarg :filter :reader filter-stream-filter)))
   > 
   >   (defmethod stream-tyo ((stream filter-output-stream) char)
   >     (stream-tyo (filter-stream-sink stream)
   >                 (funcall (filter-stream-filter stream) char)))
   > 
   >   (defmacro with-filter-stream ((var sink filter) &body body)
   >     `(with-open-stream (,var (make-instance 'filter-output-stream
   >                                :sink ,sink :filter ,filter))
   >        ,@body))
   > 
   > You might want to extend this macro to handle input streams too.
   > If not, you could give it a name with "OUTPUT" in it, like
   > WITH-FILTERED-OUTPUT, to make its purpose more obvious.
   > 
   > -- Robert

   Sorry, but this just doesn't fly on either LISP I have access to.
   CMU Common Lisp 17f, running on eutaw

   * (find-class 'output-stream)

   Error in function FIND-CLASS:  Class not yet defined:
     OUTPUT-STREAM

   I get a similar error in Symbolics LISP, and I find no documentation
   of any such class in the CL references I have, which leads me to
   believe that this must be a definition of your particular
   implementation.  Closer investigation shows that a file stream is
   implemented as a structure in CMUCL, and it's not clear (to me) how to
   define a specialized stream.

   But I am suprised that CL appears not to have something like this --
   this is what I expected because it seems like a natural.

You need the "extended streams" which are specified in the CLIM specs
and (partly) in Sonya Keene's book.

Whether there is a portable (and *efficient*) implementation around I
do not know, although I have faint memories of seeing something like
this in the CLUE/CLIO stuff.

Anybody has a clearer picture?

Cheers
-- 
Marco Antoniotti - Resistente Umano
===============================================================================
International Computer Science Institute	| ·······@icsi.berkeley.edu
1947 Center STR, Suite 600			| tel. +1 (510) 643 9153
Berkeley, CA, 94704-1198, USA			|      +1 (510) 642 4274 x149
===============================================================================
	...it is simplicity that is difficult to make.
	...e` la semplicita` che e` difficile a farsi.
				Bertholdt Brecht
From: Ralf Muschall
Subject: Re: A stream as a pipe
Date: 
Message-ID: <6CpOfonGy0B@elefant.Jena.Thur.De>
··········@nrl.navy.mil (Liam Healy)  wrote in
comp.lang.lisp <··············@apogee.nrl.navy.mil>:

> ······@MCS.COM (Robert Munyer) writes:
> >
> > In article <··············@apogee.nrl.navy.mil>,
> > Liam Healy  <··········@nrl.navy.mil> wrote:
> >
> > > I would like to create a stream in CL that plays the same role as
> > > a pipe in unix.  Specifically, I would like to apply a filter to

Maybe that this is not what you want, but it might be useful for the
purpose you need it for: You might give the *Series* package a try.
From an user's point of view, they are just like pipes. In reality they
are complicated macros that compile into loops. But they are not special
cases of streams, so you need to replace the calls to
(print data mystream) with something else.

HTH, Ralf

Btw., CLtL2 quotes 2 MIT memos (refs. [52,53]) by R. Waters apparently
giving some more information abou series than what is printed there. Are
the somehow available in a paper-free form? As I care only about usage,
not implementation, the first one seems to be the one one should
read <?>.

Ralf