Hi,
I want to be able to make my own type of stream which can act like a
filter on an other stream.
I want to be able to do something like this:
(with-open-stream (foo "myfile" :direction :output)
(with-stream-filter (bar foo (cl-ppcre:regex-replace "regexp" bar
"new-value"))
(dotimes (i 10) (read-line bar))))
Then when I read from the stream bar, every occurrence of the regexp is
replaced with "new-value" form the file "myfile".
I tried to solve this with symbol-macrolet but it did not work 100% the
way I wanted. So I probably have to make a special stream. Does anybody
have any idea on how to do this?
·······@gmail.com" <······@gmail.com> writes:
> I want to be able to make my own type of stream which can act like a
> filter on an other stream.
>
> I want to be able to do something like this:
>
>
> (with-open-stream (foo "myfile" :direction :output)
> (with-stream-filter (bar foo (cl-ppcre:regex-replace "regexp" bar
> "new-value"))
> (dotimes (i 10) (read-line bar))))
>
>
> Then when I read from the stream bar, every occurrence of the regexp is
> replaced with "new-value" form the file "myfile".
>
> I tried to solve this with symbol-macrolet but it did not work 100% the
> way I wanted. So I probably have to make a special stream. Does anybody
> have any idea on how to do this?
Use Gray Streams. This is a de-facto standard NOT included in Common Lisp.
Otherwise, if your implementation doesn't provide Gray Streams and no
Gray Stream library is available for it, then you can always write a
package where you shadow CL stream functions and you write your own,
CLOS based, to be able write your own filter stream subclass.
But it may not even be worthwhile: if you don't need to pass your
stream to existing code [of which you must have the source to be able
to substitute (:use "CL-WITH-MY-STREAMS") for (:use "COMMON-LISP")],
then you can just write a class and your own I/O function
independantly from those of Common Lisp:
(with-open-stream (foo "myfile" :direction :output)
(with-stream-filter (bar foo (cl-ppcre:regex-replace
"regexp" bar "new-value"))
(dotimes (i 10) (filter-read-line bar))))
is not too bad.
--
__Pascal Bourguignon__ http://www.informatimago.com/
So, I used simple-streams.
Now I would like some comments on this. I did this small hack to learn
more lisp.
I must point out that the code is not tested very much, I just put it
together just so it would work.
It does not work with read-sequence, and I don't know why yet. I'm also
unsure how to do the unwind-protect part the correct way. Right now a
stream can potentially not be closed correctly.
further on, I would like suggestions on how to improve the
functionality to make it more useful. The obvious part is to also make
a write filter and a replace filter.
when you use "()" in a regex the result will be whats in the
parentheses. when you don't use parentheses, the result will be the
whole line.
PS: blame google-groups for any obscure indenting
(defpackage :sfilter
(:use :common-lisp
:stream
:cl-ppcre))
(in-package :sfilter)
(eval-when (compile)
(require :simple-streams)
(require :iodefs))
(eval-when (compile load eval)
(defconstant stream-filter-buffer-size 4000))
(def-stream-class input-filter-stream
(single-channel-simple-stream)
((filter :initform (error ":filter Must have arg") :initarg :filter
:accessor filter)))
(defmethod device-open ((stream input-filter-stream) options)
;; be sure to make an raw data buffer (a string) and its count
(let* ((base-stream (getf options :base-stream))
(external-format (or (getf options :external-format)
(and base-stream
(stream-external-format
base-stream)))))
(if base-stream
(with-stream-class (input-filter-stream stream)
(unless (sm excl::buffer stream)
(setf (sm excl::buffer stream)
(make-array (device-buffer-length stream) :element-type
'character)))
(setf (sm excl::input-handle stream) base-stream)
(add-stream-instance-flags stream :string :input :simple)
(compose-encapsulating-streams stream external-format)
(stream::install-single-channel-character-strategy
(sm excl::melded-stream stream) external-format nil)
t)
(progn (error "input-filter-stream needs a :base-stream")
nil))))
(defmethod device-read ((stream input-filter-stream) buffer start end
blocking)
(with-stream-class (input-filter-stream stream)
(let ((base-stream (sm excl::input-handle stream)) ;; must be a
stream
(buffer (sm excl::buffer stream))
(new-buff (make-sequence 'string #.stream-filter-buffer-size))
(filter (filter stream)))
(with-stream-class (stream base-stream)
(multiple-value-bind (res done)
(funcall-stm-handler excl::j-read-chars base-stream new-buff
#\Newline 0
(or (and end
(- end start)
(min (- end start) #.stream-filter-buffer-size))
#.stream-filter-buffer-size)
(not (null blocking)))
(if (zerop res)
(if (eq done :eof)
-1
0)
(progn
(multiple-value-bind (str spec) (cl-ppcre::scan-to-strings filter
new-buff :start 0 :end res)
(if str ;; when we have a match
(progn
(if (zerop (length spec)) ;; And if we not have used ()
nil ;; (setf new-buff str) ;; get the result
(progn
(setf new-buff (aref spec 0)) ;; maybe we used ()
(setf res (length new-buff)))) ;; how many characters do we
have
(dotimes (i res) ;; Copy the buffer
(setf (aref buffer i)
(aref new-buff i)))
(setf (aref buffer res) #\Newline) ;; We have to add the newline
removed by j-read-chars
(if (zerop res)
0
(1+ res))) ;; Return the length of result
+ newline
0))))))))) ;; If no match return 0
read chars..
(defmacro with-open-filter-stream ((s path filter) &body body)
(let ((base (gensym)))
`(let* ((,base (make-instance 'stream::file-simple-stream :filename
,path))
(,s (make-instance 'sfilter::input-filter-stream
:base-stream ,base
:filter ,filter)))
(unwind-protect
(progn
,@body)
(progn (close ,s) (close ,base))))))
(defun test (&optional path filter)
(sfilter::with-open-filter-stream (s path filter)
(loop as x = (read-line s nil nil)
while x
collect x)))