I'm learning CL and I am writing a quick utility for use at work to
interface with RRDTool (a data logging and graphing tool). RRTTool
can run in an interactive mode where commands are read from standard
input and the result of the command is sent to standard output. I
want to use this from with lisp but I am unsure if I am using the
correct idioms and would appreciate feedback (the code below may not
be correct but I'm interested in the high-level question).
Here is how I want to use this functionality (I've capitalized the
questionable areas):
(defvar *RRD-INPUT* nil)
(defvar *RRD-OUTPUT* nil)
(defmacro WITH-RRDTOOL (&body body)
(let ((rrd (gensym)))
`(let* ((,rrd (ext:run-program "rrdtool" "-"
:input :stream
:output :stream))
(*rrd-input* (process-input ,rrd))
(*rrd-output* (process-output ,rrd)))
(unwind-protect (progn ,@body)
(process-close ,rrd)))))
(defun RRD-UPDATE (filename swin sbuf rwin rbuf)
(format *RRD-INPUT* "update ~A ~A ~A ~A~%" swin sbuf rwin rbuf)
(let ((result (read-line *RRD-OUTPUT*)))
;; Check the result
))
(defun parse-netstat-log (filename)
(WITH-RRDTOOL
(dolines (line filename)
(register-groups-bind (src dst swin sbuf rwin rbuf)
("^(\\S+)\\s+(\\S+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)"
line)
(LOG-DATA src dst swin sbuf rwin rbuf)))))
(defun LOG-DATA (src dst swin sbuf rwin rbuf)
(RRD-UPDATE (format nil "~A-~A.rrd" src dst) swin sbuf rwin rbuf))
My question is on the design of the WITH-RRDTOOL macro. Is it poor
form for my macro to set the special variables *RRD-INPUT* and
*RRD-OUTPUT* and then have all my associated RRD-* functions use those
specials?
Or should I do something like and explicitly pass around the stream
handles but giving the user the choice of the names for them
(although I don't see a reason why the user should have a need for the
handles).
(defun parse-netstat-log (filename)
(WITH-RRDTOOL (IN OUT)
(dolines (line filename)
(register-groups-bind (src dst swin sbuf rwin rbuf)
("^(\\S+)\\s+(\\S+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)"
line)
(log-data IN OUT src dst swin sbuf rwin rbuf)))))
(defmacro with-rrdtool ((IN OUT) &body body)
(let ((rrd (gensym)))
`(let* ((,rrd (ext:run-program "rrdtool" "-"
:input :stream
:output :stream))
(,in (process-input ,rrd))
(,out (process-output ,rrd)))
(unwind-protect (progn ,@body)
(process-close ,rrd)))))
(defun rrd-update (IN OUT filename swin sbuf rwin rbuf)
(format IN "update ~A ~A ~A ~A~%" swin sbuf rwin rbuf)
(let ((result (read-line OUT)))
;; Check the result
))
Or would it be better if I passed the process handle around (again,
not too sure why the user would need the process handle either.)
(defun parse-netstat-log (filename)
(WITH-RRDTOOL (PROCESS)
(dolines (line filename)
(register-groups-bind (src dst swin sbuf rwin rbuf)
("^(\\S+)\\s+(\\S+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)"
line)
(log-data PROCESS src dst swin sbuf rwin rbuf)))))
(defmacro with-rrdtool ((RRD) &body body)
`(let* ((,rrd (ext:run-program "rrdtool" "-"
:input :stream
:output :stream)))
(unwind-protect (progn ,@body)
(process-close ,rrd)))))
(defun rrd-update (PROCESS filename swin sbuf rwin rbuf)
(format (PROCESS-INPUT PROCESS) "update ~A ~A ~A ~A~%" swin sbuf rwin rbuf)
(let ((result (read-line (PROCESS-OUTPUT PROCESS)))
;; Check the result
))
I suppose the reason I am confused is that I have not seen a WITH-*
macro yet that doesn't expect the caller to pass the names of symbols
that can be used while in the WITH form. This seems to indicate to me
that I'm doing something wrong.
Although, I really don't want the caller to have access to the
internals, but with my original WITH- macro where I hide things, its
not clear to the caller that the RRD-* functions must be invoked
within the WITH-RRDTOOL form.
Thanks,
Pete
In article <··············@coco.kazmier.com>,
Pete Kazmier <··················@kazmier.com> wrote:
> I'm learning CL and I am writing a quick utility for use at work to
> interface with RRDTool (a data logging and graphing tool). RRTTool
> can run in an interactive mode where commands are read from standard
> input and the result of the command is sent to standard output. I
> want to use this from with lisp but I am unsure if I am using the
> correct idioms and would appreciate feedback (the code below may not
> be correct but I'm interested in the high-level question).
>
> Here is how I want to use this functionality (I've capitalized the
> questionable areas):
>
> (defvar *RRD-INPUT* nil)
> (defvar *RRD-OUTPUT* nil)
>
> (defmacro WITH-RRDTOOL (&body body)
> (let ((rrd (gensym)))
> `(let* ((,rrd (ext:run-program "rrdtool" "-"
> :input :stream
> :output :stream))
> (*rrd-input* (process-input ,rrd))
> (*rrd-output* (process-output ,rrd)))
> (unwind-protect (progn ,@body)
> (process-close ,rrd)))))
I'd make a slight change:
(defmacro with-rrdtool (&body body)
(let ((rrd (gensym)))
`(let (,rrd *rrd-input* *rrd-output*)
(unwind-protect
(progn (setq ,rrd (ext:run-program ...)
*rrd-input* (process-input ,rrd)
*rrd-output* (process-output ,rrd))
,@body)
(when ,rrd
(process-close ,rrd))))))
The difference is that your version can fail to call PROCESS-CLOSE if
the function is interrupted while initializing the variables in the
LET*. In mine the window is much smaller -- only if it's interrupted
between RUN-PROGRAM returning and the value being stored in the variable.
> (defun RRD-UPDATE (filename swin sbuf rwin rbuf)
> (format *RRD-INPUT* "update ~A ~A ~A ~A~%" swin sbuf rwin rbuf)
> (let ((result (read-line *RRD-OUTPUT*)))
> ;; Check the result
> ))
>
> (defun parse-netstat-log (filename)
> (WITH-RRDTOOL
> (dolines (line filename)
> (register-groups-bind (src dst swin sbuf rwin rbuf)
> ("^(\\S+)\\s+(\\S+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)\\s+(\\d+)"
> line)
> (LOG-DATA src dst swin sbuf rwin rbuf)))))
>
> (defun LOG-DATA (src dst swin sbuf rwin rbuf)
> (RRD-UPDATE (format nil "~A-~A.rrd" src dst) swin sbuf rwin rbuf))
>
> My question is on the design of the WITH-RRDTOOL macro. Is it poor
> form for my macro to set the special variables *RRD-INPUT* and
> *RRD-OUTPUT* and then have all my associated RRD-* functions use those
> specials?
Would it ever make sense to have nested calls to WITH-RRDTOOL, each
accessing a different rrdtool process? If so, using special variables
makes this harder, since you can't access the outer bindings from within
the nested macro.
>
> Or should I do something like and explicitly pass around the stream
> handles but giving the user the choice of the names for them
> (although I don't see a reason why the user should have a need for the
> handles).
A common solution is to make the variables optional, and default to the
special variables.
(defmacro with-rrdtool ((&key (input '*rdtool-input*)
(output '*rdtool-output*))
&body body)
...)
> I suppose the reason I am confused is that I have not seen a WITH-*
> macro yet that doesn't expect the caller to pass the names of symbols
> that can be used while in the WITH form. This seems to indicate to me
> that I'm doing something wrong.
WITH-OPEN-FILE and WITH-OPEN-STREAM require the user to specify the
variables to be bound.
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
Barry Margolin <······@alum.mit.edu> writes:
> In article <··············@coco.kazmier.com>,
> Pete Kazmier <··················@kazmier.com> wrote:
>
>> (defmacro WITH-RRDTOOL (&body body)
>> (let ((rrd (gensym)))
>> `(let* ((,rrd (ext:run-program "rrdtool" "-"
>> :input :stream
>> :output :stream))
>> (*rrd-input* (process-input ,rrd))
>> (*rrd-output* (process-output ,rrd)))
>> (unwind-protect (progn ,@body)
>> (process-close ,rrd)))))
>
> I'd make a slight change:
>
> (defmacro with-rrdtool (&body body)
> (let ((rrd (gensym)))
> `(let (,rrd *rrd-input* *rrd-output*)
> (unwind-protect
> (progn (setq ,rrd (ext:run-program ...)
> *rrd-input* (process-input ,rrd)
> *rrd-output* (process-output ,rrd))
> ,@body)
> (when ,rrd
> (process-close ,rrd))))))
>
> The difference is that your version can fail to call PROCESS-CLOSE
> if the function is interrupted while initializing the variables in
> the LET*. In mine the window is much smaller -- only if it's
> interrupted between RUN-PROGRAM returning and the value being stored
> in the variable.
Thank you for the suggestion. I do have one followup question based
on this advice. Is it strictly necessary to include ext:run-program
within the unwind-protect? I only ask because of two examples that
I've read recently in which the first form that yields the object we
are protecting is not included.
1) Example of using unwind-protect in PCL[1]:
(defmacro with-database-connection ((var &rest open-args) &body body)
`(let ((,var (open-connection ,@open-args)))
(unwind-protect (progn ,@body)
(close-connection ,var))))
Should the unwind-protect encompass the open-connection call?
2) Example in Lemonodor's blog entry about Exceptions[2]:
(defun http-get (url)
(let* ((host (url-host url))
(port (url-port url))
(stream (open-stream host port))
(success-p NIL))
(unwind-protect
(progn
(format stream ...)
(force-output stream)
(let ((result (list (response-read-code stream)
(response-read-headers stream)
stream)))
(setf success-p T)
result))
(unless success-p
(close stream)))))
Should the unwind-protect encompass the open-stream call?
[1] http://www.gigamonkeys.com/book/the-special-operators.html#unwinding-the-stack
[2] http://lemonodor.com/archives/001145.html
In article <··············@coco.kazmier.com>,
Pete Kazmier <··················@kazmier.com> wrote:
> Barry Margolin <······@alum.mit.edu> writes:
> > The difference is that your version can fail to call PROCESS-CLOSE
> > if the function is interrupted while initializing the variables in
> > the LET*. In mine the window is much smaller -- only if it's
> > interrupted between RUN-PROGRAM returning and the value being stored
> > in the variable.
>
> Thank you for the suggestion. I do have one followup question based
> on this advice. Is it strictly necessary to include ext:run-program
> within the unwind-protect? I only ask because of two examples that
> I've read recently in which the first form that yields the object we
> are protecting is not included.
Unfortunately, it's quite common to use this style, where you open
something in the variable initializations, outside the UNWIND-PROTECT.
It makes the code simpler, because you don't need a separate binding and
assignment, and you don't have to check for the variable being null in
the cleanup expression.
But it means the window of vulnerability is a little bigger, so I
recommend using the slightly more complex form that I showed.
>
> 1) Example of using unwind-protect in PCL[1]:
>
> (defmacro with-database-connection ((var &rest open-args) &body body)
> `(let ((,var (open-connection ,@open-args)))
> (unwind-protect (progn ,@body)
> (close-connection ,var))))
>
> Should the unwind-protect encompass the open-connection call?
>
> 2) Example in Lemonodor's blog entry about Exceptions[2]:
>
> (defun http-get (url)
> (let* ((host (url-host url))
> (port (url-port url))
> (stream (open-stream host port))
> (success-p NIL))
> (unwind-protect
> (progn
> (format stream ...)
> (force-output stream)
> (let ((result (list (response-read-code stream)
> (response-read-headers stream)
> stream)))
> (setf success-p T)
> result))
> (unless success-p
> (close stream)))))
>
> Should the unwind-protect encompass the open-stream call?
>
>
>
> [1]
> http://www.gigamonkeys.com/book/the-special-operators.html#unwinding-the-stack
> [2] http://lemonodor.com/archives/001145.html
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
Pete Kazmier wrote:
> (defmacro WITH-RRDTOOL (&body body)
There is a general syntax convention of with-* macros that there
is a first subform that pertains to the bindings or whatever
established by the macro, and that subsequent subforms are &body
forms. This covention is strong enough that it is used even for
macros that accept no modifiesr for the context established by
the macro. Programming tools such as the automatic Lisp indenter
in Emacs understand this and typically indent a with=* about which
they hve no specific information with the first subform indented.
If the fist subform is a regular body form it is misleading to
indent it. The real issue, of course, is that it is wasier for
programmers to remember the simple rule that the first subform of
any with-* operator consists of modifiers for the body but is not
part of the body. If this rule is obeyed consistently, when a
programmer can think of no necessary modifiers for the body, be
can (and should) code nil as the first subform without confusion
whether this is part of the body.
These are the macros in the CL package that are named with-*:
with-accessors (slots instance &body body)
with-compilation-unit ((&key override warnings-hook) &body body)
with-condition-restarts (condition-form restarts-form &body forms)
with-hash-table-iterator ((next-fn hash-table) &body body)
with-input-from-string ((var string &key index ...) &body body)
with-open-file ((var &rest open-args) &body body)
with-open-stream ((var stream) &body body)
with-output-to-string ((var &optional string &key ...) &body body)
with-package-iterator ((next-fn package-list &rest symbol-types)
&body body)
with-simple-restart ((restart-name format-control
&rest format-arguments)
&body forms)
with-slots (slots instance &body body)
with-standard-io-syntax (&body body)
I would argue that the four macros that don't obey this convention
have broken designs and would have beeter have been named something
not beginning with "with-". They make the language less regular and
harder to remember, not to mention learn.
I would code your macro this way:
(defmacro with-rrdtool (() &body hody) ...)
Pete Kazmier <··················@kazmier.com> writes:
> (defvar *RRD-INPUT* nil)
> (defvar *RRD-OUTPUT* nil)
>
> (defmacro WITH-RRDTOOL (&body body)
> (let ((rrd (gensym)))
> `(let* ((,rrd (ext:run-program "rrdtool" "-"
> :input :stream
> :output :stream))
> (*rrd-input* (process-input ,rrd))
> (*rrd-output* (process-output ,rrd)))
> (unwind-protect (progn ,@body)
> (process-close ,rrd)))))
...
> My question is on the design of the WITH-RRDTOOL macro. Is it poor
> form for my macro to set the special variables *RRD-INPUT* and
> *RRD-OUTPUT* and then have all my associated RRD-* functions use those
> specials?
>
> Or should I do something like and explicitly pass around the stream
> handles but giving the user the choice of the names for them
> (although I don't see a reason why the user should have a need for the
> (defmacro with-rrdtool ((IN OUT) &body body)
> (let ((rrd (gensym)))
> `(let* ((,rrd (ext:run-program "rrdtool" "-"
> :input :stream
> :output :stream))
> (,in (process-input ,rrd))
> (,out (process-output ,rrd)))
> (unwind-protect (progn ,@body)
> (process-close ,rrd)))))
This would be my preferred solution. It conforms to the general
expectations of other WITH-... macros and also provides additional
flexibility. One benefit with being able to specify the names is that
you then don't have to worry about dynamic scope.
I don't know if it would make any sense to have WITH-RRDTOOL macros
nested, but you do get extra flexibilty by being able to pass the
argument names explicitly. The problem with special variables is that
if something internal to code you call withing the WITH-RRDTOOL macro
references the special variables, they may get values they aren't
expecting.
On the other hand, it does save you passing variable names if the
implicit nature doesn't really hurt you. I've written WITH-... macros
using both of these methods of communicating, depending on situation. I
tend to use the global binding approach only if the WITH... macro is
setting up some sort of global context for operations that occur within
its body -- particularly if the effects pervade to great depth, and use
the individual variable part if the purpose is to provide the bindings
for use by functions that are called directly in the body of the
WITH... macro.
> I suppose the reason I am confused is that I have not seen a WITH-*
> macro yet that doesn't expect the caller to pass the names of symbols
> that can be used while in the WITH form. This seems to indicate to me
> that I'm doing something wrong.
As with most things, this comes down to a design question about how this
is supposed to work.
BTW, if you want to have a macro that shouldn't be nested, you can
accomplish that with special variable bindings and a test. One simple
approach would be to not provide values in the DEFVAR statements:
(defvar *RRD-INPUT*)
and use a (boundp '*RRD-INPUT*) test in the expanded macro to signal a
run-time error if nested invocations occur. I don't think there would
be a way to detect this at macro-expansion time.
--
Thomas A. Russ, USC/Information Sciences Institute