I'm working on a LW CAPI program that does some network monitoring
with a neato gui display. One of the setup screens lets me enter and
modify the list of network probes that the program will monitor.
Below, I'm using make-load-form to save out the probe list when the
program is closed, and read/eval to re-assemble the list when the
program is started.
When I started to seriously learn CL, I came across the ALU page
http://www.lisp.org/table/style.htm#efficiency
that urges beginners to avoid eval, which I've done till now. Since
the below code works, is eval correctly or at least reasonably used in
this case?
I'm aware the unprotected eval is a security hole, but the program
does not run suid/sgid and the config file is located in the user's
home directory, so I think thats good enough for the moment.
Thanks,
Greg
(defclass probe-definition ()
((name :accessor probe-name :initarg :name)
(host :accessor probe-host :initarg :hostname)
(spec :accessor probe-spec :initarg :spec)
(interval :accessor probe-interval :initarg :interval)
(cmdqueue :initform nil :accessor probe-cmdqueue)
(dataqueue :initform nil :accessor probe-dataqueue)
(rstream :initform nil :accessor probe-remote-stream)
(pktproc :initform nil :accessor probe-dataproc) ) )
(defmethod make-load-form ((obj probe-definition) &optional env)
`(create-probe ,(probe-name obj) ,(probe-host obj) ,(probe-spec obj) ,(probe-interval obj)))
(defmethod load-probes ((interface qos-interface))
(with-standard-io-syntax
(with-open-file (str +CONFIG-PATHNAME+
:direction :input
:if-does-not-exist :create)
(setf (collection-items (probelist interface))
(coerce (loop for e = (eval (read str nil nil))
while e
collect e) 'vector)) )))
(defmethod save-probes ((interface qos-interface))
(with-standard-io-syntax
(with-open-file (str +CONFIG-PATHNAME+
:direction :output
:if-exists :supersede
:if-does-not-exist :create)
(loop for e in (coerce (collection-items (probelist interface)) 'cons)
do
(print (make-load-form e) str)) )))
Greg Menke <··········@toadmail.com> writes:
> Below, I'm using make-load-form to save out the probe list when the
> program is closed, and read/eval to re-assemble the list when the
> program is started.
> When I started to seriously learn CL, I came across the ALU page
> http://www.lisp.org/table/style.htm#efficiency
> that urges beginners to avoid eval, which I've done till now. Since
> the below code works, is eval correctly or at least reasonably used in
> this case?
IMHO the warning still applies. OTOH, why bother uch?
To me it looks like you're making superfluous use of EVAL, whereas it
would be just as simple to use READ and FUNCALL/APPLY any known
creator function yourself. I.e. you'd be writing a tiny interpreter
for your custom format configuration file. It's very little
additional code.
What's the difference between writing out a file full of
(progn (create-probe '"grin" "foo.com" '(spec ...)))
and
(create-probe "grin" "foo.com" (spec ...))
? One looks like directly EVAL'able code, while the other is a data
structure, whose layout is chosen so as to be easily funcallable.
With the latter, you could still use READ, check that it's a cons
whose car/first EQ's 'CREATE-PROBE and funcall/APPLY the cdr/rest.
No EVAL.
BTW, be careful about quoting.
You don't want to
(apply 'create-probe '('"grin" '(spec ...))) ;wrong
but rather
(apply 'create-probe '("grin" (spec")))
> I'm aware the unprotected eval is a security hole,
Did you know about *READ-EVAL* as a security hole?
> (coerce (loop for e = (eval (read str nil nil))
> while e
> collect e) 'vector)) )))
You might wish to find out about adjustable vectors with a fill
pointer and VECTOR-PUSH-EXTEND instead of going through throw-away
lists. (note that here, both eval and read would generate a lot
of garbage anyway)
> (loop for e in (coerce (collection-items (probelist interface)) 'cons)
> do
> (print (make-load-form e) str)) )))
LOOP has FOR ACROSS to operate on vectors.
In addition to writing a mini-interpreter for reading your data, you
may also want to write a mini-saver. Once in a similar need, I
arranged for all my data to be easy to write out using PRINT, e.g. I
used a-lists and p-lists, numbers, strings and symbols and such, but
no hash-tables.
Regards,
Joerg Hoehle
TSI ITC-Security Technologiezentrum
Joerg-Cyril Hoehle <······@nospam.com> writes:
> Greg Menke <··········@toadmail.com> writes:
> > Below, I'm using make-load-form to save out the probe list when the
> > program is closed, and read/eval to re-assemble the list when the
> > program is started.
>
> > When I started to seriously learn CL, I came across the ALU page
> > http://www.lisp.org/table/style.htm#efficiency
> > that urges beginners to avoid eval, which I've done till now. Since
> > the below code works, is eval correctly or at least reasonably used in
> > this case?
>
> IMHO the warning still applies. OTOH, why bother uch?
>
> To me it looks like you're making superfluous use of EVAL, whereas it
> would be just as simple to use READ and FUNCALL/APPLY any known
> creator function yourself. I.e. you'd be writing a tiny interpreter
> for your custom format configuration file. It's very little
> additional code.
>
> What's the difference between writing out a file full of
> (progn (create-probe '"grin" "foo.com" '(spec ...)))
> and
> (create-probe "grin" "foo.com" (spec ...))
> ? One looks like directly EVAL'able code, while the other is a data
> structure, whose layout is chosen so as to be easily funcallable.
>
> With the latter, you could still use READ, check that it's a cons
> whose car/first EQ's 'CREATE-PROBE and funcall/APPLY the cdr/rest.
> No EVAL.
Thats where I'm headed, as the config file will have to contain more
than just a set of probe object instances, and I don't want to make it
fixed format. I guess the idea is to make a "config file
mini-language" suited to representing the data I need- then have a
little reader and a little saver.
>
> > I'm aware the unprotected eval is a security hole,
> Did you know about *READ-EVAL* as a security hole?
In what way is it an additional hole?
> > (coerce (loop for e = (eval (read str nil nil))
> > while e
> > collect e) 'vector)) )))
>
> You might wish to find out about adjustable vectors with a fill
> pointer and VECTOR-PUSH-EXTEND instead of going through throw-away
> lists. (note that here, both eval and read would generate a lot
> of garbage anyway)
I always forget them- sheesh. I've surfed them a thousand times and
promptly forgotten. I will remember one day....
Thanks,
Gregm
Greg Menke <··········@toadmail.com> writes:
> Below, I'm using make-load-form to save out the probe list when the
> program is closed, and read/eval to re-assemble the list when the
> program is started.
I don't think you really want MAKE-LOAD-FORM here. Have a look at
http://groups.google.com/groups?selm=87adbi49u9.fsf%40darkstar.cartan
Regards,
--
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."
PGP key ID 0x0655CFA0
Nils Goesche <······@cartan.de> writes:
> Greg Menke <··········@toadmail.com> writes:
>
> > Below, I'm using make-load-form to save out the probe list when the
> > program is closed, and read/eval to re-assemble the list when the
> > program is started.
>
> I don't think you really want MAKE-LOAD-FORM here. Have a look at
>
> http://groups.google.com/groups?selm=87adbi49u9.fsf%40darkstar.cartan
I looked at the posting and it is instructive. I understand that I
could write and read the object instances by hand, but I thought the
make-load-form lets me avoid fiddling with formatting and reader macro
chars. From your link;
(defmethod print-object ((self test) stream)
(write-string "#{" stream)
(write (class-name (class-of self)) :stream stream)
(dolist (slot-name (get-slots-to-dump self))
(when (slot-boundp self slot-name)
(write-char #\Space stream)
(write slot-name :stream stream)
(write-char #\Space stream)
(write (slot-value self slot-name) :stream stream)))
(write-char #\} stream)
self)
;;; Now define how to read them back in
(eval-when (:compile-toplevel :load-toplevel :execute)
(defun test-reader (stream char arg)
(declare (ignore char arg))
(let* ((class-name (read stream t nil t))
(slots (read-delimited-list #\} stream t))
(obj (make-instance class-name)))
(loop for (slot-name slot-value) on slots by #'cddr do
(setf (slot-value obj slot-name) slot-value))
obj))
(set-syntax-from-char #\} #\))
(set-dispatch-macro-character #\# #\{ #'test-reader))
Isn't hardcoding the reader chars and file representation in this way
undesirable?
If my use of make-load-form isn't desirable in this case, when is it
desirable?
Thanks,
Greg
Greg Menke <··········@toadmail.com> writes:
> Nils Goesche <······@cartan.de> writes:
>
> > Greg Menke <··········@toadmail.com> writes:
> >
> > > Below, I'm using make-load-form to save out the probe list when the
> > > program is closed, and read/eval to re-assemble the list when the
> > > program is started.
> >
> > I don't think you really want MAKE-LOAD-FORM here. Have a look at
> >
> > http://groups.google.com/groups?selm=87adbi49u9.fsf%40darkstar.cartan
>
> I looked at the posting and it is instructive. I understand that I
> could write and read the object instances by hand, but I thought the
> make-load-form lets me avoid fiddling with formatting and reader
> macro chars. From your link;
[PRINT-OBJECT method defined and read table hacked]
> Isn't hardcoding the reader chars
That's one of the best things you can do with those read tables :-)
> and file representation in this way undesirable?
Did you look at the files you generated before? You are defining the
file representation yourself, too, in your code, only that you
circumvent the default printer and reader mechanism. You are calling
MAKE-LOAD-FORM yourself; you could as well give the method some other
name.
> If my use of make-load-form isn't desirable in this case, when is it
> desirable?
It is needed when the Lisp system has to ``externalize�� an object,
notably when you call COMPILE-FILE on a file with code that has
literal objects that are instances of your newly defined classes. See
the example again. MAKE-LOAD-FORM is neither used nor needed for
printing or reading objects! If you want to know more about it, read
all of
3.2.4 Literal Objects in Compiled Files
in the HyperSpec. In general, you can ignore MAKE-LOAD-FORM until you
run into problems with COMPILE-FILE :-)
Or you could use something like this in general:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defpackage "DUMPABLE"
(:use "CL")
(:export "DUMPABLE"))
(in-package "DUMPABLE")
(defclass dumpable ()
())
(defmacro do-slot-names ((name class &optional return) &body body)
(let ((slot (gensym "SLOT")))
`(dolist (,slot (hcl:class-effective-slots ,class) ,return)
(let ((,name (hcl:slot-definition-name ,slot)))
,@body))))
(defmethod print-object ((self dumpable) stream)
(let ((class (class-of self)))
(write-string "#{" stream)
(write (class-name class) :stream stream)
(do-slot-names (name class)
(when (slot-boundp self name)
(write-char #\Space stream)
(write name :stream stream)
(write-char #\Space stream)
(write (slot-value self name) :stream stream)))
(write-char #\} stream)))
(defun dumpable-reader (stream char arg)
(declare (ignore char arg))
(let* ((class-name (read stream t nil t))
(slots (read-delimited-list #\} stream t))
(obj (make-instance class-name)))
(loop for (slot-name slot-value) on slots by #'cddr do
(setf (slot-value obj slot-name) slot-value))
obj))
(set-dispatch-macro-character #\# #\{ #'dumpable-reader)
(set-syntax-from-char #\} #\))
(defmethod make-load-form ((self dumpable) &optional env)
(make-load-form-saving-slots self :environment env))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
And then use DUMPABLE:DUMPABLE as a mixin:
CL-USER 227 > (load (compile-file "lisp:dumpable.lisp"))
;;; Compiling file lisp:dumpable.lisp ...
;;; Safety = 3, Speed = 1, Space = 1, Float = 1, Interruptible = 0
;;; Compilation speed = 1, Debug = 2, Fixnum safety = 3, GC safety = 3
;;; Source level debugging is on
;;; Source file recording is on
;;; Cross referencing is on
; (TOP-LEVEL-FORM 1)
; (DEFPACKAGE "DUMPABLE")
; (TOP-LEVEL-FORM 2)
; (DEFCLASS DUMPABLE:DUMPABLE)
; DUMPABLE::DO-SLOT-NAMES
; (METHOD PRINT-OBJECT (DUMPABLE:DUMPABLE T))
; DUMPABLE::DUMPABLE-READER
; (TOP-LEVEL-FORM 3)
; (TOP-LEVEL-FORM 4)
; (METHOD MAKE-LOAD-FORM (DUMPABLE:DUMPABLE))
; (TOP-LEVEL-FORM 5)
#P"/usr/local/data/src/lisp/dumpable.ufsl"
CL-USER 228 > (defclass foobar (dumpable:dumpable)
((x :initarg :x :reader foobar-x)))
#<STANDARD-CLASS FOOBAR 206797C4>
CL-USER 229 > (make-instance 'foobar :x 42)
#{FOOBAR X 42}
CL-USER 230 > (read-from-string "#{FOOBAR X 42}")
#{FOOBAR X 42}
14
This should work in many cases. It could be improved, though, for
instance by defining a method that will return the slots to be dumped
and can be overridden by users of the mixin. Left as an exercise :-)
Regards,
--
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."
PGP key ID 0x0655CFA0
Greg Menke <··········@toadmail.com> writes:
> I'm aware the unprotected eval is a security hole, but the program
> does not run suid/sgid and the config file is located in the user's
> home directory, so I think thats good enough for the moment.
I don't think so.
The thing that makes viruses so dangerous is not whose permissions they have
but the number of programs willing to turn "text sitting in a directory" into
"running program". If you look at most Internet worms recently, you find
that they aren't running with special perms--they are just putting foo.jpg.pif
or some such into a directory (so that it looks like a jpg, but really it's
a script of some kind) and then waiting for it to get run. Now you're saying
you're going to be the part that autoruns some code. At that point, the
ONLY thing protecting you is not the fact that it doesn't run with root perms
but the fact that most virus writers don't know Lisp. Even so, I think that's
a weak form of protection...
Kent M Pitman <······@world.std.com> writes:
> Greg Menke <··········@toadmail.com> writes:
>
> > I'm aware the unprotected eval is a security hole, but the program
> > does not run suid/sgid and the config file is located in the user's
> > home directory, so I think thats good enough for the moment.
>
> I don't think so.
>
> The thing that makes viruses so dangerous is not whose permissions they have
> but the number of programs willing to turn "text sitting in a directory" into
> "running program". If you look at most Internet worms recently, you find
> that they aren't running with special perms--they are just putting foo.jpg.pif
> or some such into a directory (so that it looks like a jpg, but really it's
> a script of some kind) and then waiting for it to get run. Now you're saying
> you're going to be the part that autoruns some code. At that point, the
> ONLY thing protecting you is not the fact that it doesn't run with root perms
> but the fact that most virus writers don't know Lisp. Even so, I think that's
> a weak form of protection...
True, however its the same default level of protection you get with
most everything else Unixy- but I get what you mean. From a previous
exchange on this thread, it seems I can avoid eval altogether anyhow.
The interesting thing about this hole is there is no need to construct
a buffer overflow case or disguise anything. All the virus writer has
to do is shove some code into the config file that naturally spawns
off some process to be exploited, without the program's user ever
being aware of it. Its even portable to any target architecture that
will run Lispworks. Much better than Outlook- No doubt I could leave
the "feature" in and the Windows users wouldn't notice the
difference... ;)
Gregm
Kent M Pitman <······@world.std.com> writes:
> At that point, the ONLY thing protecting you is not the fact that it
> doesn't run with root perms but the fact that most virus writers
> don't know Lisp.
Is using `read' or `read-from-string' alone considered to be safe? How
does one take advantage of the convenient, built-in de/serialization
facility without risking free execution of code? Is disabling
*read-eval* sufficient?
--
Steven E. Harris :: ········@raytheon.com
Raytheon :: http://www.raytheon.com
Steven E. Harris <········@raytheon.com> writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > At that point, the ONLY thing protecting you is not the fact that it
> > doesn't run with root perms but the fact that most virus writers
> > don't know Lisp.
>
> Is using `read' or `read-from-string' alone considered to be safe? How
> does one take advantage of the convenient, built-in de/serialization
> facility without risking free execution of code? Is disabling
> *read-eval* sufficient?
It is when you're using the standard readtable (or a copy of it).
See also with-standard-io-syntax, but be sure you bind *read-eval*
inside of it, not outside, since it binds *read-eval* to the unsafe value T.
Kent M Pitman <······@world.std.com> writes:
> Steven E. Harris <········@raytheon.com> writes:
>
> > Kent M Pitman <······@world.std.com> writes:
> >
> > > At that point, the ONLY thing protecting you is not the fact that it
> > > doesn't run with root perms but the fact that most virus writers
> > > don't know Lisp.
> >
> > Is using `read' or `read-from-string' alone considered to be safe? How
> > does one take advantage of the convenient, built-in de/serialization
> > facility without risking free execution of code? Is disabling
> > *read-eval* sufficient?
>
> It is when you're using the standard readtable (or a copy of it).
> See also with-standard-io-syntax, but be sure you bind *read-eval*
> inside of it, not outside, since it binds *read-eval* to the unsafe value T.
Sorry if this is a dumb or obvious question, but how does one work
with the results of reading with *read-eval* set to false? I've been
experimenting and can't get anything but "Cannot evaluate" condition.
Is the correct behavior to handle the condition and usefully decompose
whatever was read?
Thanks,
Gregm
"Greg Menke" <··········@toadmail.com> wrote in message
···················@europa.pienet...
> Kent M Pitman <······@world.std.com> writes:
> > It is when you're using the standard readtable (or a copy of it).
> > See also with-standard-io-syntax, but be sure you bind *read-eval*
> > inside of it, not outside, since it binds *read-eval* to the unsafe
value T.
>
> Sorry if this is a dumb or obvious question, but how does one work
> with the results of reading with *read-eval* set to false? I've been
> experimenting and can't get anything but "Cannot evaluate" condition.
> Is the correct behavior to handle the condition and usefully decompose
> whatever was read?
Try out this stuff below:
CL-USER 96 > (defvar good-stuff "(if (y-or-n-p \"is it safe\")
(+ 2 2)
(format t \"Bad Code\")))")
GOOD-STUFF
CL-USER 97 > (defvar bad-stuff "(if (y-or-n-p \"is it safe\")
(progn
#.(format t \"Bang, you're dead\")
(+ 2 2))
(format t \"~%Bad Code\"))")
BAD-STUFF
CL-USER 98 > (eval (read-from-string good-stuff))
4
CL-USER 99 > (eval (read-from-string bad-stuff))
Bang, you're dead ;; bad code runs before I can answer
Bad Code ;; y-or-n-p's question
NIL
CL-USER 100 > (eval (let ((*read-eval* nil)) (read-from-string good-stuff)))
4
CL-USER 101 > (eval (let ((*read-eval* nil)) (read-from-string bad-stuff)))
Error: Cannot evaluate (FORMAT T "Bang, you're dead") because *READ-EVAL* is
NIL.
1 (continue) Evaluate :FORMAT-STRING anyway.
2 Ignore form.
3 (abort) Return to level 0.
4 Return to top loop level 0.
Type :b for backtrace, :c <option number> to proceed, or :? for other
options
CL-USER 102 : 1 > :c 2
Bad Code
NIL
HTH!
--
Coby Beck
(remove #\Space "coby 101 @ big pond . com")
"Coby Beck" <·····@mercury.bc.ca> writes:
> "Greg Menke" <··········@toadmail.com> wrote in message
> ···················@europa.pienet...
> > Kent M Pitman <······@world.std.com> writes:
> > > It is when you're using the standard readtable (or a copy of it).
> > > See also with-standard-io-syntax, but be sure you bind *read-eval*
> > > inside of it, not outside, since it binds *read-eval* to the unsafe
> value T.
> >
> > Sorry if this is a dumb or obvious question, but how does one work
> > with the results of reading with *read-eval* set to false? I've been
> > experimenting and can't get anything but "Cannot evaluate" condition.
> > Is the correct behavior to handle the condition and usefully decompose
> > whatever was read?
>
> Try out this stuff below:
>
> CL-USER 96 > (defvar good-stuff "(if (y-or-n-p \"is it safe\")
> (+ 2 2)
> (format t \"Bad Code\")))")
> GOOD-STUFF
>
> CL-USER 97 > (defvar bad-stuff "(if (y-or-n-p \"is it safe\")
> (progn
> #.(format t \"Bang, you're dead\")
> (+ 2 2))
> (format t \"~%Bad Code\"))")
> BAD-STUFF
>
> CL-USER 98 > (eval (read-from-string good-stuff))
> 4
>
> CL-USER 99 > (eval (read-from-string bad-stuff))
> Bang, you're dead ;; bad code runs before I can answer
> Bad Code ;; y-or-n-p's question
> NIL
>
> CL-USER 100 > (eval (let ((*read-eval* nil)) (read-from-string good-stuff)))
> 4
>
> CL-USER 101 > (eval (let ((*read-eval* nil)) (read-from-string bad-stuff)))
>
> Error: Cannot evaluate (FORMAT T "Bang, you're dead") because *READ-EVAL* is
> NIL.
> 1 (continue) Evaluate :FORMAT-STRING anyway.
> 2 Ignore form.
> 3 (abort) Return to level 0.
> 4 Return to top loop level 0.
>
> Type :b for backtrace, :c <option number> to proceed, or :? for other
> options
>
> CL-USER 102 : 1 > :c 2
> Bad Code
> NIL
So the point is to skip the evaluation of the "Bang..." code in the
reader by throwing a condition, which I would handle as appropriate-
in this case ignoring it?
Then when reading my config file, I would use this technique to wipe
out any code that the reader would execute at read-time, leaving my
code to adequately validate the rest.
Cool.
Gregm
{{Date: 01 Aug 2003 08:21:32 -0400
From: Greg Menke <··········@toadmail.com>
So the point is to skip the evaluation of the "Bang..." code in the
reader by throwing a condition, which I would handle as appropriate-
in this case ignoring it? Then when reading my config file, I would
use this technique to wipe out any code that the reader would execute
at read-time, leaving my code to adequately validate the rest.}}
I would think if you are maintaining a configuration file on your
directory, and your software discovers that somebody has trespassed in
your directory to install a potential reader-macro virus in your
configuration file, that you would want to take some more drastic
action than just ignoring the error. The sooner you are alerted to the
problem, the best chance you have of looking at the date-last-written
of the compromised file and then looking up that date in a system log
to see what processes were running at that time to possibly detect an
intruder logged in at that time or a virus in some other file that you
accidently activated at that time.
By the way, I have been developing CGI/LISP applications, and have been
thinking of writing an application that teaches how to program in LISP.
At many places the user could enter some LISP syntax into a form and
submit it and my server application would decide whether it's valid and
then judge it based on what the student was supposed to have done at
that transaction. But I'm worried about reader macros that could cause
code to be executed without my main program ever seeing it, when my
program uses read-from-string. It's my understanding that both #. and
#, can cause arbitrary s-expressions embedded in student's input to be
evaluated, right? So I need to disable both of them, right? Recently I
saw mention of *read-eval* which can be bound to NIL to avoid this
danger. Does it work for both #. and #, or do I need something else
also to protect against both? What about #+ and #-, are they totally
safe in this regard?