Hi,
I have a question about extending applications using straight Common
Lisp as the extension language.
A while back I wrote a system in which you could add functionality
easily inside an initialization file. It was very similar to a .emacs
file in Emacs in that you had full access to Common Lisp, and every
symbol in every package. There was nothing that was somehow
off-limits in that file, since it was getting processed by #'LOAD,
essentially.
Is there a way to restrict symbol access to only one package's
*exported* symbols? I'm thinking that I'd like to give the user
access to most CL, but not all of it, and some of the things in CL I
want to package up into an interface I'm more comfortable with the
users being able to call. For example, they can only work within a
single package.
thanks,
dave
* David Bakhash wrote:
> Is there a way to restrict symbol access to only one package's
> *exported* symbols? I'm thinking that I'd like to give the user
> access to most CL, but not all of it, and some of the things in CL I
> want to package up into an interface I'm more comfortable with the
> users being able to call. For example, they can only work within a
> single package.
I think the answer is yes, with slight caveats. The caveats I'm aware
of are:
* The reader should hjave a `safe mode' where it will not blow up in
any bad way (for instance evaluating general code at read time) when
given something completely mutant -- this may be assured by language
definition, but I'm not sure. I think that for most implementations
this is the case, assuming you set some reader variables correctly
(*read-eval* is the big one here). Signalling an error is OK of
course.
* The compiler should have a `safe mode' where it will not blow up in
bad ways (again, evaluating general code at compile time is bad) --
it either needs to compile something or signal an error.
* The runtime system needs to be similarly safe. I think this is OK
in practice for most implementations. Of course there may be
termination problems, but there's not a whole bunch you can do about
that.
* You're happy dealing with source.
If this is OK then you can do the following:
Define one or more `good' packages, which export the symbols you are
interested in. I have a little thing called `conduits' which lets you
do this in quite an easy way -- you can define packages which extend
others in a fairly flexible way -- an example being the CL/CONDUITS
package which is the same as CL except that a things like EXPORT now
come from another package and do the stuff that the conduits system
needs... Things you want to avoid in `good' packages, apart from the
obvious ones, are READ (because someone can then read bad things from
a string), symbol-lookup and creation functions (because they can then
get at bad symbols), and various other things. It's probably hard to
get this right in general.
Now, read the user input with the reader in `safe' mode, being careful
to catch any errors.'
Walk over the resulting form, checking that it contains no bad things.
Bad things are things like symbols which aren't exported by your
`good' packages above, or other stuff that you don't know what it is.
I think in my toy version of this I allowed good symbols, conses,
numbers, strings and characters. If you allow any other compound type,
like general arrays, you have to be willing to walk over them checking
they have no bad things in. The walker needs to do an occurs check.
Compile the resulting form, catching errors.
Call the function you just got, catching errors.
Once you are done all this, you also want to `clean' various user-type
packages to get rid of any extra symbols that got interned. This is
important if several people might use the system and you don't want
them communicating with each other secretly.
Of course, formal people will be horrified by the slapdash nature of
this scheme, but I think it can be made to work. We convinced
ourselves that it would be OK for something like a web application
where the user would be allowed to specify little bits of code to be
run to extend the system dynamically.
I'd be interested if anyone can see any really obvious holes in it!
I can put the conduit packages stuff up for ftp if need be (I think I
posted various versions of it here a couple of times, as well).
--tim
Tim Bradshaw <···@cley.com> writes:
> I can put the conduit packages stuff up for ftp if need be (I think I
> posted various versions of it here a couple of times, as well).
I'd like to see that package. Please post it if that's okay with
you. You can tar/gzip/uuencode it, of course, if it's really big, or
if it has lots of long lines, etc.
Now, about your solution...
It seems kinda severe. I realized early on that this would be no
simple task, but more and more it seems like using CL's #'READ,
#'LOAD, #'COMPILE-FILE are problematic for application extension,
unless you *really* trust your users. Plus, I don't even think that
the commercial implementations will let you call #'COMPILE-FILE, but
that's a separate issue, and there's always the work-around of using
#'COMPILE.
The solution I seem to be hearing is to start with a code walker,
check the code, and then maybe shove it through. The code walker
might be able to get away with using #'READ, with *READ-EVAL* set to
nil. This "checker" would have to be very smart, and like you said,
it would probably take serious effort to get it right. This is
probably why companies write their own languages and parsers.
Anyway, I appreciate the help.
dave
* David Bakhash wrote:
> Tim Bradshaw <···@cley.com> writes:
> I'd like to see that package. Please post it if that's okay with
> you. You can tar/gzip/uuencode it, of course, if it's really big, or
> if it has lots of long lines, etc.
OK, I'll probably put it up for ftp, as it's easier somehow...
> It seems kinda severe. I realized early on that this would be no
> simple task, but more and more it seems like using CL's #'READ,
> #'LOAD, #'COMPILE-FILE are problematic for application extension,
> unless you *really* trust your users. Plus, I don't even think that
> the commercial implementations will let you call #'COMPILE-FILE, but
> that's a separate issue, and there's always the work-around of using
> #'COMPILE.
You don't need load & compile-file. In fact if you're happy not to
compile you can just call eval.
> The solution I seem to be hearing is to start with a code walker,
> check the code, and then maybe shove it through. The code walker
> might be able to get away with using #'READ, with *READ-EVAL* set to
> nil. This "checker" would have to be very smart, and like you said,
> it would probably take serious effort to get it right. This is
> probably why companies write their own languages and parsers.
If you're willing to live with an fairly restrictive subset of CL I
don't think it's that hard. You need to just disallow hairy literals
(no #S(...) in code!). If, for instance you just allow LAMBDA and
IF, say, then you have a small (complete? I forget) subset of Lisp
which is pretty obviously safe.
The proof of concept thing I did for this was pretty small.
Unfortunately I think it relied on a version of the conduit system I
no longer have... I'll try and dig it out.
It's one of the great beauties of Lisp that this kind of thing *can*
be done, and done without heroic effort.
--tim
Tim Bradshaw <···@cley.com> writes:
> The proof of concept thing I did for this was pretty small.
> Unfortunately I think it relied on a version of the conduit system I
> no longer have... I'll try and dig it out.
thanks. I'd like to see that. I'm still not so sure exactly what
kinds of things it does that are beyond what I do regularly with
packages, like (shadowing) imports, export, etc. The real key I was
looking for was pointed out by KMP, which has to do with having #'READ
signal a condition when it tries to access a symbol in a package other
than the one you're in. I think a code-walker should handle this,
though I'd much rather not have to pre-walk the code, make sure it's
safe, and then walk back over it with #'READ. That just seems less
optimal and more work than if there were some generalized boolean
variable that could be set to handle this.
> It's one of the great beauties of Lisp that this kind of thing *can*
> be done, and done without heroic effort.
Depends on what you consider heroic, but the fact that it's even a
possible consideration is nice. Don't mean to be disparaging of other
languages, but they tend to suck in this respect, as compared to CL.
But considering that the C guys, for example, would just yacc their
way out of this mess, and have been for years, it's not like they
don't have a solution; it's just a different solution. Why redefine a
grammar if you don't have to? Why not use built-in tools that already
work?
Anyway, I wrote an application a while back that loaded an init file
which was pure CL, and made that file load from a particular package.
But I really didn't want users to do things like (CL::DEFCLASS ...) in
there, since I wrote my own special macros that did extra
bookkeeping.
dave
* David Bakhash wrote:
> Tim Bradshaw <···@cley.com> writes:
>> The proof of concept thing I did for this was pretty small.
>> Unfortunately I think it relied on a version of the conduit system I
>> no longer have... I'll try and dig it out.
> thanks. I'd like to see that.
OK, it's at the end of this article. The DEFINE-SAFE-LISP-PACKAGE
macro is what ended up, after some elaboration, as the conduit-package
system I now have (you can see why it got called that if you read the
comments...). However the conduit stuff doesn't have the `package
cleaning' stuff that is in here. The form checker hangs off
FORM-SAFE-P, and it's extensible in an obvious way. (It's also less
than 50 lines, which meets my definition of `not heroic'.)
TS-CLEANER gets you into a little REP loop where you can type a tiny
subset of CL (the subset being defined by the contents of the SL-PLAY
package), and it will tell you if it's safe and then evaluate it if it
is.
Note this was a proof-of-concept I wrote in an hour. It doesn't catch
errors or anything like that.
I have to clean the real conduits stuff up but I'll put that up for
ftp shortly.
--tim
--cut--
;; -*- Mode: LISP; Base: 10; Syntax: Ansi-common-lisp; Package: CL-USER -*- ;;
;; File - verify-form.lisp
;; Description - toy form checker
;; Author - Tim Bradshaw (tfb at lostwithiel)
;; Created On - Feb 27, 1998
;; Last Modified On - Wed Sep 6 18:29:58 2000
;; Last Modified By - Tim Bradshaw (tfb at lostwithiel)
;; Update Count - 2
;; Status - Unknown
;;
;; $Id$
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;; Play checking forms for safety.
;;;
;;; the idea is to walk over a form checking that it has no bad things
;;; in, where `bad' is either a data type we don't know about or a
;;; symbol from a package we don't like.
(in-package :cl-user)
(defun symbol-directly-present-in-package-p (s p)
;; is S actually in P, not just inherited.
(multiple-value-bind (sym stype)
(find-symbol (symbol-name s) p)
(case stype
((:external :internal)
(values sym stype))
(otherwise
(values nil nil)))))
(defmacro define-safe-lisp-package (name &body import-clauses)
;; each import-clause should be (:FROM pack . syms/sym-names). All
;; these symbols are imported and then exported directly -- this
;; package is really just a conduit.
`(defpackage ,name
(:use)
,@(loop for clause in
(mapcar #'(lambda (clause)
(destructuring-bind (from pack . sym/names) clause
(if (and (eq from ':from) (find-package pack))
(cons (package-name (find-package pack))
(mapcar #'(lambda (sym/name)
(etypecase sym/name
(symbol
(symbol-name sym/name))
(string sym/name)))
sym/names)))))
import-clauses)
collect `(:import-from ,@clause)
collect `(:export ,@(cdr clause)))))
;;; Package cleaning hack
;;;
;;; This is pretty dubious because it uses an alist of packages, not
;;; names, and who knows whether DEFPACKAGE modifies a package or makes
;;; a new one? All this needs to be made better.
(defvar *cp-package-alist* '())
(defun register-cleanable-package (p &key (ptype ':current))
(let ((p (find-package p)))
(let* ((p-found-p (assoc p *cp-package-alist*))
(pdesc (or p-found-p (list p))))
(ecase ptype
(:current
;; stash current state
(let ((phash (or (cdr p-found-p)
(make-hash-table))))
(clrhash phash)
;; we're only interested in external symbols because this is
;; only for conduit packages which export everything (above).
(do-external-symbols (s p)
(setf (gethash s phash) t))
(setf (cdr pdesc) phash)))
(:empty
(setf (cdr pdesc) nil)))
(when (not p-found-p)
(push pdesc *cp-package-alist*))
p)))
(defun clean-package (p)
(let ((p (find-package p)))
(let ((pdesc (assoc p *cp-package-alist*)))
(when (not pdesc)
(error "package ~A is not registered for cleaning" p))
(if (cdr pdesc)
(let ((phash (cdr pdesc)))
(do-symbols (s p)
(unless (gethash s phash)
(unintern s p))))
(do-symbols (s p)
(when (symbol-directly-present-in-package-p s p)
(unintern s p))))
p)))
(defun clean-all-cleanable-packages ()
;; clean all packages we know about. This needs to be fixed to be
;; linear.
(loop for pdesc in *cp-package-alist*
do
(clean-package (car pdesc)))
(values))
(define-safe-lisp-package :sl-play
;; basic things
(:from "CL" nil lambda if and or not function quote labels)
;; arithmetic
(:from "CL" + - / * = > < >= <=))
(defpackage :sl-play-user
(:use :sl-play))
(register-cleanable-package :sl-play :ptype :current)
(register-cleanable-package :sl-play-user :ptype :empty)
(defvar *sl-package* (find-package :sl-play))
(defvar *sl-user-package* (find-package :sl-play-user))
(defun form-safe-p (form &key
(sl-package *sl-package*)
(sl-user-package *sl-user-package*)
(occurs-ht (make-hash-table)))
(clrhash occurs-ht)
(check-form-safe-p form (list (find-package sl-package)
(find-package sl-user-package))
occurs-ht))
;;; The real function.
;;;
;;; Trust: numbers, strings, chars nil, conses whose car & cdr we trust,
;;; symbols which are in a good package.
(defgeneric check-form-safe-p (form packages occurs-ht))
(defmethod check-form-safe-p ((form t) packages occurs-ht)
(declare (ignore packages occurs-ht))
nil)
(defmethod check-form-safe-p ((form number) packages occurs-ht)
(declare (ignore packages occurs-ht))
t)
(defmethod check-form-safe-p ((form string) packages occurs-ht)
(declare (ignore packages occurs-ht))
t)
(defmethod check-form-safe-p ((form character) packages occurs-ht)
(declare (ignore packages occurs-ht))
t)
(defmethod check-form-safe-p ((form null) packages occurs-ht)
(declare (ignore packages occurs-ht))
t)
(defmethod check-form-safe-p ((form cons) packages occurs-ht)
(if (gethash form occurs-ht)
t
(progn
(setf (gethash form occurs-ht) form)
(and (check-form-safe-p (car form) packages occurs-ht)
(check-form-safe-p (cdr form) packages occurs-ht)))))
(defmethod check-form-safe-p ((form symbol) packages occurs-ht)
(declare (ignore occurs-ht))
(some #'(lambda (p)
(symbol-directly-present-in-package-p form p))
packages))
(defun ts-cleaner (&key (sl-user-package *sl-user-package*)
(sl-package *sl-package*))
(clean-all-cleanable-packages)
(with-standard-io-syntax
(loop with *package* = sl-user-package
with *print-circle* = t
with *print-readably* = nil
with *read-eval* = nil
with occurs-ht = (make-hash-table)
for form = (progn
(format t "~&Form: ")
(read))
while form
do
(if (form-safe-p form
:sl-package sl-package
:sl-user-package sl-user-package
:occurs-ht occurs-ht)
(format t "~&Safe. Result: ~S~%"
(eval form))
(format t "~&Unsafe~%"))
(clean-all-cleanable-packages)))
(values))
I think that with the way you defined your generic function
#'CHECK-FORM-SAFE-P, one can add some extra stuff for structures and
arrays. I think that for arrays you can travese them without resorting
to implementation-specific stuff, just using array-rank and stuff, but
with structures, it's probably different, since you'll need to access
slot definitions and such.
Essentially, your program is a recursive code-walker and it looks good.
I don't like having to add an additional pass to the code, but it's
probably worthwhile; after all, it avoids one from having to get into
the guts of #'READ. Nice work. I think you sold me. This way is
totally standard.
dave
David Bakhash <·····@alum.mit.edu> writes:
> Essentially, your program is a recursive code-walker and it looks good.
> I don't like having to add an additional pass to the code, but it's
> probably worthwhile; after all, it avoids one from having to get into
> the guts of #'READ.
I'm only spot-checking this thread, so didn't see what this refers to.
If you are implementing this by doing (CHECK (READ)) and having the
CHECK look for bad symbols, the only issue is that if #. is turned on,
there can be dependencies that slip by unchecked in the #. call, which
will not be checked. You might want to set up your own #. which is like
the system one but which also does (CHECK (READ)) to avoid read-time
computation done on strange symbols.
I also didn't see how you were checking symbols. Don't look for
SYMBOL-PACKAGE. That can be misleading if the symbols are inherited
into the package of choice. Using FIND-SYMBOL on the symbol name and
the package you want to restrict yourself to and making sure they come
out the same will give best results. (Or put a property on the good
symbols and check those for better performance.)
* Kent M Pitman wrote:
> I'm only spot-checking this thread, so didn't see what this refers to.
> If you are implementing this by doing (CHECK (READ)) and having the
> CHECK look for bad symbols, the only issue is that if #. is turned on,
> there can be dependencies that slip by unchecked in the #. call, which
> will not be checked. You might want to set up your own #. which is like
> the system one but which also does (CHECK (READ)) to avoid read-time
> computation done on strange symbols.
I turn #. off, too dangerous!!
> I also didn't see how you were checking symbols. Don't look for
> SYMBOL-PACKAGE. That can be misleading if the symbols are inherited
> into the package of choice. Using FIND-SYMBOL on the symbol name and
> the package you want to restrict yourself to and making sure they come
> out the same will give best results. (Or put a property on the good
> symbols and check those for better performance.)
I use a trick whereby I import and then reexport symbols, so I
basically grow my own `custom CL' packages, and then have a function
SYMBOL-DIRECTLY-PRESENT-IN-PACKAGE-P, which uses FIND-SYMBOL as you
say. Using a property might be better (so long as the subset you
allow access to doesn't allow property manipulation!).
--tim
* David Bakhash wrote:
> I don't see `T'. Considering that it took me longer to write this post
> than to just check to see if `T' would be accepted or not is probably
> not a good sign, though, now that I think about it. But I like to
> really dive into code before using it so I don't _think_ it works just
> because it handles the cases I test.
I don't think it handles T. But you could just define a package that
has T in.
> Also, I don't think that Tim expected people to use his :sl-play
> package; I think he was just using that as an example of generally safe
> CL symbols (esp since he didn't export any but the most obvious math
> symbols).
yes, the whole sl-play thing was just there so I could test it. None
of the code is meant to be used in anger.
--tim
For anyone who's interested in somehow using the code by Tim, you may
want to add this to the methods. It just handles reading arrays
(since there's no reason to disallow arrays, as far as I know):
(defmethod check-form-safe-p ((form array) packages occurs-ht)
(if (gethash form occurs-ht)
t
(progn
(setf (gethash form occurs-ht) form)
(loop for i below (array-total-size form)
always (check-form-safe-p (row-major-aref form i))))))
dave
Tim Bradshaw <···@cley.com> writes:
> * David Bakhash wrote:
> > Tim Bradshaw <···@cley.com> writes:
> > It seems kinda severe. I realized early on that this would be no
> > simple task, but more and more it seems like using CL's #'READ,
> > #'LOAD, #'COMPILE-FILE are problematic for application extension,
> > unless you *really* trust your users. Plus, I don't even think that
> > the commercial implementations will let you call #'COMPILE-FILE, but
> > that's a separate issue, and there's always the work-around of using
> > #'COMPILE.
>
> You don't need load & compile-file. In fact if you're happy not to
> compile you can just call eval.
instead of:
(eval form)
there's always:
(funcall (compile nil #'(lambda () form)))
I would guess, however, that there are very few likely FORMs that
would execute faster the 2nd way, since compilation is a relatively
time-consuming process. Also, I remember from another recent post
that compilation is can be dubious in situations like these, because
of environment stuff (e.g. the example with multiprocessing and
variable scope). I just think it's cool that you can compile a form
so easily in Common Lisp.
dave
* David Bakhash wrote:
> (eval form)
> there's always:
> (funcall (compile nil #'(lambda () form)))
> I would guess, however, that there are very few likely FORMs that
> would execute faster the 2nd way, since compilation is a relatively
> time-consuming process. Also, I remember from another recent post
> that compilation is can be dubious in situations like these, because
> of environment stuff (e.g. the example with multiprocessing and
> variable scope). I just think it's cool that you can compile a form
> so easily in Common Lisp.
That's not clear I think. Some Lisps essentially implement EVAL like
this, and it's not an unreasonable approach as it means you have
potentially less code to maintain.
Also if FORM is a defining form of some kind it can be well worth it
to point the compiler at it so whatever it defines gets compiled,
because that (if it's a function) might get called a lot...
I think my theory was that in the kind of situation I envisaged this
being used -- where information is coming in down some kind of
network stream, the compilation overhead is likely to be negligible cf
network delays.
--tim
David Bakhash <·····@alum.mit.edu> writes:
> Is there a way to restrict symbol access to only one package's
> *exported* symbols?
I think this is a slightly confused thing to ask for. For example,
suppose I told you that the CL package was made up of symbols from
CL-IO, CL-GLUE, CL-SEQ, and CL-MISC? Suddenly, you find you have access
to five packages...
In other words, what would it mean to have access only to one package,
other than to say you wanted access only to a particular set of
symbols. To do that, you could just IMPORT the relevant symbols and
then turn off ":" altogether, I guess...
For the most part, though, I think it makes much more sense to ask to
have a package guard itself from use than to insist that a programmer
not use functionality already present.
A rare exception is "development for deployment in another environment"
where you need to bound the whole package universe to a fixed set of packages
and make others invisible because they won't be visible in the target.
I implemented this on Genera, but it couldn't be done from within CL.
We had to replace all the myriad places that packages can get named
and created a whole suite of things called SYS:FIND-PACKAGE-FOR-SYNTAX, etc.
and the inverses like SYS:PACKAGE-NAME-FOR-SYNTAX and all that.
so that you could not only bound the set of packages but rename them within
a given syntax. In that way, ANSI CL and CLTL1 and about 4 other dialects
including original Zetalisp were able to be simultaneously present even though
they sometimes used incompatible package names. We used the syntax
ZL:::USER:FOO to refer to "the symbol FOO in the USER package of the syntax
ZL". Those lines that you see in some files that say
-*- Mode: LISP; Syntax: ANSI-Common-Lisp; Package: CL-USER -*-
are what cause you to get a syntax in which CL and CL-USER refer to what
Zetalisp would ordinarily call FUTURE-COMMON-LISP and FUTURE-COMMON-LISP-USER.
CL specifically leaves mulitple colons to the implementation, which is why
we picked that ::: syntax.
Kent M Pitman <······@world.std.com> writes:
> David Bakhash <·····@alum.mit.edu> writes:
> > Is there a way to restrict symbol access to only one package's
> > *exported* symbols?
> ...To do that, you could just IMPORT the relevant symbols and then
> turn off ":" altogether, I guess...
Okay. How do you turn this off? Is that possible? That's a first
step towards what I want.
> We used the syntax ZL:::USER:FOO to refer to "the symbol FOO in the
> USER package of the syntax ZL".[...] CL specifically leaves mulitple
> colons to the implementation, which is why we picked that :::
> syntax.
Do you mean that it leaves that open for the implementation? or for
the developer? I ask because I don't know how to make use of these
multiple colons (for example, in the way you mentioned above with
`:::').
dave
David Bakhash <·····@alum.mit.edu> writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > David Bakhash <·····@alum.mit.edu> writes:
>
> > > Is there a way to restrict symbol access to only one package's
> > > *exported* symbols?
>
> > ...To do that, you could just IMPORT the relevant symbols and then
> > turn off ":" altogether, I guess...
>
> Okay. How do you turn this off? Is that possible? That's a first
> step towards what I want.
Just change its syntax to be anything other than its default and you'll
probably cause it to not be what it originally was.
> > We used the syntax ZL:::USER:FOO to refer to "the symbol FOO in the
> > USER package of the syntax ZL".[...] CL specifically leaves mulitple
> > colons to the implementation, which is why we picked that :::
> > syntax.
>
> Do you mean that it leaves that open for the implementation? or for
> the developer? I ask because I don't know how to make use of these
> multiple colons (for example, in the way you mentioned above with
> `:::').
Just what I said--implementation. There is no useful way for the programmer
to define anything to do with multiple colons without clobbering the meaning
of : and ::, nor is there any way for the user to define anything like : and
::.
Of course, just to add to the utility of this discussion, another even
more useful (and pretty obvious use) of this is use straight Common
Lisp over a socket stream inside a protocol. So one application would
talk to another by just sending it some specialized Lisp to evaluate.
If you've ever used the Festival system for TTS, or even gnuserv in
Emacs, you've probably seen this with Scheme and elisp respectively.
It's probably not the most efficient way to design a protocol, but it
works, and it's a simple-as-hell hack.
If there were tools to trim down CL, protect against obvious security
holes, and such, this would be great thing to have handy.
dave
From: Tim Bradshaw
Subject: Re: more uses (was: protected packages, etc.)
Date:
Message-ID: <ey3aedlxvyd.fsf@cley.com>
* David Bakhash wrote:
> Of course, just to add to the utility of this discussion, another even
> more useful (and pretty obvious use) of this is use straight Common
> Lisp over a socket stream inside a protocol. So one application would
> talk to another by just sending it some specialized Lisp to evaluate.
> If you've ever used the Festival system for TTS, or even gnuserv in
> Emacs, you've probably seen this with Scheme and elisp respectively.
> It's probably not the most efficient way to design a protocol, but it
> works, and it's a simple-as-hell hack.
Right. This is what the verify-form thingy I just posted was meant to
end up as.
> If there were tools to trim down CL, protect against obvious security
> holes, and such, this would be great thing to have handy.
Yup. I claim (without real proof) that such a checked and safe
CL-subset is relatively achievable, especially if you are willing to
be cautious.
--tim
Tim Bradshaw <···@cley.com> writes:
> * David Bakhash wrote:
> > Of course, just to add to the utility of this discussion, another even
> > more useful (and pretty obvious use) of this is use straight Common
> > Lisp over a socket stream inside a protocol. So one application would
> > talk to another by just sending it some specialized Lisp to evaluate.
>
> Right. This is what the verify-form thingy I just posted was meant to
> end up as.
So I never got to see the conduit thing, and same goes for the
verify-form thing you're mentioning. Do you have an FTP or web site
where one can see these things?
> > If there were tools to trim down CL, protect against obvious security
> > holes, and such, this would be great thing to have handy.
>
> Yup. I claim (without real proof) that such a checked and safe
> CL-subset is relatively achievable, especially if you are willing to
> be cautious.
I don't know if it's acheivable w/o writing your own #'READ and/or a
code walker with some serious checks, which is exactly what I was trying
to prevent.
Again, just to say it one last time:
My claim is that if #'READ can be made to operate in only a single
package, and that symbols from other packages were inaccessible, then I
think I could manage the rest, which is the heart of the problem
(i.e. figuring out what is and is not safe for a particular program to
use). For example, I wouldn't want anyone to even _see_ things such as
MP::*ALL-PROCESSES*, and much less be able to modify that variable.
Well, it was a wish from the start, so I'm not too upset that it doesn't
seem to exist, and that no one has really done it yet. I think you have
to get under the skin of the implementation. I think that having the
source for #'READ can be a big win, so that you can change its
sematics.
I wonder if package locks have anything to do with this problem. For
example, in ACL, there's package locking, which kinda is on the right
track of what I want:
<URL:http://www.franz.com/support/documentation/5.0.1/doc/cl/packages.htm#5.1 Package locking>
But of course, the user can just locally enclose his or her forms inside
the EXCL:WITHOUT-PACKAGE-LOCKS macro, and you're back to square one.
Package locks in ACL only tend to prevent redefinition of some aspect of
the packagg -- not accessing the symbols.
I think that the implemntors would be doing the right thing to consider
these additional features into their work, and I know I'd use it for
sure, if it were well-designed and clean.
dave
David Bakhash <·····@alum.mit.edu> writes:
> So I never got to see the conduit thing, and same goes for the
> verify-form thing you're mentioning. Do you have an FTP or web site
> where one can see these things?
okay. Just found your verify-form thing. tx.
dave
From: Pekka P. Pirinen
Subject: Re: more uses (was: protected packages, etc.)
Date:
Message-ID: <ixya0nqdwk.fsf@harlequin.co.uk>
David Bakhash <·····@alum.mit.edu> writes:
> Tim Bradshaw <···@cley.com> writes:
> > * David Bakhash wrote:
> > > If there were tools to trim down CL, protect against obvious security
> > > holes, and such, this would be great thing to have handy.
> >
> > Yup. I claim (without real proof) that such a checked and safe
> > CL-subset is relatively achievable, especially if you are willing to
> > be cautious.
>
> I don't know if it's acheivable w/o writing your own #'READ and/or a
> code walker with some serious checks, which is exactly what I was trying
> to prevent.
I'm fairly certain that it isn't, at least for some values of "safe"
(security isn't a single, unambiguous concept; usually security
efforts start by defining who's authorized to do what, and then
defining threat models to describe how we expect unauthorized parties
to attempt to circumvent this and how much effort we're willing to
spend to counter that).
In general, it's much harder to make a system safe by stripping out
functionality from a general system than to start from nothing and
only add what you know/think is safe. Especially if you didn't write
that general system in the first place.
> My claim is that if #'READ can be made to operate in only a single
> package, and that symbols from other packages were inaccessible, then I
> think I could manage the rest, which is the heart of the problem
> (i.e. figuring out what is and is not safe for a particular program to
> use). For example, I wouldn't want anyone to even _see_ things such as
> MP::*ALL-PROCESSES*, and much less be able to modify that variable.
Yes, but even if you hack the reader so that this symbol cannot be
read, and suppress INTERN and other functions that could provide
direct access to it, are you sure it cannot be found on the property
list of some accessible symbol? Under any conditions? Clearly not.
Also, there are symbols you have to make accessible, but whose value
must not be changed. Say, T. Your Lisp implementation probably
protects T for you, because the consequences of setting T are so
painful -- take it from one who has tried -- but if it didn't, you'd
find it quite complicated to add that. You would also need to
consider modifyability of function cells, property lists,... in fact,
any cells of any accessible structure. An easy way out of that would
be to only allow a pure functional subset, but that's not CL anymore.
Now, this isn't to say that conduit packages and such tricks cannot be
useful. If your threat model says that you're worried about
non-expert users crashing the program by mistake, but malicious
hacking is not an issue, then this approach is probably good enough.
> Well, it was a wish from the start, so I'm not too upset that it doesn't
> seem to exist, and that no one has really done it yet. I think you have
> to get under the skin of the implementation.
Yes, and if you need to do that a lot, it might well be easier to
write your own reader and interpreter, than to try to modify someone
else's. Much more portable, too.
--
Pekka P. Pirinen, Harlequin Limited, Cambridge, UK
What you don't see is what gets you. - various hackers
·····@harlequin.co.uk (Pekka P. Pirinen) writes:
> In general, it's much harder to make a system safe by stripping out
> functionality from a general system than to start from nothing and
> only add what you know/think is safe. Especially if you didn't
> write that general system in the first place.
Yeah. That was my point about writing your own #'READ versus using
the built-in. But after more thought I think something like conduits
can help out a lot. For example:
> Yes, but even if you hack the reader so that this symbol cannot be
> read, and suppress INTERN and other functions that could provide
> direct access to it, are you sure it cannot be found on the property
> list of some accessible symbol? Under any conditions? Clearly not.
excellent point. We might consider that "there's no need for clients
to need #'SYMBOL-PLIST, so just don't export that."
On the other hand, if they have SETF and try to do so, then you might
run into problems. Things like this are definitely important to check
out.
> Also, there are symbols you have to make accessible, but whose value
> must not be changed. Say, T. Your Lisp implementation probably
> protects T for you, because the consequences of setting T are so
> painful -- take it from one who has tried -- but if it didn't, you'd
> find it quite complicated to add that. You would also need to
> consider modifyability of function cells, property lists,... in fact,
> any cells of any accessible structure. An easy way out of that would
> be to only allow a pure functional subset, but that's not CL anymore.
again, good point. A lot has to do with SETQ and SETF here. You
might shadow them, and redefine these safely.
dave
From: Tim Bradshaw
Subject: Re: more uses (was: protected packages, etc.)
Date:
Message-ID: <ey3bsxiu8du.fsf@cley.com>
* David Bakhash wrote:
> ·····@harlequin.co.uk (Pekka P. Pirinen) writes:
>> In general, it's much harder to make a system safe by stripping out
>> functionality from a general system than to start from nothing and
>> only add what you know/think is safe. Especially if you didn't
>> write that general system in the first place.
> Yeah. That was my point about writing your own #'READ versus using
> the built-in. But after more thought I think something like conduits
> can help out a lot. For example:
My original thought I think was that you'd use the conduits stuff as a
safe front-end to a language which wasn't just a CL subset. Although
I since decided that you could just `export' a CL subset and it would
be safe(-ish), and the toy demo did that, I think the original thing
was to write a little language in the normal Lisp way, and then use
conduits to avoid having to write READ and so on.
Of course, since then I've spent more time using conduits to define
custom versions of CL itself -- the conduits code itself defines a
nicely recursive CL/CONDUITS package which makes DEFPACKAGE &c be the
conduits version...
--tim
Tim Bradshaw <···@cley.com> writes:
> My original thought I think was that you'd use the conduits stuff as
> a safe front-end to a language which wasn't just a CL subset.
> Although I since decided that you could just `export' a CL subset
> and it would be safe(-ish), and the toy demo did that, I think the
> original thing was to write a little language in the normal Lisp
> way, and then use conduits to avoid having to write READ and so on.
that's pretty much the same thing, except when you say "little
language", you're probably just talking about defining some additional
functions, macros, classes, etc. That's exactly what I wanted to do.
conduits is really neat, and I believe that it will prove useful, at
least for how I want to use it.
dave