From: Andy Chambers
Subject: loading data files
Date: 
Message-ID: <0f46f68f-37d4-43aa-a311-94f6d86d8816@e1g2000hsh.googlegroups.com>
I've defined a bunch of macros so that my program can load its data-
structures by just calling `load' on a file that has something like

(item 1 "test" :a 1 :b 2)
(item 2 "test-2" :a 1 :b 2)

This works fine but now I'm thinking about how to handle "untrusted"
files.  Is there some way of "sandboxing" load to allow only certain
white-listed functions/macros?

Cheers,
Andy

From: Russell McManus
Subject: Re: loading data files
Date: 
Message-ID: <87r6iqrnhj.fsf@thelonious.cl-user.org>
Andy Chambers <··············@googlemail.com> writes:

> I've defined a bunch of macros so that my program can load its data-
> structures by just calling `load' on a file that has something like
> 
> (item 1 "test" :a 1 :b 2)
> (item 2 "test-2" :a 1 :b 2)
> 
> This works fine but now I'm thinking about how to handle "untrusted"
> files.  Is there some way of "sandboxing" load to allow only certain
> white-listed functions/macros?

As others have stated recently in this forum, you need to write a
trivial compiler to achieve much safety in this scenario.  The
compiler really is just a grammar checker.

Your compiler can work on data that has passed through READ (make sure
that *READ-EVAL* is bound to nil).  Once you're working with Lisp
data, writing the grammar checker / compiler should be pretty easy.

After you've verified that the forms in the file match a fixed, safe
grammar, you can leverage the COMPILE function to obtain full speed.

-russ
From: Alan Crowe
Subject: Re: loading data files
Date: 
Message-ID: <86ve8259xv.fsf@cawtech.freeserve.co.uk>
Andy Chambers <··············@googlemail.com> writes:

> I've defined a bunch of macros so that my program can load its data-
> structures by just calling `load' on a file that has something like
> 
> (item 1 "test" :a 1 :b 2)
> (item 2 "test-2" :a 1 :b 2)
> 
> This works fine but now I'm thinking about how to handle "untrusted"
> files.  Is there some way of "sandboxing" load to allow only certain
> white-listed functions/macros?
> 

Sandboxing in general is a difficult problem, but if you
only want to white-list some data definition operators you
can avoid the difficulties.

LOAD merely works through a file, reading and evaling each
form in turn, so you can write your own load that walks the
executable form built by read before evaling it.

You will need to bind *read-eval* to NIL.

CL-USER> (defparameter *allowed*
           '(foo bar))
*ALLOWED*

CL-USER> (defun walk (code)
           (typecase code
             (atom code)
             (list (if (member (first code)
                               *allowed*)
                       (mapcar #'walk code)
                       (error "Operator ~S is forbidden." 
                              (first code))))))
WALK

CL-USER> (walk (read))
(foo 3 (bar 5 7) 11)
(FOO 3 (BAR 5 7) 11)

CL-USER> (walk (read))
(foo 13 (bar (fool 17) 19))
; Evaluation aborted

You might want to do something like this anyway, to validate
the format of the input. It offers a good opportunity to
write much better error reporting than LOAD offers by
default.

Alan Crowe
Edinburgh
Scotland
From: Andy Chambers
Subject: Re: loading data files
Date: 
Message-ID: <10bb3ef7-f274-40da-8c65-8ad659e5a29e@c29g2000hsa.googlegroups.com>
On Nov 16, 6:20 pm, Alan Crowe <····@cawtech.freeserve.co.uk> wrote:
> Andy Chambers <··············@googlemail.com> writes:
> > I've defined a bunch of macros so that my program can load its data-
> > structures by just calling `load' on a file that has something like
>
> > (item 1 "test" :a 1 :b 2)
> > (item 2 "test-2" :a 1 :b 2)
>
> > This works fine but now I'm thinking about how to handle "untrusted"
> > files.  Is there some way of "sandboxing" load to allow only certain
> > white-listed functions/macros?
>
> Sandboxing in general is a difficult problem, but if you
> only want to white-list some data definition operators you
> can avoid the difficulties.
>
> LOAD merely works through a file, reading and evaling each
> form in turn, so you can write your own load that walks the
> executable form built by read before evaling it.

I thought that might be the case.  Didn't realize how easy it would be
though.

Cheers,
Andy
From: Rob Warnock
Subject: Re: loading data files
Date: 
Message-ID: <vJOdnQDI1ZBX_aPanZ2dnUVZ_gWdnZ2d@speakeasy.net>
Andy Chambers  <··············@googlemail.com> wrote:
+---------------
| Alan Crowe <····@cawtech.freeserve.co.uk> wrote:
| > Andy Chambers <··············@googlemail.com> writes:
| > > I've defined a bunch of macros so that my program can load its data-
| > > structures by just calling `load' on a file that has something like
| > >    (item 1 "test" :a 1 :b 2)
| > >    (item 2 "test-2" :a 1 :b 2)
| > > This works fine but now I'm thinking about how to handle "untrusted"...
...
| > LOAD merely works through a file, reading and evaling each
| > form in turn, so you can write your own load that walks the
| > executable form built by read before evaling it.
| 
| I thought that might be the case.  Didn't realize how easy it would be
| though.
+---------------

Might even be something as simple as this [though with ITEM
being a *function* in this case, rather than a macro!]:

    (defun my-data-load (file)
      (with-open-file (s file)
	(let ((*read-eval* nil))
	  (loop with eof = (list nil)
		for form = (read s nil eof)
		until (eq form eof) do
	    (if (and (consp form) (eq (car form) 'item))
	      (apply #'item (cdr form))
	      (error "Bad form in input: ~s" form))))))


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607