Hello all,
I would be happy if someone (maybe Kent Pitman :-)) could explain me
how to solve the following problem.
I want to read Lisp code to make some statistical analysis. The read
function is the obvious solution. The problem is that the code
may contains symbols belonging to packages which do not exist yet.
Here is an example:
USER(60): (read)
(defun foo ()
(bar:baz ...))
Error: Package "BAR" not found.
[condition type: READER-ERROR]
[1] USER(61):
The reader function complains about an undefined package, but the
signaled condition is of type READER-ERROR. At this moment, I would
like to identify the missing package, create it, and retry the read
operation. Is this possible (without accessing the printed
representation of the condition)?
There are other kinds of errors that I want to deal with. For
example, in the previous example, if I create the missing package,
another error will be signaled:
Error: Symbol "BAZ" not found in the BAR package.
[condition type: READER-ERROR]
[1] USER(89):
Obviously, I now export BAZ from BAR and retry.
So, is there a generic solution?
Thanks in advance,
Ant=F3nio Leit=E3o.
In article <··············@gia.ist.utl.pt> Antonio Leitao <···@gia.ist.utl.pt> writes:
> I would be happy if someone (maybe Kent Pitman :-)) could explain me
> how to solve the following problem. I want to read Lisp code to make
> some statistical analysis. The read function is the obvious solution.
> The problem is that the code may contains symbols belonging to
> packages which do not exist yet.
Ola' Antonio. Desde que voce^ mencionou o meu nome especificamente,
e sinto um pouco de responsibilidade para o sistema de condico~es de CL,
tentarei explicar. Se ainda ha' alguma problema depois, me pode
mandar e-mail para mais informac,a~o.
--Kent (···@harlequin.com)
> So, is there a generic solution?
Yes and no. Within any given implementation, you can PROBABLY write a
generic solution. There is no portable generic solution because the
precise nature of the conditions and restarts is not presently
standardized. But the number of implementations at present is small,
so coming up with a generic solution for the implementation(s) you're
dealing with isn't an impossible task.
Fortunately, you asked for me to show you HOW to solve the problem,
you didn't actually ask for the solution. :-)
The two things that will vary among implementations are the
representation of the continuation and the set of available restarts.
I will here show you some worked examples of how to figure out both
of those.
(1) Figure out the class of error that is signaled in your implementation.
Although the class will vary by implementation, most implementations
will provide enough of a stack debugger for you to find the frame
with the call to ERROR and figure out how to recognize the class
of error and how to pick apart that error.
So suppose some code were being used like this
(with-open-file (s "somefile.lisp")
(loop (let ((form (read s nil nil)))
(if (not form) (return) (print form)))))
and suppose your file somefile.lisp contains:
(defun foo (x) (bar:baz x))
This code will blow out at some point and you can use the debugger
to see the erring stack frame. In LispWorks, we apparently don't
signal a very high-tech error, but you can still see what condition
to expect and get the data:
Call to CERROR
CONDITIONS::CONTINUE-STRING : "Make a new symbol"
CONDITIONS::DATUM : CONDITIONS:SIMPLE-READER-ERROR
CONDITIONS::ARGUMENTS : (:FORMAT-STRING "Symbol ~S not found at all in the ~A package." :FORMAT-ARGUMENTS ("BAZ" "BAR"))
or if the symbol exists and not exported you'll get:
Call to CERROR
CONDITIONS::CONTINUE-STRING : "use symbol anyway."
CONDITIONS::DATUM : CONDITIONS:SYMBOL-NOT-EXTERNAL-READER-ERROR
CONDITIONS::ARGUMENTS : (:SYMBOL "BAZ" :PACKAGE "BAR")
So now we know the two types of conditions that will be potentially made.
There is also a slight problem of figuring out how to access those.
You may have to poke around in documentation to learn that a
CONDITIONS:SIMPLE-ERROR (a mixin to CONDITIONS:SIMPLE-READER-ERROR above)
uses accessors CONDITIONS:FORMAT-STRING and CONDITIONS:FORMAT-ARGUMENTS
(in CLtL2) or CONDITIONS:FORMAT-CONTROL and CONDITIONS:FORMAT-ARGUMENTS
(in ANSI CL). (Of course, the debugger above showed us :FORMAT-STRING
being used, so you can tell I'm using a CLtL2-compliant Lisp.)
It wasn't obvious to me what the accessors were for
CONDITIONS:SYMBOL-NOT-EXTERNAL-READER-ERROR
so I'm going to use SLOT-VALUE, which works in LispWorks. It might require
some help from the people who support your Lisp to find out what accessor you
need for the condition signalled in your Lisp.
In some implementations, the inspector can also be helpful in figuring out
accessors (or how to bypass them). To invoke the inspector, you could try:
(handler-bind ((error #'(lambda (c) (inspect c))))
(with-open-file (s "somefile.lisp")
(loop (let ((form (read s nil nil)))
(if (not form) (return) (print form))))))
This won't head off debugger entry, but it will cause you to enter the debugger
first in case you can't figure out how to get your hands on the actual
condition object in your debugger--some debuggers make that harder than others.
(2) Figure out what restarts are available. In this case, the simplest way is:
(handler-bind ((error #'(lambda (c)
(dolist (r (compute-restarts))
(format t "~&~S - ~A~%" (restart-name r) r)))))
(with-open-file (s "somefile.lisp")
(loop (let ((form (read s nil nil)))
(if (not form) (return) (print form))))))
What you want to know is the symbol tag that's used to identify the restart
you usually select, so you can use FIND-RESTART and/or INVOKE-RESTART with
that tag. If it has a tag of NIL in your implementation, you can't use
FIND-RESTART and INVOKE-RESTART on the NIL, but you can still map over the
result of COMPUTE-RESTARTS calling (PRINC-TO-STRING restart) and seeing if
the restart's printed form is the string you want. It's clumsy, but it will
mostly work.
OK! Supposing you got this far, you just have to put that information back together
into a handler. A handler gets control at the time of the call to ERROR and is allowed
to take an action that will head off entry to the debugger. In this case, for
LispWorks, the result looks like:
(handler-bind (#+LispWorks
(conditions:simple-reader-error
#'(lambda (c)
(when (string-equal (conditions:simple-condition-format-string c) "Symbol ~S not found at all in the ~A package.")
(destructuring-bind (symbol-name package-name)
(conditions:simple-condition-format-arguments c)
;... maybe intern symbol-name in package-name so this doesn't happen again...
;e.g.: (intern symbol-name package-name)
(invoke-restart 'continue)))))
#+LispWorks
(conditions:symbol-not-external-reader-error
#'(lambda (c)
;maybe do something with (slot-value c 'symbol) and (slot-value c 'package)
;to export the symbol so this won't happen again
;e.g.: (export (slot-value c 'symbol) (slot-value c 'package))
(invoke-restart 'continue))))
(with-open-file (s "somefile.lisp")
(loop (let ((form (read s nil nil)))
(if (not form) (return) (print form))))))
Since the condition handlers are not portable, I've marked them with #+ for the
appropriate implementation. To find what features to use to identify your Lisp,
see the *features* list.
For brevity in this long message, and because I don't know your personal background,
I've omitted explanations of what many of these operators do. I encourage you to
download a copy of the Common Lisp HyperSpec(TM) from
http://www.harlequin.com/books/HyperSpec/
so that you have an easy reference to the language for whatever operators I've
mentioned that you might not be familiar with.
Now I can't let you go without one word of warning:
The above code allows you to read code that mentions exported symbols
that the writer of the original code thought were exported but you didn't
know were supposed to be exported. That means the two of you do not have
an agreement about the function protocol to use. In most cases,
auto-exporting is NOT the answer to this problem. It can be done,
but that doesn't make it advisable. The errors you get from referencing
external symbols that are not exported are safeguards, and you're writing
code that will be pushing past safeguards without asking. Be sure that's
really what you want.
Espero que essa explicac,a~o vai da-lhe muitas ide'ias para
experimentar mais. Ate' logo...
--Kent (···@harlequin.com)
From: Vassil Nikolov
Subject: Re: Help on Common Lisp condition system.
Date:
Message-ID: <33BD094A.7C03@lri.fr>
Kent M Pitman wrote:
#<detailed example on handling exceptions omitted>
> Now I can't let you go without one word of warning:
>
> The above code allows you to read code that mentions exported symbols
> that the writer of the original code thought were exported but you didn't
> know were supposed to be exported. That means the two of you do not have
> an agreement about the function protocol to use. In most cases,
> auto-exporting is NOT the answer to this problem. It can be done,
> but that doesn't make it advisable. The errors you get from referencing
> external symbols that are not exported are safeguards, and you're writing
> code that will be pushing past safeguards without asking. Be sure that's
> really what you want.
Perhaps I might continue this line a little further, since it _does_
seem that the problem is not about handling the exceptions, but whether
a more basic change is needed. If this is a code walker, shouldn't it
take care to process EVAL-WHEN forms? So far, there is a problem with
non-existent packages/symbols; one day there will be a problem with
undefined macro characters, and so on.
(I know code walkers are hard to write, but solving problems on
a piece-meal basis might turn out to be more expensive eventually...)
--
Vassil Nikolov, visitor at LRI:
Laboratoire de Recherche en Informatique, Universite Paris-Sud
Until 9 July 1997: ··············@lri.fr
Normally: ········@bgearn.acad.bg
In article <·············@lri.fr> Vassil Nikolov
<··············@lri.fr> writes:
> If this is a code walker, shouldn't it take care to process
> EVAL-WHEN forms? So far, there is a problem with non-existent
> packages/symbols; one day there will be a problem with undefined
> macro characters, and so on.
I think this is a very astute observation. However, I received more
information by e-mail about why the question had been asked and am now
satisfied that for the limited situation he has, what he's doing is
less likely to run out of control in that way.
If one were dealing with code-walkers, I agree with you that EVAL-WHEN
is problematic, however there isn't really a good general-purpose
solution since often the whole reason to use a code-walker is to avoid
evaluation. What might really be wished for is a way to isolate
`evaluation' (even in the global environment) to a localizable effect
that can be discarded afterward. (There are deep philosophical
reasons why this is hard in Lisp, and many of them have to do with the
way certain things have meaning only when interned into the REAL
package system and the REAL class system, and would not make equal
sense in an isolated "clean" environment.)
Consider just the case of code-walking (or compiling) a file with one
set of read syntax definitions and then doing the same with a set of
incompatible definitions, perhaps even ones that required on the previous
side-effects not having happened. It is presently beyond the scope of
CL to deal with these issues of shared/inherited global environments,
and if you look carefully at the spec, you'll see the issue of how this
relationship is established and what "compile time evaluation" means
is danced around carefully and left considerably to the implementation.
The advantage of handling each error as a special case is that you can
also write code to undo the effect after you're done, so that later
runs are not polluted. While calling EVAL manages the general case of
macro characters and other things in the forward direction, it loses
control of reversability...
More and more I'm convinced:
In the real world there are no solutions, only trade-offs.
From: Vassil Nikolov
Subject: Re: Help on Common Lisp condition system.
Date:
Message-ID: <33BE2CB6.1BD3@lri.fr>
I would like to continue this a little further, in more general
terms than the original specific situation.
Kent M Pitman wrote:
>
#<text omitted>
>
> If one were dealing with code-walkers, I agree with you that EVAL-WHEN
> is problematic, however there isn't really a good general-purpose
> solution since often the whole reason to use a code-walker is to avoid
> evaluation. What might really be wished for is a way to isolate
> `evaluation' (even in the global environment) to a localizable effect
> that can be discarded afterward. (There are deep philosophical
> reasons why this is hard in Lisp, and many of them have to do with the
> way certain things have meaning only when interned into the REAL
> package system and the REAL class system, and would not make equal
> sense in an isolated "clean" environment.)
Yes, writing appropriate exception handlers, especially if the
problem is limited, is certainly less painful than writing a code
walker. (Perhaps it would be nice to have a general-purpose
off-the-shelf code walker with a lot of hooks, but I'm just
fantasizing.) What annoys me here is that the principle of
`code is data' does not quite apply when just reading printed
representation of code (vs. reading and evaluating same).
The Lisp reader could be extended with a lot more functionality,
though it would be difficult to prove that the increased complexity
is really worth it. For example, there might be a hook in the
reader just after en extended token is read and before it is converted
into a number or a symbol is interned. By default, that is what would
happen, but some code walkers might prefer to interfere at this
point to handle non-existent packages, etc., thus avoiding the need
to process EVAL-WHEN forms. This, of course, just ignores the
problem of `local isolation of evaluation effects,' but would
improve the situation as regards to `code is data when read.'
(One could also wish to be able to bind just individual character
macro definitions, rather than copy the whole readtable, modify
the copy, and bind *READTABLE*, but maybe that's overkill.)
> Consider just the case of code-walking (or compiling) a file with one
> set of read syntax definitions and then doing the same with a set of
> incompatible definitions, perhaps even ones that required on the previous
> side-effects not having happened. It is presently beyond the scope of
> CL to deal with these issues of shared/inherited global environments,
> and if you look carefully at the spec, you'll see the issue of how this
> relationship is established and what "compile time evaluation" means
> is danced around carefully and left considerably to the implementation.
Obviously that was one of the hard jobs when the language definition
was produced: have a meaningful EVAL-WHEN without overspecifying it.
(It is also a hard job to grasp what EVAl-WHEN really does.)
The important issue here is `shared/inherited global environments,'
and it may well need a next generation of the language to introduce
this concept. The save/restore operations of PostScript come to mind,
but then introducing such a thing into Lisp is not straightforward (to
put it mildly). Perhaps---just dreaming again---the next generation
of Lisp will have virtual machines (or has it been done already?).
If creating a new virtual machine is not an expensive operation, a code
walker could easily do evaluations with its own VM, and not worry
about the global effect. VMs would also provide a possible basis
for standardised multi-tasking in Lisp. (Who knows, maybe one day
we'll have the WITH-VM special form?)
> The advantage of handling each error as a special case is that you can
> also write code to undo the effect after you're done, so that later
> runs are not polluted. While calling EVAL manages the general case of
> macro characters and other things in the forward direction, it loses
> control of reversability...
Once again, you've hit the point! A *truly great* programming
environment
ought to have reversability. Unfortunately, it is the programmer's
responsibility to ensure this. (In a perfect world, maybe we would have
automatic reversability, where one could have APPLY-IN-REVERSE. ':-)')
>
> More and more I'm convinced:
> In the real world there are no solutions, only trade-offs.
And the trade-offs are often suboptimal.
--
Vassil Nikolov, visitor at LRI:
Laboratoire de Recherche en Informatique, Universite Paris-Sud
Until 9 July 1997: ··············@lri.fr
Normally: ········@bgearn.acad.bg