Greetings. I'm wondering if anyone knows of a "generally accepted" Lisp
idiom for gracefully killing a thread and allowing it to "clean up" before
dieing. Perhaps an example would help; in this case the Lisp implementation
is Allegro.
We've a test suite that exercises a Lisp server process we've written. The
suite creates a number of threads that beat on the server indefinitely; to
stop the test one evaluates a function in the suite's interface. Currently,
this function simply issues an MP:PROCESS-KILL. Unfortunately, threads in
the test suite hold open streams to the server and these streams are not
being closed, at least in a timely fashion [ perhaps the streams would
eventually be closed during some GC cycle, but I'm sure this is
implementation dependent ]
I can think of a couple of alternatives here; one perhaps using
MP:PROCESS-INTERRUPT to raise a handled condition within the process's run
function (not sure about this though). However, since I'm certain this is a
somewhat common situation I'd like to know if there is a semi-standard idiom
for handling the problem. I realize that any solution will likely be
implementation-specific, but I'd like some ideas regardless. Thanks much.
Regards,
--
------------------------------------------
David E. Young
········@computer.org
http://lisa.sourceforge.net
"But all the world understands my language."
-- Franz Joseph Haydn (1732-1809)
From: ···@itasoftware.com
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <heuhk58x.fsf@itasoftware.com>
"David E. Young" <········@computer.org> writes:
> Greetings. I'm wondering if anyone knows of a "generally accepted" Lisp
> idiom for gracefully killing a thread and allowing it to "clean up" before
> dieing. Perhaps an example would help; in this case the Lisp implementation
> is Allegro.
>
> We've a test suite that exercises a Lisp server process we've written. The
> suite creates a number of threads that beat on the server indefinitely; to
> stop the test one evaluates a function in the suite's interface. Currently,
> this function simply issues an MP:PROCESS-KILL. Unfortunately, threads in
> the test suite hold open streams to the server and these streams are not
> being closed, at least in a timely fashion [ perhaps the streams would
> eventually be closed during some GC cycle, but I'm sure this is
> implementation dependent ]
I've argued this with Franz over and over again. Their answer appears
to be ``Don't use MP:PROCESS-KILL to terminate a process.''
There are a few ways to solve this. You could have each process poll
for a death signal (yuck), or you could fix the interaction between
process-kill and unwind-protect. The basic idea is to defer handling
of the process-kill signal during the cleanup forms of the
unwind-protect. I have some code that you might find useful.
<···@itasoftware.com> wrote in message ·················@itasoftware.com...
> "David E. Young" <········@computer.org> writes:
>
> > Greetings. I'm wondering if anyone knows of a "generally accepted" Lisp
> > idiom for gracefully killing a thread and allowing it to "clean up"
before
> > dieing. Perhaps an example would help; in this case the Lisp
implementation
> > is Allegro...
>
> I've argued this with Franz over and over again. Their answer appears
> to be ``Don't use MP:PROCESS-KILL to terminate a process.''
I cc'd the original post to ····@franz.com; according to Franz,
"mp:process-kill does clean-up of the process including that of running
unwind-protects. Consequently if you want files opened in the process to be
closed when the process is killed you would have to provide the
unwind-protects in which the files would be closed."
> ...you could fix the interaction between process-kill and unwind-protect.
So, are you saying the interaction between these two is broken in some way?
I haven't yet tried their suggestion so I don't know. I'd be interested in
seeing your code to handle this situation. Thanks much.
Regards,
--
------------------------------------------
David E. Young
········@computer.org
http://lisa.sourceforge.net
"But all the world understands my language."
-- Franz Joseph Haydn (1732-1809)
From: ···@itasoftware.com
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <zo89ieqo.fsf@itasoftware.com>
"David E. Young" <········@computer.org> writes:
> <···@itasoftware.com> wrote in message ·················@itasoftware.com...
> > "David E. Young" <········@computer.org> writes:
> >
> > > Greetings. I'm wondering if anyone knows of a "generally accepted" Lisp
> > > idiom for gracefully killing a thread and allowing it to "clean up"
> before
> > > dieing. Perhaps an example would help; in this case the Lisp
> implementation
> > > is Allegro...
> >
> > I've argued this with Franz over and over again. Their answer appears
> > to be ``Don't use MP:PROCESS-KILL to terminate a process.''
>
> I cc'd the original post to ····@franz.com; according to Franz,
>
> "mp:process-kill does clean-up of the process including that of running
> unwind-protects. Consequently if you want files opened in the process to be
> closed when the process is killed you would have to provide the
> unwind-protects in which the files would be closed."
>
> > ...you could fix the interaction between process-kill and unwind-protect.
>
> So, are you saying the interaction between these two is broken in some way?
> I haven't yet tried their suggestion so I don't know. I'd be interested in
> seeing your code to handle this situation. Thanks much.
I believe that Franz's implementation of unwind-protect is
insufficiently protective in the context of Franz's implementation of
multitasking. I base this belief on my experience writing a
multithreaded web-based application that required that all
unwind-protects be executed if a process were aborted. Using the
basic unwind-protect code provided by Franz would frequently cause
database handles to be `lost' requiring a exiting and reloading of the
lisp. Using a carefully coded replacement for unwind-protect solved
the bulk of the problems. (It was still possible to die in a
spectacular way, but it was far, far less likely).
jkf and I went back and forth on this in July. The gist was this:
jrm
Franz's implementation of unwind-protect is not safe from
asynchronous interrupts and thus not useful in a multitasking
environment.
jkf
This statement is false.
jrm
The implementation of unwind-protect in Allegro CL does not defer
handling of asynchronous interrupts during the `cleanup' forms of
an unwind-protect. This means that it is possible for an
ill-timed asynchronous event (control-c or process-kill, for
example) to cause execution of the cleanup forms in an
unwind-protect to be aborted.
In a multitasking environment, it is absolutely critical that
there be an unwind-protect mechanism that cannot be asynchronously
aborted. This means that another thread should not be able to
cause a non-local exit within this thread while the cleanup forms
are being run.
jkf
It's not critical or even necessary if you don't allow one thread
to process-interrupt another thread. You can control whether your
own code calls functions that would cause a non local transfer of
control of another thread.
jrm
Granted. If an asynchronous abort never occurs, it doesn't matter
if unwind-protect can't protect against it.
But I can't control whether another piece of code might want to
process-kill my process. Suppose I were using a web server
provided by a third party, and it spawned a worker thread for each
request, and it kept careful track of the amount of time spent in
the worker thread, killing it when a set limit is exceeded. This
seems to me a very plausible and reasonable design. However, if
my worker thread is beginning to clean up when the master kills
the thread, the cleanup forms *that are being executed* get
aborted, but the other ones further down the stack do not.
jkf
I don't think that it is a good design, it requires that code over
which you have no control be written to a unusually strict
standard, namely the standard that it leave nothing trashed should
execution of the thread cease at any point.
-----
We never did go much further than this, but as you can see, the
general idea is that you should not go around calling MP:PROCESS-KILL
and expect to recover from it.
Franz replied to you:
> "mp:process-kill does clean-up of the process including that of
> running unwind-protects. Consequently if you want files opened
> in the process to be closed when the process is killed you would
> have to provide the unwind-protects in which the files would be
> closed."
This isn't quite true. MP:PROCESS-KILL does clean-up of the process
including that of any *PENDING* unwind-protects. If the process being
killed is currently executing the cleanup form of an unwind-protect,
that cleanup is aborted. So in the case of a WITH-OPEN-FILE, which
expands into:
(COMMON-LISP:LET ((FOO (OPEN ....)) (#:G174410 T))
(COMMON-LISP:UNWIND-PROTECT
(MULTIPLE-VALUE-PROG1 (PROGN ...) (SETQ #:G174410 NIL))
(WHEN (STREAMP FOO) (CLOSE FOO :ABORT #:G174410))))
should the process kill arrive during the call to close (which is a
non-atomic call) the file will not be closed. This may be the effect
you are seeing.
You can test this rather easily. Try this version of unwind-protect:
(defmacro unwind-protect-without-interrupts (protected-form &body cleanup-forms)
(let ((interrupt-enables (gensym (symbol-name :interrupt-enables-))))
`(LET ((,interrupt-enables EXCL::*WITHOUT-INTERRUPTS*)
(EXCL::*WITHOUT-INTERRUPTS* t))
(UNWIND-PROTECT
(LET ((EXCL::*WITHOUT-INTERRUPTS* ,interrupt-enables))
,protected-form)
;; interrupts are off during cleanup
,@cleanup-forms))))
in the code that needs to clean up your stream (if you are using
WITH-OPEN-FILE or WITH-OPEN-STREAM, you'll have to write your own
expansion). If the problem goes away, then you have been bitten by
the unwind-protect bug.
The solution of disabling interrupts completely may or may not be
appropriate for your application (should be ok for testing). If you
cannot tolerate turning interrutps off for that period of time, it is
possible to write a version that allows multitasking to proceed, but
disallows asynchronous process kills during unwind-protect cleanups.
Let me know if the experiment with disabling interrupts works, and if
it does, we can deal with safely re-enabling and deferring them.
<···@itasoftware.com> wrote in message ·················@itasoftware.com...
>
> I believe that Franz's implementation of unwind-protect is
> insufficiently protective in the context of Franz's implementation of
> multitasking...
> We never did go much further than this, but as you can see, the
> general idea is that you should not go around calling MP:PROCESS-KILL
> and expect to recover from it.
>
> Franz replied to you:
> > "mp:process-kill does clean-up of the process including that of
> > running unwind-protects. Consequently if you want files opened
> > in the process to be closed when the process is killed you would
> > have to provide the unwind-protects in which the files would be
> > closed."
>
> This isn't quite true. MP:PROCESS-KILL does clean-up of the process
> including that of any *PENDING* unwind-protects. If the process being
> killed is currently executing the cleanup form of an unwind-protect,
> that cleanup is aborted...
Ah, I see. Yes, this seems to be a problem if indeed Allegro behaves this
way. This predicament is similar to the way Java behaves (or used to)
whenever STOP is (was) invoked on a thread, and why that function (and a few
related others) are deprecated and their use strongly discouraged.
Well, gosh, so I guess the disagreement is whether a) it's poor design to
kill a thread in this manner; or b) PROCESS-KILL is an acceptable interface
and ACL should guarantee proper cleanup in the presence of asynchronous
events.
>
> Let me know if the experiment with disabling interrupts works, and if
> it does, we can deal with safely re-enabling and deferring them.
>
I'll investigate and let you know the results. Thanks very much for the
detailed description.
Regards,
--
------------------------------------------
David E. Young
········@computer.org
http://lisa.sourceforge.net
"But all the world understands my language."
-- Franz Joseph Haydn (1732-1809)
From: ···@itasoftware.com
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <vgiwifgn.fsf@itasoftware.com>
"David E. Young" <········@computer.org> writes:
>
> Well, gosh, so I guess the disagreement is whether a) it's poor design to
> kill a thread in this manner; or b) PROCESS-KILL is an acceptable interface
> and ACL should guarantee proper cleanup in the presence of asynchronous
> events.
>
Pretty much.
My belief is that it ought to be possible to write code that can be
aborted asynchronously. I know it isn't easy, but it can be done with
care. I believe that there ought to be a mechanism for initiating
asynchronous actions that perform non-local exits (like process-kill
or a control-c interrupt). I believe that there ought to be a
mechanism for synchronizing with these asynchronous actions.
I think that the people at Franz agree with me on most of these points
(they argue that the ability to have an asynchronous action initiate a
non-local exit is just asking for trouble, but try debugging code
without ever aborting from a control-c). The big point we disagree on
is whether the mechanism for synchronizing with asynchronous events is
a new construct or simply the extension of UNWIND-PROTECT into the
multithreaded domain.
In my experience, I have found that people naturally expect
UNWIND-PROTECT to *always* run the cleanup form. After all, that is
what the Hyperspec says it does:
UNWIND-PROTECT ... guarantees that cleanup-forms are executed
before unwind-protect exits, whether it terminates normally or is
aborted by a control transfer of some kind. UNWIND-PROTECT is
intended to be used to make sure that certain side effects take
place after the evaluation of protected-form.
UNWIND-PROTECT protects against *all* attempts to exit from
protected-form, including go, handler-case, ignore-errors,
restart-case, return-from, throw, and
with-simple-restart. [emphasis mine]
Most people are suprised to find out that under certain circumstances,
in certain implementations, that *none* of the cleanup forms run
(consider an asynchronous interrupt occurring just as the cleanup
forms are entered, but before any real work is done), and it usually
takes a careful explanation of why this might be the case.
Understanding the problem does not lead to an obvious solution,
however. The macro I posted:
(defmacro unwind-protect (protected-form &body cleanup-forms)
(let ((interrupt-enables (gensym (symbol-name :interrupt-enables-))))
`(LET ((,interrupt-enables EXCL::*WITHOUT-INTERRUPTS*)
(EXCL::*WITHOUT-INTERRUPTS* t))
(CL:UNWIND-PROTECT
(LET ((EXCL::*WITHOUT-INTERRUPTS* ,interrupt-enables))
,protected-form)
;; interrupts are off during cleanup
,@cleanup-forms))))
relies on two non-portable details of implementation: the fact that
special variable binding and unbinding is atomic with respect to
interrupts, and that interrupt enables are controlled via the value of
a special variable. If either of these were not the case, it might
not even be possible to create a mechanism to synchronize with
asynchronous events (fortunately, the above is true under Allegro and
Lucid Common Lisp). Even so, it is probably not immediately obvious
why there have to be two nested special variable bindings (it took me
a bit of time to figure out that trick). So it would seem to me that
the onus is upon the *vendor* to supply the appropriate
synchronization mechanism. After all, they ought to know the
subtleties of their own implementation better than anyone else.
Supposing that the vendor *did* supply the appropriate mechanisms
(Lucid provided "interruptions-inhibited-unwind-protect"). Now, in
order to `port' your code to a multitasking environment, you simply
change all calls to UNWIND-PROTECT to
INTERRUPTIONS-INHIBITED-UNWIND-PROTECT. But chances are you didn't
develop your code in a single-threaded non-interruptable lisp. If
this is the case, you *always* want to use
INTERRUPTIONS-INHIBITED-UNWIND-PROTECT, and *never* use the
single-threaded UNWIND-PROTECT. But if UNWIND-PROTECT already did
what INTERRUPTIONS-INHIBITED-UNWIND-PROTECT does, the effort of
`porting' to a multithreaded environment is NIL, and many programs
that run in a single-threaded lisp could run unchanged in a
multithreaded lisp (provided that only one instance of the program is
running).
It seems to me that making UNWIND-PROTECT safe against asynchronous
throws in the general case (and providing a way to make it unsafe
should one really desire that) is a lot more in the spirit of ``the
right thing'' than leaving it up to the end user to discover this
nasty property and try to engineer a solution himself.
First let me say that I feel that your summary of our discussion has
been accurate and fair.
I'd just like to show how I read these parts of the spec
> UNWIND-PROTECT ... guarantees that cleanup-forms are executed
> before unwind-protect exits, whether it terminates normally or is
> aborted by a control transfer of some kind. UNWIND-PROTECT is
> intended to be used to make sure that certain side effects take
> place after the evaluation of protected-form.
What does it mean for forms to be 'executed'? Well it means that
lisp starts to evaluate them in the normal way. If a throw
is done, then control leaves the exit forms. If an interrupt
occurs then it's handled normally.
> UNWIND-PROTECT protects against *all* attempts to exit from
> protected-form, including go, handler-case, ignore-errors,
> restart-case, return-from, throw, and
> with-simple-restart. [emphasis mine]
>
What the standard doesn't say though is that unwind-protect protects
against all attempts to exit from the *cleanup-forms*. I think
that's a meaningful omission. Even not considering MP there can
be errors in cleanup forms and errors could cause an early
exit from the cleanup forms as control passes to enclosing
condition handler.
From: ···@itasoftware.com
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <wv3c8er2.fsf@itasoftware.com>
John Foderaro <···@xspammerx.franz.com> writes:
> First let me say that I feel that your summary of our discussion has
> been accurate and fair.
Thanks!
> I'd just like to show how I read these parts of the spec
>
> > UNWIND-PROTECT ... guarantees that cleanup-forms are executed
> > before unwind-protect exits, whether it terminates normally or is
> > aborted by a control transfer of some kind. UNWIND-PROTECT is
> > intended to be used to make sure that certain side effects take
> > place after the evaluation of protected-form.
>
> What does it mean for forms to be 'executed'? Well it means that
> lisp starts to evaluate them in the normal way. If a throw
> is done, then control leaves the exit forms. If an interrupt
> occurs then it's handled normally.
There's the rub: interrupts are not part of the ANSI spec, so there
is no definition of `normal' handling for them.
> > UNWIND-PROTECT protects against *all* attempts to exit from
> > protected-form, including go, handler-case, ignore-errors,
> > restart-case, return-from, throw, and
> > with-simple-restart. [emphasis mine]
> >
>
> What the standard doesn't say though is that unwind-protect protects
> against all attempts to exit from the *cleanup-forms*. I think
> that's a meaningful omission. Even not considering MP there can
> be errors in cleanup forms and errors could cause an early
> exit from the cleanup forms as control passes to enclosing
> condition handler.
Agreed, but those sort of errors are synchronous. (And I'm not
suggesting that *broken* code ought to magically start working in a MT
environment).
In the case where the cleanup forms *do* work correctly in a
non-multithreaded environment (i.e, they would normally run to
completion no matter what), I'd like them to continue working
correctly in a multithreaded environment.
We do treat interrupts as a kind of condition which
I think makes sense given that we have the
mechanism of the condition system already in place
to handle them.
> In the case where the cleanup forms *do* work correctly in a
> non-multithreaded environment (i.e, they would normally run to
> completion no matter what), I'd like them to continue working
> correctly in a multithreaded environment.
I think that 'no matter what' has to be specified carefully.
Your cleanup form could involve looking at special variables
or data structures and while that would work fine in a
non MP lisp when you go to a MP lisp you could have other
threads getting some CPU time and altering those
specials or data structures. You may say that when
a cleanup form is started no other thread can run and
interrupts are disabled. What if that cleanup form
has to do some serious cleaning up and this takes a while?
Do you want to really freeze out all other threads?
Can the cleanup form say "I'm going to be here awhile,
it's ok if other processes are run"? If so, how?
One could design a series of unwind-protect variants that
were more and more strict as regards what they would
let occur during the cleanup forms. You've already
written one variant.
I think that the current unwind-protect is the 'primitive'
on to which the variants can be built and it shouldn't
be changed.
[I speak only for myself, others here where I work
may disagree].
From: Kent M Pitman
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <sfwd75417pd.fsf@world.std.com>
John Foderaro <···@xspammerx.franz.com> writes:
> We do treat interrupts as a kind of condition which
> I think makes sense given that we have the
> mechanism of the condition system already in place
> to handle them.
Oh, please, please, please. I have worked so hard to get this terminology
right.
Some languages do indeed confuse conditions and interrupts.
The interrupting of a process is a primitive action that typically
halts the ongoing execution of the process, saving any relevant
ephemeral state for later restore (e.g., register state) and forcing
a new function call upon the process at the current point.
Interrupting is NOT a condition. It can't be. A condition is a static
object, not an action.
Interrupting is NOT a signal (in the Lisp sense). It can't be. A signal
is a synchronous invocation of the condition handling facility.
Interrupting is an imperative preemption invocation of a given function
in the current dynamic state of another process.
Now, what is commonly called a "keyboard interrupt" may be implemented by
a "process interrupt" (the thing I've described above) in which the
thing the interrupting function does is to (once it has done its
asynchronous thing and asserted a synchronous foothold) finally signal
a condition as part of the function which was given to process-interrupt.
This is probably what you meant. I just want to be careful because a lot
of people don't understand the CL condition system and they are confused
by the blurring of the term "signal"/"condition"/etc. that happens in other
languages where its all one atomic thing and doesn't have the quark-like
decomposition that CL has.
> > In the case where the cleanup forms *do* work correctly in a
> > non-multithreaded environment (i.e, they would normally run to
> > completion no matter what), I'd like them to continue working
> > correctly in a multithreaded environment.
>
> I think that 'no matter what' has to be specified carefully.
> Your cleanup form could involve looking at special variables
> or data structures and while that would work fine in a
> non MP lisp when you go to a MP lisp you could have other
> threads getting some CPU time and altering those
> specials or data structures.
Not bound specials. Yes global variables. Yes heap state.
Further, if people have used locks, they can rely on whatever structures
are locked being consistent even in global vars and heap state that is
protected by such locks.
> You may say that when
> a cleanup form is started no other thread can run and
> interrupts are disabled. What if that cleanup form
> has to do some serious cleaning up and this takes a while?
> Do you want to really freeze out all other threads?
There is an intermediate position possible here, depending on what kind of
discipline one uses. Even with MP enabled, the program may know that only
the current process will be trying to get at certain data EVEN IF no locks
were used. This is a design-time lock. Or they may use dynamic locks.
Or they may have designed their data to just not care. e.g., an attempt
to do an abort close is either going to succeed or perhaps is going to be
beaten out by some other process doing an abort-close on the same stream, or
there may even be I/O going on until the abort-close happens, but maybe it
all amounts to the same.
> Can the cleanup form say "I'm going to be here awhile,
> it's ok if other processes are run"? If so, how?
It can go into a WITHOUT-ABORTS state. I think that's what the LispM does.
You can do multiprocessing, but you may not abort. If you try, you end up
in the debugger and have to use a lower-level unwind operation to fish your
way out if that's what you really want.
> One could design a series of unwind-protect variants that
> were more and more strict as regards what they would
> let occur during the cleanup forms. You've already
> written one variant.
If there were several such protection methods (without-aborts,
without-preemption, without-interrupts) then indeed the user could decide,
but absent such portable means, users rely somewhat on the vendor to provide
them at least some cover if they're going to claim to have a conforming
implementation that is also multi-tasking...
On the LispM, if you try to interactively abort an unwind, for example, you
get an interactive query about the fact that a without-aborts is active and
asking how serious you are about the abort.
> I think that the current unwind-protect is the 'primitive'
> on to which the variants can be built and it shouldn't
> be changed.
>
> [I speak only for myself, others here where I work
> may disagree].
In article <···············@world.std.com>, ······@world.std.com says...
> This is probably what you meant.
Indeed.
I could have explained how ACL is informed of interrupts and how
it keeps track of that information and when ACL chooses to
interrupt user code and then what process is done to
process that interrupt and what the default behavior
is, but I just went right to the end and said that
(by default) the interrupt is represented as a condition
(and I didn't even mentioned the word 'signalled'
since I felt that the fact that the condition was
signalled was implied).
From: ···@itasoftware.com
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <sne0838f.fsf@itasoftware.com>
Kent M Pitman <······@world.std.com> writes:
> > > In the case where the cleanup forms *do* work correctly in a
> > > non-multithreaded environment (i.e, they would normally run to
> > > completion no matter what), I'd like them to continue working
> > > correctly in a multithreaded environment.
> >
> > I think that 'no matter what' has to be specified carefully.
> > Your cleanup form could involve looking at special variables
> > or data structures and while that would work fine in a
> > non MP lisp when you go to a MP lisp you could have other
> > threads getting some CPU time and altering those
> > specials or data structures.
>
> Not bound specials. Yes global variables. Yes heap state.
>
> Further, if people have used locks, they can rely on whatever structures
> are locked being consistent even in global vars and heap state that is
> protected by such locks.
Exactly.
> It can go into a WITHOUT-ABORTS state. I think that's what the LispM does.
> You can do multiprocessing, but you may not abort. If you try, you end up
> in the debugger and have to use a lower-level unwind operation to fish your
> way out if that's what you really want.
Yes. At LUCID we called it an `interruptions inhibited' state.
Multiprocessing still occurs, but you could not cause a thread in the
WITHOUT-ABORTS state (interruptions inhibited state) to perform any
asynchronous non-local exits.
> > One could design a series of unwind-protect variants that
> > were more and more strict as regards what they would
> > let occur during the cleanup forms. You've already
> > written one variant.
>
> If there were several such protection methods (without-aborts,
> without-preemption, without-interrupts) then indeed the user could decide,
> but absent such portable means, users rely somewhat on the vendor to provide
> them at least some cover if they're going to claim to have a conforming
> implementation that is also multi-tasking...
Right. And the problem is that it is *very hard* to write a reliable
version of these, even if you don't care about portability, even if
you have access to the source code and guts of the system, even if you
like to think about this sort of stuff. A more naive user would be
way out of his depth in this case.
> On the LispM, if you try to interactively abort an unwind, for example, you
> get an interactive query about the fact that a without-aborts is active and
> asking how serious you are about the abort.
Thank you, Kent, for reminding me what this feature was called on the
LispM.
From: Kent M Pitman
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <sfwofooe0tq.fsf@world.std.com>
John Foderaro <···@xspammerx.franz.com> writes:
> First let me say that I feel that your summary of our discussion has
> been accurate and fair.
>
> I'd just like to show how I read these parts of the spec
>
> > UNWIND-PROTECT ... guarantees that cleanup-forms are executed
> > before unwind-protect exits, whether it terminates normally or is
> > aborted by a control transfer of some kind. UNWIND-PROTECT is
> > intended to be used to make sure that certain side effects take
> > place after the evaluation of protected-form.
>
> What does it mean for forms to be 'executed'?
Not to disagree with John here, but merely as "auxiliary info", I recommend
a side-trip to X3J13 issue EXIT-EXTENT (available through the HyperSpec),
which discusses this issue in detail.
There was huge amounts of discussion about this at the X3J13 level.
I found the link for everyone who's interested:
http://www.xanalys.com/software_tools/reference/HyperSpec/Issues/iss152-
writeup.html
This goes to show that even in the non MP case the question of
what kind of world cleanup forms should live in is tricky.
ACL implements what this proposal calls the "medium" approach
which make cleanup forms see the same environment as they would
if the unwind-protect were replaced with a progn (I.e.
the cleanup forms are not treated specially).
<···@itasoftware.com> wrote in message ·················@itasoftware.com...
> It seems to me that making UNWIND-PROTECT safe against asynchronous
> throws in the general case (and providing a way to make it unsafe
> should one really desire that) is a lot more in the spirit of ``the
> right thing'' than leaving it up to the end user to discover this
> nasty property and try to engineer a solution himself.
Ok. So, suppose I want a practical solution that will be portable across the
"major" MP Lisps; say, ACL, LispWorks and CMUCL (omission from this list
does not imply a Lisp isn't "major"). Also, let's assume that we're not
going to worry about asynchronous interrupts other than perhaps PROCESS-KILL
(forget the keyboard, for example). Now, it seems to me that 1) PROCESS-KILL
is dangerous; and 2) UNWIND-PROTECT, out of the box, will not work reliably
because code could *in* the "finally" forms when the PROCESS-KILL arrives
and not complete.
Can I, perhaps, have the function given to PROCESS-INTERRUPT raise a
condition that's handled by the thread's run function? Hmm; I suppose the
PROCESS-INTERRUPT function doesn't run in the same context as the thread's
run function does it? I need to test this.
It sounds like there are no standard idioms to deal with this situation in
Common Lisp.
Regards,
--
------------------------------------------
David E. Young
········@computer.org
http://lisa.sourceforge.net
"But all the world understands my language."
-- Franz Joseph Haydn (1732-1809)
From: ···@itasoftware.com
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <66av864f.fsf@itasoftware.com>
"David E. Young" <········@computer.org> writes:
> <···@itasoftware.com> wrote in message ·················@itasoftware.com...
>
> > It seems to me that making UNWIND-PROTECT safe against asynchronous
> > throws in the general case (and providing a way to make it unsafe
> > should one really desire that) is a lot more in the spirit of ``the
> > right thing'' than leaving it up to the end user to discover this
> > nasty property and try to engineer a solution himself.
>
> Ok. So, suppose I want a practical solution that will be portable across the
> "major" MP Lisps; say, ACL, LispWorks and CMUCL (omission from this list
> does not imply a Lisp isn't "major"). Also, let's assume that we're not
> going to worry about asynchronous interrupts other than perhaps PROCESS-KILL
> (forget the keyboard, for example). Now, it seems to me that 1) PROCESS-KILL
> is dangerous; and 2) UNWIND-PROTECT, out of the box, will not work reliably
> because code could *in* the "finally" forms when the PROCESS-KILL arrives
> and not complete.
PROCESS-KILL *might* be dangerous. It depends on whether the Lisp
allows asynchronous aborts during an unwind-protect cleanup. I know
that ACL does not. I know that LCL (nee Lucid) will if you use
interruptions-inhibited-unwind-protect. I know that the LispM can
(but I don't know if it is the default). I suspect that neither
LispWorks and CMUCL protect against this, though.
If you have a lisp with this problem and that there is no obvious
vendor-supplied solution, you *may* be SOL. It would depend on a
number of factors. However, most Lisps that have multitasking have a
mechanism for turning it off. So it is likely that you could
determine what would be the equivalent of
(let ((old-value <is-multitasking-enabled>))
(<without-multitasking>
(unwind-protect
(<maybe-with-multitasking>
(body))
(cleanup))))
> Can I, perhaps, have the function given to PROCESS-INTERRUPT raise a
> condition that's handled by the thread's run function? Hmm; I suppose the
> PROCESS-INTERRUPT function doesn't run in the same context as the thread's
> run function does it? I need to test this.
It depends on how PROCESS-INTERRUPT (and/or PROCESS-KILL) is
implemented. A common way of implementing multithreading is to
timeslice the lisp process. Each thread takes turns running for a
brief period of time with a policy determined by a `scheduler'. When
the scheduler yields control to a particular thread, that thread first
checks a list of `pending interrupts' before continuing in the task
that it was doing. In this sort of implementation, the interrupt
function is run by the thread in the dynamic context of the thread
(whatever that might have been when the thread was forced to yield).
But this is not the only way to implement this sort of thing.
I have written a much hairier mechanism that may be a bit more
portable. It does not perform as well, so it is not suitable for use
in tight loops. On the other hand, it does not prohibit multitasking
during the cleanup, just aborts, so it is useful when the cleanup form
might be long-running or require locks. It works much like you
described. Every process, when launched, establishes a signal handler
for managing asynchronous events. The `standard' PROCESS-KILL and
PROCESS-INTERRUPT are shadowed by functions that interrupt the other
process and invoke the signal handler with a thunk. The signal
handler examines a special variable that indicates whether the process
is abortable. If it is, it immediately runs the thunk. If it is
*not*, though, it places the thunk on a list of pending interrupts and
invokes the appropriate restart handler for the signal.
When leaving a `nonabortable' context, the code is careful to check
the pending interrupt list. If that list contains pending thunks,
they are then run.
There are a few extra bells and whistles: if the interrupt is caused
by a control-C, rather than simply enqueue a thunk, a counter is
incremented. When the counter reaches a specified number, a warning
is printed that you are about to interrupt a process that is in a
`nonabortable' state. The next control-c will cause the debugger to
be entered. This also has the nice effect of `coalescing' multiple
deferred control-Cs (so pounding control-C a couple of times doesn't
cause you end up deep in the debugger).
When a process-kill is encountered, the handler goes into a
`process-is-being-killed' state. Further process interrupts are
prohibited, and the process makes the effort to unwind to top level.
> It sounds like there are no standard idioms to deal with this situation in
> Common Lisp.
No *standard* idioms that I am aware of, but the above mechanisms have
worked for me in a number of cases.
My personal belief is that the `stack-group' based model of
multitasking is way too low-level and that it ought to be replaced
with a higher-level interface. Among the things that would be nice to
see is a notion of an `independent' process vs. a `child' process,
and the notion of a `multithreaded server'.
From: Martin Simmons
Subject: Re: MP Lisp: idioms for cleanly killing a process
Date:
Message-ID: <1000149082.726025@itn>
"David E. Young" <········@computer.org> wrote in message
······························@typhoon.southeast.rr.com...
>
> <···@itasoftware.com> wrote in message ·················@itasoftware.com...
>
> > It seems to me that making UNWIND-PROTECT safe against asynchronous
> > throws in the general case (and providing a way to make it unsafe
> > should one really desire that) is a lot more in the spirit of ``the
> > right thing'' than leaving it up to the end user to discover this
> > nasty property and try to engineer a solution himself.
>
> Ok. So, suppose I want a practical solution that will be portable across the
> "major" MP Lisps; say, ACL, LispWorks and CMUCL (omission from this list
> does not imply a Lisp isn't "major"). Also, let's assume that we're not
> going to worry about asynchronous interrupts other than perhaps PROCESS-KILL
> (forget the keyboard, for example). Now, it seems to me that 1) PROCESS-KILL
> is dangerous; and 2) UNWIND-PROTECT, out of the box, will not work reliably
> because code could *in* the "finally" forms when the PROCESS-KILL arrives
> and not complete.
>
> Can I, perhaps, have the function given to PROCESS-INTERRUPT raise a
> condition that's handled by the thread's run function? Hmm; I suppose the
> PROCESS-INTERRUPT function doesn't run in the same context as the thread's
> run function does it? I need to test this.
It almost certainly won't matter what the context is: the function given to
PROCESS-INTERRUPT might run at an inappropriate time, so handling a condition
will cause exactly the same problems as PROCESS-KILL.
> It sounds like there are no standard idioms to deal with this situation in
> Common Lisp.
I think the only safe way to deal with this portably is to maintain an "abort
flag" for the process and check that periodically at safe places in the code.
--
Martin Simmons, Xanalys Software Tools
······@xanalys.com
rot13 to reply.
"Martin Simmons" <······@xanalys.com> wrote in message
······················@itn...
> It almost certainly won't matter what the context is: the function given
to
> PROCESS-INTERRUPT might run at an inappropriate time, so handling a
condition
> will cause exactly the same problems as PROCESS-KILL.
Agreed.
> I think the only safe way to deal with this portably is to maintain an
"abort
> flag" for the process and check that periodically at safe places in the
code.
Yes, probably so. Thanks very much...
--
------------------------------------------
David E. Young
········@computer.org
http://lisa.sourceforge.net
"But all the world understands my language."
-- Franz Joseph Haydn (1732-1809)