Hello,
I just wanted to use MP:WITH-TIMEOUT from CMU CL on my FreeBSD system
running CMU CL 18c. But it does not abort the body fully disregarding
the timeout value.
Can somebody verify this?
PORT:WITH-TIMEOUT from CLOCC does not work either.
Regards,
Julian
--
Um meinen oeffentlichen Schluessel zu erhalten:
To get my public key:
http://math-www.uni-paderborn.de/pgp/
Julian Stecklina <··········@web.de> wrote in message news:<··············@blitz.comp.com>...
> I just wanted to use MP:WITH-TIMEOUT from CMU CL on my FreeBSD system
> running CMU CL 18c. But it does not abort the body fully disregarding
> the timeout value.
>
> Can somebody verify this?
I had problems with this too.
A couple of ideas:
1) Did you turn on SIGALRM thread interrupts with (mp:start-sigalrm-yield ..)?
If so, this may still be dangerous in cmucl, and I _think_ this is when
WITH-TIMEOUT failed. You might want to try avoiding SIGALRM interrupts,
and writing your threads to yield themselves.
2) If your threads yield with (mp:process-yield) rather than
relying on the risky SIGALRM scheduler, it should be possible to
to write a wrapper for (mp:process-yield) that aborts the process
if (> (mp::process-%real-time your-process) *your-timeout*)
Sorry if this is vague, but I don't remember the details, since I gave up
on with-timeout a while ago, deciding that it didn't work, and wasn't really
the right thing to do, anyway. Unfortunately, the only way to understand
CMUCL threads is to read the source in multi-proc.lisp.
> PORT:WITH-TIMEOUT from CLOCC does not work either.
I also decided that CLOCC was a little oversimplified, especially
for things like threads and sockets, because it glosses over a lot of
implementation-dependent subtleties.
Julian Stecklina <··········@web.de> writes:
> I just wanted to use MP:WITH-TIMEOUT from CMU CL on my FreeBSD system
> running CMU CL 18c. But it does not abort the body fully disregarding
> the timeout value.
>
> Can somebody verify this?
>
> PORT:WITH-TIMEOUT from CLOCC does not work either.
The general WITH-TIMEOUT is a pretty bad solution to implementing
timeouts. I think it was some engineer at Xanalys that made me aware
of the fact that a straightforward WITH-TIMEOUT is inherently
dangerous, since it may kill a process while it is executing
UNWIND-PROTECT cleanup forms.
--
(espen)
Espen Vestre <·····@*do-not-spam-me*.vestre.net> writes:
> Julian Stecklina <··········@web.de> writes:
>
> > I just wanted to use MP:WITH-TIMEOUT from CMU CL on my FreeBSD system
> > running CMU CL 18c. But it does not abort the body fully disregarding
> > the timeout value.
> >
> > Can somebody verify this?
> >
> > PORT:WITH-TIMEOUT from CLOCC does not work either.
>
> The general WITH-TIMEOUT is a pretty bad solution to implementing
> timeouts. I think it was some engineer at Xanalys that made me aware
> of the fact that a straightforward WITH-TIMEOUT is inherently
> dangerous, since it may kill a process while it is executing
> UNWIND-PROTECT cleanup forms.
The with-timeout contains communication through a
socket. socket-cleanup is outside of with-timeout.
If you claim with-timeout to be bad, what would you suggest instead?
And why isn't with-timeout working... (the original quesion)
Regards,
Julian
--
Um meinen oeffentlichen Schluessel zu erhalten:
To get my public key:
http://math-www.uni-paderborn.de/pgp/
Julian Stecklina <··········@web.de> writes:
> The with-timeout contains communication through a
> socket. socket-cleanup is outside of with-timeout.
>
> If you claim with-timeout to be bad, what would you suggest instead?
> And why isn't with-timeout working... (the original quesion)
I don't think I'm familiar enough with CMU CL to give you the complete
answer, but often process-wait-with-timeout may be a good solution to
timeout implementations.
--
(espen)
Julian Stecklina <··········@web.de> writes:
> The with-timeout contains communication through a
> socket. socket-cleanup is outside of with-timeout.
>
> If you claim with-timeout to be bad, what would you suggest instead?
> And why isn't with-timeout working... (the original quesion)
If the timeout really occurs only due to socket communication, I'd
suggest using the :timeout argument to make-fd-stream instead, which
seems like a much better/safer solution. This will cause read/write
operations on the stream to abort with an error of type
system:io-timeout if the specified timeout is exceeded, which can be
usefully handled by your cleanup code.
I don't know why with-timeout isn't working, we'll have to look into
that.
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
Espen Vestre <·····@*do-not-spam-me*.vestre.net> writes:
> Julian Stecklina <··········@web.de> writes:
>
> > I just wanted to use MP:WITH-TIMEOUT from CMU CL on my FreeBSD system
> > running CMU CL 18c. But it does not abort the body fully disregarding
> > the timeout value.
> >
> > Can somebody verify this?
> >
> > PORT:WITH-TIMEOUT from CLOCC does not work either.
>
> The general WITH-TIMEOUT is a pretty bad solution to implementing
> timeouts. I think it was some engineer at Xanalys that made me aware
> of the fact that a straightforward WITH-TIMEOUT is inherently
> dangerous, since it may kill a process while it is executing
> UNWIND-PROTECT cleanup forms.
I think this is a bug in the casting of the CL spec into multiprocessing.
Certainly the spec does not say what a conforming use of
UNWIND-PROTECT is during multiprocessing, so this is all just personal
opinion, but IMO a system should always try very hard to execute cleanups,
even on a process kill.
I remember this was an issue on MIT Multics, ages ago, since one was
charged for computer time moment by moment, and there was a risk that
what amounted to an unwind-protect (Multics PL/1 had such things, as I
recall) in your logout init file could leave you unable to log out and
still being billed. As I recall, their solution was to have the attempt
to kill just time out after a while. I guess this is more or less the same
conclusion reached by Microsoft when you kill a process from the task
manager -- give it a chance to exit cleanly and then ask interactively
about harsher means.
Bad things happen when you don't respect UNWIND-PROTECT, though, and people
should not be encouraged to assume they are unsafe.
"Kent M Pitman" <······@world.std.com> wrote in message
····················@shell01.TheWorld.com...
> Espen Vestre <·····@*do-not-spam-me*.vestre.net> writes:
> > The general WITH-TIMEOUT is a pretty bad solution to implementing
> > timeouts. I think it was some engineer at Xanalys that made me aware
> > of the fact that a straightforward WITH-TIMEOUT is inherently
> > dangerous, since it may kill a process while it is executing
> > UNWIND-PROTECT cleanup forms.
>
> I think this is a bug in the casting of the CL spec into multiprocessing.
IMHO it is bug in the expectations of typical CL multiprocessing packages/users.
Most threading systems (pthreads, Win32, Java etc) don't have primitives that
execute arbitrary code asychronously in a thread (as PROCESS-INTERRUPT does),
simply because it is uncontrollably dangerous. Unfortunately, WITH-TIMEOUT is
usually implemented like that (i.e. WITH-TIMED-ASYNCHRONOUS-THROW).
> Certainly the spec does not say what a conforming use of
> UNWIND-PROTECT is during multiprocessing, so this is all just personal
> opinion, but IMO a system should always try very hard to execute cleanups,
> even on a process kill.
The issue with WITH-TIMEOUT is what happens if the timeout occurs in the cleanup
forms, e.g. in the SLEEP below.
(let (stream)
(unwind-protect
(progn
(setq stream (open file))
(long-computation stream))
(sleep wasting-time)
(close stream)))
--
Martin Simmons
······@xanalys.com
rot13 to reply
Martin Simmons wrote:
> "Kent M Pitman" <······@world.std.com> wrote in message
> ····················@shell01.TheWorld.com...
>> Espen Vestre <·····@*do-not-spam-me*.vestre.net> writes:
>> > The general WITH-TIMEOUT is a pretty bad solution to implementing
>> > timeouts. I think it was some engineer at Xanalys that made me aware
>> > of the fact that a straightforward WITH-TIMEOUT is inherently
>> > dangerous, since it may kill a process while it is executing
>> > UNWIND-PROTECT cleanup forms.
>>
>> I think this is a bug in the casting of the CL spec into multiprocessing.
>
> IMHO it is bug in the expectations of typical CL multiprocessing
> packages/users. Most threading systems (pthreads, Win32, Java etc) don't
> have primitives that execute arbitrary code asychronously in a thread (as
> PROCESS-INTERRUPT does),
> simply because it is uncontrollably dangerous. Unfortunately,
> WITH-TIMEOUT is usually implemented like that (i.e.
> WITH-TIMED-ASYNCHRONOUS-THROW).
Is there an alternative for implementing a facility like WITH-TIMEOUT?
I'm really interested since if there would be a way to implement it safer I
could try that for ACL-COMPAT. I do not really have the option to get rid
of WITH-TIMEOUT since the whole purpose of ACL-COMPAT is to be as compatible
as possible to the ACL API.
In regard to AllegroServe and it's uses of WITH-TIMEOUT I could rewrite
this parts of it but this would mean further diverging from the original
codebase and therefore should be thought upon carefully.
The problem seems to be that you asynchronously break out of a computation
that may not like that. To me it seems that the whole purpose of
WITH-TIMEOUT is to break out of an arbitrary computation. So I don't think
that WITH-TIMEOUT is wrong but more that it is not very suited for some
uses.
If someone uses a thing like WITH-TIMEOUT he should be aware of the dangers
that can arise when using cleanup code inside of it.
>> Certainly the spec does not say what a conforming use of
>> UNWIND-PROTECT is during multiprocessing, so this is all just personal
>> opinion, but IMO a system should always try very hard to execute
>> cleanups, even on a process kill.
>
> The issue with WITH-TIMEOUT is what happens if the timeout occurs in the
> cleanup forms, e.g. in the SLEEP below.
>
> (let (stream)
> (unwind-protect
> (progn
> (setq stream (open file))
> (long-computation stream))
> (sleep wasting-time)
> (close stream)))
Yes I have already shown such an example using my WITH-TIMEOUT code in
another post in this thread.
How about rewriting your example to something like that (untested):
(let (stream)
(flet ((clean-up () (when stream (close stream) (setf stream nil))))
(acl-mp:with-timeout (secs (clean-up))
(unwind-protect
(progn
(setq stream (open file))
(long-computation stream))
(sleep wasting-time)
(clean-up)))))
The critical point in this example is the point between closing the stream
and setting the binding to nil. If we are at this point when the clean-up
forms in the unwind-protect run then the timeout code will again try to
close the already closed stream.
The Hyperspec says at the page for CLOSE:
"It is permissible to close an already closed stream, but in that case the
result is implementation-dependent"
So in this particular example this code might workaround the problem but it
is certainly an ugly hack (and I might have overseen some important facts).
I fear that it gets even worser if there are multiple nested
WITH-TIMEOUTs...
ciao,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> How about rewriting your example to something like that (untested):
>
> (let (stream)
> (flet ((clean-up () (when stream (close stream) (setf stream nil))))
> (acl-mp:with-timeout (secs (clean-up))
> (unwind-protect
> (progn
> (setq stream (open file))
> (long-computation stream))
> (sleep wasting-time)
> (clean-up)))))
>
> The critical point in this example is the point between closing the stream
> and setting the binding to nil. If we are at this point when the clean-up
> forms in the unwind-protect run then the timeout code will again try to
> close the already closed stream.
>
> The Hyperspec says at the page for CLOSE:
> "It is permissible to close an already closed stream, but in that case the
> result is implementation-dependent"
>
> So in this particular example this code might workaround the problem but it
> is certainly an ugly hack (and I might have overseen some important facts).
> I fear that it gets even worser if there are multiple nested
> WITH-TIMEOUTs...
Why not using a WITHOUT-SCHEDULING or WITHOUT-INTERRUPTS in the unwind
body?
--
Frederic Brunel
Software Engineer
In-Fusio, The Mobile Fun Connection
Frederic Brunel wrote:
> Jochen Schmidt <···@dataheaven.de> writes:
>
>> How about rewriting your example to something like that (untested):
>>
>> (let (stream)
>> (flet ((clean-up () (when stream (close stream) (setf stream nil))))
>> (acl-mp:with-timeout (secs (clean-up))
>> (unwind-protect
>> (progn
>> (setq stream (open file))
>> (long-computation stream))
>> (sleep wasting-time)
>> (clean-up)))))
>>
>> The critical point in this example is the point between closing the
>> stream and setting the binding to nil. If we are at this point when the
>> clean-up forms in the unwind-protect run then the timeout code will again
>> try to close the already closed stream.
>>
>> The Hyperspec says at the page for CLOSE:
>> "It is permissible to close an already closed stream, but in that case
>> the result is implementation-dependent"
>>
>> So in this particular example this code might workaround the problem but
>> it is certainly an ugly hack (and I might have overseen some important
>> facts). I fear that it gets even worser if there are multiple nested
>> WITH-TIMEOUTs...
>
> Why not using a WITHOUT-SCHEDULING or WITHOUT-INTERRUPTS in the unwind
> body?
Yes this was the second solution I proposed in
http://groups.google.com/groups?q=WITH-TIMEOUT&hl=en&selm=a503ff%245l0%241%40rznews2.rrze.uni-erlangen.de&rnum=2
But in the above example the (sleep wasting-time) means that you stop all
interrupts for at least wasting-time. The above example using some kind of
"cleanup-flag" does not interfere with the rest of the system that way.
You cannot use WITHOUT-INTERRUPTS only in (clean-up) because the timeout
could already occur in (sleep wasting-time).
ciao,
Jochen
--
http://www.dataheaven.de
>>>>> "JS" == Jochen Schmidt <···@dataheaven.de> writes:
[...]
JS> Is there an alternative for implementing a facility like
JS> WITH-TIMEOUT? [...]
I have not looked too deeply into this, but if the implementation allows
you to make UNIX signal handlers to signal lisp conditions then you can
use SIGALARM and signal a 'timeout' condition from your SIGALARM handler.
I don't know if there are good reasons for the dynamic lisp context to be
available to UNIX signal handlers but it seemed to work when I tried it.
I had test code for CMUCL at some point but I can't find it. I believe
Dave Bakhash tried the idea for LW and confirmed it worked when we were
protoyping something. These were both on x86 so maybe they worked because
of some paculiarity of the architecture requiring signal handler stacks
to be stacked on lisp stacks.
In any event _if_ you can manage to get this to work you can have
nesting with-timeout calls and such by being clever about always
setting the one alarm timer you have to the soonest timeout.
There is a buggy but probably fixable example of signalling lisp
conditions from UNIX signal handlers in:
http://www.bagley.org/~doug/shootout/bench/echo/echo.cmucl
There was also a thread here last year or so about the same topic, but I
don't remember if there was any solid code mentioned in it.
cheers,
BM
Bulent Murtezaoglu wrote:
>>>>>> "JS" == Jochen Schmidt <···@dataheaven.de> writes:
> [...]
> JS> Is there an alternative for implementing a facility like
> JS> WITH-TIMEOUT? [...]
>
> I have not looked too deeply into this, but if the implementation allows
> you to make UNIX signal handlers to signal lisp conditions then you can
> use SIGALARM and signal a 'timeout' condition from your SIGALARM handler.
This sounds like the timers you already can create using LispWorks'
multiprocessing package (see the documentation for MP:MAKE-TIMER)
But the problem is not setting up the timer but the use of UNWIND-PROTECT
within the body of such a WITH-TIMEOUT form. If the timeout triggers the
actual running code is interrupted regardless where it were. If it were in
the clean-up forms of an UNWIND-PROTECT this clean-up forms get interrupted.
I have to say that I do not know what original ACLs WITH-TIMEOUT does in
this situation. I suppose that it interrupts the cleanup-forms too - but I
have not tested this.
> I don't know if there are good reasons for the dynamic lisp context to be
> available to UNIX signal handlers but it seemed to work when I tried it.
> I had test code for CMUCL at some point but I can't find it. I believe
> Dave Bakhash tried the idea for LW and confirmed it worked when we were
> protoyping something. These were both on x86 so maybe they worked because
> of some paculiarity of the architecture requiring signal handler stacks
> to be stacked on lisp stacks.
Why didn't you use the timers that already come with LispWorks? (For the LW
version I mean)
> In any event _if_ you can manage to get this to work you can have
> nesting with-timeout calls and such by being clever about always
> setting the one alarm timer you have to the soonest timeout.
The problem with nested WITH-TIMEOUT calls I saw was not because of nesting
the timeouts but more that an outer WITH-TIMEOUT can throw you out of the
timeout-forms of an inner timeout. So you could not even use the
"cleanup-flag" approach I outlined. The only doable solution would be
WITHOUT-INTERRUPTs around the clean-up forms (with all consequences...).
ciao,
Jochen
--
http://www.dataheaven.de
>>>>> "JS" == Jochen Schmidt <···@dataheaven.de> writes:
[...]
JS> But the problem is not setting up the timer but the use of
JS> UNWIND-PROTECT within the body of such a WITH-TIMEOUT form. If
JS> the timeout triggers the actual running code is interrupted
JS> regardless where it were. If it were in the clean-up forms of
JS> an UNWIND-PROTECT this clean-up forms get interrupted.
You are absolutely right. I missed the point. I was assuming that
by signalling a lisp condition we'd avoid complications. Not so, since
the clean-up forms in unwind-protect are not protected against non-local
exists even when they are caused by lisp mechanisms. Sorry.
[... on testing the signal/alarm hack on LW]
JS> Why didn't you use the timers that already come with
JS> LispWorks? (For the LW version I mean)
I didn't do that bit. I think Dave just wanted to experiment.
sorry 'bout the confusion,
BM
>>>>> On Thu, 21 Feb 2002 19:17:49 GMT, Bulent Murtezaoglu ("Bulent") writes:
Bulent> I was assuming that by signalling a lisp condition we'd
Bulent> avoid complications. Not so, since the clean-up forms in
Bulent> unwind-protect are not protected against non-local exists
Bulent> even when they are caused by lisp mechanisms. Sorry.
UNWIND-PROTECT protects against non-local exits such as GO, THROW, RETURN,
and signalled conditions, but the _cleanup_ forms are _not_ protected.
You could wrap the cleanup forms in a nested UNWIND-PROTECT like this:
(block nil
(unwind-protect
(return 1) ;non-local exit
(unwind-protect
(return 2) ;non-local exit while cleaning up
(return 3)))) ;better do this in that case
Maybe your nested cleanup form could at least set some state
indicating that things were known to be an invalid.
If you want to be more careful about not blowing away a computation
with a PROCESS-INTERRUPT, there are several strategies.
The cleanup-forms could bind a handler for the interrupt condition.
Or, better, rather than using PROCESS-INTERRUPT to signal a condition,
you could have it set just a flag in a special variable that's noticed
by your control loop (which is the sort of very minimal thing that an
interrupt handler would typically do).
Even though the multiprocessing/interrupt and condition system are
abstract and seem high-level, don't be misled into thinking that this
solves all the problems with multiprocessing. You still must think
about all those issues: it's just that you have been given some
primitives that are halfway reasonable to deal with.
"Christopher C. Stacy" <······@theworld.com> wrote in message
··················@theworld.com...
> You could wrap the cleanup forms in a nested UNWIND-PROTECT like this:
>
> (block nil
> (unwind-protect
> (return 1) ;non-local exit
> (unwind-protect
> (return 2) ;non-local exit while cleaning up
> (return 3)))) ;better do this in that case
Unfortunately, that still makes some assumptions, e.g. that UNWIND-PROTECT can't
be interrupted just before it executes the first cleanup.
> Maybe your nested cleanup form could at least set some state
> indicating that things were known to be an invalid.
If you can guarantee to set state atomically w.r.t. interrupts.
--
Martin Simmons, Xanalys Software Tools
······@xanalys.com
rot13 to reply
From: Michael J. Ferrador
Subject: Re: WITH-TIMEOUT and CMU CL
Date:
Message-ID: <3C7661B1.F756178B@orn.com>
Martin Simmons wrote:
>
> "Christopher C. Stacy" <······@theworld.com> wrote in message
> ··················@theworld.com...
> > You could wrap the cleanup forms in a nested UNWIND-PROTECT like this:
> >
> > (block nil
> > (unwind-protect
> > (return 1) ;non-local exit
> > (unwind-protect
> > (return 2) ;non-local exit while cleaning up
> > (return 3)))) ;better do this in that case
>
> Unfortunately, that still makes some assumptions, e.g. that UNWIND-PROTECT can't
> be interrupted just before it executes the first cleanup.
major redesign - But, queueable interrupts ?
or more generally - multiple event queues (1 per process/thread) ?
I've seen this requested per app or window for the X window system.
CMUCL does have an event system? or is it CLM's?
> > Maybe your nested cleanup form could at least set some state
> > indicating that things were known to be an invalid.
>
> If you can guarantee to set state atomically w.r.t. interrupts.
"Michael J. Ferrador" <·····@orn.com> wrote in message
······················@orn.com...
> Martin Simmons wrote:
> >
> > "Christopher C. Stacy" <······@theworld.com> wrote in message
> > ··················@theworld.com...
> > > You could wrap the cleanup forms in a nested UNWIND-PROTECT like this:
> > >
> > > (block nil
> > > (unwind-protect
> > > (return 1) ;non-local exit
> > > (unwind-protect
> > > (return 2) ;non-local exit while cleaning up
> > > (return 3)))) ;better do this in that case
> >
> > Unfortunately, that still makes some assumptions, e.g. that UNWIND-PROTECT
can't
> > be interrupted just before it executes the first cleanup.
>
> major redesign - But, queueable interrupts ?
>
>
> or more generally - multiple event queues (1 per process/thread) ?
> I've seen this requested per app or window for the X window system.
>
> CMUCL does have an event system? or is it CLM's?
Yes, that is effectively what Christopher was suggesting, i.e. handle the
interrupt at a well-defined safe place rather than asychronously.
--
Martin Simmons, Xanalys Software Tools
······@xanalys.com
rot13 to reply
>>>>> On Fri, 22 Feb 2002 13:05:08 -0000, Martin Simmons ("Martin") writes:
Martin> Unfortunately, that still makes some assumptions, e.g. that
Martin> UNWIND-PROTECT can't be interrupted just before it executes
Martin> the first cleanup.
Yes, the nested UNWIND-PROTECT example was only intended to show
how you can protect cleanups, but not in the face of multiprocessing
interrupts. That's why I suggested _not_ using PROCESS-INTERRUPT to
cause a non-local exit, and to use it only to carefully manage an
interrupt service state vector (processed at a different level).
For any readers who somehow missed the point that Martin and everyone
is trying to make: Common Lisp is not defined for multiprocessing,
and in particular there are no guarantees about operations being atomic
(beyond not corrupting the given implementation).
"Jochen Schmidt" <···@dataheaven.de> wrote in message
·················@rznews2.rrze.uni-erlangen.de...
> The problem with nested WITH-TIMEOUT calls I saw was not because of nesting
> the timeouts but more that an outer WITH-TIMEOUT can throw you out of the
> timeout-forms of an inner timeout. So you could not even use the
> "cleanup-flag" approach I outlined. The only doable solution would be
> WITHOUT-INTERRUPTs around the clean-up forms (with all consequences...).
Yes, WITHOUT-INTERRUPTS/WITHOUT-SCHEDULING is another rather a bad thing to have
in a threading API.
--
Martin Simmons, Xanalys Software Tools
······@xanalys.com
rot13 to reply
"Jochen Schmidt" <···@dataheaven.de> wrote in message
·················@rznews2.rrze.uni-erlangen.de...
> Martin Simmons wrote:
> > IMHO it is bug in the expectations of typical CL multiprocessing
> > packages/users. Most threading systems (pthreads, Win32, Java etc) don't
> > have primitives that execute arbitrary code asychronously in a thread (as
> > PROCESS-INTERRUPT does),
> > simply because it is uncontrollably dangerous. Unfortunately,
> > WITH-TIMEOUT is usually implemented like that (i.e.
> > WITH-TIMED-ASYNCHRONOUS-THROW).
>
> Is there an alternative for implementing a facility like WITH-TIMEOUT?
Not that I know of.
> The problem seems to be that you asynchronously break out of a computation
> that may not like that. To me it seems that the whole purpose of
> WITH-TIMEOUT is to break out of an arbitrary computation. So I don't think
> that WITH-TIMEOUT is wrong but more that it is not very suited for some
> uses.
> If someone uses a thing like WITH-TIMEOUT he should be aware of the dangers
> that can arise when using cleanup code inside of it.
Yes, that's true. That means you need to avoid calling functions which might
use cleanups, but CL doesn't specify which functions (or even special operators)
that might be so it won't be portable.
> > (let (stream)
> > (unwind-protect
> > (progn
> > (setq stream (open file))
> > (long-computation stream))
> > (sleep wasting-time)
> > (close stream)))
>
> Yes I have already shown such an example using my WITH-TIMEOUT code in
> another post in this thread.
Sorry, I missed that.
> How about rewriting your example to something like that (untested):
>
> (let (stream)
> (flet ((clean-up () (when stream (close stream) (setf stream nil))))
> (acl-mp:with-timeout (secs (clean-up))
> (unwind-protect
> (progn
> (setq stream (open file))
> (long-computation stream))
> (sleep wasting-time)
> (clean-up)))))
>
> The critical point in this example is the point between closing the stream
> and setting the binding to nil. If we are at this point when the clean-up
> forms in the unwind-protect run then the timeout code will again try to
> close the already closed stream.
>
> The Hyperspec says at the page for CLOSE:
> "It is permissible to close an already closed stream, but in that case the
> result is implementation-dependent"
True, but this kind of solution is only available when the cleanups are known
when you write the WITH-TIMEOUT body.
Also, if the timeout happens inside CLOSE then it will get hairy. I think the
Hyperspec is assuming that CLOSE returns normally before you call it again.
Wrapping CLOSE in WITHOUT-INTERRUPTS might not be good for performance if the
file is remote.
> I fear that it gets even worser if there are multiple nested
> WITH-TIMEOUTs...
Don't even think about it :-)
--
Martin Simmons, Xanalys Software Tools
······@xanalys.com
rot13 to reply
Espen Vestre wrote:
> Julian Stecklina <··········@web.de> writes:
>
>> I just wanted to use MP:WITH-TIMEOUT from CMU CL on my FreeBSD system
>> running CMU CL 18c. But it does not abort the body fully disregarding
>> the timeout value.
>>
>> Can somebody verify this?
>>
>> PORT:WITH-TIMEOUT from CLOCC does not work either.
>
> The general WITH-TIMEOUT is a pretty bad solution to implementing
> timeouts. I think it was some engineer at Xanalys that made me aware
> of the fact that a straightforward WITH-TIMEOUT is inherently
> dangerous, since it may kill a process while it is executing
> UNWIND-PROTECT cleanup forms.
AllegroServe uses ACLs MP:WITH-TIMEOUT at several places.
I found no notes about particular dangers in regard to UNWIND-PROTECT (or
how they handle it at all) in their documentation.
In ACL-COMPAT I use the following code for LispWorks:
(defun invoke-with-timeout (timeout bodyfn timeoutfn)
(block timeout
(let* ((process mp:*current-process*)
(timer (mp:make-timer
#'(lambda ()
(mp:process-interrupt process
#'(lambda ()
(return-from timeout
(funcall timeoutfn))))))))
(mp:schedule-timer-relative timer timeout)
(unwind-protect (funcall bodyfn)
(mp:unschedule-timer timer)))))
(defmacro with-timeout ((seconds &body timeout-forms) &body body)
"Execute BODY; if execution takes more than SECONDS seconds, terminate
and evaluate TIMEOUT-FORMS."
`(invoke-with-timeout ,seconds #'(lambda () ,@body)
#'(lambda () ,@timeout-forms)))
I wrote a little testfunction to show the issue with this particular
implementation:
(defun test-timeout (secs)
(acl-mp:with-timeout (secs (format t "Timeout!~%"))
(unwind-protect (progn (sleep 5)
(format t "Body completed!~%"))
(progn
(format t "Begin protected code~%")
(sleep 5)
(format t "End protected code~%")))))
CL-USER 110 > (test-timeout 15) ; Enough timeout for all forms
Body completed!
Begin protected code
End protected code
NIL
CL-USER 111 > (test-timeout 2) ; Timeout while the UNWIND-PROTECT body runs
Timeout!
Protected code
End protected code
NIL
CL-USER 112 > (test-timeout 7) ; Timeout in UNWIND-PROTECT cleanup forms
Body completed!
Begin protected code
Timeout!
NIL
As we can see this WITH-TIMEOUT executes the cleanup forms if the timeout
ends in the UNWIND-PROTECT body.
This means that this WITH-TIMEOUT does not guarantee to return within the
time given to it because the cleanup-forms may take some time too (like
here). But this is not the real problem which manifests in the third
example. As we are already in the cleanup forms there is no thing that
prevents the timeout of being interrupted.
I see to ways to workaround this:
Use a "cleaned-up" flag in the unwind-protect so that the timeout-forms can
resume the cleanup if we get thrown out in the cleanup-forms.
Use something like WITHOUT-INTERRUPTS around the cleanup-forms.
AFAIK ACL added timeouts on I/O operations. This approach seems not to have
this problem with interrupted cleanup-forms because the timeouts can only
occur in the I/O functions within the body and are not triggered by some
external source. LW4.2 added a "read-timeout" property to socket-streams
which might allow a similar approach.
ciao,
Jochen
--
http://www.dataheaven.de
Jochen Schmidt <···@dataheaven.de> writes:
> external source. LW4.2 added a "read-timeout" property to socket-streams
> which might allow a similar approach.
Yes, this feature is really useful. I think it solves most of the
i/o-related timeout-problems (together with process-wait-with-timeout).
(Of course one might want to use timeouts on completely different
operations than i/o-related ones, and then WITH-TIMEOUT may be useful,
if used with care.)
--
(espen)
Jochen Schmidt <···@dataheaven.de> writes:
...
> As we can see this WITH-TIMEOUT executes the cleanup forms if the timeout
> ends in the UNWIND-PROTECT body.
> This means that this WITH-TIMEOUT does not guarantee to return within the
> time given to it because the cleanup-forms may take some time too (like
> here).
...
I ran into this problem in the MOO language, which allocates time to
programs in hard quanta, so if you don't get done in time, you get
blown away. The question is what to do with the out-of-time
signal--should the unwinds be run? My feeling is that an optimally
designed language that cares about such timing issues (and that would
include a multiprocessing Lisp that wants predictable time
performance) should be doing the moral analog of what window systems
do when they do all that painful arithmetic about borders to make sure
windows with scroll bars in windows with borders still draw in the
right fclipping region: they should laboriously compute the difference
between the central area and the fringe space. So when set up an
unwind, you should reserve time for the execution of it. That means
you should do _very_ simple (or at least _very_ predictable) things in
your unwinds, and a manner should be created for predicting the time
those simple things will take, and and time allocated to execution of
a process should be able to ask the process how long it needs to
finish; and highly optimized programs would infer or predict this info
at compile time.