I'm working on a 4th iteration on monetd-mapi
http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
it's basically a library build with cffi http://common-lisp.net/project/cffi/
to interface with monetdb http://monetdb.cwi.nl/
The problem is that foreign code interaction has (sometimes) nasty
idea to block the whole lisp process ,
sometimes even crushing it, rare but it happens.
So I was thinking to provide some Erlang style *process-protection*
and CL-MUPROC came to my mind.
Anybody has some idea how well could it help for managing lost
processes, or has some other experiences with it?
Or any other suggestion for my problem will be appreciated?
thanks
Slobodan
On Mon, 24 Sep 2007 19:35:58 -0000, Slobodan Blazeski
<·················@gmail.com> wrote:
>I'm working on a 4th iteration on monetd-mapi
>http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
>it's basically a library build with cffi http://common-lisp.net/project/cffi/
>to interface with monetdb http://monetdb.cwi.nl/
>
>The problem is that foreign code interaction has (sometimes) nasty
>idea to block the whole lisp process ,
>sometimes even crushing it, rare but it happens.
>So I was thinking to provide some Erlang style *process-protection*
>and CL-MUPROC came to my mind.
>Anybody has some idea how well could it help for managing lost
>processes, or has some other experiences with it?
>Or any other suggestion for my problem will be appreciated?
I don't have any experience with CL-MUPROC, but because all of its
"processes" exist in the same Lisp image, I would be careful. If you
already have one or more FFI calls that can kill the serial image,
they might also be able to kill a threaded one ... it will depend on
your particular Lisp+FFI implementation.
A safer, albeit slower, solution is to remote the calls by routing
them though a separate OS process which can be monitored and restarted
if it crashes.
George
--
for email reply remove "/" from address
George Neuner wrote:
> On Mon, 24 Sep 2007 19:35:58 -0000, Slobodan Blazeski
> <·················@gmail.com> wrote:
>
>> I'm working on a 4th iteration on monetd-mapi
>> http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
>> it's basically a library build with cffi http://common-lisp.net/project/cffi/
>> to interface with monetdb http://monetdb.cwi.nl/
>>
>> The problem is that foreign code interaction has (sometimes) nasty
>> idea to block the whole lisp process ,
>> sometimes even crushing it, rare but it happens.
>> So I was thinking to provide some Erlang style *process-protection*
>> and CL-MUPROC came to my mind.
>> Anybody has some idea how well could it help for managing lost
>> processes, or has some other experiences with it?
>> Or any other suggestion for my problem will be appreciated?
>
> I don't have any experience with CL-MUPROC, but because all of its
> "processes" exist in the same Lisp image, I would be careful. If you
> already have one or more FFI calls that can kill the serial image,
> they might also be able to kill a threaded one ... it will depend on
> your particular Lisp+FFI implementation.
>
> A safer, albeit slower, solution is to remote the calls by routing
> them though a separate OS process which can be monitored and restarted
> if it crashes.
Yes, any "in-process" solution will have the same issue of hanging
when the CFFI code borks. Many implementations (SBCL, CMUCL, ...)
have some support for native threads. Maybe you could spawn a
background thread as a "watchdog timer" and have it kill/restart the
main thread when one of these incidents occurs? That way
multithreading doesn't interact with the normal code.
- Daniel
On Sep 25, 6:37 am, D Herring <········@at.tentpost.dot.com> wrote:
> George Neuner wrote:
> > On Mon, 24 Sep 2007 19:35:58 -0000, Slobodan Blazeski
> > <·················@gmail.com> wrote:
>
> >> I'm working on a 4th iteration on monetd-mapi
> >>http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
> >> it's basically a library build with cffihttp://common-lisp.net/project/cffi/
> >> to interface with monetdbhttp://monetdb.cwi.nl/
>
> >> The problem is that foreign code interaction has (sometimes) nasty
> >> idea to block the whole lisp process ,
> >> sometimes even crushing it, rare but it happens.
> >> So I was thinking to provide some Erlang style *process-protection*
> >> and CL-MUPROC came to my mind.
> >> Anybody has some idea how well could it help for managing lost
> >> processes, or has some other experiences with it?
> >> Or any other suggestion for my problem will be appreciated?
>
> > I don't have any experience with CL-MUPROC, but because all of its
> > "processes" exist in the same Lisp image, I would be careful. If you
> > already have one or more FFI calls that can kill the serial image,
> > they might also be able to kill a threaded one ... it will depend on
> > your particular Lisp+FFI implementation.
I want a portable solution supporting at least Allegro, LW, SBCL,
OpenMCL & OpenMCL.
>
> > A safer, albeit slower, solution is to remote the calls by routing
> > them though a separate OS process which can be monitored and restarted
> > if it crashes.
>
> Yes, any "in-process" solution will have the same issue of hanging
> when the CFFI code borks. Many implementations (SBCL, CMUCL, ...)
> have some support for native threads. Maybe you could spawn a
> background thread as a "watchdog timer" and have it kill/restart the
> main thread when one of these incidents occurs? That way
> multithreading doesn't interact with the normal code.
>
> - Daniel- Hide quoted text -
>
> - Show quoted text -
I was planning to start a few processes as a cffi connection pool and
manage them halt/kill/restart
but I need some library to do that automatically as a portable
solution, I'm not in a mood to learn
process quirks of a 5 different implementations.
On 25 Sep., 09:10, Slobodan Blazeski <·················@gmail.com>
wrote:
> On Sep 25, 6:37 am, D Herring <········@at.tentpost.dot.com> wrote:
>
>
>
> > George Neuner wrote:
> > > On Mon, 24 Sep 2007 19:35:58 -0000, Slobodan Blazeski
> > > <·················@gmail.com> wrote:
There is such a library in the Portable Allegro Server at
http://portableaserve.sourceforge.net/ - look in the acl-compat
directory.
On Sep 25, 3:22 pm, ·············@gmail.com wrote:
> On 25 Sep., 09:10, Slobodan Blazeski <·················@gmail.com>
> wrote:
>
> > On Sep 25, 6:37 am, D Herring <········@at.tentpost.dot.com> wrote:
>
> > > George Neuner wrote:
> > > > On Mon, 24 Sep 2007 19:35:58 -0000, Slobodan Blazeski
> > > > <·················@gmail.com> wrote:
>
> There is such a library in the Portable Allegro Server athttp://portableaserve.sourceforge.net/- look in the acl-compat
> directory.
acl-mp-package ? Thanks I will look at it.
From: Frank Goenninger
Subject: Re: How to protect your lisp foreign code interaction ?
Date:
Message-ID: <fd96ib$5c0$00$1@news.t-online.com>
On 2007-09-24 21:35:58 +0200, Slobodan Blazeski
<·················@gmail.com> said:
> I'm working on a 4th iteration on monetd-mapi
> http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
> it's basically a library build with cffi http://common-lisp.net/project/cffi/
> to interface with monetdb http://monetdb.cwi.nl/
>
> The problem is that foreign code interaction has (sometimes) nasty
> idea to block the whole lisp process ,
> sometimes even crushing it, rare but it happens.
Hmm. Difficult to see how to help you with that scarce description.
FFI? Pointers! Hard experience ... CFFI helps in handling them - still
they are a PITA.
Anything more specific available from your side?
For a more general approach as you outline:
> So I was thinking to provide some Erlang style *process-protection*
> and CL-MUPROC came to my mind.
> Anybody has some idea how well could it help for managing lost
> processes, or has some other experiences with it?
> Or any other suggestion for my problem will be appreciated?
Can't see how this helps, really. Yeah, you could run your
FFI-depending code in a seperate "process" - only thing is it's the
same image and not even a real separate process. Anything happens: Most
likely you'll be affected nevertheless.
OTOH I have no experience with CL-MUPROC as such.
>
> thanks
>
> Slobodan
Frank
--
Frank Goenninger
frgo(at)goenninger(dot)net
"Don't ask me! I haven't been reading comp.lang.lisp long enough to
really know ..."
On Sep 24, 10:26 pm, Frank Goenninger <····@goenninger.net> wrote:
> On 2007-09-24 21:35:58 +0200, Slobodan Blazeski
> <·················@gmail.com> said:
>
> > I'm working on a 4th iteration on monetd-mapi
> >http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
> > it's basically a library build with cffihttp://common-lisp.net/project/cffi/
> > to interface with monetdbhttp://monetdb.cwi.nl/
>
> > The problem is that foreign code interaction has (sometimes) nasty
> > idea to block the whole lisp process ,
> > sometimes even crushing it, rare but it happens.
>
> Hmm. Difficult to see how to help you with that scarce description.
> FFI? Pointers! Hard experience ... CFFI helps in handling them - still
> they are a PITA.
> Anything more specific available from your side?
I use CFFI as a portability layer to monetdb mapi library which is a c
code.
I want a portable solution supporting at least the big 5 : Allegro,
Lispworks, SBCL, CMUCL & OpenMCL.
I talk with monetdb through pointers. What kind of more specific
information you want.
>
> For a more general approach as you outline:
>
> > So I was thinking to provide some Erlang style *process-protection*
> > and CL-MUPROC came to my mind.
> > Anybody has some idea how well could it help for managing lost
> > processes, or has some other experiences with it?
> > Or any other suggestion for my problem will be appreciated?
>
> Can't see how this helps, really. Yeah, you could run your
> FFI-depending code in a seperate "process" - only thing is it's the
> same image and not even a real separate process. Anything happens: Most
> likely you'll be affected nevertheless.
Crup.
>
> OTOH I have no experience with CL-MUPROC as such.
>
>
>
> > thanks
>
> > Slobodan
>
> Frank
>
> --
> Frank Goenninger
>
> frgo(at)goenninger(dot)net
>
> "Don't ask me! I haven't been reading comp.lang.lisp long enough to
> really know ..."
Slobodan Blazeski <·················@gmail.com> writes:
> On Sep 24, 10:26 pm, Frank Goenninger <····@goenninger.net> wrote:
>> On 2007-09-24 21:35:58 +0200, Slobodan Blazeski
>> <·················@gmail.com> said:
>>
>> > I'm working on a 4th iteration on monetd-mapi
>> >http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
>> > it's basically a library build with cffihttp://common-lisp.net/project/cffi/
>> > to interface with monetdbhttp://monetdb.cwi.nl/
>>
>> > The problem is that foreign code interaction has (sometimes) nasty
>> > idea to block the whole lisp process ,
>> > sometimes even crushing it, rare but it happens.
>>
>> Hmm. Difficult to see how to help you with that scarce description.
>> FFI? Pointers! Hard experience ... CFFI helps in handling them - still
>> they are a PITA.
>> Anything more specific available from your side?
>
> I use CFFI as a portability layer to monetdb mapi library which is a c
> code.
I figured that, yes.
> I want a portable solution supporting at least the big 5 : Allegro,
> Lispworks, SBCL, CMUCL & OpenMCL.
Fine.
> I talk with monetdb through pointers. What kind of more specific
> information you want.
Code that crashes. defcfuns that make problems. Show us the CFFI stuff
you did and where you see something /unusual/.
>>
>> For a more general approach as you outline:
>>
>> > So I was thinking to provide some Erlang style *process-protection*
>> > and CL-MUPROC came to my mind.
>> > Anybody has some idea how well could it help for managing lost
>> > processes, or has some other experiences with it?
>> > Or any other suggestion for my problem will be appreciated?
>>
>> Can't see how this helps, really. Yeah, you could run your
>> FFI-depending code in a seperate "process" - only thing is it's the
>> same image and not even a real separate process. Anything happens: Most
>> likely you'll be affected nevertheless.
>
> Crup.
Crup? I am not a native speaker but the only word I can come up with
is "Crap". If so, well, thanks for being elaborate ...
Frank
--
Frank Goenninger
http://www.goenninger.net
frgo(at)mac(dot)com
"Don't ask me! I haven't been reading comp.lang.lisp long enough to
really know ..."
On Sep 25, 6:41 pm, Frank Goenninger DG1SBG <·············@nomail.org>
wrote:
> Slobodan Blazeski <·················@gmail.com> writes:
> > On Sep 24, 10:26 pm, Frank Goenninger <····@goenninger.net> wrote:
> >> On 2007-09-24 21:35:58 +0200, Slobodan Blazeski
> >> <·················@gmail.com> said:
>
> >> > I'm working on a 4th iteration on monetd-mapi
> >> >http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
> >> > it's basically a library build with cffihttp://common-lisp.net/project/cffi/
> >> > to interface with monetdbhttp://monetdb.cwi.nl/
>
> >> > The problem is that foreign code interaction has (sometimes) nasty
> >> > idea to block the whole lisp process ,
> >> > sometimes even crushing it, rare but it happens.
>
> >> Hmm. Difficult to see how to help you with that scarce description.
> >> FFI? Pointers! Hard experience ... CFFI helps in handling them - still
> >> they are a PITA.
> >> Anything more specific available from your side?
>
> > I use CFFI as a portability layer to monetdb mapi library which is a c
> > code.
>
> I figured that, yes.
>
> > I want a portable solution supporting at least the big 5 : Allegro,
> > Lispworks, SBCL, CMUCL & OpenMCL.
>
> Fine.
>
> > I talk with monetdb through pointers. What kind of more specific
> > information you want.
>
> Code that crashes. defcfuns that make problems. Show us the CFFI stuff
> you did and where you see something /unusual/.
>
>
>
> >> For a more general approach as you outline:
>
> >> > So I was thinking to provide some Erlang style *process-protection*
> >> > and CL-MUPROC came to my mind.
> >> > Anybody has some idea how well could it help for managing lost
> >> > processes, or has some other experiences with it?
> >> > Or any other suggestion for my problem will be appreciated?
>
> >> Can't see how this helps, really. Yeah, you could run your
> >> FFI-depending code in a seperate "process" - only thing is it's the
> >> same image and not even a real separate process. Anything happens: Most
> >> likely you'll be affected nevertheless.
>
> > Crup.
>
> Crup? I am not a native speaker but the only word I can come up with
> is "Crap". If so, well, thanks for being elaborate ...
>
> Frank
Frank I'm not a native speaker either and I didn't wanted to offend
anybody, actually I'm very gratefull for all the good replies in this
replies. Crup was a reaction for being unable to find a good way to
avoid being affected.
I'm not able to point specific cffi code because it doesn't happen in
deterministic way, it's more of a random and seldom occurence, it
might be lisp or it might be monetdb , I don't know , that's why I
want to control the processes the Erlang way, creating something like
http://yarivsblog.com/articles/2006/09/13/erlang-mysql-driver-reloaded/.
I'm in a refactoring & clean up process so maybe things will clear up
afterward.
cheers
bobi
From: Frank Goenninger
Subject: Re: How to protect your lisp foreign code interaction ?
Date:
Message-ID: <fdelv7$98p$01$1@news.t-online.com>
On 2007-09-26 21:44:17 +0200, Slobodan Blazeski
<·················@gmail.com> said:
>> Crup? I am not a native speaker but the only word I can come up with
>> is "Crap". If so, well, thanks for being elaborate ...
>>
>> Frank
>
> Frank I'm not a native speaker either and I didn't wanted to offend
> anybody, actually I'm very gratefull for all the good replies in this
> replies. Crup was a reaction for being unable to find a good way to
> avoid being affected.
Hey hey - no offense taken. I just wanted to be a good cll citizen and
give the post an edge ;-)
Never lose your humor on that very special newsgroup - or that's what I
/try/ to, anyway :-)
> I'm not able to point specific cffi code because it doesn't happen in
> deterministic way, it's more of a random and seldom occurence, it
> might be lisp or it might be monetdb , I don't know , that's why I
> want to control the processes the Erlang way, creating something like
> http://yarivsblog.com/articles/2006/09/13/erlang-mysql-driver-reloaded/.
> I'm in a refactoring & clean up process so maybe things will clear up
> afterward.
Oh - I wish you good luck! I have been doing quite a lot of CFFI /
foreign code debugging so if you feel you can throw something over,
just do so. I'd be glad to help.
>
> cheers
> bobi
Cheers
Frank
--
Frank Goenninger
frgo(at)goenninger(dot)net
"Don't ask me! I haven't been reading comp.lang.lisp long enough to
really know ..."
On Sep 27, 12:20 am, Frank Goenninger <····@goenninger.net> wrote:
> On 2007-09-26 21:44:17 +0200, Slobodan Blazeski
> <·················@gmail.com> said:
>
> >> Crup? I am not a native speaker but the only word I can come up with
> >> is "Crap". If so, well, thanks for being elaborate ...
>
> >> Frank
>
> > Frank I'm not a native speaker either and I didn't wanted to offend
> > anybody, actually I'm very gratefull for all the good replies in this
> > replies. Crup was a reaction for being unable to find a good way to
> > avoid being affected.
>
> Hey hey - no offense taken. I just wanted to be a good cll citizen and
> give the post an edge ;-)
> Never lose your humor on that very special newsgroup - or that's what I
> /try/ to, anyway :-)
>
> > I'm not able to point specific cffi code because it doesn't happen in
> > deterministic way, it's more of a random and seldom occurence, it
> > might be lisp or it might be monetdb , I don't know , that's why I
> > want to control the processes the Erlang way, creating something like
> >http://yarivsblog.com/articles/2006/09/13/erlang-mysql-driver-reloaded/.
> > I'm in a refactoring & clean up process so maybe things will clear up
> > afterward.
>
> Oh - I wish you good luck! I have been doing quite a lot of CFFI /
> foreign code debugging so if you feel you can throw something over,
> just do so. I'd be glad to help.
Thanks for your offer Frank I might just do that.
I only dream that someday people will be building foreign interfaces
to lisp instead of vice versa.
I hope I will seen that someday come true.
>
>
>
> > cheers
> > bobi
>
> Cheers
> Frank
>
> --
> Frank Goenninger
>
> frgo(at)goenninger(dot)net
>
> "Don't ask me! I haven't been reading comp.lang.lisp long enough to
> really know ..."
Slobodan Blazeski wrote:
> I'm working on a 4th iteration on monetd-mapi
> http://slobodan.blazeski.googlepages.com/Monetdb-mapi.html
> it's basically a library build with cffi http://common-lisp.net/project/cffi/
> to interface with monetdb http://monetdb.cwi.nl/
>
> The problem is that foreign code interaction has (sometimes) nasty
> idea to block the whole lisp process ,
> sometimes even crushing it, rare but it happens.
> So I was thinking to provide some Erlang style *process-protection*
> and CL-MUPROC came to my mind.
> Anybody has some idea how well could it help for managing lost
> processes, or has some other experiences with it?
> Or any other suggestion for my problem will be appreciated?
>
> thanks
>
> Slobodan
>
Like several posters have already pointed out, cl-muproc relies on the multiprocessing facilities of the implementation,
and so does not offer additional protection from blocking within a FFI call.
-Klaus
[author of cl-muproc]