Writing Web-application and multiprocessing are closely related
subjects. Each new HTTP request gets processed in a separate thread in
AllegroServe (portable). Here comes the difficulties:
1. One foreign call hangs up all threads (no one of them works until
foreign call is done, actually the whole Lisp world is flat-out
hung for a while).
2. Even if a thread doesn't foreign call but computes something w/o
making any I/O all other threads also have to wait.
Visible outcome of the first mentioned point is that if one of the
users of the Web-application does something that causes issuing
foreign call he and all other users of the same application can't do
nothing until it is done.
Is there a way to overcome these difficulties in the CL implementation
with non :os-threads threading model (CMUCL for example)?
Yes, it is possible to use separate OS processes (instead of Lisp
threads) to handle HTTP requests in AllegroServe, in that case
however, there is no easy way for mutual communication among them no
more, no shared variables for example and the size of the each process
is some 30M, which is rather expensive.
One possible solution of the problem might be switching to another
Lisp with native threading like SBCL (for Linux, not for
FreeBSD). However it seems too much luxurious, especially if you
already have a bunch of applications written for another Lisp like
CMUCL.
The overcoming of #2 may be achieved by inserting MP:PROCESS-YIELD,
or (SLEEP 0), or whatever which 1) implies some knowledge of the code
being changed and 2) unfortunately it has nothing to do with the
difficulty #1
I'd appreciate if you shared some of your thoughts on this
subject. What is the proper and reasonable way to get robust web
application with green threads ?
--
Vladimir Zolotykh
Vladimir Zolotykh <······@eurocom.od.ua> writes:
> Yes, it is possible to use separate OS processes (instead of Lisp
> threads) to handle HTTP requests in AllegroServe, in that case
> however, there is no easy way for mutual communication among them no
> more, no shared variables for example and the size of the each process
> is some 30M, which is rather expensive.
This is not entirely true.
You can fork lisp processes, and use shared memory to communicate. You
can even manage this shared memory as a garbage collected heap of lisp
objects.
See COM.INFORMATIMAGO.COMMON-LISP.HEAP
and COM.INFORMATIMAGO.CLISP.RAW-MEMORY
See http://www.informatimago.com/develop/lisp/index.html
for CVS checkout instructions.
(The index.html is not up-to-date, I should have it built
automatically from the lisp sources...)
You could even share memory between different lisp implementations!
--
__Pascal Bourguignon__ http://www.informatimago.com/
Cats meow out of angst
"Thumbs! If only we had thumbs!
We could break so much!"
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: Green threads and Web-application
Date:
Message-ID: <87d5dsyhcz.fsf@qrnik.zagroda>
Vladimir Zolotykh <······@eurocom.od.ua> writes:
> Is there a way to overcome these difficulties in the CL implementation
> with non :os-threads threading model (CMUCL for example)?
There is a nice design of hybrid green and OS threads:
http://www.haskell.org/~simonmar/papers/conc-ffi.pdf
but it requires changes to the scheduler, even though
it's easily adaptable to pure green thread designs.
--
__("< Marcin Kowalczyk
\__/ ······@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Vladimir Zolotykh <······@eurocom.od.ua> writes:
> 1. One foreign call hangs up all threads (no one of them works until
> foreign call is done, actually the whole Lisp world is flat-out
> hung for a while).
> I'd appreciate if you shared some of your thoughts on this
> subject. What is the proper and reasonable way to get robust web
> application with green threads ?
I use lispworks, which has the problems you mention above.
To do this you basically have to bite the bullet, and make
all foreign code non blocking. This can be done in two ways:
1) use non blocking api (e.g. in PostgreSQL's libpq, you can
write code which never blocks, then your lisp can basically
to an mp:process-wait on a function which polls libpq
2) you use a multiplexer in a standalone C thread, and get lisp
to send it requests, then do an mp:process-wait on a polling
function. Meanwhile, your C code is running in another native
C thread.
Yes -- it all _totally_ sucks.
I'm eagerly awaiting Lispworks 5 to see if this will finally be
fixed.