mod_lisp 2.3 is available for beta testing. Contact me if you want to test
it.
The major improvement is to force Apache to read all the Lisp reply before
eventually closing the socket. This should avoid writing to the Apache
process when it has closed because the client browser has closed the
connection.
Thanks to Edi Weitz for finding this one.
Marc
Marc Battyani <·············@fractalconcept.com> wrote:
+---------------
| mod_lisp 2.3 ... The major improvement is to force Apache to
| read all the Lisp reply before eventually closing the socket.
| This should avoid writing to the Apache process when it has
| closed because the client browser has closed the connection.
+---------------
Just curious...
1. Does this in any way affect the ability (very important in
some applications!) to "stream" data through Apache to the
client browser, so that the beginning of the output gets to
the browser early to give the user something to look at while
the rest is being computed (e.g., with a slow database access)?
I don't think it *has* to -- you could be simply "draining
the pipe" if the client aborts -- but I wanted to check.
2. What are the implications for a "runaway" Lisp process that
generates infinite output? Normally when that happens and
the user (say) manually hits "Stop" (aborting the connection),
the abort would propagate back to the Lisp process as a
SIGPIPE (which could be caught & handled), would it not?
Have you perhaps traded one evil for another?
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
> Marc Battyani <·············@fractalconcept.com> wrote:
> +---------------
> | mod_lisp 2.3 ... The major improvement is to force Apache to
> | read all the Lisp reply before eventually closing the socket.
> | This should avoid writing to the Apache process when it has
> | closed because the client browser has closed the connection.
> +---------------
>
> Just curious...
>
> 1. Does this in any way affect the ability (very important in
> some applications!) to "stream" data through Apache to the
> client browser, so that the beginning of the output gets to
> the browser early to give the user something to look at while
> the rest is being computed (e.g., with a slow database access)?
>
> I don't think it *has* to -- you could be simply "draining
> the pipe" if the client aborts -- but I wanted to check.
>
> 2. What are the implications for a "runaway" Lisp process that
> generates infinite output? Normally when that happens and
> the user (say) manually hits "Stop" (aborting the connection),
> the abort would propagate back to the Lisp process as a
> SIGPIPE (which could be caught & handled), would it not?
> Have you perhaps traded one evil for another?
I think both of these concerns don't apply to the way mod_lisp works.
1. mod_lisp communicates with the Lisp image through a socket
connection. This is _not_ the connection between Apache and the
browser (and thus an aborted http connection won't propagate
through to the Lisp image as a SIGPIPE). The paragraph above from
Marc talks about the mod_lisp/Apache socket.
2. mod_lisp requires the Lisp image to provide a correct
"Content-length" header _before_ sending the body, i.e. in almost
all cases you won't be able to send data to the client before you
have finished computing your dynamic page. (This should also answer
the question about "runaway" processes with infinite output. They
won't affect Apache because the usual way to use mod_lisp is to
write your output into a string, then compute its length, then hand
your data to mod_lisp.)
HTH,
Edi.
"Edi Weitz" <···@agharta.de> wrote
> ····@rpw3.org (Rob Warnock) writes:
>
> > Marc Battyani <·············@fractalconcept.com> wrote:
> > +---------------
> > | mod_lisp 2.3 ... The major improvement is to force Apache to
> > | read all the Lisp reply before eventually closing the socket.
> > | This should avoid writing to the Apache process when it has
> > | closed because the client browser has closed the connection.
> > +---------------
> >
> > Just curious...
> >
> > 1. Does this in any way affect the ability (very important in
> > some applications!) to "stream" data through Apache to the
> > client browser, so that the beginning of the output gets to
> > the browser early to give the user something to look at while
> > the rest is being computed (e.g., with a slow database access)?
> >
> > I don't think it *has* to -- you could be simply "draining
> > the pipe" if the client aborts -- but I wanted to check.
> >
> > 2. What are the implications for a "runaway" Lisp process that
> > generates infinite output? Normally when that happens and
> > the user (say) manually hits "Stop" (aborting the connection),
> > the abort would propagate back to the Lisp process as a
> > SIGPIPE (which could be caught & handled), would it not?
Yes, it is what is happening now and what we want to avoid.
In fact there are 2 ways of using mod_lisp.
The most performant one is to keep the Apache to Lisp socket open between
request. For this the Lisp process has to send the content length to Apache
before sending the data. This generally implies that the reply has already
been computed.
In the other mode, the one you refer too, the Lisp process does not knows
the content length and it will output the reply content while computing it.
The end of the data is when Lisp closes the socket. In that case we still
want to get the SIGPIPE error.
> > Have you perhaps traded one evil for another?
No both modes can coexist on a per reply basis.
Except that now they don't anymore as the unknown reply length case has been
dropped.
So I put it back for people who use slow databases... ;)
Version 2.31 is now available for testing...
> I think both of these concerns don't apply to the way mod_lisp works.
In fact they apply but only in the case where you don't reuse the
Apache/Lisp connection.
> 1. mod_lisp communicates with the Lisp image through a socket
> connection. This is _not_ the connection between Apache and the
> browser (and thus an aborted http connection won't propagate
> through to the Lisp image as a SIGPIPE). The paragraph above from
> Marc talks about the mod_lisp/Apache socket.
>
> 2. mod_lisp requires the Lisp image to provide a correct
> "Content-length" header _before_ sending the body, i.e. in almost
> all cases you won't be able to send data to the client before you
> have finished computing your dynamic page. (This should also answer
> the question about "runaway" processes with infinite output. They
> won't affect Apache because the usual way to use mod_lisp is to
> write your output into a string, then compute its length, then hand
> your data to mod_lisp.)
Indeed, this is the prefered way.
Marc
Marc Battyani <·············@fractalconcept.com> wrote:
+---------------
| "Edi Weitz" <···@agharta.de> wrote
| > ····@rpw3.org (Rob Warnock) writes:
| > > 1. Does this in any way affect the ability (very important in
| > > some applications!) to "stream" data through Apache ...
...
| In fact there are 2 ways of using mod_lisp.
...
| In the other mode, the one you refer too, the Lisp process does not knows
| the content length and it will output the reply content while computing it.
| The end of the data is when Lisp closes the socket. In that case we still
| want to get the SIGPIPE error.
+---------------
Ahhh! Thanks, a very helpful clarification of Edi's answer.
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
> Marc Battyani <·············@fractalconcept.com> wrote:
> +---------------
> | "Edi Weitz" <···@agharta.de> wrote
> | > ····@rpw3.org (Rob Warnock) writes:
> | > > 1. Does this in any way affect the ability (very important in
> | > > some applications!) to "stream" data through Apache ...
> ...
> | In fact there are 2 ways of using mod_lisp.
> ...
> | In the other mode, the one you refer too, the Lisp process does not knows
> | the content length and it will output the reply content while computing it.
> | The end of the data is when Lisp closes the socket. In that case we still
> | want to get the SIGPIPE error.
> +---------------
>
> Ahhh! Thanks, a very helpful clarification of Edi's answer.
Let me add to my defense that I didn't know this other mode... :)
It wasn't in 2.2 which I'm currently using but I think it was in older
versions. Marc just re-introduced it in the 2.3 beta release.
Cheers,
Edi.
Rearranging the quotes, since I think you actually replied to
my #1 question with your #2 answer, and vice-versa...
Edi Weitz <···@agharta.de> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > 1. Does this in any way affect the ability (very important in
| > some applications!) to "stream" data through Apache to the
| > client browser, so that the beginning of the output gets to
| > the browser early to give the user something to look at while
| > the rest is being computed (e.g., with a slow database access)?
|
| 2. mod_lisp requires the Lisp image to provide a correct
| "Content-length" header _before_ sending the body, i.e. in almost
| all cases you won't be able to send data to the client before you
| have finished computing your dynamic page.
+---------------
Gee, that's too bad. The ability to "stream" data (HTML, at least)
to the client without knowing in advance how long it is it quite
useful in many situations.
Does "mod_lisp" at least let you use HTTP/1.1 "chunked" transfer
encoding to achieve such incremental output? And can you do that
even if it's an HTTP/1.0 client?
+---------------
| > 2. What are the implications for a "runaway" Lisp process that
| > generates infinite output? Normally when that happens and
| > the user (say) manually hits "Stop" (aborting the connection),
| > the abort would propagate back to the Lisp process as a
| > SIGPIPE (which could be caught & handled), would it not?
| I think both of these concerns don't apply to the way mod_lisp works.
| 1. mod_lisp communicates with the Lisp image through a socket
| connection. This is _not_ the connection between Apache and the
| browser (and thus an aborted http connection won't propagate
| through to the Lisp image as a SIGPIPE).
+---------------
Well, that's up to "mod_lisp" to decide, which is why I asked.
Certainly one could design an API that *did* pass the abort
through -- it's one of the available design decisions. And if
you support "chunked" transfer encoding (see other question),
then there are some cases where you'd *want* to pass it through,
to avoid wasting lots of expensive resources computing massive
amounts of data that "mod_lisp" would just throw on the floor
(after a client abort). [E.g., a suitably-malicious client could
launch a DoS attack against a "mod_lisp"-based server by making
*large* numbers of "expensive" requests and then immediately
aborting them.]
Anyway, thanks for your answers (even if they weren't entirely
what I had hoped to hear)...
-Rob
-----
Rob Warnock, PP-ASEL-IA <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
····@rpw3.org (Rob Warnock) writes:
> 2. What are the implications for a "runaway" Lisp process that
> generates infinite output? Normally when that happens and
> the user (say) manually hits "Stop" (aborting the connection),
> the abort would propagate back to the Lisp process as a
> SIGPIPE (which could be caught & handled), would it not?
> Have you perhaps traded one evil for another?
This is something of a tangent, but I've never found a use for SIGPIPE
except in the simplest of programs which only have one interesting
file handle - reason being that it's asynchronous, so unless you have
SA_SIGINFO signal support to tell you which fd is dead, you don't
know. And SA_SIGINFO (a) still doesn't exist on all systems, (b) can
be badly broken when it does. (e.g. threaded programs on any Linux
system that still uses LinuxThreads - which is most of them). It's
easier overall to check the return value of write() - even easier when
your Lisp implementation does that for you, too :-)
Your general point I agree with, though...
-dan
--
http://www.cliki.net/ - Link farm for free CL-on-Unix resources
"Tibor Simko" <···········@cern.ch> wrote
> On Mon, 6 Jan 2003, Marc Battyani wrote:
> > mod_lisp 2.3 is available for beta testing.
>
> Does it work already with Apache 2?
No not yet. BTW is Apache 2 really used yet ?
I use Debian stable (woody) and it does not have Apache 2 in its packages.
It's only available in the testing distribution.
Marc
On Wed, 8 Jan 2003, Marc Battyani wrote:
> BTW is Apache 2 really used yet ?
Dunno. Personally we use it here mainly because apache2+mod_python3.0
works much better than apache1+mod_python2.7 for our problems. (BTW
apache2 source compiles and runs well on Woody.)
T.