From: David Steuber
Subject: Your favorite Lisp on Debian?
Date: 
Message-ID: <m2y8pwqdql.fsf@david-steuber.com>
On my Debian box, I've installed the packages cmucl, sbcl, and
sbcl-mt.  This isn't an exhaustive list as there are suppport
packeges also installed.

What I would like to know, if there can be a reasonable answer, is
which Lisp would be preferable for a web application that has
propertys like I described in "Ye Old Time Sharing System".

I'm kind of curious as to why sbcl and sbcl-mt exist as seperate
packages.  That seems odd.  Why would someone not want threading
available?

To recap on features, I'm talking about an application that has both
an HTTP view via Apache + mod_lisp and an X11 or curses view via
either SSH or RSH with X11 Forwarding/tunneling.

I'm running Debian/testing.

-- 
Those who do not remember the history of Lisp are doomed to repeat it,
badly.

> (dwim x)
NIL

From: Edi Weitz
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <m3d6779a7p.fsf@bird.agharta.de>
On Sat, 20 Mar 2004 05:38:57 GMT, David Steuber <·············@verizon.net> wrote:

> I'm kind of curious as to why sbcl and sbcl-mt exist as seperate
> packages.  That seems odd.  Why would someone not want threading
> available?

Probably because a) MT is rather new and still in development, and b)
it needs special hardware (x86) and a recent kernel. Debian is
supposed to run on non-Intel machines also.

Edi.
From: Chris Hall
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <87fzc3s88y.fsf@naia.homelinux.net>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm a LIsp n00b (done most of "Gentle Intro" and I'm on Chapter 7 of
Paul Graham's "ANSI Common Lisp"), but I also run on Debian and have
CMUCL and SBCL installed, and am fully intending on running web apps
via Lisp as well, so I am *very* interested in this. :-D

I've just started trying to get CL-HTTP running _because_ of the
threading, so that database queries don't appear to make the web
server unresponsive to users.

I feel very strongly about free (as in 'libre') software, so I am a
little concerned about CL-HTTP's licensing, but I couldn't find
another Lisp-based web server that claimed to do threads.  Plus
Araneida seems a little too immature yet for this sort of thing?  What
are you using/planning to use?  mod_lisp on Apache?  Wouldn't that
start a new Lisp for each request?  Would you use the CMUCL 'small'
core for that?

I've been using Python for my web app purposes - there is an
asynch-type framework (very complete and rather well designed by all
appearances) called Twisted that I'm quite taken with.  Twisted uses a
'select'-type polling method for web requests, and manages a separate
thread pool for database connection - one uses one of the many Python
database connectivity packages in conjunction w/the framework, as long
as they conform to the Python DBI(?) standard.  (I'm a big fan of
PostgreSQL - one can write stored procedures in Python! - I've even
come to prefer it to Oracle, though for certain clients Oracle or DB2
is definitely better.)

I've been seriously considering porting the asynch-methods, perhaps
even the db connection pooling to Lisp, but I still need good web
server - any suggestions?

And as to threading, from what I understand in the Linux workd
threading is still something of moving target - native vs. POSIX
(pthreads?), plus they are working on MOSIX for parallel processing as
well.  So I guess the mixed state of Lisp support for threads on Linux
isn't too surprising, though in that respect Linux seems to actually
lead other platforms?

+Chris

- -- 

Democracy: The worship of jackals by jackasses.
- -- H.L. Mencken
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.6 <http://mailcrypt.sourceforge.net/>

iD8DBQFAXIfErZy455Pig6QRArR8AJ9VcTz8pa4bP9Sxs+VArU6KZaxkDgCgkJn9
08i+IFkWYZ8/guaySIieerA=
=4LJS
-----END PGP SIGNATURE-----
From: Edi Weitz
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <m33c838dkk.fsf@bird.agharta.de>
On Sat, 20 Mar 2004 18:06:54 GMT, Chris Hall <·······@verizon.net> wrote:

> mod_lisp on Apache?  Wouldn't that start a new Lisp for each
> request?

No. You start CMUCL once and on each request Apache spawns a tiny C
process which talks to your single CMUCL image. (Which, BTW, offers a
wealth of new opportunities compared to, say, mod_perl and PHP.) You
can even restart Apache without affecting the Lisp running behind
it. See <http://lisp.t2100cdt.kippona.net/lispy/home>.

Whether you have multiple threads within your Lisp or not depends
solely on the CL implementation. On Linux you'll have cooperative
("green") threads managed by the Lisp system with LispWorks,
AllegroCL, or CMUCL. With SBCL or Scieneer you can have OS-level
threads. With CLISP you don't get threads at all.[1]

> I still need good web server - any suggestions?

I'm quite happy with mod_lisp - I've used it successfully for a couple
of projects.

Edi.

[1] With CLISP a better option than mod_lisp might be FastCGI:

      <http://clisp.cons.org/impnotes/modules.html#fastcgi>
From: Rob Warnock
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <I_qdndkr04PgL8Hd3czS-g@speakeasy.net>
Edi Weitz  <···@agharta.de> wrote:
+---------------
| Chris Hall <·······@verizon.net> wrote:
| > mod_lisp on Apache?  Wouldn't that start a new Lisp for each request?
| 
| No. You start CMUCL once and on each request Apache spawns a tiny C
| process which talks to your single CMUCL image.
+---------------

Actually, it doesn't even "spawn a tiny C process"!! All it does is that
one of the already pre-spwaned Apache processes opens a socket connection[1]
to the already-running Lisp process and pushes a request down the socket,
and then the Lisp process pushes a response back down the socket which
the Apache process throws back over its shoulder to the user's browser.

Normally the Apache "MaxRequestsPerChild" setting is set to some fairly
high number (possibly "infinity"), so that the when the Apache process
is done with that "mod_lisp" request it *doesn't* die, but gets re-used
for later requests.  So all a "mod_lisp" request really costs is a single
socket open/close.[2]


-Rob

[1] Unless one is already open, see [2].

[2] And maybe not even thats! The socket might even stay open and be used
    for several more requests, depending on the way you've configured
    "mod_lisp" and your Lisp server, and the specific type of request
    it was (that is, whether the response included the HTTP/1.1 response
    header "Connection: Open"). See the "mod_lisp" distribution for details,
    and RFC 2616 "HTTP/1.1", section "8.1 Persistent Connections".

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Chris Hall
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <87brmejidf.fsf@naia.homelinux.net>
····@rpw3.org (Rob Warnock) writes:

> Edi Weitz  <···@agharta.de> wrote:
> +---------------
> | Chris Hall <·······@verizon.net> wrote:
> | > mod_lisp on Apache?  Wouldn't that start a new Lisp for each request?
> | 
> | No. You start CMUCL once and on each request Apache spawns a tiny C
> | process which talks to your single CMUCL image.
> +---------------
> 
> Actually, it doesn't even "spawn a tiny C process"!! All it does is that
> one of the already pre-spwaned Apache processes opens a socket connection[1]
> to the already-running Lisp process and pushes a request down the socket,
> and then the Lisp process pushes a response back down the socket which
> the Apache process throws back over its shoulder to the user's browser.
> 
> Normally the Apache "MaxRequestsPerChild" setting is set to some fairly
> high number (possibly "infinity"), so that the when the Apache process
> is done with that "mod_lisp" request it *doesn't* die, but gets re-used
> for later requests.  So all a "mod_lisp" request really costs is a single
> socket open/close.[2]
> 
> 
> -Rob
> 
> [1] Unless one is already open, see [2].
> 
> [2] And maybe not even thats! The socket might even stay open and be used
>     for several more requests, depending on the way you've configured
>     "mod_lisp" and your Lisp server, and the specific type of request
>     it was (that is, whether the response included the HTTP/1.1 response
>     header "Connection: Open"). See the "mod_lisp" distribution for details,
>     and RFC 2616 "HTTP/1.1", section "8.1 Persistent Connections".
> 
> -----
> Rob Warnock			<····@rpw3.org>
> 627 26th Avenue			<URL:http://rpw3.org/>
> San Mateo, CA 94403		(650)572-2607
> 

Thanks all, this was exactly the information I was looking for.

(Sorry for the delay in my response - I've been busy getting CL-HTTP
and pAServe running, learning about defsystem, asdf, etc., etc. and
have been neglecting my newsgroups :^/ )

I'm still weighing the various options in respect to actual
application I have in mind.

P.S. CL-HTTP on CMUCL has pathname problems in the defsytem - I had to
hardcode in my local paths in a couple of places, but after that
seemed to build and run the demos just fine.

-- 
If we had less statemanship we could get along with fewer battleships.
-- Mark Twain (1835 - 1910)
From: Alex Mizrahi
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <c3jj08$28jngp$1@ID-177567.news.uni-berlin.de>
(message (Hello 'Chris)
(you :wrote  :on '(Sat, 20 Mar 2004 18:06:54 GMT))
(

 CH> What are you using/planning to use?  mod_lisp on Apache?  Wouldn't
 CH> that start a new Lisp for each request?  Would you use the CMUCL
 CH> 'small'
 CH> core for that?

no, it will not.. it can use multithreading..
you will not be happy with CMUCL multithreading - it works only when you
specially call a function(as i remember yield)..
however there is a hack - it can call automatically this function each 0.5
secs, for example, basing on some signal.
this solution is not too stable(for example, you can have a lot of troubles
with debuger and exceptions, as i remember). but it worked for us..
i think there is no much need in more precious timing than 0.5 secs for web
purposes. it could even kick a thread that did not finish in 30 seconds..
as for stability, we've tested it doing requests from 4 machines on LAN as
fast as possible.. we were quite satisfied with results. some pages were
lost - one of 1000, or 1 of 10000, i don't remember exactly - and i have no
clue wheter it's a defect of CMUCL, mod_list, Apache or Perl client, but
this is quite insignificant percentage, i think..

)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
(prin1 "Jane dates only Lisp programmers"))
From: Chris Hall
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <877jx2jhgk.fsf@naia.homelinux.net>
"Alex Mizrahi" <·········@xhotmail.com> writes:

> (message (Hello 'Chris)
> (you :wrote  :on '(Sat, 20 Mar 2004 18:06:54 GMT))
> (
> 
>  CH> What are you using/planning to use?  mod_lisp on Apache?  Wouldn't
>  CH> that start a new Lisp for each request?  Would you use the CMUCL
>  CH> 'small'
>  CH> core for that?
> 
> no, it will not.. it can use multithreading..
> you will not be happy with CMUCL multithreading - it works only when you
> specially call a function(as i remember yield)..
> however there is a hack - it can call automatically this function each 0.5
> secs, for example, basing on some signal.
> this solution is not too stable(for example, you can have a lot of troubles
> with debuger and exceptions, as i remember). but it worked for us..
> i think there is no much need in more precious timing than 0.5 secs for web
> purposes. it could even kick a thread that did not finish in 30 seconds..
> as for stability, we've tested it doing requests from 4 machines on LAN as
> fast as possible.. we were quite satisfied with results. some pages were
> lost - one of 1000, or 1 of 10000, i don't remember exactly - and i have no
> clue wheter it's a defect of CMUCL, mod_list, Apache or Perl client, but
> this is quite insignificant percentage, i think..
> 
> )
> (With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
> (prin1 "Jane dates only Lisp programmers"))
> 
> 

Thanks - I'm familiar with some aspects of 'yield' issues from using
Twisted, a Python 'select'-based asynchronous communication server
framework.

Twisted uses something they call 'Deferred's to aid in this, and for
things like database queries the framework manages a pool of OS
threads so that the comm server(s) don't block.

So, one creates a 'Deferred' query object and returns it to the
framework, which manages the actual query.  Thus when, for a web
server, an HTTP request comes in, the framework starts building your
HTML page, sees the 'Deferred', drops it the database thread pool, and
goes to look for and service other HTTP requests until the database is
done, at which point it resumes building your page for you and sends
it off the client.

One can also chain 'Deferred's, and they have 'Errback's in case
something went wrong - these can sometimes be used to recover as well.

It's a bit complicated to explain (and I am sure in implementation),
but once one 'groks' it, simple.  It seems to work quite well and
manages some very respectable throughput.

I've also been tinkering with CMUCL's co-op multi-threads in the
context of Apps/Listener in McCLIM distribution - on my slow box the
Listener is pretty much unusable in threaded mode, and when I run it
straight from the REPL my CPU stays at 100%, though Listener is quite
responsive and usable.

-- 
If we had less statemanship we could get along with fewer battleships.
-- Mark Twain (1835 - 1910)
From: Paolo Amoroso
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <878yhuv6is.fsf@plato.moon.paoloamoroso.it>
Chris Hall <·······@verizon.net> writes:

> little concerned about CL-HTTP's licensing, but I couldn't find
> another Lisp-based web server that claimed to do threads.  Plus
> Araneida seems a little too immature yet for this sort of thing?  What

These sites, plus a commercial one that sells concert tickets, are
based on Araneida:

  http://www.cliki.net
  http://alu.cliki.net
  http://mcclim.cliki.net


Paolo
-- 
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
From: Daniel Barlow
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <878yhu83o4.fsf@noetbook.telent.net>
Paolo Amoroso <·······@mclink.it> writes:

> Chris Hall <·······@verizon.net> writes:
>
>> little concerned about CL-HTTP's licensing, but I couldn't find
>> another Lisp-based web server that claimed to do threads.  Plus
>> Araneida seems a little too immature yet for this sort of thing?  What
>
> These sites, plus a commercial one that sells concert tickets, are
> based on Araneida:

[snipped]

I'm pretty confident in the stability of Araneida in its serve-event
configuration (it hooks into the SBCL event loop to provide the
illusion - and most of the benefits - of multithreading), but I'd
agree the threaded version has not really been beaten on in the same
way.

If your application is written in such a way that it could be deployed
happily on an event-based sever - which is mostly a question of (1)
making sure each page is computed and output speedily on the server
side, (2) using Apache mod_proxy to buffer for slow clients, it's
actually possible to switch between the threaded and unthreaded
versions just by changing a couple of class names.  Obviously it's not
so great if you have requests that take a long time to come back,
though.


-dan

-- 
"please make sure that the person is your friend before you confirm"
From: Chris Hall
Subject: Re: Your favorite Lisp on Debian?
Date: 
Message-ID: <873c7qjh06.fsf@naia.homelinux.net>
Daniel Barlow <···@telent.net> writes:

> Paolo Amoroso <·······@mclink.it> writes:
> 
> > Chris Hall <·······@verizon.net> writes:
> >
> >> little concerned about CL-HTTP's licensing, but I couldn't find
> >> another Lisp-based web server that claimed to do threads.  Plus
> >> Araneida seems a little too immature yet for this sort of thing?  What
> >
> > These sites, plus a commercial one that sells concert tickets, are
> > based on Araneida:
> 
> [snipped]
> 
> I'm pretty confident in the stability of Araneida in its serve-event
> configuration (it hooks into the SBCL event loop to provide the
> illusion - and most of the benefits - of multithreading), but I'd
> agree the threaded version has not really been beaten on in the same
> way.
> 
> If your application is written in such a way that it could be deployed
> happily on an event-based sever - which is mostly a question of (1)
> making sure each page is computed and output speedily on the server
> side, (2) using Apache mod_proxy to buffer for slow clients, it's
> actually possible to switch between the threaded and unthreaded
> versions just by changing a couple of class names.  Obviously it's not
> so great if you have requests that take a long time to come back,
> though.
> 
> 
> -dan
> 
> -- 
> "please make sure that the person is your friend before you confirm"

Thanks, and yes, I am comfortable with event-based environments - I've
used the Python-based Twisted comm server framework (see earlier post).

In fact, the more I learned about asynchronous, select-based server
type stuff, the more taken I was with it - I've come to prefer it
wherever possible, not least because actual multi-threading is so
issue-laden - implementaion differences, OS differences, the additional
overhead required when it is available, etc.

Maybe I should give Araneida a look since I am in evaluation mode
anyway, and I already have SBCL installed.

Araneida requires SBCL?

-- 
If we had less statemanship we could get along with fewer battleships.
-- Mark Twain (1835 - 1910)