As soon as my DSL gets installed, I'm planning on hosting my Web site
from my own machine, and I've got a number of innovative ideas on
features I could have. I'm planning to write my own server code in
Common Lisp, and I'm wondering what the best starting point would be.
So far, I've only used CLISP, which I've been very pleased with, but
it looks like I'll be using CMUCL, since threads are kind of a
necessity for a Web server. I've seen comments in places about Steel
Bank Lisp or something supposedly being easier to set up and having a
few other features. Is this true, and how compatible is it with CMUCL
really?
Secondly, for the software itself. I have, of course, heard of
CL-HTTP, but haven't had a chance to play with it. I'm honestly
somewhat leery of it, though, since every time in the past I've had to
integrate someone else's code with my own, I end up working around
their code and wishing I didn't have to use it. How much of a server
framework does CL-HTTP provide, and what are the actual benefits of
using it instead of starting from scratch? (HTTP isn't _that_
complicated, after all...)
Any feedback would be greatly appreciated.
--
vsync
http://quadium.net/ - last updated Mon Sep 18 18:09:16 PDT 2000
(cons (cons (car (cons 'c 'r)) (cdr (cons 'a 'o))) ; Orjner
(cons (cons (car (cons 'n 'c)) (cdr (cons nil 's))) nil)))
vsync <·····@quadium.net> writes:
> I'm planning to write my own server code in Common Lisp, and I'm
> wondering what the best starting point would be.
the best starting point is http://allegroserve.sourceforge.net
Of course, this only works on ACL (and Corman Lisp) right now, from
what I hear, but it's simple enough and well-designed, so that it
would be a good place to start. Who knows? It might even have some
of these features you were thinking about (though CL-HTTP has many
more).
In addition, it would be nice to see a port for CMUCL.
dave
From: Chris Double
Subject: Re: best approach for writing web server?
Date:
Message-ID: <wkr96edbf6.fsf@double.co.nz>
David Bakhash <·····@alum.mit.edu> writes:
> vsync <·····@quadium.net> writes:
>
> > I'm planning to write my own server code in Common Lisp, and I'm
> > wondering what the best starting point would be.
>
> the best starting point is http://allegroserve.sourceforge.net
>
> Of course, this only works on ACL (and Corman Lisp) right now, from
> what I hear, but it's simple enough and well-designed, so that it
> would be a good place to start.
I started by writing my own simple HTTP server in Lisp. It works for
simple cases and serves files and dynamic pages. But then I started
wanting to implement a few more of the HTTP/1.1 features and
eventually decided that it would be easier to look at CL-HTTP or
AllegroServe. Why do all that work when someone else has already done
it.
I ported AllegroServe to CormanLisp and it was relatively easy. Most of it
is portable Lisp, and the remaining bits require implementing portions
of the ACL specific routines. I hope to make it available this weekend
on my site. The nice thing about AllegroServe is it is small and easy
to grasp.
I'd like to port CL-HTTP too as that has quite a few higher level
features (Like the web presentation system). I didn't start with that
as it is quite a bit bigger and not as easy to grasp so quickly.
Chris.
--
http://www.double.co.nz/cl
On 20 Sep 2000 02:49:05 -0700, vsync <·····@quadium.net> wrote:
>So far, I've only used CLISP, which I've been very pleased with, but
>it looks like I'll be using CMUCL, since threads are kind of a
>necessity for a Web server.
Have a look at the simple tcl web server, described in Dr. Dobbs
Journal september 1999:
EVENT-BASED SERVERS IN TCL
by Stephen Uhler
Event-based programming is a powerful alternative to threads when
building high-speed network servers. Stephen implements a web server in
Tcl that is based on events and callbacks. Additional resources include
tclserv.txt (listings) and tclserv.zip (source code).
--
------------------------------------------------------------------------------
email: marc dot hoffmann at users dot whh dot wau dot nl
------------------------------------------------------------------------------
······@No-Such-Domain.anywhere (see.signature) writes:
> On 20 Sep 2000 02:49:05 -0700, vsync <·····@quadium.net> wrote:
>
> >So far, I've only used CLISP, which I've been very pleased with, but
> >it looks like I'll be using CMUCL, since threads are kind of a
> >necessity for a Web server.
>
> Have a look at the simple tcl web server, described in Dr. Dobbs
> Journal september 1999:
>
> Event-based programming is a powerful alternative to threads when
> building high-speed network servers. Stephen implements a web server
> in Tcl that is based on events and callbacks. Additional resources
> include tclserv.txt (listings) and tclserv.zip (source code).
I think the truth of the matter is that different OS's favor different
styles of performing tasks that appear to be multithreaded. I've
heard that Linux favors forking over multiple threads, etc. Then, you
think about it, if a Lisp implementation, e.g. ACL or LW on Linux,
performs the scheduling inside Lisp itself, then not only doesn't it
get some of the benefits of using the low-level code in the kernel,
but there's time spent in scheduling itself, and since the OS is
already doing something similar, it's wasteful.
Another problem with Lisp systems is that some of the neat features of
threading and handling file descriptors (select(), fcntl()) is
abstraced and defaulting. You don't get the kind of control you'd get
if you were writing in C, for example, unless you somehow figure out a
way to do so, and this can get nasty if you really need to get down to
that level. Perl lets you at the very guts of C, but from a Perl
interface. But the fact is that if you can avoid it, you should.
I'd have a hard time believing that the Tcl server above isn't
forking-based. There are non-forking servers, but they're complicated
as hell. If they're non-forking, then they use select() and special
non-blocking directives, and as far as I know, no Lisp-based socket
system lets you go there.
Forking is simply not an option for Lisp servers. It's either gotta
be multi-threaded or use non-blocking I/O. This is to take advantage
of the "Lisp World" thing, and sharing data between client handlers.
Only the former has been developed so far in Lisp, as far as I know,
and I'd love to know otherwise. Also, multithreading is vastly
simpler from a programming perspective, though I guess the non-simple
part can be abstracted away.
dave
From: Chris Double
Subject: Re: best approach for writing web server?
Date:
Message-ID: <wkvgvqdblj.fsf@double.co.nz>
David Bakhash <·····@alum.mit.edu> writes:
> Then, you think about it, if a Lisp implementation, e.g. ACL or LW
> on Linux, performs the scheduling inside Lisp itself, then not only
> doesn't it get some of the benefits of using the low-level code in
> the kernel, but there's time spent in scheduling itself, and since
> the OS is already doing something similar, it's wasteful.
Corman Lisp and (I think) ACL on Windows using OS based threads. It
would be interesting to compare something like AllegroServe which uses
threads on an OS threads system vs. an internal scheduling system.
> If they're non-forking, then they use select() and special
> non-blocking directives, and as far as I know, no Lisp-based socket
> system lets you go there.
I've just finished re-writing the socket routines I wrote for Corman
Lisp to use non blocking sockets internally. This is so I can cleanly
cancel out of a socket blocking on a receive or accept - or do some
other processing in Lisp while waiting.
I've been considering ways of making the non-blocking-ness visible to
the user as well - the non block Windows sockets allow a kind of event
driven programming style where you get notifed of the arrival of data,
conncetions, disconnections, etc. I want to come up with a reasonable
Lisp interface to this - any pointers for design inspiration?
The new non-blocking based library will be available after a bit of
testing this weekend probably.
Chris.
--
http://www.double.co.nz/cl
Chris Double <·····@double.co.nz> writes:
> David Bakhash <·····@alum.mit.edu> writes:
>
> > If they're non-forking, then they use select() and special
> > non-blocking directives, and as far as I know, no Lisp-based
> > socket system lets you go there.
>
> I've just finished re-writing the socket routines I wrote for Corman
> Lisp to use non blocking sockets internally. This is so I can
> cleanly cancel out of a socket blocking on a receive or accept - or
> do some other processing in Lisp while waiting.
Yes. It's definitely true that if you're willing to really dive into
the details of a a networking application, you can optimize bits by
using non-blocking. I think threads do that just fine, though with
less efficient things like #'MP:PROCESS-RUN-REASONS and
#'MP:PROCESS-ARREST-REASONS (both SETF-able) and more importantly
#'MP:PROCESS-WAIT. When that process (thread) is waiting on I/O (or
some other event in your system), it can just sleep.
I have only seen people use select() and fcntl() when writing
non-threaded servers (neither forking nor real threads). I am
interested to know how such additions can improve performance without
incurring a totally different style to writing CL-style servers.
> I've been considering ways of making the non-blocking-ness visible
> to the user as well - the non block Windows sockets allow a kind of
> event driven programming style where you get notifed of the arrival
> of data, conncetions, disconnections, etc. I want to come up with a
> reasonable Lisp interface to this - any pointers for design
> inspiration?
If you're really intersted in a nice API for this, then contact me
directly. I can expound on it, but only if you're really interested.
dave
David Bakhash <·····@alum.mit.edu> wrote:
+---------------
| There are non-forking servers, but they're complicated as hell.
| If they're non-forking, then they use select() and special non-blocking
| directives, and as far as I know, no Lisp-based socket system lets
| you go there.
+---------------
Actually, MzScheme (the engine underneath DrScheme) provides select()-based
scheduling in its thread system. You write code within a Scheme thread
that *looks* likes it's blocking, but instead of blocking in the socket
I/O, the internal scheduler adds that fd to the FD_SET of the select()
call and runs some other thread.
Back on July 13th, Shriram Krishnamurthi <·······@cs.rice.edu> posted
to comp.lang.scheme a tiny (couple of pages) web server written in
MzScheme that leaves the top-level listener running while servicing HTTP
requests -- useful for debugging. The server itself was single-threaded,
but moving the "thread" call until after the "tcp-accept" would make it
run a thread per request. See <························@sun.cs.rice.edu>
for the code.
[O.k., it's Scheme, not Lisp, but the point is that you *can* seamlessly
integrate a socket facility with Scheme threads, and probably with (some)
Lisps, too.]
-Rob
-----
Rob Warnock, 41L-955 ····@sgi.com
Network Engineering http://reality.sgi.com/rpw3/
Silicon Graphics, Inc. Phone: 650-933-1673
1600 Amphitheatre Pkwy. PP-ASEL-IA
Mountain View, CA 94043
In article <··············@piro.quadium.net>, vsync <·····@quadium.net>
wrote:
> As soon as my DSL gets installed, I'm planning on hosting my Web site
> from my own machine, and I've got a number of innovative ideas on
> features I could have. I'm planning to write my own server code in
> Common Lisp, and I'm wondering what the best starting point would be.
Reading the HTTP standards, maybe?
> it looks like I'll be using CMUCL, since threads are kind of a
> necessity for a Web server.
Threads in CMUCL I have seen only for x86 and I guess they
need a bit more work - but it is a starting point.
> Secondly, for the software itself. I have, of course, heard of
> CL-HTTP, but haven't had a chance to play with it. I'm honestly
> somewhat leery of it, though, since every time in the past I've had to
> integrate someone else's code with my own, I end up working around
> their code and wishing I didn't have to use it. How much of a server
> framework does CL-HTTP provide, and what are the actual benefits of
> using it instead of starting from scratch? (HTTP isn't _that_
> complicated, after all...)
Well, HTTP 1.1 isn't that easy. CL-HTTP is ported to a wide
variety of systems and contains the knowledge how to
interface to processes/streams/locks/TCP/... on those
platforms in a more or less efficient way. CL-HTTP is
kind of a larger framework for web applications. It
does support HTTP 1.0 / HTTP 1.1, client and server and proxy,
and quite a few libraries on top of that. Various
people have contributed over time, so not everything
is ideally packaged - but so is real life software.
It uses a lot of CL and beyond, so be prepared
for some steep learning curve. Some Vendors had to put
in quite some fixes into their Lisp, to improve the
implementation of their Lisp system.
CL-HTTP for example runs www.lisp.org (static pages,
on ACL), www.publications.whitehouse.gov and my server
at home. ;-) I even use it as my web proxy at home
(hey, a Lisp machine as a web proxy ;-) ). Currently
we are looking for some people who might be interested
to develop applications on top of the proxy capability.
A lot of time is going into making the proxy work
(using HTTP 1.1 connections on both sides for example).
So this is screenshot of how CL-HTTP looks like on the
Mac using MCL (excuse the crowded screen):
http://corporate-world.lisp.de/mcl/cl-http-on-mcl.jpg
Rainer Joswig
--
Rainer Joswig, Hamburg, Germany
Email: ·············@corporate-world.lisp.de
Web: http://corporate-world.lisp.de/
"vsync" <·····@quadium.net> wrote in message
···················@piro.quadium.net...
> So far, I've only used CLISP, which I've been very pleased with, but
> it looks like I'll be using CMUCL, since threads are kind of a
> necessity for a Web server.
Let's start right there. Threads are not a necessity if you only have a
couple of hits an hour. I'd start with XP's dictum of "doing the simplest
thing that works". At that point, you can accurately assess whether threads
are necessary. If you do need them, and you're running a Windows box, try
Corman Lisp (www.corman.net).
faa
On 20 Sep 2000 02:49:05 -0700, vsync <·····@quadium.net> wrote:
> features I could have. I'm planning to write my own server code in
> Common Lisp, and I'm wondering what the best starting point would be.
You may want to check:
http://ww.telent.net/cliki/Web/
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
Paolo Amoroso <·······@mclink.it> writes:
> You may want to check:
>
> http://ww.telent.net/cliki/Web/
Have you tried this URL recently? It doesn't work for me; the "ww"
entry seems to have disappeared from the domain...
Colin Walters <·······@cis.ohio-state.edu> writes:
> Paolo Amoroso <·······@mclink.it> writes:
>
> > You may want to check:
> >
> > http://ww.telent.net/cliki/Web/
>
> Have you tried this URL recently? It doesn't work for me; the "ww"
> entry seems to have disappeared from the domain...
There was downtime on telent.net announced recently, IIRC because of a
move of ISPs or somesuch.
Christophe
--
Jesus College, Cambridge, CB5 8BL +44 1223 524 842
(FORMAT T "(·@{~w ········@{~w~^ ~})" 'FORMAT T "(·@{~w ········@{~w~^ ~})")
On 21 Sep 2000 00:10:24 +0100, Christophe Rhodes <·····@cam.ac.uk> wrote:
> There was downtime on telent.net announced recently, IIRC because of a
> move of ISPs or somesuch.
Yes. CLiki will be down until further notice. I posted the URL because I
thought there might have been some more opportunities to visit the site
before the announced downtime.
Let's hope Daniel Barlow finds a new home for CLiki soon. In the meantime,
Dan: thanks for your excellent work on CLiki.
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/