So programming.reddit.com has had quite a few links to Seaside
articles the last couple days. Seaside is the predominant web app
programming framework for Smalltalk and it's claim to fame is
continuations. I know that Common Lisp has a continuation based
Application Server, namely UnCommon Web. I've noticed that the
majority of web app talk with regards to CL is all based around
Hunchentoot/Araneida/AllegroServe/CL-HTTP. Is there a reason for
this? Is UnCommon Web a secret weapon? Or only used by Marco? ;)
-dustin
my take : (this is in no way to criticize ucw, just something that i
observed)
1. steep learning curve
so you thought you finally get a handle of CL, and now you're
presented with a whole new set of things to learn call/cc / cps
transformation, etc
2. small community
you might not be able to get help quickly.
but the major problem is that since there are not many people using
it, there's no pressure to make it run across all major
implementations (acl / lispworks / cmucl / sbcl / openmcl / mcl / ecl)
x os combinations (linux / windows / macosx). this is often the most
frustrating thing that can put users off (it's a common problem across
most of the cl libraries, not just ucw)
3. documentation is thin
for most laymen (that's me) they can follow the example codes, tweak
it a bit, but usually they can't get far.
4. leaky abstraction (http://www.joelonsoftware.com/articles/
LeakyAbstractions.html)
well, at least with scheme, you know what a continuation is (but it
already introduce a lot of problems that you don't have otherwise).
with ucw, you need to understand how its kall/cc different from
scheme's call/cc and when and where you can't use it.
the worse thing with leaky abstraction is that, if the app works fine,
it's ok. but if it drops to the debugger, you better understand ucw
_really_ well.
that said, it didn't stop very smart people like drewc (who creates
LOL) and folks from para-agent (who made a killer app w/ ucw) to build
something really complex with it.
i guess ucw is like cl, if it falls on the hands of an expert, it can
be really powerful.
my 0.02
Dustin Withers <·········@gmail.com> writes:
> Is there a reason for this? Is UnCommon Web a secret weapon? Or only
> used by Marco? ;)
I think one of the reasons may be that UnCommon Web is the only Lisp
framework that people know of :)
[SHAMELESS PLUG]
A little over one month ago I left my job. I have spent a huge chunk
of my time writing a web framework in Common Lisp. In retrospect, not
looking at UnCommon Web was a NIH syndrome but I've accomplished so
much in this time that the thought of throwing it away gives me
chills. I hope to release the framework in the coming months. You can
see some screenshots here:
http://www.defmacro.org/temp/shot1.png
http://www.defmacro.org/temp/shot2.png
This is my first major Common Lisp project and I can say without any
reservations - the benefits I've reaped blow every other language
known to me out of the water (and I know quite a few). I have a few
widgets I have to finish, at which point I'll arrive at one of the
crucial milestones - designing modal interaction.
I've left a few doors open to take the framework into one of many
possible directions in this respect. As I think about this more and
more I keep coming to the conclusion that it's best to limit the
framework to support modal development without continuations.
[/SHAMELESS PLUG]
Here is what I want to support:
(defun authenticatedp ()
(if *authenticatedp*
t
(do-modal #'login-or-register)))
(defun some-action ()
(when (authenticatedp)
(do-action)))
The key is 'do-modal' - a construct that shows a login page (which can
manage rounds of user interaction), and returns the result of the
login operation as if it were a simple function call.
I don't need continuations to achieve this functionality. The trick,
saving the stack, can be done with threads. Whenever 'do-modal' is
called, it can spawn a thread and assign it to the current
session. All interaction will then be executed by that thread while
'do-modal' blocks the original thread and waits for some signal from
the new one.
Naturally it's necessary to map a session to the current thread and
take care of the back button. This seems rather simple - assign a
number to each request and whenever 'do-modal' is called, store the
number in some data structure. If the user hits the back button enough
times that he leaves the "modal" part of the page, the framework can
detect that and simply signal the waiting threads to resume. I don't
need support for restartable computations to support the back
button. For instance, if the user hits pages A, B, C, D, then hits the
back button twice and executes some action from B, pages C and D are
gone forever (which will be simulated by unwinding of the stack via
unblocking threads).
The only time continuations *may* be useful is when the user clones
the session in another browser, but that's a very moot point. First,
cloning isn't generally an expectation since you can only easily clone
a session when cookies aren't supported and the session ID is
rewritten on the URI. Second, even if you do take that path,
continuations that maintain UI state will gradually grow out of sync
with the backend data you modify through PUT requests. The user will
end up seeing a UI for a non-existant back-end state - IMO a very
undesirable situation.
As far as I can tell the common "integer counter" example is very
contrived. You have a state zero, you proceed to click "increment"
four times, then hit the back button twice, hit "increment", and
expect to see a three, not a five. Assuming this is desireable (you
wouldn't want stuff disappearing from the shopping cart), you've now
introduced a problem similar to cloning - the data in the database
you've commited with PUT requests is out of sync with your widget
state. I'm not sure how one would handle such issues in practice, but
if you really want to you don't need continuations to do this either -
simply store user actions in idempotent closures (set to value,
instead of toggle).
Disclaimer: my ideas about modal web application programming are
untested. I will soon implement everything discussed above and report
back my findings.
--
Regards,
Slava Akhmechet.
Slava Akhmechet <·········@gmail.com> writes:
> Here is what I want to support:
>
> (defun authenticatedp ()
> (if *authenticatedp*
> t
> (do-modal #'login-or-register)))
>
> (defun some-action ()
> (when (authenticatedp)
> (do-action)))
>
> The key is 'do-modal' - a construct that shows a login page (which can
> manage rounds of user interaction), and returns the result of the
> login operation as if it were a simple function call.
>
> I don't need continuations to achieve this functionality. The trick,
> saving the stack, can be done with threads. Whenever 'do-modal' is
> called, it can spawn a thread and assign it to the current
> session. All interaction will then be executed by that thread while
> 'do-modal' blocks the original thread and waits for some signal from
> the new one.
It can work.
The problem you have is that there is a limited number of threads
possible in a process (around 300 or 400). Therefore you can handle
only a limited number of concurent modal sessions (around 150 or 200
if you call recursively do-modal only once per session).
--
__Pascal Bourguignon__ http://www.informatimago.com/
NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.
On May 31, 2:33 pm, Pascal Bourguignon <····@informatimago.com> wrote:
> Slava Akhmechet <·········@gmail.com> writes:
> > Here is what I want to support:
>
> > (defun authenticatedp ()
> > (if *authenticatedp*
> > t
> > (do-modal #'login-or-register)))
>
> > (defun some-action ()
> > (when (authenticatedp)
> > (do-action)))
>
> > The key is 'do-modal' - a construct that shows a login page (which can
> > manage rounds of user interaction), and returns the result of the
> > login operation as if it were a simple function call.
>
> > I don't need continuations to achieve this functionality. The trick,
> > saving the stack, can be done with threads. Whenever 'do-modal' is
> > called, it can spawn a thread and assign it to the current
> > session. All interaction will then be executed by that thread while
> > 'do-modal' blocks the original thread and waits for some signal from
> > the new one.
>
> It can work.
>
> The problem you have is that there is a limited number of threads
> possible in a process (around 300 or 400). Therefore you can handle
> only a limited number of concurent modal sessions (around 150 or 200
> if you call recursively do-modal only once per session).
>
> --
> __Pascal Bourguignon__ http://www.informatimago.com/
>
> NOTE: The most fundamental particles in this product are held
> together by a "gluing" force about which little is currently known
> and whose adhesive power can therefore not be permanently
> guaranteed.- Hide quoted text -
>
> - Show quoted text -
Slightly out of topic , how do you handle number of concurrent
threads. (Purely theoretical) I was thinking of something like
prespecified maximum number slots and a waiting list .
Pascal Bourguignon <···@informatimago.com> writes:
> The problem you have is that there is a limited number of threads
> possible in a process (around 300 or 400). Therefore you can handle
> only a limited number of concurent modal sessions (around 150 or 200
> if you call recursively do-modal only once per session).
That's a bit pessimistic. Statistically I think do-modal stretches
will be rather brief - you call one to authenticate, or call one with
some dialog, and then within a few seconds get out. For some features
do-modal calls can get rather long and recursive but it's unclear how
many users will be in such a state at any given time. I think you
could easily have thousands of users if you put them into the modal
state judiciously.
Of course some Lisp implementations would fit into this model better
than others. Initially I experimented with green thread systems
(Haskell, particularily GHC, has excellent support for these, as well
as Erlang), but I quickly found that with these languages I don't have
the freedom I'm given with Lisp (perhaps that's why they're so
scalable).
Anyway, if the app on top of such system gets really huge, one can
always hire a few hackers to complete the Erlisp project :)
--
Regards,
Slava Akhmechet.
On Jun 1, 5:06 pm, Pascal Bourguignon <····@informatimago.com> wrote:
> Nicolas Neuss <········@mathematik.uni-karlsruhe.de> writes:
> > fireblade <·················@gmail.com> writes:
>
> >> Yes but MUPROC runs on top of lisp implementation threading system with
> >> limited threads facilities (compared with leader Erlang and lisp cousin
> >> Termite/Gambit which support tens of thousands threads).
>
> > Not that I need that many threads at the moment, but:
1.
> > does anyone know which CL implementation allows the largest number of threads?
2.
> > And how many would that be?
>
3.
> Is there any CL implementation that includes a stack usage predictor,
> which would allow the thread creation function to know exactly how
> much stack to allocate for the given thread function?
>
4.
> Or is there any CL implementation that allocates the stack frames on
> the heap, so that thread stacks can use only the needed space and not
> a fixed amount which limits artificially the number of threads that
> can be allocated?
>
> --
> __Pascal Bourguignon__ http://www.informatimago.com/
>
I'm also very curious about above questions so if anybody has some
idea,experience and especially numbers (even some rough
approximations) I would be glad to hear .
fireblade wrote:
> On Jun 1, 5:06 pm, Pascal Bourguignon <····@informatimago.com> wrote:
> 4.
>> Or is there any CL implementation that allocates the stack frames on
>> the heap, so that thread stacks can use only the needed space and not
>> a fixed amount which limits artificially the number of threads that
>> can be allocated?
>>
>> --
>> __Pascal Bourguignon__
>> http://www.informatimago.com/
>>
>
> I'm also very curious about above questions so if anybody has some
> idea,experience and especially numbers (even some rough
> approximations) I would be glad to hear .
Well, Pascal's above would probably be best,
but I just did a fairly pointless experiment with SBCL's conventional
threading:
A standard build of SBCL on my system supports somewhere between 500
and 600 threads before the (relatively generous*) amount of per-thread
memory allocated for each thread blocks creation of further threads.
By reducing the allocations drastically**, I was able to get it up to
6000, but these allocations might be too low to do anything useful (in
my case, each thread was basically just sleeping) - though SBCL
itself and slime still appeared to build+work (somewhat to my surprise,
given it was a first stab), and the system was rather sluggish (not too
surprising) - if I had messed with linux process scheduling (e.g. CK
patches) I suspect I could have reined in the threads and kept the rest
of the system a bit more responsive. But like I said, fairly
pointless, and my attention has now wandered.
On a typical modern system, Linux itself notionally supports at least
tens of thousands of native-threads/processes*** - though there may be
a few ulimits and sysctls that need adjusting, e.g. "ulimit -u"
and "sysctl kernel.threads-max". This is practically limited by the
available memory even in the per-thread-stack case (there's a system
memory overhead for each native thread). By default, the per-user
process count ulimit on linux may be set quite low (at the whim of your
distro maker), usually the sysctl defaults to something reasonable for
your system IIRC.
* Back in the day we had 4k stacks, and liked it dammit, uphill
both ways in the snow...
**
src/runtime/thread.c;ALIEN_STACK_SIZE (1M -> 8k)
src/runtime/validate.h;BINDING_STACK_SIZE (1M -> 64k)
src/runtime/validate.h;THREAD_CONTROL_STACK_SIZE (2M -> 64k)
*** On linux, both processes and threads are implemented with the same
kernel process primitives anyway, threads just share access to memory
and other other resources (use "ps -eL" to see the hidden processes).
Using multiple memory-protected processes for an application is a
pretty normal implementation strategy on linux. N.B. Don't mistakenly
generalise experience of windows ultra-heavyweight-processes to linux.
Ex-windows people on linux often jump to multithreaded programming
unnecessarily, thinking that windows processes suck, so all processes
must suck (you get similar logic about command lines - cmd.exe sucks =>
command lines suck).
Hmm. I haven't examined the (cow-)fork-friendliness of SBCL's GC
in particular, though one would expect the generational
GC at least should be reasonably fork-friendly...
On 2007-06-04, fireblade <·················@gmail.com> wrote:
> On Jun 1, 5:06 pm, Pascal Bourguignon <····@informatimago.com> wrote:
>> Nicolas Neuss <········@mathematik.uni-karlsruhe.de> writes:
>> > fireblade <·················@gmail.com> writes:
>>
>> >> Yes but MUPROC runs on top of lisp implementation threading
>> >> system with limited threads facilities (compared with leader
>> >> Erlang and lisp cousin Termite/Gambit which support tens of
>> >> thousands threads).
>>
>> > Not that I need that many threads at the moment, but:
> 1.
>> > does anyone know which CL implementation allows the largest
>> > number of threads?
> 2.
>> > And how many would that be?
>>
> 3.
>> Is there any CL implementation that includes a stack usage
>> predictor, which would allow the thread creation function to know
>> exactly how much stack to allocate for the given thread function?
>>
> 4.
>> Or is there any CL implementation that allocates the stack frames
>> on the heap, so that thread stacks can use only the needed space
>> and not a fixed amount which limits artificially the number of
>> threads that can be allocated?
>
> I'm also very curious about above questions so if anybody has some
> idea,experience and especially numbers (even some rough
> approximations) I would be glad to hear .
Data point: Lispworks Pro for Linux, 5.0.2, stock, run from the IDE:
(loop for n below 1000 do
(mp:process-run-function
(gensym) ; name
nil ; keywords
#'(lambda (s n) ; function
(format s "running thread ~D~%" n)
(sleep 10))
*standard-output* n)) ; arguments to function
... throws an error at 288.
This may be worse than no data, in that it may be really easy to tune
LWL so that it allows more. For example, a DELIVERed application may
allow lots more threads. I am not a LWL (or Lisp or threading)
expert. Your mileage may vary.
-- Larry
On May 31, 3:54 am, Dustin Withers <·········@gmail.com> wrote:
> Is UnCommon Web a secret weapon? Or only used by Marco? ;)
If i have to choose i'd go for secret weapon. The second option at
least is false. From the top of my head i know from folks in Canada,
USA, Italy, India, Hungary (not 100 percent it was Hungary) and Turkey
who use it. Oddly enough they all use it commercially and most of them
are teams. Coincidentally I myself will start hacking full-time (and
payed!!) on a ucw app so we might add Sweden as well. I actually think
that's quite impressive for a lisp framework that's got the reputation
of being impenetrable.
As for the status of ucw: there's now a _dev and _ajax branch. Forget
about the _dev branch. It seems to be kinda dead. Drew Crampsie had
started a major cleanup of the codebase, taking a from the ground up
approach, and to my surprise when checking it's status for this post,
it seems to have been merged with ucw_ajax a few days ago. So in good
ucw tradition you can't really call it stable.
On the documentation front i will once again refer to my tutorial:
http://trac.common-lisp.net/ucw/wiki/UcwIntro which is for the ucw_dev
branch.
I'm getting a proxy error trying to reach it atm; i sent an email to
common-lisp.net. As noted ucw_dev seems to have been abandoned; The
last patch was three months ago. But recently i had to jump ship from
_dev to _ajax, cause the project i'm working on now is written in
ajax. I didn't really have to adjust my knowledge to be productive.
The new cleanup has changed the api somewhat i saw on darcsweb
( http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=ucw-ucw_ajax;a=summary
); defaction was dropped for example and contextl has been added to
the standard toolbox, but diffing the tutorial with the _ajax examples
should clear stuff up i hope. Since i'm gonna spend quite a bit of
time with ucw in the next few months i might redo the tutorial to
reflect the current situation, but i'm quite lazy by nature.
Admittedly, the two preceding paragraphs don't really speak in favour
(who would want cleanups eh) of ucw, but i would like to break a lance
for this framework: I've been doing some work for my new work for a
couple of months now with ucw and elephant, and i'd say that once the
learning curve starts to straighten it gets real easy to get work
done, and i find the stuff that has already been written to be very
maintainable and very shapeable. I don't have to much experience with
the ajax integration but it seems to be nice and nifty. To take a
simplified view of things, you can view a page of a ucw application as
a tree of components. As i understand it you can just specify for
every component if it will be an ajax component or not and presto...
it's done.
As should be clear: i'm having a blast with ucw, but your mileage may
vary.
/Ties