From: ··········@gmail.com
Subject: Lisp Web Development and Application State
Date: 
Message-ID: <1157255460.163818.134390@74g2000cwt.googlegroups.com>
How do people around here save state in Lisp-based web apps?  I have
been playing around with tbnl/cl-who, but most of the stuff I have been
doing is pretty standard.  I want to put together a form application
that lets me easily create multi-page forms where the back-button
works.

I have looked at UCW and read Paul Graham's writings on ViaWeb.  UCW is
great, and PG's stuff would work if I spawned a new process for each
user (closures with multiple users and one lisp process are, well,
closures.  They save one state).  UCW is huge and I am looking for a
simple idea to prototype.

So, how would you build a simple web application that saves state
between pages in Lisp?

From: Rob Warnock
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <NdSdnXQTV6Vg_WfZnZ2dnUVZ_sCdnZ2d@speakeasy.net>
··········@gmail.com <··········@gmail.com> wrote:
+---------------
| How do people around here save state in Lisp-based web apps?  I have
| been playing around with tbnl/cl-who, but most of the stuff I have been
| doing is pretty standard.  I want to put together a form application
| that lets me easily create multi-page forms where the back-button works.
+---------------

Well, from one point of view, it's not so much the "Back" button per se
that's the problem, but the "Reload/Refresh" and "Forward" buttons.  ;-}

+---------------
| I have looked at UCW and read Paul Graham's writings on ViaWeb.
| UCW is great, and PG's stuff would work if I spawned a new process
| for each user (closures with multiple users and one lisp process are,
| well, closures.  They save one state).
+---------------

Not necessarily. See the recent long thread here[1] that eventually
got around to an interesting discussion of "web continuations"[2]
with a complaint by me about needing multiple continuations [that
mirrors your "closures...save one state" complaint] and a delightful
response by Anton van Straaten showing a simple way to provide a web
page with multiple continuations by constructing them all before
painting the page [and then abandoning the default continuation for
the page, which is the clever bit].

+---------------
| So, how would you build a simple web application that saves state
| between pages in Lisp?
+---------------

Hidden variables in the HTML forms, that is, <INPUT TYPE="hidden"
NAME="state" VALUE="8NM83gdUrqI8TsHe"> or similar. These provide
the hook for you to map back to wherever on the server you stored
the full state [whether in memory or an SQL database or whatever],
as I showed previously.[3]

In the limit, this can extend so far as to store *all* of the state
in the displayed pages, which has the distinct advantage of not
needing to handle any timeouts on session state on the server,
as well as allowing pages to be "restarted" even after having been
saved to the user's local disk(!) [but in that case you will probably
want to store a cryptographically-secure hash with the state to avoid
modify/replay attacks.


-Rob

[1] Under the somewhat flamebait subject "Lisp or Scheme?".

[2] The meat of the "web continuations" portion started at about
    message #53 or so, with a post by Ken Tilton near here:

      <http://groups.google.com/group/comp.lang.lisp/tree/browse_frm/thread/25a9a0b117e725e1/c3b89e08206f9298?rnum=51&_done=%2Fgroup%2Fcomp.lang.lisp%2Fbrowse_frm%2Fthread%2F25a9a0b117e725e1%2Fa49a359a1461a39b%3Ftvc%3D1%26#doc_b39acefc408ff0e3>

    containing the phrase "Face it, a continuation is just a cute trick
    for managing state", and then the dsicussion that followed.

[3] An article of mine about three weeks ago:

      <·············································································································@speakeasy.net>

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: ··········@gmail.com
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <1157296472.656890.172680@e3g2000cwe.googlegroups.com>
> Hidden variables in the HTML forms, that is, <INPUT TYPE="hidden"
> NAME="state" VALUE="8NM83gdUrqI8TsHe"> or similar. These provide
> the hook for you to map back to wherever on the server you stored
> the full state [whether in memory or an SQL database or whatever],
> as I showed previously.[3]
>
> In the limit, this can extend so far as to store *all* of the state
> in the displayed pages, which has the distinct advantage of not
> needing to handle any timeouts on session state on the server,
> as well as allowing pages to be "restarted" even after having been
> saved to the user's local disk(!) [but in that case you will probably
> want to store a cryptographically-secure hash with the state to avoid
> modify/replay attacks.
>

ASP.net does this, and I never thought all of the reasons why they went
with this method.  At first, I thought, why not just store the data in
a session or hack together a continuations/closures system that stores
the data on the server.  Sessions are great, but not for my particular
problem which is multi-page forms where the user might walk away for an
hour to get more information.

There are also all sorts of other cool little things that can be done
with storing data in a hidden variable:

- Save the "state" to a single session variable for small forms.
- Write the "state" to the database in a single field.  Also, just save
parts of the state to a field.
- Write out the "state" easily to a file.
- The reasons you pointed out.

The database structure of the application is very, very abstract so
that I can create new forms on the fly without having to create new
tables.

Thanks for the well-thought-out response.

Chris
From: Rafal Strzalinski
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <8de16$44faa8e7$d4ba5e95$16591@news.chello.pl>
··········@gmail.com napisał(a):
> How do people around here save state in Lisp-based web apps?  I have
> been playing around with tbnl/cl-who, but most of the stuff I have been
> doing is pretty standard. 

> I want to put together a form application
> that lets me easily create multi-page forms where the back-button
> works.

Forget about it. I have not seen any application with working 'back'
button. Just provide your own back/next buttons on page.

> So, how would you build a simple web application that saves state
> between pages in Lisp?

Maybe you should use 'session' feature:

 http://weitz.de/tbnl/#sessions



-- 
Pozdrawiam,
Rafal Strzalinski (nabla)
http://rafal.strzalinski.pl
From: Alex Mizrahi
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <44fac052$0$75033$14726298@news.sunsite.dk>
(message (Hello ···········@gmail.com)
(you :wrote  :on '(2 Sep 2006 20:51:00 -0700))
(

 m> closures.  They save one state).  UCW is huge and I am looking for a
 m> simple idea to prototype.

if you like continuations, you can extract it from UCW -- it's quite small 
at itself. i used old CPS-transformer extracted from arnesi. afair Marco 
replaced it with interpreter then.

 m> So, how would you build a simple web application that saves state
 m> between pages in Lisp?

what state do you need to save?

continuations are great for wizard-style pages -- where you enter params and 
go from one page to another.
but i think most application do not work in that way -- they act more like 
accessor to database, displaying information for some URL, and sometimes 
mutate this information for POST requests, there's no much state there. i 
believe it's original design of the Web, using forms to go from one page to 
another is some kind of perversion :)

if you need your web application to mock ordinary GUI application with 
dialogs etc, you won't be satisfied with full page reload anyway.
there are modern technologies to help in this case -- client-side 
scripting -- JavaScript. thus, you can build all fancy dialogs with 
JavaScript letting server serve as back-end -- doing database queries or 
whatever.
but if you for some reasons do not work JavaScript to do most logic but act 
only as GUI layer, you might need continuations to manage that GUI. but are 
you sure you want this? :)

there is a problem with keeping client stuff on server -- it easily gets 
bloated. it's ok if only few persons use your site, but if it's publically 
available, some people might want to do DoS on it. each continuations easily 
require 100+ KB memory, so opening 10000 sessions will consume 1 GB. even if 
you shot some sessions, you can shot sessions of live people -- so service 
would be broken anyway.

)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"People who lust for the Feel of keys on their fingertips (c) Inity") 
From: John Thingstad
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <op.tfcn8jcopqzri1@pandora.upc.no>
On Sun, 03 Sep 2006 05:51:00 +0200, ··········@gmail.com  
<··········@gmail.com> wrote:

> How do people around here save state in Lisp-based web apps?  I have
> been playing around with tbnl/cl-who, but most of the stuff I have been
> doing is pretty standard.  I want to put together a form application
> that lets me easily create multi-page forms where the back-button
> works.
>
> I have looked at UCW and read Paul Graham's writings on ViaWeb.  UCW is
> great, and PG's stuff would work if I spawned a new process for each
> user (closures with multiple users and one lisp process are, well,
> closures.  They save one state).  UCW is huge and I am looking for a
> simple idea to prototype.
>
> So, how would you build a simple web application that saves state
> between pages in Lisp?
>

Well the old way would be by setting cookies on the client.
With allegro-serve and ActionScript you can have server side
data for the user well.
A third way is to encode the information in the URL.
Then there is the abillity to update only parts of the page (AJAX).
This is not really a Lisp issue. I suggest a book on web development.

-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
From: Pierre THIERRY
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <pan.2006.09.04.14.05.34.848509@levallois.eu.org>
Le Sat, 02 Sep 2006 20:51:00 -0700, ··········@gmail.com a écrit :
> So, how would you build a simple web application that saves state
> between pages in Lisp?

With a running Lisp image instead of scripts run by the Web server, part
of the problem just vanishes. In my case, I added Elephant so that I can
just consider my data persistent, even accross reboots of the image.

Partly,
Nowhere man
-- 
···········@levallois.eu.org
OpenPGP 0xD9D50D8A
From: ··········@gmail.com
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <1157393841.532015.74280@p79g2000cwp.googlegroups.com>
Pierre THIERRY wrote:
> Le Sat, 02 Sep 2006 20:51:00 -0700, ··········@gmail.com a écrit :
> > So, how would you build a simple web application that saves state
> > between pages in Lisp?
>
> With a running Lisp image instead of scripts run by the Web server, part
> of the problem just vanishes. In my case, I added Elephant so that I can
> just consider my data persistent, even accross reboots of the image.
>
> Partly,
> Nowhere man
> --
> ···········@levallois.eu.org
> OpenPGP 0xD9D50D8A

How do you know when to let go of the user data?  For example, if a
user starts to fill out a multi-page form and never finishes, how long
do you keep the data on the server?
From: howard yeh
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <1157402264.518570.210390@m73g2000cwd.googlegroups.com>
··········@gmail.com wrote:
> Pierre THIERRY wrote:
> > Le Sat, 02 Sep 2006 20:51:00 -0700, ··········@gmail.com a écrit :
> > > So, how would you build a simple web application that saves state
> > > between pages in Lisp?
> >
> > With a running Lisp image instead of scripts run by the Web server, part
> > of the problem just vanishes. In my case, I added Elephant so that I can
> > just consider my data persistent, even accross reboots of the image.
> >
> > Partly,
> > Nowhere man
> > --
> > ···········@levallois.eu.org
> > OpenPGP 0xD9D50D8A
>
> How do you know when to let go of the user data?  For example, if a
> user starts to fill out a multi-page form and never finishes, how long
> do you keep the data on the server?

The most straightforward solution is probably using timeout to reap old
continuations. Elephant is an object store, it persists arbitrary lisp
data
by writing to a Berkeley DB.

Incidentally, my google Summer of Code project might interest you.
http://common-lisp.net/project/wispylisp/tutorial.html
It has continuation enabled form, which lets you program functionally
across the server/client barrier.
From: ··········@gmail.com
Subject: Re: Lisp Web Development and Application State
Date: 
Message-ID: <1157469469.318838.225550@m79g2000cwm.googlegroups.com>
howard yeh wrote:
> ··········@gmail.com wrote:
> > Pierre THIERRY wrote:
> > > Le Sat, 02 Sep 2006 20:51:00 -0700, ··········@gmail.com a écrit :
> > > > So, how would you build a simple web application that saves state
> > > > between pages in Lisp?
> > >
> > > With a running Lisp image instead of scripts run by the Web server, part
> > > of the problem just vanishes. In my case, I added Elephant so that I can
> > > just consider my data persistent, even accross reboots of the image.
> > >
> > > Partly,
> > > Nowhere man
> > > --
> > > ···········@levallois.eu.org
> > > OpenPGP 0xD9D50D8A
> >
> > How do you know when to let go of the user data?  For example, if a
> > user starts to fill out a multi-page form and never finishes, how long
> > do you keep the data on the server?
>
> The most straightforward solution is probably using timeout to reap old
> continuations. Elephant is an object store, it persists arbitrary lisp
> data
> by writing to a Berkeley DB.
>
> Incidentally, my google Summer of Code project might interest you.
> http://common-lisp.net/project/wispylisp/tutorial.html
> It has continuation enabled form, which lets you program functionally
> across the server/client barrier.

I am actually testing out storing all of the data as a base64 encoded
plists in a hidden field called "state".  This stores all of the data
on the client, but I can pretty easily store the data on the server -
especially if the user has authenticated to the system.  The forms
themselves are rendered from plists, and so it is natural to store the
data in the same sort of format.  The whole idea is to be able to take
a definition file that is a plist, combine it with a state file which
is a plist, and get the entire state of the application at any time.

I have looked at wispy lisp and I am pretty excited that you are
bringing Rails to Lisp.

Chris