From: Blaine
Subject: Advice on mod_lisp-based web application.
Date: 
Message-ID: <1177352568.447958.52800@l77g2000hsb.googlegroups.com>
I've got a lisp simulation program that serves its interface over
html.  After
all kinds of trouble with IT folks at my organization - paperwork,
having to
migrate from Portable Allegro Serve to mod_lisp/Apache, I have a beta
version
on our intranet.

The beta works fine for one user, but, understandably, it doesn't work
for
multiple users.  It's understandable because 1) multiple concurrent
users
isn't (at lease wasn't) a requirement and 2) I'm just a noob.

My kludgey and hopefully short term solution strategy is to allow only
one
user at a time.  There are only a few (less than five) potential
users, so
it's not as bad a solution as it sounds.

What I was hoping for is some advice on implmentation.  At the very
least, it
seems, I have two desiderata:

1.  Persistence.  My server needs to be able to maintain connection
with a
    single, unique user while that user is using the simulation, and
politely
    block other users.

and

2. Termination.  After some pre-specified idle period, terminate the
"session"
   (not sure what word I should be using here), thus allowing a new
user
   access.

As for the Persistence issue, I was thinking I could use remote-ip-
address
key/value pair that comes with every modlisp request to identify the
user.
Will this work, or will I have to do something with cookies?

As for the termination issue, I've been tinkering around with a
timeout.  One
of the options when you start a modlisp server is a timeout.  When
that
timeout is reached, a timeout condition is signaled.  Presumably, I
could
write code that could handle that condition, offering the user a
choice to
continue using the simulation, and in the event that the user doesn't
respond,
close out the user's "session".  I'm not sure this is an appropriate
use of
the condition system, and furthermore, I don't know where I would put
the
handling code (a handler-bind form, I belive).

My guess is I have more problems than just Persistence and
Termination.  Any
comments on either of these or the other problems that I'm not yet
aware will
be much appreciated.


Thanks in advance,
Blaine


PS A while back I wrote a little noob  modlisp install guide that may
provide some details that may help answer my questions:
http://www.blaino.com/guide/modlisp-pcl-guide.html

From: Alex Mizrahi
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <462d1e6b$0$90269$14726298@news.sunsite.dk>
(message (Hello 'Blaine)
(you :wrote  :on '(23 Apr 2007 11:22:48 -0700))
(

 B> The beta works fine for one user, but, understandably, it doesn't work
 B> for
 B> multiple users.

hmm, since HTTP is stateless protocol it's kinda hard to make single-user 
application with it..
unless it's something extremely special, it should be very easy to convert 
it to multi-user -- if you have state in some global variables, move them to 
session variables etc. it should be lots easier than dealing with 
'persistence' and 'termination' delirium you're describing below.

the only thing i can imagine that makes it's hard to make multi-user 
application is having some calculations running persistently in the thread.. 
but to serve web requests in such situation you need multithreading anyway, 
so adding more users should be straightforward -- spawn more threads..

  B> As for the Persistence issue, I was thinking I could use remote-ip-
 B> address
 B> key/value pair that comes with every modlisp request to identify the
 B> user.
 B> Will this work, or will I have to do something with cookies?

cookies is a standard way of doing this kind of things. i think it's always 
better to use standard things.
there's also another technique for this that works everywhere -- put session 
info (session id or whatever) into URLs.

 B> close out the user's "session".  I'm not sure this is an appropriate
 B> use of
 B> the condition system, and furthermore, I don't know where I would put
 B> the
 B> handling code (a handler-bind form, I belive).

as you describe, you're using HTTP protocol in a stateful fashion, and 
moreover you can send info from server to client even if client isn't 
requesting it.. how?
certainly there's stuff like AJAX, but you haven't mentioned it..

)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"I am everything you want and I am everything you need") 
From: Tim X
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <87slaqvz3l.fsf@lion.rapttech.com.au>
Blaine <···············@gmail.com> writes:

> I've got a lisp simulation program that serves its interface over
> html.  After
> all kinds of trouble with IT folks at my organization - paperwork,
> having to
> migrate from Portable Allegro Serve to mod_lisp/Apache, I have a beta
> version
> on our intranet.
>
> The beta works fine for one user, but, understandably, it doesn't work
> for
> multiple users.  It's understandable because 1) multiple concurrent
> users
> isn't (at lease wasn't) a requirement and 2) I'm just a noob.
>
> My kludgey and hopefully short term solution strategy is to allow only
> one
> user at a time.  There are only a few (less than five) potential
> users, so
> it's not as bad a solution as it sounds.
>
> What I was hoping for is some advice on implmentation.  At the very
> least, it
> seems, I have two desiderata:
>
> 1.  Persistence.  My server needs to be able to maintain connection
> with a
>     single, unique user while that user is using the simulation, and
> politely
>     block other users.
>
> and
>
> 2. Termination.  After some pre-specified idle period, terminate the
> "session"
>    (not sure what word I should be using here), thus allowing a new
> user
>    access.
>
> As for the Persistence issue, I was thinking I could use remote-ip-
> address
> key/value pair that comes with every modlisp request to identify the
> user.
> Will this work, or will I have to do something with cookies?
>
> As for the termination issue, I've been tinkering around with a
> timeout.  One
> of the options when you start a modlisp server is a timeout.  When
> that
> timeout is reached, a timeout condition is signaled.  Presumably, I
> could
> write code that could handle that condition, offering the user a
> choice to
> continue using the simulation, and in the event that the user doesn't
> respond,
> close out the user's "session".  I'm not sure this is an appropriate
> use of
> the condition system, and furthermore, I don't know where I would put
> the
> handling code (a handler-bind form, I belive).
>
> My guess is I have more problems than just Persistence and
> Termination.  Any
> comments on either of these or the other problems that I'm not yet
> aware will
> be much appreciated.
>
>
> Thanks in advance,
> Blaine
>
>
> PS A while back I wrote a little noob  modlisp install guide that may
> provide some details that may help answer my questions:
> http://www.blaino.com/guide/modlisp-pcl-guide.html
>

I think you may need to re-think your paradigm. It seems to me that you are
approaching the problem from the wrong direction. HTTP is essentially a
"stateless" protocol rather than a stateful connected protocol. The use of
cookies etc, can simulate a stateful interaction, but its not like other more
connection oriented protocols, wehre both ends are always connected and
communicating over the same connection. While Apache and other servers to
provide some support for maintaining connections and connection pooling, this
is more related to improved performance in establishing a connection rather
than maintaining a continuous bidirectional communication channel. 

You said your applicaiton would not work with more than one user. I'd
concentrate on fixing that issue. Approach the whole client/server interaction
as one which is stateless and then use cookies (or even REMOTE_USER) as a way
of connecting states and maybe have your application save/load past states of
your lisp simulation before/after dealing with a client request. This will
allow you to avoid the whole issue of trying to restrict access to a specific
user in a serial type interaction model and as a side effect, you will be able
to support multiple users at the same time (ignoring performance/resource
issues of course). 

Once you assume a stateless interaction model in which you cannot know the
order or sequence of requests you will recieve, lots of problems just vanish.
Most of the time, problems I've seen people having with web based applications
are due to incorrect assumptions about the order/sequence of interactions you
will get from the same and different clients. this is an understandable mistake
because from an end user perspective, it does appear to be a well defined
sequence/series of interactions. 

Tim

   
-- 
tcross (at) rapttech dot com dot au
From: Blaine
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <1177436807.760766.256330@c18g2000prb.googlegroups.com>
Alex and Tim,

Thanks for the advice.  If a stateless interaction model will make my
problems vanish then I'm definitely interested in trying to change
things.

I do have a good deal of state in global variables.  The
recommendation is to migrate those to "session variables".  Might you
say a little more about session variables?  Are these instances of a
data structure that includes session id information along with the
variable's value?  Or something else?

Also, Alex wrote, "the only thing i can imagine that makes it hard to
make multi-user
application is having some calculations running persistently in the
thread.."  The simulation (discrete event monte carlo type thing),
when run lots of times which I have to do to get statistically sound
results, can take a few minutes to run.  Is this the sort of thing you
imagine as being problem?  Or, might it be the case that if I migrate
state-preserving global variables to session variables, I will be
okay?

I guess another way of asking the question is this.  Is the
elimination of global variables sufficient to allow multithreading?
(I'm using SBCL and the Debian common-lisp-controller).

Again, thank you,
Blaine
From: Alex Mizrahi
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <462f09a0$0$90264$14726298@news.sunsite.dk>
(message (Hello 'Blaine)
(you :wrote  :on '(24 Apr 2007 10:46:47 -0700))
(

 B> I do have a good deal of state in global variables.  The
 B> recommendation is to migrate those to "session variables".  Might you
 B> say a little more about session variables?  Are these instances of a
 B> data structure that includes session id information along with the
 B> variable's value?  Or something else?

you can have, for example, a hash table *sessions* with keys being 
session-ids and values being a class, struct, hash-table or plist.
if you create new session, you generate a new ID and put into this 
hash-table:
  (setf (gethash new-session-id *sessions*) (make-session-context :param pam 
..))

when you recieve a request, if you have session id you find a context for 
this session:
    (let ((*cur-context* (gethash session-id *sessions*)))
      ;;and you can use it
        (print (session-context-param1 *cur-context*)))

certainly you might want to have shorter names for retrieving this 
variables.
if you are using this variables intensively, you might want to just bind 
them in thread:

(let ((*my-param1* (session-context-param1 *cur-context*))
       (*my-param2* ...

in most implementations that works find even in multithreaded scenario.

 B> results, can take a few minutes to run.  Is this the sort of thing you
 B> imagine as being problem?  Or, might it be the case that if I migrate
 B> state-preserving global variables to session variables, I will be
 B> okay?

how do you implement this now? user presses button in browser, and waits for 
a few minutes?
if that works it's not really a problem, but browsers might have a timeout 
for longer delays..
i think it would be better to run calculations in thread, setting some 
session variables on completion, for example.
then on requests from browser you can check that variables, and if 
computation is complete you can display results, if not -- display 'please 
wait' message.

 B> I guess another way of asking the question is this.  Is the
 B> elimination of global variables sufficient to allow multithreading?

almost. you'll need a synchronization to be sure you access variables only 
from one thread at time.
in your case you can have just a single mutex and lock it when you read or 
write any session variables.

if you'll have a separate thread doing computations, your requests 
processing will be fast, so you can lock this mutex on all request 
processing time. also you'll need to lock mutex when you return results from 
the thread

 B> (I'm using SBCL and the Debian common-lisp-controller).

if that's relatively new versions (Debian with kernel 2.6) it should be 
fine.

)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"I am everything you want and I am everything you need") 
From: Tim X
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <877is0wuuo.fsf@lion.rapttech.com.au>
Blaine <···············@gmail.com> writes:

> Alex and Tim,
>
> Thanks for the advice.  If a stateless interaction model will make my
> problems vanish then I'm definitely interested in trying to change
> things.
>
> I do have a good deal of state in global variables.  The
> recommendation is to migrate those to "session variables".  Might you
> say a little more about session variables?  Are these instances of a
> data structure that includes session id information along with the
> variable's value?  Or something else?
>
> Also, Alex wrote, "the only thing i can imagine that makes it hard to
> make multi-user
> application is having some calculations running persistently in the
> thread.."  The simulation (discrete event monte carlo type thing),
> when run lots of times which I have to do to get statistically sound
> results, can take a few minutes to run.  Is this the sort of thing you
> imagine as being problem?  Or, might it be the case that if I migrate
> state-preserving global variables to session variables, I will be
> okay?
>
> I guess another way of asking the question is this.  Is the
> elimination of global variables sufficient to allow multithreading?
> (I'm using SBCL and the Debian common-lisp-controller).
>

Alex has already addressed your questions quite adequately, so I doubt I can
add much more. You seem to have the general idea - any variable or information
which is specific to a particular client request needs to be kept in a state
variable of some type and you need some mechanism to link client requests
together in order to know which state variables correspond with which client.
Any structure that meets your need, hash table, sequence, structure etc, can be
used. 

If you don't have many state variables, you can even try using variables in web
forms. The server returns whatever result it generates and possibly records
state information in variables embedded in the form it returns. The user then
uses a submit button to send the next bit of data back to the server. The
server reads the (hidden) variables from the form to re-initiate state and then
processes the new bit of data. 

Obviously, this will only work well when there are not too many state variables
to track. In some ways, its a poor 'cookies' alternative, but will work even
when users have cookies disabled. 

Another approach which is beginning to gain increasing ground is to use
javascript and maintain more of the state init stuff in the browser and send it
to the server. However, as there are still considerable variations in
javascript engines, this can be more complex. However, it does allow things
like pages that are only partially updated rather than totally updated etc. 

If your application is really quite complex, with lots of variables etc, you
can even use a database backend and have the information keyed to some unique
value that the server retrieves from the client (via a form variable, cookie or
possibly even url/path). You then have routines to save/restore state from the
database. 

Note carefully Alex's comment about using a mutex or some other mechanism to
ensure serial access to variables. If you have multiple threads/processes
running, you can never rely on what sequence events can occur in. Generally,
you need to always keep this in mind when working with web apps. You have no
guarantee of number or order of client requests and therefore, cannot make any
assumptions about when resources are used. The way the server works can also be
something you need to keep in mind. I'm not sure about mod_lisp, but some
systems just execute a whole new instance, in which keeping things separate is
a bit easier, but communicating information between requests may be more
difficult. Other systems are a master system that just forks of threads or
processes, in which case you need more care, but communication between requests
is easier. I suspect mod_lisp is more like the latter as these systems tend to
be more responsive and less resource intensive. 

Tim


-- 
tcross (at) rapttech dot com dot au
From: Blaine
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <1177516389.659623.147900@r35g2000prh.googlegroups.com>
Tim, Alex, thanks for the advice.  Problems vanishing does sound
nice.

I have much state in global variables, and Alex's recommendation is to
migrate those to session variables.  Could you say a little more about
these, about their implementation?  I'm imagining some sort of
structure that includes, along with the variable value, a session id.
This structure, while more complex than the prior global variable is
still global.  Am I thinking about this correctly?

Thanks again.  I really appreciate it.
From: Blaine
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <1177586546.587476.134200@o40g2000prh.googlegroups.com>
Fellas,

Thank you.

Sorry about my last - I was having trouble with usenet and thought the
former had not posted.

- Blaine
From: Tim X
Subject: Re: Advice on mod_lisp-based web application.
Date: 
Message-ID: <87y7kfv7ml.fsf@lion.rapttech.com.au>
Blaine <···············@gmail.com> writes:

> Tim, Alex, thanks for the advice.  Problems vanishing does sound
> nice.
>
> I have much state in global variables, and Alex's recommendation is to
> migrate those to session variables.  Could you say a little more about
> these, about their implementation?  I'm imagining some sort of
> structure that includes, along with the variable value, a session id.
> This structure, while more complex than the prior global variable is
> still global.  Am I thinking about this correctly?
>
> Thanks again.  I really appreciate it.
>

Yep, you seem to be on the right track. How you implement it is really
dependent on what you need to track, but essentially some sort of structure
with some sort of 'session key'. A hash, as suggested by Alex is often a good
choice. It still has to be global because you need to be able to retrieve it
during a latter request (you could store it in a database if complex/large
enough to justify the overhead). Essentially, any value which is specific to a
user's session needs to be in this structure and used by subsequent requests. 

Tim

-- 
tcross (at) rapttech dot com dot au