From: CoffeeMug
Subject: Rolling out updates to a live CL server
Date: 
Message-ID: <7c170b28-6aae-43e4-85c0-e71d32aa9c06@13g2000yql.googlegroups.com>
When people talk about Common Lisp they usually mention the
interactivity factor and the ability to fix bugs on live servers. I am
trying to figure out how to do this now, beyond making very small and
obvious changes.

I have a pretty standard setup - SBCL and Hunchentoot behind Apache
(using mod_lisp2), a swank server (I tunnel over ssh), and detachtty.
I can make trivial changes through slime, but what about rolling out
larger updates?

Overall, it seems that I would have to diff the tip of my branch with
the revision that's on the live server. I'd then parse the diff file
(an extension to ediff?) and send each chunk of code in the diff
through slime. Of course simply doing this straight out won't work -
I'll have to figure out full context from the diff (get whole function
body), possibly cherry pick chunks of code, and reorder them because
they might depend on each other.

Has anyone done something like this? Writing the tools to do it isn't
trivial at all, it looks like a very time consuming project. If anyone
has experience doing this or some chunks of code laying around, they'd
be very much appreciated.

From: Zach Beane
Subject: Re: Rolling out updates to a live CL server
Date: 
Message-ID: <m3tz60pe6u.fsf@unnamed.xach.com>
CoffeeMug <·········@gmail.com> writes:

> Overall, it seems that I would have to diff the tip of my branch with
> the revision that's on the live server. I'd then parse the diff file
> (an extension to ediff?) and send each chunk of code in the diff
> through slime. Of course simply doing this straight out won't work -
> I'll have to figure out full context from the diff (get whole function
> body), possibly cherry pick chunks of code, and reorder them because
> they might depend on each other.

Espen Vestre has written about his approach in the past. It involves not
diffing, but loading files as sexps and using set-difference on them,
then tweaking things by hand if necessary.

Zach
From: Peder O. Klingenberg
Subject: Re: Rolling out updates to a live CL server
Date: 
Message-ID: <ksmybspi5x.fsf@netfonds.no>
CoffeeMug <·········@gmail.com> writes:

> I can make trivial changes through slime, but what about rolling out
> larger updates?

We have a number of long-running servers based on Lispworks.  Our
environment predates Slime, so we don't use swank, but we do of course
have REPL access.

My rule of thumb is that if it's a single function change, I paste it
in the REPL.  If it's larger than that, but still confined to a single
source file, I compile the source file and ship the .fasl to the
server, then load it in the running image.  If it's even larger, it's
usually easier to restart the system.

Regardless of the size of the change, a new image has to be built,
tested and installed anyway, to ensure the system will come up cleanly
if neccessary.

This works well enough that I've never seen the need to mess around
with diff.

...Peder...
-- 
I wish a new life awaited _me_ in some off-world colony.
From: Pascal Costanza
Subject: Re: Rolling out updates to a live CL server
Date: 
Message-ID: <71pe0qFmgmilU1@mid.individual.net>
Peder O. Klingenberg wrote:
> CoffeeMug <·········@gmail.com> writes:
> 
>> I can make trivial changes through slime, but what about rolling out
>> larger updates?
> 
> We have a number of long-running servers based on Lispworks.  Our
> environment predates Slime, so we don't use swank, but we do of course
> have REPL access.
> 
> My rule of thumb is that if it's a single function change, I paste it
> in the REPL.  If it's larger than that, but still confined to a single
> source file, I compile the source file and ship the .fasl to the
> server, then load it in the running image.  If it's even larger, it's
> usually easier to restart the system.
> 
> Regardless of the size of the change, a new image has to be built,
> tested and installed anyway, to ensure the system will come up cleanly
> if neccessary.
> 
> This works well enough that I've never seen the need to mess around
> with diff.

What's important to know here is that many/most definitions in Common 
Lisp can be redefined in such a way that the redefinition is seen by all 
the other running code. This is very obvious for class and method 
redefinition, but also for plain functions, variables, etc.

This is unusual when compared to other programming languages, and takes 
a little while to be grasped.


Pascal

-- 
ELS'09: http://www.european-lisp-symposium.org/
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Chun Tian (binghe)
Subject: Re: Rolling out updates to a live CL server
Date: 
Message-ID: <541efc3e-09b4-40ff-a3cc-146f260a5a8b@z8g2000prd.googlegroups.com>
Things become complicated when you try to delete a method from generic
function, change the structure of a class, or even update a macro
which may be used by other functions. But I'm quite sure Common Lisp
supplies all facility to do any change in a live Lisp image.

Generating loadable lisp patch from source code diff is a very hard
mission, but even though, the Symbolics Patch Facility which found in
Lisp Machine has done this over 20 years ago, and this part was mainly
written by Daniel Weinreb, the ILC 09 Chairman.

On Mar 11, 5:19 pm, Pascal Costanza <····@p-cos.net> wrote:
> Peder O. Klingenberg wrote:
> > CoffeeMug <·········@gmail.com> writes:
>
> >> I can make trivial changes through slime, but what about rolling out
> >> larger updates?
>
> > We have a number of long-running servers based on Lispworks.  Our
> > environment predates Slime, so we don't use swank, but we do of course
> > have REPL access.
>
> > My rule of thumb is that if it's a single function change, I paste it
> > in the REPL.  If it's larger than that, but still confined to a single
> > source file, I compile the source file and ship the .fasl to the
> > server, then load it in the running image.  If it's even larger, it's
> > usually easier to restart the system.
>
> > Regardless of the size of the change, a new image has to be built,
> > tested and installed anyway, to ensure the system will come up cleanly
> > if neccessary.
>
> > This works well enough that I've never seen the need to mess around
> > with diff.
>
> What's important to know here is that many/most definitions in Common
> Lisp can be redefined in such a way that the redefinition is seen by all
> the other running code. This is very obvious for class and method
> redefinition, but also for plain functions, variables, etc.
>
> This is unusual when compared to other programming languages, and takes
> a little while to be grasped.
>
> Pascal
>
> --
> ELS'09:http://www.european-lisp-symposium.org/
> My website:http://p-cos.net
> Common Lisp Document Repository:http://cdr.eurolisp.org
> Closer to MOP & ContextL:http://common-lisp.net/project/closer/
From: Vsevolod
Subject: Re: Rolling out updates to a live CL server
Date: 
Message-ID: <44ff30f1-7be9-4b35-9522-88c1479573e6@p11g2000yqe.googlegroups.com>
On Mar 11, 8:30 am, CoffeeMug <·········@gmail.com> wrote:
> When people talk about Common Lisp they usually mention the
> interactivity factor and the ability to fix bugs on live servers. I am
> trying to figure out how to do this now, beyond making very small and
> obvious changes.
>
> I have a pretty standard setup - SBCL and Hunchentoot behind Apache
> (using mod_lisp2), a swank server (I tunnel over ssh), and detachtty.
> I can make trivial changes through slime, but what about rolling out
> larger updates?
>
> Overall, it seems that I would have to diff the tip of my branch with
> the revision that's on the live server. I'd then parse the diff file
> (an extension to ediff?) and send each chunk of code in the diff
> through slime. Of course simply doing this straight out won't work -
> I'll have to figure out full context from the diff (get whole function
> body), possibly cherry pick chunks of code, and reorder them because
> they might depend on each other.
>
> Has anyone done something like this? Writing the tools to do it isn't
> trivial at all, it looks like a very time consuming project. If anyone
> has experience doing this or some chunks of code laying around, they'd
> be very much appreciated.

Why don't you consider the simple option of packaging everything in an
ASDF system and re-loading it in the live image?
From: Pascal J. Bourguignon
Subject: Re: Rolling out updates to a live CL server
Date: 
Message-ID: <7ctz5ybofp.fsf@pbourguignon.anevia.com>
Vsevolod <········@gmail.com> writes:

> On Mar 11, 8:30 am, CoffeeMug <·········@gmail.com> wrote:
>> When people talk about Common Lisp they usually mention the
>> interactivity factor and the ability to fix bugs on live servers. I am
>> trying to figure out how to do this now, beyond making very small and
>> obvious changes.
>>
>> I have a pretty standard setup - SBCL and Hunchentoot behind Apache
>> (using mod_lisp2), a swank server (I tunnel over ssh), and detachtty.
>> I can make trivial changes through slime, but what about rolling out
>> larger updates?
>
> Why don't you consider the simple option of packaging everything in an
> ASDF system and re-loading it in the live image?

Which, basically if you take care of distinguishing correctly DEFVAR
and DEFPARAMETER should go smoothly enough.  You may also need to
write some convenience functions to hot-reinitialize the server, but
for simple servers things go well enough just reloading everything.

-- 
__Pascal Bourguignon__