From: Willy
Subject: Proper way to do Interactive Development
Date: 
Message-ID: <94059dac.0311052233.1005c72a@posting.google.com>
Hi,

I am a beginner and still leaning the beauty of Lisp especially in the
interactive development. After did some search on the net, I'm still
not very clear how to use it in the real world case.

What I am thinking about interactive development is that I can run a
program and if it has problem, I can fix it at realtime and then
continue the program without the need of restart. So all values inside
variables still maintain. I hope I got the idea right in the first
place.

But the problem is, when I do a fix, should I do it on the program
source files, then reload, or I should do it on the system itself and
save the core, let the core grow with this way and forget all about
the source files.


Please let me make it more clear,

Method 1:
    While the program is running, and you find a bug. Fix it and
record the changes onto the source file. And then reload the source to
make sure the changes are right in place.

Method 2:
    Locate the bug, fix it and keep the system running. Forget all
about the source files because the changes are going to record to the
core image when the system do a (save-core). So the whole program will
build base on the core file.



The problem with Method 1 is that I need to do the changes twice, one
on the running system first. See whether the problem fixed. If it is,
record down all the changes AGAIN on the source files. And do a reload
to make sure all changes are ok. But then, this will not allow me to
maintain the state of the running system. And if the fix includes lots
of source files, recording changes become not that easy. It will get
even worst if the loading sequence of sources is important.

While with the Method 2, it is good but I cannot rebuild the system
all over again because I don't record down all the changes onto source
files. The only thing I have is the core image and it will get bigger
and bigger with full of other unuse functions.

And, I am not sure whether there has another way to do interactive
development properly because the reference about it on the net is very
limitted.

Would somebody help me? Or any suggestion?



Cheers,
- Willie

From: Peter Seibel
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <m3r80m55ne.fsf@javamonkey.com>
··········@yahoo.com (Willy) writes:

> Hi,
> 
> I am a beginner and still leaning the beauty of Lisp especially in the
> interactive development. After did some search on the net, I'm still
> not very clear how to use it in the real world case.
> 
> What I am thinking about interactive development is that I can run a
> program and if it has problem, I can fix it at realtime and then
> continue the program without the need of restart. So all values inside
> variables still maintain. I hope I got the idea right in the first
> place.
> 
> But the problem is, when I do a fix, should I do it on the program
> source files, then reload, or I should do it on the system itself and
> save the core, let the core grow with this way and forget all about
> the source files.
> 
> 
> Please let me make it more clear,
> 
> Method 1:
>     While the program is running, and you find a bug. Fix it and
> record the changes onto the source file. And then reload the source to
> make sure the changes are right in place.

Method 1a:

While the program is running, and you find a bug. Fix it and record
the changes onto the source file. And then send the changed forms to
your Lisp to be evaluated.

Your Lisp environment (either Emacs or the built in editor of your
Lisp's IDE) should have an obvious way to "send" a form to Lisp. For
example in Emacs with Allegro Common Lisp (using their ELI emacs
integration) I make a change to a function definition and type C-c C-x
which sends the form to Lisp to be compiled, just as if I had typed it
at the REPL, and then called COMPILE.

> Method 2:
>     Locate the bug, fix it and keep the system running. Forget all
> about the source files because the changes are going to record to the
> core image when the system do a (save-core). So the whole program will
> build base on the core file.

It's not clear that you want to do this unless you have a way to get
back the source code. At least I wouldn't. As I understand it,
Interlisp (one of the main Common Lisp predecessors) worked very much
this way (Googling for Lisp and "structure editor" may find something)
but there was a way to get back to source which most Common Lisp these
days don't support, that I know of.

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Stefan Scholl
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <14nyniut6mkmz$.dlg@parsec.no-spoon.de>
On 2003-11-06 07:33:18, Willy wrote:

> But the problem is, when I do a fix, should I do it on the program
> source files, then reload, or I should do it on the system itself and
> save the core, let the core grow with this way and forget all about
> the source files.

Both. :-)

The IMHO best way is to use screen
<http://www.gnu.org/software/screen/> + emacs + ILISP + some
supported Lisp implementation.

When it's some kind of server program you can log on to the server,
reattach the screen and continue using emacs. Just change some lines
of code, save and reevaluate the changed parts of the code.
From: Pascal Bourguignon
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <87u15hnc8s.fsf@thalassa.informatimago.com>
··········@yahoo.com (Willy) writes:
> Method 1:
>     While the program is running, and you find a bug. Fix it and
> record the changes onto the source file. And then reload the source to
> make sure the changes are right in place.
> 
> Method 2:
>     Locate the bug, fix it and keep the system running. Forget all
> about the source files because the changes are going to record to the
> core image when the system do a (save-core). So the whole program will
> build base on the core file.


Method 3 (like in smalltalk):
    Locate the bug, fix it and keep the system running.
    At a convienient time, dump the sources to external text files.

This imply that you keep the sources around in the image, not only the
sexp, because you may want to keep the comments and other reader input
too.  Some work on the reader and the REPL would be needed.


    
-- 
__Pascal_Bourguignon__
http://www.informatimago.com/
From: Espen Vestre
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <kwr80l3lcp.fsf@merced.netfonds.no>
··········@yahoo.com (Willy) writes:

> But the problem is, when I do a fix, should I do it on the program
> source files, then reload, or I should do it on the system itself and
> save the core, let the core grow with this way and forget all about
> the source files.

Here's what I do when I fix bugs in my servers:

1) If the fix is really, really urgent, I just fix it in the running
   server and think about the source files later :-)

2) If it's a simple fix, I just record it in my "patch-server.lisp"
   file which the servers consult upon startup.

3) If I have a little more time and the changes are somewhat
   substantial, I do the following:

   a) Fix it in the source files and interactively try it out in my
      test server (which runs inside the LispWorks IDE)

   b) Check the changes into CVS and tag it with a version number

   c) Export the CVS snapshot that the server is based on

   d) Automagically create a s-expression level diff between the new
      version and that snapshot, check it and possibly do some manual
      amendments, then compile it

   e) Load the compiled files into the running servers

Alt. 3 may sound like a lot of work, but it is actually not as bad at
it sounds: Mostly the cvs export is already there, the tagging is done 
almost automaticaly from lisp, etc., so sometimes the diff fasl-file
is produced in a minute or less.
-- 
  (espen)
From: Will Hartung
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <boehi5$1cefu8$1@ID-197644.news.uni-berlin.de>
"Espen Vestre" <·····@*do-not-spam-me*.vestre.net> wrote in message
···················@merced.netfonds.no...
>    e) Load the compiled files into the running servers
>
> Alt. 3 may sound like a lot of work, but it is actually not as bad at
> it sounds: Mostly the cvs export is already there, the tagging is done
> almost automaticaly from lisp, etc., so sometimes the diff fasl-file
> is produced in a minute or less.

How do you handle any potential synchronization problems with running the
updated code on the running server, particularly if it's changing code mid
transaction. Seems to me that this is a potentially nasty race condition.

Do you actually do anything about it? Or do you simply punt and not worry
about it at all?

Regards,

Will Hartung
(·····@msoft.com)
From: Espen Vestre
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <kw4qxgwmjy.fsf@merced.netfonds.no>
"Will Hartung" <·····@msoft.com> writes:

> How do you handle any potential synchronization problems with running the
> updated code on the running server, particularly if it's changing code mid
> transaction. Seems to me that this is a potentially nasty race condition.
> 
> Do you actually do anything about it? Or do you simply punt and not worry
> about it at all?

Well, I guess one useful trick is to update the software _often_, to
make each patch so small that its effects on running code is
relatively easy to foresee. But of course, sometimes you have to
restructure your code in a way that can make running code break (say
you have a function f that calls another function g and the old
version of f can't call the new version of g). But even major
restructuring is often done in backwards-compatible ways which will
make the old code finish happily. (When developing server software you
have backwards-compatibility in mind all the time anyway, since you
want old clients to be able to use the new server, except if you're a
C++ CORBA-head of course).
-- 
  (espen)
From: Tim Daly Jr.
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <87oevo5wf8.fsf@simple.intern>
Espen Vestre <·····@*do-not-spam-me*.vestre.net> writes:

        ...
>    d) Automagically create a s-expression level diff between the new
>       version and that snapshot, check it and possibly do some manual
>       amendments, then compile it
        ...

Could you clue me in on a good way to create a sexp-level diff?  That
would be handy.

-Tim


-- 
Man must shape his tools lest they shape him. -- Arthur R. Miller
From: Espen Vestre
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <kwhe1gv5w1.fsf@merced.netfonds.no>
···@tenkan.org (Tim Daly Jr.) writes:

> >    d) Automagically create a s-expression level diff between the new
> >       version and that snapshot, check it and possibly do some manual
> >       amendments, then compile it
>         ...
> 
> Could you clue me in on a good way to create a sexp-level diff?  That
> would be handy.

I don't have any revolutionary code for this. 
See this recent article: <··············@merced.netfonds.no>
-- 
  (espen)
From: Alan Crowe
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <86u15elpkj.fsf@cawtech.freeserve.co.uk>
I'm also learning Lisp. I've only been writing student
programs so far. I've been using Emacs as a front end to
CMUCL. I do "ctrl-x 3" to split my screen vertically. On the
right I do ctrl-x ctrl-f to fetch my source file. On the
left I do meta-x run-lisp to start CMUCL.

I start of with making some planned changes. I save the
source. I (load "source.lisp") to read the whole file into
Lisp.

As I find bugs, I fix the source file, then I send the new
definition across to CMUCL with crtl-meta-x, and try it out.
At this point the source and image are still synchronised.

When I hit trouble I type at the REPL. I try stuff
out. Maybe a new defun. If I'm on the wrong track, no
problem, I've not mucked up my source file. If I realise
that the new defun actually belongs in the source file, that
is not a problem either. The Emacs inferior Lisp window
keeps all the history. I carefully cut and past the final
version of the defun into the source file. No double typing.

Perhaps more experience Lisp users will be horrified by my
style of working. I'll post and find out :-)

Alan
From: mikel
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <vKhrb.25541$wD2.24741@newssvr27.news.prodigy.com>
Alan Crowe wrote:
> I'm also learning Lisp. I've only been writing student
> programs so far. I've been using Emacs as a front end to
> CMUCL. I do "ctrl-x 3" to split my screen vertically. On the
> right I do ctrl-x ctrl-f to fetch my source file. On the
> left I do meta-x run-lisp to start CMUCL.
> 
> I start of with making some planned changes. I save the
> source. I (load "source.lisp") to read the whole file into
> Lisp.
> 
> As I find bugs, I fix the source file, then I send the new
> definition across to CMUCL with crtl-meta-x, and try it out.
> At this point the source and image are still synchronised.
> 
> When I hit trouble I type at the REPL. I try stuff
> out. Maybe a new defun. If I'm on the wrong track, no
> problem, I've not mucked up my source file. If I realise
> that the new defun actually belongs in the source file, that
> is not a problem either. The Emacs inferior Lisp window
> keeps all the history. I carefully cut and past the final
> version of the defun into the source file. No double typing.
> 
> Perhaps more experience Lisp users will be horrified by my
> style of working. I'll post and find out :-)

That is a mighty fine way to do interactive development.

Lately I've been doing a bunch of work on Cocoa (Mac OS X) apps in 
Common Lisp. For reasons too grungy to get into, it is not particularly 
convenient to build a Cocoa app that is easy to start from an inferior 
lisp in Emacs. So I turn that paradigm inside out.

My Cocoa apps are lisp images that support remote telnet connections 
(using Sven Van Caekenberghe's very useful remote-repl hack for 
openmcl). So I can start up my work-in-progress and connect to it on a 
socket from Emacs (nothing fancy right now; I just use an inferior 
shell). So that way the Cocoa app is running and looking all Mac OS 
X-native, and meanwhile I'm clanking around in its belly in the remote 
repl, inspecting data structures and redefining defun and the like. And 
as you say, when I get something good I can copy it back into the source 
file used to build it.

Crude but effective.
From: KanZen
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <10eb079f.0311091114.70a309b6@posting.google.com>
Alan Crowe <····@cawNOtech.freeSPAMserve.co.uk> wrote in message news:<··············@cawtech.freeserve.co.uk>...
> I'm also learning Lisp. I've only been writing student
> programs so far. I've been using Emacs as a front end to
> CMUCL. I do "ctrl-x 3" to split my screen vertically. On the
> right I do ctrl-x ctrl-f to fetch my source file. On the
> left I do meta-x run-lisp to start CMUCL.
> 
> I start of with making some planned changes. I save the
> source. I (load "source.lisp") to read the whole file into
> Lisp.
> 
> As I find bugs, I fix the source file, then I send the new
> definition across to CMUCL with crtl-meta-x, and try it out.
> At this point the source and image are still synchronised.

Okay, this is what I (as a beginner) have been doing. So how do we
attach to a CMUCL process that it already running? (e.g. maybe it's
a web server or something) Is there some tricky emacs/ilisp command?
From: Daniel Barlow
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <87smkxjmjl.fsf@noetbook.telent.net>
······@mail.com (KanZen) writes:

> Okay, this is what I (as a beginner) have been doing. So how do we
> attach to a CMUCL process that it already running? (e.g. maybe it's
> a web server or something) Is there some tricky emacs/ilisp command?

There's no support in ilisp for this directly.  You could start the
process from screen or detachtty, or have the server run a listener on
a tcp port and connect to it with telnet, or you could try SLIME
instead of ilisp: it does all its talking to lisp on a socket instead
of using standard io channels.  It's early days yet for slime, though.

I like detachtty, because I wrote it - or perhaps, vice versa.
<http://www.cliki.net/detachtty>, or use the Debian packages.


-dan

-- 

   http://web.metacircles.com/cirCLe_CD - Free Software Lisp/Linux distro
From: Håkon Alstadheim
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <m0y8uodyx8.fsf@alstadhome.dyndns.org>
Daniel Barlow <···@telent.net> writes:

> ······@mail.com (KanZen) writes:
>
>> Okay, this is what I (as a beginner) have been doing. So how do we
>> attach to a CMUCL process that it already running? (e.g. maybe it's
>> a web server or something) Is there some tricky emacs/ilisp command?
>
> There's no support in ilisp for this directly.  You could start the
> process from screen or detachtty, or have the server run a listener on
> a tcp port and connect to it with telnet, or you could try SLIME
> instead of ilisp: it does all its talking to lisp on a socket instead
> of using standard io channels.  It's early days yet for slime, though.
>
> I like detachtty, because I wrote it - or perhaps, vice versa.
> <http://www.cliki.net/detachtty>, or use the Debian packages.

The non-obvious (for a newbie) wanting to use ilisp with detachtty is
that you can put something like this in your .emacs:
 ---
(require 'ilisp)
(setq cmua-program "attachtty /var/run/cld/socket")
(defdialect cmua "attach to cmucl running as cld" cmulisp)
 ---

This gives me a new inferior lisp mode which I invoke with M-x cmua. I
use it whenever I'm working on my server app. This idea can be
extended to any number of different servers. Just remember to do M-x
<the-inferior-lisp-you-want> whenever you start working on the source
of a different project. This will make ilisp use the correct
lisp-listener when evaluating stuff.



-- 
H�kon Alstadheim, hjemmepappa.
From: Daniel Barlow
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <87k768gdnw.fsf@noetbook.telent.net>
······@online.no (H�kon Alstadheim) writes:

> The non-obvious (for a newbie) wanting to use ilisp with detachtty is
> that you can put something like this in your .emacs:
>  ---
> (require 'ilisp)
> (setq cmua-program "attachtty /var/run/cld/socket")
> (defdialect cmua "attach to cmucl running as cld" cmulisp)

Yeah.  I originally wrote detachtty with that in mind, but then found
that I didn't trust ilisp not to send something exciting at startup
when connecting to my production services: a confused ilisp during
development is annoying but not going to affect anyone else very much,
but a confused ilisp on a public server could easily lead to the end
of the world (pun intended).  So I tend to use attachtty either from
the command line or in a plain vanilla comint.

Also I like to run emacs locally and access files across the network
with tramp, and didn't fancy working out how to make M-. understand
that the filenames it should be editing should be prefixed with
········@www.example.com: or something.


-dan

-- 

   http://web.metacircles.com/cirCLe_CD - Free Software Lisp/Linux distro
From: Tim Lavoie
Subject: Re: Proper way to do Interactive Development
Date: 
Message-ID: <87d6c0u6e5.fsf@theasylum.dyndns.org>
>>>>> "kanzen" == kanzen  <······@mail.com> writes:

    kanzen> Okay, this is what I (as a beginner) have been doing. So
    kanzen> how do we attach to a CMUCL process that it already
    kanzen> running? (e.g. maybe it's a web server or something) Is
    kanzen> there some tricky emacs/ilisp command?

If you start the CMUCL process with a utility such as detachtty or
screen, then you can detach from a running session and reattach later,
without the original process ever knowing that you'd gone.

-- 
Teach children to be polite and courteous in the home, and, when they grow up,
they won't be able to edge a car onto a freeway.