From: zenshade
Subject: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <1126201194.210973.298500@f14g2000cwb.googlegroups.com>
Here's the problem I am trying to solve.  I'm a junior programmer
working for a small ASP company that provides a webapp for a number of
energy companies in the United States.  Our architecture is
J2EE/Oracle.  The company has been moderately successful up to this
point, but is now poised to completely dominate this particular market
that our web app services.  All the signs are there that this may begin
to happen within a couple of months, and when the snowball starts
rolling it's going to be moving very fast.  The problem is, we do not
presently have a good enough support and testing infrastructure to
handle even the moderate growth we are now experiencing.

Let me give a few more details to paint a clearer picture.  Our basic
business model is that we provide a core app which every new customer
gets.  Then we have "features" that we sell to customers at extra cost.
 New features development is typically driven by a customer coming to
us and saying we really need the app to do X.  So we develop X, and
then either make it part of the core, or repackage it as a sellable
feature to other customers.  Now, this process has been going on for a
couple of years through about 20 different customers, with a new
customer being added about every 2 months, so as you can imagine, the
"feature creep" in the core has gotten rather pronounced and every
customer has their own set of unique functionality that is not in the
core.

Apart from the enormous development headaches that ensue from being
hand-tied by J2EE (a dragon I'm not yet in a position to slay), our
biggest difficulties are in support and testing.  Believe it or not,
both of these activities are conducted in a mostly ad hoc, almost word
of mouth manner.  There never was a good documentation process put in
place to map features to customers and tests to features.  These things
do get documented, but in an inconsistent manner that is spread out in
different directories and not easy to find.

Typical scenario: Feature X gets developed for customer Y.  A tester is
assigned to test it.  His basic instructions are "make sure it works".
He plays with it for awhile.  If he's good, he checks the edge cases
and finds the bugs.  He adds an entry into a testing spreadsheet that
amounts to not much more than saying "check feature X for customer Y"
for future regression testing.

Couple of months later.  Customer Y calls and says feature X does not
appear to be working.  Support person: "Let me look into it and I'll
call you back".  Because he doesn't know how it's supposed to work.  He
realizes he's going to have to dig for that info.  If the existing
documentation doesn't suffice (and it usually doesn't), he'll have to
go ask the developer and/or test it and figure it out for himself,
possibly even having to dig into the code just to see what is supposed
to happen.  The same type of scenario happens repeatedly for simple
things like configuration issues.

What I find completely amazing is that this system of support/testing
hasn't completely collapsed already.  Being somewhat in the middle
between the support/testers and development, however, I can see the
cracks in the system getting wider every day.  There probably should
have been a significant investment in something like LoadRunner a long
time ago.  But there wasn't.  Knowing the CEO/owner, there won't be,
either, at least not until the problems become so critical that
customers start dumping the product.  If nothing is done know, what I
foresee happening is at some point in the not too distant future lots
of new bodies will be thrown at the problem.  Which will of course make
matters significantly worse without an adequate training/documentation
framework in place.  Not a company I want to be working for.

So I find myself in the position of being able to make a very
significant contribution both to the company and my own future working
environment if I can find a way to solve these difficulties.  What I
have envisioned doing is developing an integrated application that will
track customer features and tie them to test cases and test results,
and also serve as a central location for documentation of functionality
and configuration options.  In an ideal world, developers would be
writing the test cases when features are developed, but I know that
isn't going to happen in my company.  So the testers have to be the
ones filling in the test details.  And whoever writes the feature specs
(sometimes a manager, sometimes me) has to fill in the functionality
descriptions.  The app would enforce these activities and ensure the
documentation gets stored in an easy and quick to find manner.
Additionally, the app would also kick off automated web testing based
on a minilanguage the testers would use.  I've already written a test
engine that automates IE as a COM object.  The app would just be
driving the engine and collecting results.

Having used lisp and java for a couple of years now, I'm pretty sure
using lisp is the only way I'm going to be able to tackle a project of
this size and pull it off on my own in the short amount of time that I
have (a couple of months, at most).  I have a fairly clear picture in
my mind of how the interface needs to work.  What I don't seem to be
able to get a good grasp on is a good model for relating dynamic and
static elements of an application.  Static elements would be things
like configuration options and web page elements.  Dynamic elements
could either be a single feature, or the interactions of a set of
features, with features being a set of static elements that change in
time.

I'm very interested in getting the opinions of others who may have have
used lisp to tackle similar problems, and what kinds of models they
used.  My ultimate hope is that if I can pull this off in lisp, I may
have the opportunity to use lisp for other webapps when we attempt to
expand into other markets.  Nirvanna.

From: Duane Rettig
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <4hdcv8f4n.fsf@franz.com>
"zenshade" <········@gmail.com> writes:

 [ ... ]

> Apart from the enormous development headaches that ensue from being
> hand-tied by J2EE (a dragon I'm not yet in a position to slay), our
> biggest difficulties are in support and testing.  Believe it or not,
> both of these activities are conducted in a mostly ad hoc, almost word
> of mouth manner.  There never was a good documentation process put in
> place to map features to customers and tests to features.  These things
> do get documented, but in an inconsistent manner that is spread out in
> different directories and not easy to find.

 [ ... ]

> Having used lisp and java for a couple of years now, I'm pretty sure
> using lisp is the only way I'm going to be able to tackle a project of
> this size and pull it off on my own in the short amount of time that I
> have (a couple of months, at most).  I have a fairly clear picture in
> my mind of how the interface needs to work.  What I don't seem to be
> able to get a good grasp on is a good model for relating dynamic and
> static elements of an application.  Static elements would be things
> like configuration options and web page elements.  Dynamic elements
> could either be a single feature, or the interactions of a set of
> features, with features being a set of static elements that change in
> time.

There are aspects of this that require more complexity than a simple
test harness can provide.  However, for the pieces that _can_ be
tested automatically by a test harness, you might look here:

http://opensource.franz.com/test/index.html

It is all lisp code, and does not have any of the higher-level option
configurations that you're looking for, but it may get you started.

> I'm very interested in getting the opinions of others who may have have
> used lisp to tackle similar problems, and what kinds of models they
> used.  My ultimate hope is that if I can pull this off in lisp, I may
> have the opportunity to use lisp for other webapps when we attempt to
> expand into other markets.  Nirvanna.

We try to automate as much as we can.  What automation we accomplish
finds regressions immediately.  For other pieces that can't be tested
automatically, like timing issues, a QA person is needed.  The QA is
necessarily different than the developer, and there is always some
tension between the two - a developer can never adequately QA his
own product.  When no QA exists at the company, the customers become
the QA, and that tension results in less than desirable customer
relations.  For configuration-sensitive issues, we set up as many
of these scenarios as possible, but it is impossible to set all of
the possible combinations up; in that case the best way to ensure
that things are working is to "eat your own dog food", i.e. become
a consumer of your own product.

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Joe Marshall
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <slwf454p.fsf@alum.mit.edu>
"zenshade" <········@gmail.com> writes:

> I'm a junior programmer working for a small ASP company that
> provides a webapp for a number of energy companies in the United
> States.  The problem is, we do not presently have a good enough
> support and testing infrastructure to handle even the moderate
> growth we are now experiencing.  There probably should have been a
> significant investment in something like LoadRunner a long time ago.
> But there wasn't.  Knowing the CEO/owner, there won't be, either, at
> least not until the problems become so critical that customers start
> dumping the product.

> What I have envisioned doing is developing an integrated application
> that will track customer features and tie them to test cases and
> test results, and also serve as a central location for documentation
> of functionality and configuration options.  My ultimate hope is
> that if I can pull this off in lisp, I may have the opportunity to
> use lisp for other webapps when we attempt to expand into other
> markets.

You're screwed.

Your configuration and test database is a good idea, but the problem
your company has isn't technological in nature.  It is clear from your
description that the CEO is unable to manage a software product.
Providing him with better tools will not fix this.

Unfortunately, the most likely future is the one you outline.  The
company will stagger along with an ever more unwieldy product until
you are unable to get more customers because you haven't the resources
to support the existing ones.  At some point, the competition will
simply buy you out.

But the configuration management/regression test tracking idea that
you have is well worth pursuing.  Don't put your hopes on it saving
the company, though.

~jrm
From: zenshade
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <1126240593.668145.231060@g44g2000cwa.googlegroups.com>
Joe Marshall wrote:
> You're screwed.

Thanks for the encouragement. :)

> Your configuration and test database is a good idea, but the problem
> your company has isn't technological in nature.  It is clear from your
> description that the CEO is unable to manage a software product.
> Providing him with better tools will not fix this.
>
> Unfortunately, the most likely future is the one you outline.  The
> company will stagger along with an ever more unwieldy product until
> you are unable to get more customers because you haven't the resources
> to support the existing ones.  At some point, the competition will
> simply buy you out.

You are probably right in the long run, but the sheer number of
contracts presently lined up pretty much guarantees this company will
stagger along incompetently for at least a couple of more years.  That,
and an almost complete lack of real competition.  The "competition" is
using visual basic and C++ and has generally done a terrible job at
finding out and meeting customer needs.  They are years behind us, and
their language choices will likely ensure they stay there.  It has
crossed my mind many times that a company using a dynamic language (be
it lisp, python or even perl) could clean our clocks with a far more
flexible and easier to use application within about three months.  It
has even occurred to me that my best option may be to find another job,
wait out my 2 year non-compete, and then do precisely that.

> But the configuration management/regression test tracking idea that
> you have is well worth pursuing.

Thanks for the real encouragement.  Even if it ends up being nothing
more than a resume stuffer, the time will be well spent (and loads of
fun!).
From: Sylvain
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <ZeWdnacCS4jVtLzeRVn-1A@speakeasy.net>
zenshade wrote:
> ...  It has
> crossed my mind many times that a company using a dynamic language (be
> it lisp, python or even perl) could clean our clocks with a far more
> flexible and easier to use application within about three months.  It
> has even occurred to me that my best option may be to find another job,
> wait out my 2 year non-compete, and then do precisely that.

considering the nature of this discussion I wouldn't want to
ask too specific a question,  but would you happen to work in
California (not a completely unlikely guess for a high tech
job)?  if this is the case,  I suggest you look into the
California Business and Professions Code,  Section 16600
(consider consulting with a lawyer) -- other states have
similar regulations by the way;  then keep a keen eye on
potential competitors using the right technology (i.e., keep
checking out their 'careers' ads -- from home, of course :-))
and polish up the good ol' resume :-)

--Sylvain
From: Joe Marshall
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <u0gujb5c.fsf@alum.mit.edu>
"zenshade" <········@gmail.com> writes:

> Joe Marshall wrote:
>> You're screwed.
>
> Thanks for the encouragement. :)

I apologize.  I don't want to discourage you, but I've been in similar
situations numerous times and it has always gone poorly.  Rather than
be discouraged, however, I suggest planning for the inevitable.
From: Pascal Bourguignon
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <87vf1bbf1c.fsf@thalassa.informatimago.com>
"zenshade" <········@gmail.com> writes:
> I'm very interested in getting the opinions of others who may have have
> used lisp to tackle similar problems, and what kinds of models they
> used.  My ultimate hope is that if I can pull this off in lisp, I may
> have the opportunity to use lisp for other webapps when we attempt to
> expand into other markets.  Nirvanna.

Think "regression testing".  Have them tester provide not a check into
Excell, but a script with an expected output file.

Then you can automatically run the scripts and diff the expected output
against the actual output, and generate a test report for each
configuration.


Of course, it's best to write these test scripts in CL.  For one
thing, you can apply profitably CLOS and build a nice inheritance tree
for most tests are similar or share the same infrastructure and the
same interfaces to the application.

Once upon a time, I did that in emacs lisp (with eieio CLOS look
alike).  But CL will be better. 


If you must test a GUI, you can attack the application either adding
scriptability  (AppleEvents, Distibuted Objects, etc), or use a
scriptable event recording/playback tool.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
From: Peter Seibel
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <m2zmqn5i2m.fsf@gigamonkeys.com>
"zenshade" <········@gmail.com> writes:

> Having used lisp and java for a couple of years now, I'm pretty sure
> using lisp is the only way I'm going to be able to tackle a project
> of this size and pull it off on my own in the short amount of time
> that I have (a couple of months, at most).  I have a fairly clear
> picture in my mind of how the interface needs to work.  What I don't
> seem to be able to get a good grasp on is a good model for relating
> dynamic and static elements of an application.  Static elements
> would be things like configuration options and web page elements.
> Dynamic elements could either be a single feature, or the
> interactions of a set of features, with features being a set of
> static elements that change in time.
>
> I'm very interested in getting the opinions of others who may have
> have used lisp to tackle similar problems, and what kinds of models
> they used.  My ultimate hope is that if I can pull this off in lisp,
> I may have the opportunity to use lisp for other webapps when we
> attempt to expand into other markets.  Nirvanna.

While Joe Marshall is likely right that your company is just screwed,
it's also possible that the main reason this bug/test-tracking
database hasn't happened yet is that nobody who cares about it has the
time or energy to do it. But here you are--you care and you believe
you have the time and energy. (Of course you have a secret
weapon--Lisp!) So I say go for. Worst case scenario, is you learn
something before the company goes under. (Unless of course there's
something else you're supposed to be doing in which case worst case is
that you get fired for not doing your actual job.)

I'd suggest you write to Franz and tell them you want to write this
stealth app to introduce Lisp into your company and ask them if you
can get on on the Allegrocache beta. Then build a decent object model
of the problem--you have features, customers, tests (probably not a
lot of those just yet), and configurations (or whatever). Build
objects to represent the actual knowledge about your company's
software and store them in Allegrocache. Then write some programs to
do things that will help your support folks and/or developers do a
better job--for instance, if you store whatever documentation is
available for each feature in the feature objects you could use
Allegroserve to provide a web interface for browsing that
documentation. And if you can gather up the information about what
customers have what non-core features you can provide another web
interface for browsing from a particular customer to their associated
problem reports and from there to the information about the features
they have.

However, you should expect that you're going to have to do a fair bit
of the grunt work of gathering and entering data, in addition to
actually writing the tool, in order to get it into a state where it's
value is obvious to the other folks in your company. And then you may
only be able to get a few folks, probably some support folks who are
feeling the most pain under the current (non-) system, interested. So
make the tool useful for them and they'll start entering data that
will make the app more valuable. Then you can branch out and add some
features that make it actually useful for the developers. Eventually
you'll have everyone using it and the CEO will want to claim it was
his idea all along. But that's okay--you'll know and we'll know where
it really came from. (And he may try and buy your silence with a nice
bonus or something.)

-Peter

-- 
Peter Seibel           * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp  * http://www.gigamonkeys.com/book/
From: zenshade
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <1126243518.872871.202920@g43g2000cwa.googlegroups.com>
Peter Seibel wrote:


> While Joe Marshall is likely right that your company is just screwed,
> it's also possible that the main reason this bug/test-tracking
> database hasn't happened yet is that nobody who cares about it has the
> time or energy to do it. But here you are--you care and you believe
> you have the time and energy. (Of course you have a secret
> weapon--Lisp!) So I say go for. Worst case scenario, is you learn
> something before the company goes under. (Unless of course there's
> something else you're supposed to be doing in which case worst case is
> that you get fired for not doing your actual job.)

I've already realized that this is a project I will probably have to
tackle largely on my own time.  As you say, if anyone in the company
believed it was important, someone would have already allocated the
resources to do it.  I think the biggest reason it hasn't happened is
the CEO and CTO look at that kind of project and try to extrapolate
from their J2EE experience how much effort it would take.  They
conclude the resource expenditure is about an order of magnitude more
than they can tolerate.  They simply can't believe it could be
accomplished an order of magnitude more cheaply.  I know better.

> Then write some programs to
> do things that will help your support folks and/or developers do a
> better job--for instance, if you store whatever documentation is
> available for each feature in the feature objects you could use
> Allegroserve to provide a web interface for browsing that
> documentation. And if you can gather up the information about what
> customers have what non-core features you can provide another web
> interface for browsing from a particular customer to their associated
> problem reports and from there to the information about the features
> they have.

Good ideas.

> However, you should expect that you're going to have to do a fair bit
> of the grunt work of gathering and entering data, in addition to
> actually writing the tool, in order to get it into a state where it's
> value is obvious to the other folks in your company. And then you may
> only be able to get a few folks, probably some support folks who are
> feeling the most pain under the current (non-) system, interested. So
> make the tool useful for them and they'll start entering data that
> will make the app more valuable. Then you can branch out and add some
> features that make it actually useful for the developers.

Actually, if I can map out a very clear object hierarchy and hand over
a working app, there's a decent chance the CTO would get behind it and
enforce it's use.  He's actually a very competent guy (albeit within
the narrow confines of J2EE) who wrote most of the webapp now poised to
take over the market.  I might even be able to approach him with just
the object model and convince him to let me work on it on company time.
 But I'm not going to do that because of one over-riding fear.  Not
that he'd say no. That he'd say yes, but do it in J2EE. ARRRGG.  Death
of a thousand small cuts.

> Eventually
> you'll have everyone using it and the CEO will want to claim it was
> his idea all along. But that's okay--you'll know and we'll know where
> it really came from. (And he may try and buy your silence with a nice
> bonus or something.)

Yep, I can see that happening. :)

BTW, I like your book a lot.  Haven't bought it yet, but it is
definitely on my list.  Chapter 3 alone is worth the price of
admission.  It's what finally made me say to myself, "Oh, that's why
they speak of macros with such reverential tones!"
From: Tim X
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <871x3yn5pf.fsf@tiger.rapttech.com.au>
"zenshade" <········@gmail.com> writes:

> Peter Seibel wrote:
> 
> 
> 
> > However, you should expect that you're going to have to do a fair bit
> > of the grunt work of gathering and entering data, in addition to
> > actually writing the tool, in order to get it into a state where it's
> > value is obvious to the other folks in your company. And then you may
> > only be able to get a few folks, probably some support folks who are
> > feeling the most pain under the current (non-) system, interested. So
> > make the tool useful for them and they'll start entering data that
> > will make the app more valuable. Then you can branch out and add some
> > features that make it actually useful for the developers.
> 
> Actually, if I can map out a very clear object hierarchy and hand over
> a working app, there's a decent chance the CTO would get behind it and
> enforce it's use.  He's actually a very competent guy (albeit within
> the narrow confines of J2EE) who wrote most of the webapp now poised to
> take over the market.  I might even be able to approach him with just
> the object model and convince him to let me work on it on company time.
>  But I'm not going to do that because of one over-riding fear.  Not
> that he'd say no. That he'd say yes, but do it in J2EE. ARRRGG.  Death
> of a thousand small cuts.
> 

Just wanted to echo Peter's warning. I've been in exactly this
situation. I also managed, once the app was done, to convince
management of its value - which wasn't too hard once they saw the
system in place. However, the whole thing still nearly failed because
the other staff just didn't see anything but more work for them to
do. A few threatened sackings later and after a few months, we had
enough data in the system to allow anyone to see it was more than just
additional administration overhead and was of real value. 

I'm now working in my own time on a tool to assist us manage things
with my new employer. This time, in addition to the actual design of
the application, I'm actually designing with an eye on how we can
automate data gathering and maintenance as much as possible. 

Oh, and I'm doing it in Lisp using cl-sql, postgres, aserve and
html-gen, though I may also move to html-template. Progress so far has
been very slow, but that is mainly me still coming to grips with Lisp
- the basics are OK, its more knowing what is available within CL and
hesitation on my own part due mainly to my still developing confidence
with CL. I find I spend a little too much time second guessing myself
rather than just getting in and coding - not a Lisp specific weakness
- I had the same problem at first with most new languages I've
learnt. Still, its been a lot of fun so far and I'm doing it in my own
time so that I can do it with Lisp - if I convinced management to do
it, I too would run the risk of being told its a great idea, but don't
use Lisp, use Java, Oracle and PL/SQL. I'm also using this as an
opportunity to factor in some text-to-speech functionality. I'll
probably disable this or make it an option in the end, but its fun to
add this new dimension and has the potential of giving it a slight
'wow' factor, which is often useful. 

Tim

-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!
From: Ray Dillinger
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <GqEUe.13021$p%3.54949@typhoon.sonic.net>
zenshade wrote:

> What I find completely amazing is that this system of support/testing
> hasn't completely collapsed already.  Being somewhat in the middle
> between the support/testers and development, however, I can see the
> cracks in the system getting wider every day.  There probably should
> have been a significant investment in something like LoadRunner a long
> time ago.  But there wasn't.  Knowing the CEO/owner, there won't be,
> either, at least not until the problems become so critical that
> customers start dumping the product.  If nothing is done know, what I
> foresee happening is at some point in the not too distant future lots
> of new bodies will be thrown at the problem.  Which will of course make
> matters significantly worse without an adequate training/documentation
> framework in place.  Not a company I want to be working for.

Okay, you have a substantial problem here; this is a software
QA crisis in the making, especially as your company gets bigger.

When a company is small, and the programmers are all in the same
room or just down the hall from each other, (and provided there
is very low programmer turnover) you don't really need major
organization to keep the QA thing under control; informal "track
down the guy who did this and ask him about it" or "run around
documenting things that aren't documented" kinds of efforts
actually are the most efficient methods.  But they don't scale.
If the company tries to grow, and especially when rapidly hiring
and trying to orient programmers, the lack of major, formal
organization and central repositories of institutional
knowledge will really _really_ hurt.

Your CTO (or maybe a better one or one with more power to
make impressions about what's needed on the CEO/owner) can
still save your company;  It's unlikely that anyone else can.

The application you are considering will help; but remember,
it will *slow down* the QA effort in its first few months
of adoption.  Once it's "up to speed" it will start to help,
but there's an initial period where it's going to have to
be pushed.  In other words, it will work only if people who
are in the middle of a crisis that requires all their time
can "spare" the time to use it and update it and fill it
with the documentation and cases and scripts and scattered
resources it needs.

And the chances of that, even if you had it in hand today
and it was bugfree, are slim.  it could happen if your
CEO/CTO got behind it and rode people's butts to use it,
starting well before the crisis. But I wouldn't bet on
that happening.  By the time the crisis is visible at
its full scale, it will most likely be too late for the
application to help.

In order for a company that provides software to sustain
rapid growth, it has to be *ready* for rapid growth, and
that readiness takes the form of having software that has
been planned and implemented with Documentation, QA and
debugging capabilities in mind and having a coherent,
integrated, documented, and ideally automated QA testing
suite.

				Bear
From: Larry Clapp
Subject: Re: Building a framework in lisp for testing and feature documentation?
Date: 
Message-ID: <slrndi8go5.3tg.larry@theclapp.ddts.net>
On 2005-09-08, zenshade <········@gmail.com> wrote:
[Much snippage]
> [...] our biggest difficulties are in support and testing.  Believe
> it or not, both of these activities are conducted in a mostly ad
> hoc, almost word of mouth manner.  There never was a good
> documentation process put in place to map features to customers and
> tests to features.  These things do get documented, but in an
> inconsistent manner that is spread out in different directories and
> not easy to find.
>
> Typical scenario: Feature X gets developed for customer Y.  A tester
> is assigned to test it.  His basic instructions are "make sure it
> works".  He plays with it for awhile.  If he's good, he checks the
> edge cases and finds the bugs.  He adds an entry into a testing
> spreadsheet that amounts to not much more than saying "check feature
> X for customer Y" for future regression testing.
>
> Couple of months later.  Customer Y calls and says feature X does
> not appear to be working.  Support person: "Let me look into it and
> I'll call you back".  Because he doesn't know how it's supposed to
> work.  He realizes he's going to have to dig for that info.  If the
> existing documentation doesn't suffice (and it usually doesn't),
> he'll have to go ask the developer and/or test it and figure it out
> for himself, possibly even having to dig into the code just to see
> what is supposed to happen.  The same type of scenario happens
> repeatedly for simple things like configuration issues.

This tool sounds kinda handy, though I haven't tried it myself (and I
don't think it uses Lisp):
http://fit.c2.com/wiki.cgi?IntroductionToFit

    Fit is a tool for enhancing communication and collaboration. Fit
    creates a feedback loop between customers and programmers. Fit
    allows customers and testers to use tools like Microsoft Office to
    give examples of how a program should behave--without being
    programmers themselves. Fit automatically checks those examples
    against the actual program, thus building a simple and powerful
    bridge between the business and software engineering worlds. 

    With Fit, customers can provide more guidance in development
    process by lending their subject matter expertise and imagination
    to the effort. The added visibility customers get into what is
    currently happening in a product allows them to steer projects the
    moment they begin to drift off-target.

If nothing else, it gives you a good place to start.

-- Larry