From: William Bland
Subject: Code coverage
Date: 
Message-ID: <pan.2004.05.28.03.26.41.645003@abstractnonsense.com>
I'm currently learning Lisp, using SLIME in Emacs, and SBCL (CMUCL on my
other computer!).  I'm really enjoying it, especially the way SLIME
highlights compiler errors and warnings in the source code.

The way it does that got me to thinking, would it be possible to make
SLIME highlight each line of source code that has been executed, so that
when testing code I could see which lines of code my tests had exercised?

Thinking about it, I believe I know how it could be done.  Keep in mind
that I'm still very much a Lisp newbie and I may have this all wrong -
feel free to set me straight if I have!

I'm thinking, before passing your code to Lisp to be compiled, SLIME could
transform it, adding some extras code that keeps track of what has been
executed.  So it would replace

(defun foo (x)
	(format t "hello!~%")
	(format t "The value is: ~A" x)
	x)

with

(defun foo (x)
	(push 1 *lines-executed*) (format t "hello!~%")
	(push 2 *lines-executed*) (format t "The value is: ~A" x)
	(push 3 *lines-executed*) x)

The transformation could presumably be done by wrapping the code in a
macro?

It would be really neat if, each time you typed something in the repl,
SLIME looked at *lines-executed*, and added highlighting to any lines that
had been executed.  You'd want a way to reset all the lines to an
un-highlit state, after you were finished testing, and a way to tell SLIME
to switch the feature on and off too.

What do people think about this?  I'm almost tempted to try doing it
myself, despite my Lisp newbieness and the fact that the only Emacs
hacking I've ever done is on my own .emacs file!

I have a rather nasty feeling that someone is going to shout at me for not
knowing that Lisp or SLIME already has this built in (pretty much
everything else I've wanted so far has been!).  Ah well, I couldn't find
it with a google so maybe it hasn't been done yet after all.

Cheers,
	Bill.
-- 
Dr. William Bland.
It would not be too unfair to any language to refer to Java as a
stripped down Lisp or Smalltalk with a C syntax.   (Ken Anderson).

From: Kenny Tilton
Subject: Re: Code coverage
Date: 
Message-ID: <EHytc.64780$mX.21053926@twister.nyc.rr.com>
William Bland wrote:

> I'm currently learning Lisp, using SLIME in Emacs, and SBCL (CMUCL on my
> other computer!).  I'm really enjoying it, especially the way SLIME
> highlights compiler errors and warnings in the source code.
> 
> The way it does that got me to thinking, would it be possible to make
> SLIME highlight each line of source code that has been executed, so that
> when testing code I could see which lines of code my tests had exercised?
> 
> Thinking about it, I believe I know how it could be done.  Keep in mind
> that I'm still very much a Lisp newbie and I may have this all wrong -
> feel free to set me straight if I have!
> 
> I'm thinking, before passing your code to Lisp to be compiled, SLIME could
> transform it, adding some extras code that keeps track of what has been
> executed.  So it would replace
> 
> (defun foo (x)
> 	(format t "hello!~%")
> 	(format t "The value is: ~A" x)
> 	x)
> 
> with
> 
> (defun foo (x)
> 	(push 1 *lines-executed*) (format t "hello!~%")
> 	(push 2 *lines-executed*) (format t "The value is: ~A" x)
> 	(push 3 *lines-executed*) x)

But Lispniks do not write lines of code, they write trees of it:

(defun aaa (x)
    (if x
        (when (bbb x)
           (zzz))
      (www)))

> 
> The transformation could presumably be done by wrapping the code in a
> macro?

You mean:

(with-code-coverage ()
  (defun aaa (x)
    (if x
        (when (bbb x)
           (zzz))
      (www)))

icck. Something else?

-- 
Home? http://tilton-technology.com
Cells? http://www.common-lisp.net/project/cells/
Cello? http://www.common-lisp.net/project/cello/
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
Your Project Here! http://alu.cliki.net/Industry%20Application
From: William Bland
Subject: Re: Code coverage
Date: 
Message-ID: <pan.2004.05.28.04.42.07.459009@abstractnonsense.com>
On Fri, 28 May 2004 03:52:04 +0000, Kenny Tilton wrote:
> 
> William Bland wrote:
>> 
>> I'm thinking, before passing your code to Lisp to be compiled, SLIME could
>> transform it, adding some extras code that keeps track of what has been
>> executed.  So it would replace
>> 
>> (defun foo (x)
>> 	(format t "hello!~%")
>> 	(format t "The value is: ~A" x)
>> 	x)
>> 
>> with
>> 
>> (defun foo (x)
>> 	(push 1 *lines-executed*) (format t "hello!~%")
>> 	(push 2 *lines-executed*) (format t "The value is: ~A" x)
>> 	(push 3 *lines-executed*) x)
> 
> But Lispniks do not write lines of code, they write trees of it:
> 
> (defun aaa (x)
>     (if x
>         (when (bbb x)
>            (zzz))
>       (www)))
> 

OK, so couldn't I give each subexpression a label, that SLIME remembers,
and that gets pushed onto *subexpressions-evaluated* when the
corresponding subexpression gets evaluated?

So:

(progn (push 1 *subexpressions-evaluated*)
       (defun aaa (x)
	 (progn (push 2 *subexpressions-evaluated*)
		(if (progn (push 3 *subexpressions-evaluated*) x)
		    (progn (push 4 *subexpressions-evaluated*)
			   (when (progn (push 5 *subexpressions-evaluated*)
					(bbb x))
			     (progn (push 6 *subexpressions-evaluated*) (zzz)))
			   (progn (push 7 *subexpressions-evaluated*) (www)))))))

Wouldn't that work?  I did it by hand, but I think I could write the code
to do it...

Cheers,
	Bill.
-- 
Dr. William Bland.
It would not be too unfair to any language to refer to Java as a
stripped down Lisp or Smalltalk with a C syntax.   (Ken Anderson).
From: Svein Ove Aas
Subject: Re: Code coverage
Date: 
Message-ID: <HqCtc.4424$eH3.77878@news4.e.nsc.no>
William Bland wrote:

> On Fri, 28 May 2004 03:52:04 +0000, Kenny Tilton wrote:
>> 
>> William Bland wrote:
>>> 
>>> I'm thinking, before passing your code to Lisp to be compiled, SLIME
>>> could transform it, adding some extras code that keeps track of what
>>> has been
>>> executed.  So it would replace
>>> 
>>> (defun foo (x)
>>> (format t "hello!~%")
>>> (format t "The value is: ~A" x)
>>> x)
>>> 
>>> with
>>> 
>>> (defun foo (x)
>>> (push 1 *lines-executed*) (format t "hello!~%")
>>> (push 2 *lines-executed*) (format t "The value is: ~A" x)
>>> (push 3 *lines-executed*) x)
>> 
>> But Lispniks do not write lines of code, they write trees of it:
>> 
>> (defun aaa (x)
>>     (if x
>>         (when (bbb x)
>>            (zzz))
>>       (www)))
>> 
> 
> OK, so couldn't I give each subexpression a label, that SLIME remembers,
> and that gets pushed onto *subexpressions-evaluated* when the
> corresponding subexpression gets evaluated?
> 
> So:
> 
> (progn (push 1 *subexpressions-evaluated*)
>        (defun aaa (x)
> (progn (push 2 *subexpressions-evaluated*)
> (if (progn (push 3 *subexpressions-evaluated*) x)
> (progn (push 4 *subexpressions-evaluated*)
> (when (progn (push 5 *subexpressions-evaluated*)
> (bbb x))
> (progn (push 6 *subexpressions-evaluated*) (zzz)))
> (progn (push 7 *subexpressions-evaluated*) (www)))))))
> 
> Wouldn't that work?  I did it by hand, but I think I could write the
> code to do it...

Ick. Ick, I say.

Well, you could; there are a limited number of special forms, and not all
of those would require special handling. Beyond that, it's a matter of
wrapping each and every function call in a (progn (push /subnr/
*sybexpr-eval*) (/original statement/)).

It should work, except for macros - they essentially define new syntax, so
automated parsing of them is, um, hard.

You have four options:
1: Macroexpand the macros, and do stuff on the result. *Severe* ick.
2: Wrap the entire macro, ignore the contents. (But what about LOOP?)
3: Don't do this kind of thing in the first place. Unfortunately, it might
be useful for something...
4: Do it properly, and write a rewindable Common Lisp interpreter. Holy
Grail(tm), but it would be *very* useful.
From: Kenny Tilton
Subject: Re: Code coverage
Date: 
Message-ID: <ayGtc.64806$mX.21160201@twister.nyc.rr.com>
William Bland wrote:

> On Fri, 28 May 2004 03:52:04 +0000, Kenny Tilton wrote:
> 
>>William Bland wrote:
>>
>>>I'm thinking, before passing your code to Lisp to be compiled, SLIME could
>>>transform it, adding some extras code that keeps track of what has been
>>>executed.  So it would replace
>>>
>>>(defun foo (x)
>>>	(format t "hello!~%")
>>>	(format t "The value is: ~A" x)
>>>	x)
>>>
>>>with
>>>
>>>(defun foo (x)
>>>	(push 1 *lines-executed*) (format t "hello!~%")
>>>	(push 2 *lines-executed*) (format t "The value is: ~A" x)
>>>	(push 3 *lines-executed*) x)
>>
>>But Lispniks do not write lines of code, they write trees of it:
>>
>>(defun aaa (x)
>>    (if x
>>        (when (bbb x)
>>           (zzz))
>>      (www)))
>>
> 
> 
> OK, so couldn't I give each subexpression a label, that SLIME remembers,
> and that gets pushed onto *subexpressions-evaluated* when the
> corresponding subexpression gets evaluated?
> 
> So:
> 
> (progn (push 1 *subexpressions-evaluated*)
>        (defun aaa (x)
> 	 (progn (push 2 *subexpressions-evaluated*)
> 		(if (progn (push 3 *subexpressions-evaluated*) x)
> 		    (progn (push 4 *subexpressions-evaluated*)
> 			   (when (progn (push 5 *subexpressions-evaluated*)
> 					(bbb x))
> 			     (progn (push 6 *subexpressions-evaluated*) (zzz)))
> 			   (progn (push 7 *subexpressions-evaluated*) (www)))))))
> 
> Wouldn't that work?  I did it by hand, but I think I could write the code
> to do it...
> 
> Cheers,
> 	Bill.

You are going to have Slime do the rewriting? How will the editor know 
which tags were assigned to each form, and get access to the coverage 
results? And what happens when you start editing? will all the mappings 
following the change suddenly be invalidated and disappear or show garbage?

well, i do not know anything about slime/ilisp and how much info 
they/emacs have access to, so whadoiknow? certainly you can write a 
codewalker. give it a try.

kenny


-- 
Home? http://tilton-technology.com
Cells? http://www.common-lisp.net/project/cells/
Cello? http://www.common-lisp.net/project/cello/
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
Your Project Here! http://alu.cliki.net/Industry%20Application
From: Christian Lynbech
Subject: Re: Code coverage
Date: 
Message-ID: <ofpt8pj90d.fsf@situla.ted.dk.eu.ericsson.se>
>>>>> "William" == William Bland <····@abstractnonsense.com> writes:

William> The way it does that got me to thinking, would it be possible
William> to make SLIME highlight each line of source code that has
William> been executed, so that when testing code I could see which
William> lines of code my tests had exercised?

I am pretty sure that I once came across a package that did exactly
this sort of thing. I think it worked by redefining some central
special forms such as `defun' and then running the program would make
the program output such extra info that then would be gobbled by some
emacs code and turned into highlighting.

I am a bit unsure exactly where it was I found it, but it mgith have a
part of Lisp Debug:

        http://www.marclisp.bewoner.antwerpen.be/intro1.html

It would indeed be a nice addition to SLIME.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian ··@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - ·······@hal.com (Michael A. Petonic)
From: Alexander Schmolck
Subject: Re: Code coverage
Date: 
Message-ID: <yfsr7t4trwu.fsf@black132.ex.ac.uk>
William Bland <····@abstractnonsense.com> writes:

> The way it does that got me to thinking, would it be possible to make
> SLIME highlight each line of source code that has been executed, so that
> when testing code I could see which lines of code my tests had exercised?

I think you might find this interesting:

http://www.merl.com/papers/TR91-04/

'as
From: William Bland
Subject: Re: Code coverage
Date: 
Message-ID: <pan.2004.06.02.19.00.51.711823@abstractnonsense.com>
On Fri, 28 May 2004 17:53:05 +0100, Alexander Schmolck wrote:

> William Bland <····@abstractnonsense.com> writes:
> 
>> The way it does that got me to thinking, would it be possible to make
>> SLIME highlight each line of source code that has been executed, so that
>> when testing code I could see which lines of code my tests had exercised?
> 
> I think you might find this interesting:
> 
> http://www.merl.com/papers/TR91-04/
> 
> 'as

Thanks!  I did find it very interesting.  The approach seems similar to
the one I was suggesting (although obviously much more sophisticated).  I
found the source code here:

http://www-cgi.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/testing/cover/

and am pleased to see the license seems one that could co-exist with SLIME.

I probably won't get much time to work on this in the next week or so,
since I'll be installing a new OS on my laptop (Fedora killed my
computer!).  The first stumbling block though, is that SBCL doesn't like
the first line of the source file:

	(IN-PACKAGE "COVER" :USE '("LISP"))

Hyperspec only has one argument for in-package.  Is the :USE a pre-ansi
thing?

Cheers,
	Bill.
-- 
Dr. William Bland.
It would not be too unfair to any language to refer to Java as a
stripped down Lisp or Smalltalk with a C syntax.   (Ken Anderson).
From: Carl Shapiro
Subject: Re: Code coverage
Date: 
Message-ID: <ouyy8n5wygt.fsf@panix3.panix.com>
William Bland <····@abstractnonsense.com> writes:

> 	(IN-PACKAGE "COVER" :USE '("LISP"))
> 
> Hyperspec only has one argument for in-package.  Is the :USE a pre-ansi
> thing?

Yes, that's the cltl1 IN-PACKAGE.
From: Zach Beane
Subject: Re: Code coverage
Date: 
Message-ID: <m38yf5rbro.fsf@unnamed.xach.com>
William Bland <····@abstractnonsense.com> writes:

> The first stumbling block though, is that SBCL doesn't like
> the first line of the source file:
>
> 	(IN-PACKAGE "COVER" :USE '("LISP"))
>
> Hyperspec only has one argument for in-package.  Is the :USE a pre-ansi
> thing?

IN-PACKAGE used to be used to define packages; CLtL1 had no
DEFPACKAGE. There are a couple relevant cleanup documents:

   http://www.lispworks.com/reference/HyperSpec/Issues/iss195_w.htm

   http://www.lispworks.com/reference/HyperSpec/Issues/iss108_w.htm

Zach
From: William Bland
Subject: Re: Code coverage
Date: 
Message-ID: <pan.2004.06.03.03.30.27.303258@abstractnonsense.com>
On Wed, 02 Jun 2004 15:38:19 -0400, Zach Beane wrote:

> William Bland <····@abstractnonsense.com> writes:
> 
>> The first stumbling block though, is that SBCL doesn't like
>> the first line of the source file:
>>
>> 	(IN-PACKAGE "COVER" :USE '("LISP"))
>>
>> Hyperspec only has one argument for in-package.  Is the :USE a pre-ansi
>> thing?
> 
> IN-PACKAGE used to be used to define packages; CLtL1 had no
> DEFPACKAGE. There are a couple relevant cleanup documents:
> 
>    http://www.lispworks.com/reference/HyperSpec/Issues/iss195_w.htm
> 
>    http://www.lispworks.com/reference/HyperSpec/Issues/iss108_w.htm
> 
> Zach

Thanks very much.

I seem to have the code in an almost usable state, but I just noticed the
following in the copyright notice:

| Permission  to  use,  copy, modify, and distribute this software  and  its |
| documentation for any purpose  and without fee is hereby granted, provided |
| that this copyright  and  permission  notice  appear  in  all  copies  and |
| supporting  documentation,  and  that  the  name  of M.I.T. not be used in |
| advertising or  publicity  pertaining  to  distribution  of  the  software |
| without   specific,   written   prior   permission.      M.I.T.  makes  no |
| representations  about  the  suitability of this software for any purpose. |
| It is provided "as is" without express or implied warranty.                |

Is this similar to the "advertising clause" that gave the original BSD
license so many problems?  I know it's usual to ask people to leave
copyright notices in code, but I haven't seen people ask for them to
appear in supporting documentation before and I'm a little worried that
this code might not be usable for many people because of this clause in
the license.

I only really know my licenses up to the standard GPL and BSD.  Could
someone with a little more experience of license issues offer any advice?

Thanks,
	Bill.
-- 
Dr. William Bland.
It would not be too unfair to any language to refer to Java as a
stripped down Lisp or Smalltalk with a C syntax.   (Ken Anderson).
From: Karl A. Krueger
Subject: Re: Code coverage
Date: 
Message-ID: <c9nd1g$lrm$1@baldur.whoi.edu>
William Bland <····@abstractnonsense.com> wrote:
> I seem to have the code in an almost usable state, but I just noticed the
> following in the copyright notice:
> 
> | Permission  to  use,  copy, modify, and distribute this software  and  its |
> | documentation for any purpose  and without fee is hereby granted, provided |
> | that this copyright  and  permission  notice  appear  in  all  copies  and |
> | supporting  documentation,  and  that  the  name  of M.I.T. not be used in |
> | advertising or  publicity  pertaining  to  distribution  of  the  software |
> | without   specific,   written   prior   permission.      M.I.T.  makes  no |
> | representations  about  the  suitability of this software for any purpose. |
> | It is provided "as is" without express or implied warranty.                |
> 
> Is this similar to the "advertising clause" that gave the original BSD
> license so many problems?

No.

The BSD advertising clause read as follows:

| All advertising materials mentioning features or use of this software
| must display the following acknowledgement:
|     This product includes software developed by the University of
|     California, Berkeley and its contributors.  

The MIT license above simply asks you to duplicate the copyright and
license statement in the documentation -- where, presumably, it is
already found.  So basically it says, "Don't remove our attribution,
and don't remove -this license-, since it makes clear to all comers
that they have the same rights you do with regards to this code."

The old BSD license, on the other hand, requires that if you distribute
Berkeley code together with other code (say, in a BSD Unix distribution)
and mention any BSD feature in the -advertisements-, you have to mention
Berkeley in the adverts as well.  In other words, you -may not-, for
instance, run a short classified ad:

	CHEAP UNIX DISTRIBUTION featuring Fast File System.
	Send $5 to Church of the Macro-Kernel, Dallas, TX.

... since that ad mentions a BSD feature (FFS) but doesn't contain the
acknowledgement.

For more information:

	http://www.gnu.org/philosophy/bsd.html

-- 
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped.  s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
From: Rob Warnock
Subject: Re: Code coverage
Date: 
Message-ID: <0J6dnd9o6uDSRyLd4p2dnA@speakeasy.net>
Karl A. Krueger <········@example.edu> wrote:
+---------------
| For more information:
| 	http://www.gnu.org/philosophy/bsd.html
+---------------

Note that this page still contains a small but important omission:

    Dean Hal Varian at the University of California took up the cause,
    and championed it with the administration. In June 1999, after two
    years of discussions, the University of California removed this
    clause from the license of BSD.

    Thus, there is now a new BSD license which does not contain the
    advertising clause.

More importantly, when the Regents of the University of California
approved the change, they also announced that it was *retroactive*
to all pre-existing UCB-developed code. That is, one may safely ignore
the advertising clause in any pre-1999 direct BSD copyright notice,
as you can see from this excerpt from the official announcement
[heavily edited for brevity]:

    July 22, 1999
    To All Licensees, Distributors of Any Version of BSD:

    As you know, certain of the Berkeley Software Distribution ("BSD")
    source code files require that further distributions... acknowledge
    within their advertising materials that such products contain
    software developed by UC Berkeley and its contributors.
    ...

    Effective immediately, licensees and distributors are no longer
    required to include the acknowledgement within advertising materials.
    Accordingly, the foregoing paragraph of those BSD Unix files
    containing it is hereby deleted in its entirety.

    William Hoskins
    Director, Office of Technology Licensing
    University of California, Berkeley

Although, as the above-referenced URL notes, this did not entirely fix
the problem:

    Unfortunately, this does not eliminate the legacy of the advertising
    clause: similar clauses are still present in the licenses of many
    packages which are not part of BSD. The change in license for BSD
    has no effect on the other packages which imitated the old BSD
    license; only the developers who made them can change them.
    ...
    So if you have a favorite package which still uses the BSD license
    with the advertising clause, please ask the maintainer to look at
    this web page, and consider making the change.

Note also that they suggest using the MIT X license instead of the
"new" BSD license, just to avoid accidentally picking up an "old" BSD
license by mistake, though that seems a bit condescending to me.

IMHO & AFAIK, there is no serious reason to prefer one over the other.
If you're adding to something that already uses an "X11-style" license,
just use that one to keep it simple for later developers, and conversely
if you're adding to code with a "new BSD-style" license.


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607