I'm working alone on an LWW project and I'm convinced that I'm not
using the IDE to its fullest extent. If I were in a different
environment, I would be able to look over somebody's shoulder or they
would look over mine and tell me - "hey, it would be better if you did
that this way". It's the same with Emacs - one gets stuck in a rut
using the same set of operations one is familiar with until someone
else gives you an out-of-the-blue comment.
I'm looking for comments on how I could use the LWW (LispWorks under
Windows 4.2, soon to be under Linux) more productively. I wonder if I
can impose on some of you to look over my shoulder as I describe my
actions in ascii...
Sorry for the length of this posting - I'm trying to be explicit about
how I do this and am trying to avoid describing my operations in a way
that is biased on my (possibly incorrect) assumptions. At least I've
chopped the posting up into a set of short (possibly biased)
questions, followed by a set of descriptions of how I abuse the
environment. What's really needed is a small video clip of me doing
what I do or a webcam session with someone :-).
Thanx in advace!
Executive Summary
-----------------
- I wonder if really know what the minimum set of IDE operations to
make a source code change, to save it, to re-compile and test it,
are.
- I don't get defsystem. I want to view it as a makefile, but certain
things don't seem to fit this mental model.
- I wonder if I've been polluted by the C / Unix model. I vaguely
think that compile generally produces .fsl files and load loads .fsl
files into the ide environment and that you have to do things in
this order (compile, then (re)load). If I do misunderstand this,
then maybe I'm using the ide in an overly-klunky way.
- I keep wishing that there was a way to single-step lisp at the
source level (this possibly comes from using C ide's).
- I tend to debug by inserting (break) sexprs into the code, then
recompiling. When debugging, I often want to grab intermediate
results off the stack, and try experiments with them in the
Listener. Right-clicking and turning on Trace doesn't give me this
effect - trace blows by the place I want to stop at.
- I want the compiler to pessimize maximally - I want as much debug
info as possible and don't care a whit about performance. At the
time I start up LWW, I immediately cut/paste:
(proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
into the listener and hit RETURN. This line doesn't seem to have
any effect if I stick it into a .lisp file - I have to do it
manually in the Listener.
- None of the code I've written uses packages or namespaces. Hmmm.
- I don't save workspace images. I generally start from scratch every
morning. Hmmm.
Intro
-----
For background, my project consists of a C-based compiler for a
java-subset language, and a whole whack of lisp code that implements a
graphical programming language (picture compiler) and windows-based
simulator. The final target is a byte-coded machine running on a
small embedded processor. (The use of C is political, not technical).
The project has now grown to some 70 .lisp files arranged into 11
sub-directories.
Initial Startup
---------------
I start up a stock LWW.
Then open the file make.lisp at the top level directory. It contains
a series of simple lines like:
(compile-file (current-pathname "util.lisp") :load t)
one, for each .lisp file in the directory and one for each make.lisp
in each sub-directory. There is a make.lisp in each subdirectory,
constructed in a similar manner. (I didn't understand defsystem when
I started, and I still don't really get it. I don't use the System
Browser at all - hmmm).
The top level make.lisp contains one extra line
(proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
which I manually cut, then paste into the Listener and execute it (I
want all compilations for all time to be done with the maximum level
of debuggability. Performance seems not to be an issue at all).
Then, I compile and load the *whole* shebang by pulling down the
File>>Compile and Load menu. This takes half a minute or so.
Making Changes
--------------
Every time I make a set of changes to a file, I save them (^X^S) and
perform FILE>>Compile and Load.
Sometimes, when I think that the change is very temporary, I
right-click on the definition and click Compile. I'll run a test or
two with this, but soon, I will File>>Compile and Load the whole file
"just to be sure".
Running / Debugging
-------------------
I run the test program by typing a simple sexpr into the Listener.
The program runs until a bug (or assertion failure) is found. I poke
around with the visual debugger.
I double-click on items in the stack frame which pops me into the
editor highlighting source code at that point. I found that if I use
macros heavily or forget to proclaim pessimization, the editor
sometimes shows me the wrong spot.
I examine and test variables by clicking on a stack frame and playing
with them in the Listener.
In LWW4.1, I often got :DONT-KNOW as the value of many variables (I
haven't seen this yet in my one week of 4.2 use). I assume that the
compiler has optimized the variable away, even though I asked it
nicely to not do so.
I often find that I want to stop at a certain spot in the code, so
that I may examine the local environment (it is a simple way of
testing - use the debugger to confirm that a data structure conforms
with what I think it should be; it is also a way of debugging
confusion - I took the car of xyz here and it didn't give me the
result I had expected, let me re-discover what the structure of the
data really is at this point).
To do this, I insert "(break)" and recompile/load. Later, when
satisfied, I delete the "(break)" and recompile/load.
Code Structure
--------------
Everything I've written is in the same namespace (i.e. I haven't used
any in-package's or the like). My experience says that it is time to
reconsider the organization of this project. I'm willing to do this,
but I'm not sure that I understand how best to do this in CL.
C DLL's
-------
The C portion is in its own subdirectory containing an MSVC project.
It produces a .dll which I move to a distinguished place on my disk
when I'm satisfied that it works.
Debugging C vs. Lisp
--------------------
Other than debugging the C in the MSVC environment as much as
possible, there seems to be no way to debug C while in the lisp
environment. When an error occurs in the .dll while running lisp, I
get a dialog (usually a program fault) and then try to deduce what
part of the C it was in by looking at the lisp stack trace. I then
unload the .dll and switch to the MSVC environment to find / fix the
bug. I don't believe that it is possible to invoke the MS debugger
while running the .dll in lisp.
Delivering
----------
I've got a .bat file that starts up LWW from the command line to
produce an executable. This is pretty much like the manual says.
Thanks
Paul Tarvydas
>
> Debugging C vs. Lisp
> --------------------
> Other than debugging the C in the MSVC environment as much as
> possible, there seems to be no way to debug C while in the lisp
> environment. When an error occurs in the .dll while running lisp, I
> get a dialog (usually a program fault) and then try to deduce what
> part of the C it was in by looking at the lisp stack trace. I then
> unload the .dll and switch to the MSVC environment to find / fix the
> bug. I don't believe that it is possible to invoke the MS debugger
> while running the .dll in lisp.
While I wait with baited breath for answers to some of the questions
you originally posted, I might be able to help with this one.
I've debugged .dlls in the past, I would open the .dll project in
MSVC, then fill in the name of the program that will use this dll
someplace I don't recall in the Project's debug preferences. When you
"run" the .dll, it will start the program you've identified, then wait
till execution hits your breakpoints, etc... This approach has worked for C
apps calling .dlls or VB programs too- it only requires the calling
program to be an executable.
Gregm
In article <··············@mindspring.com>, Greg Menke wrote:
>> Debugging C vs. Lisp
>> --------------------
>> Other than debugging the C in the MSVC environment as much as
>> possible, there seems to be no way to debug C while in the lisp
>> environment. When an error occurs in the .dll while running lisp, I
>> get a dialog (usually a program fault) and then try to deduce what
>> part of the C it was in by looking at the lisp stack trace. I then
>> unload the .dll and switch to the MSVC environment to find / fix the
>> bug. I don't believe that it is possible to invoke the MS debugger
>> while running the .dll in lisp.
>
> While I wait with baited breath for answers to some of the questions
> you originally posted, I might be able to help with this one.
>
> I've debugged .dlls in the past, I would open the .dll project in
> MSVC, then fill in the name of the program that will use this dll
> someplace I don't recall in the Project's debug preferences. When you
> "run" the .dll, it will start the program you've identified, then wait
> till execution hits your breakpoints, etc... This approach has worked for C
> apps calling .dlls or VB programs too- it only requires the calling
> program to be an executable.
It's been years since I last debugged anything on Windows, but
I always used the SoftICE debugger from Numega. It runs at a very
low level, at every crash you are immediately in SoftICE and can see
what's going on. You can also insert
__asm int 3
lines (I don't remember the exact syntax) in your sources that will
jump directly into SoftICE. Also, the frequent bluescreens become
less of a nuisance, because instead of a bluescreen you'll see the
usual SoftICE screen and by reading the assembler code of the offending
driver you can often patch some memory addresses, skip some instructions
and actually make Winblows run again so you can safe your work and
reboot. SoftICE is the only thing I miss on Linux.
Regards,
--
Nils Goesche
Ask not for whom the <CONTROL-G> tolls.
PGP key ID 0xC66D6E6F
From: Simon Andr�s
Subject: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <vcdpu3d7hem.fsf_-_@russell.math.bme.hu>
Greg Menke <···········@mindspring.com> writes:
> >
> > Debugging C vs. Lisp
> > --------------------
> > Other than debugging the C in the MSVC environment as much as
> > possible, there seems to be no way to debug C while in the lisp
> > environment. When an error occurs in the .dll while running lisp, I
> > get a dialog (usually a program fault) and then try to deduce what
> > part of the C it was in by looking at the lisp stack trace. I then
> > unload the .dll and switch to the MSVC environment to find / fix the
> > bug. I don't believe that it is possible to invoke the MS debugger
> > while running the .dll in lisp.
>
> While I wait with baited breath for answers to some of the questions
> you originally posted, I might be able to help with this one.
>
> I've debugged .dlls in the past, I would open the .dll project in
> MSVC, then fill in the name of the program that will use this dll
> someplace I don't recall in the Project's debug preferences. When you
> "run" the .dll, it will start the program you've identified, then wait
> till execution hits your breakpoints, etc... This approach has worked for C
> apps calling .dlls or VB programs too- it only requires the calling
> program to be an executable.
While we are at it: how does one debug C called from lisp on Unix? I'm
especially interested in C called from ACL and CMU CL on Linux.
(There's a curious M-x fi:gdb thing in eli that looks like a
constant function alwas returning
Cannot get pathname and pid: Received signal number 11 (Segmentation violation)
:-)
Andras
>
> Gregm
From: Paolo Amoroso
Subject: Re: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <tH9mPB7J7NYORxx2aQd9HXQ25PL8@4ax.com>
On 10 Feb 2002 11:36:17 +0100, ······@math.bme.hu (Simon Andr�s) wrote:
> While we are at it: how does one debug C called from lisp on Unix? I'm
> especially interested in C called from ACL and CMU CL on Linux.
The paper "Instruction-Level Breakpoint Stepping in the Current Process" by
Duane Rettig, presented at LUGM '99 and published in the conference
proceedings, may be relevant.
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://www.paoloamoroso.it/ency/README
[http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/]
From: Bill Clementson
Subject: Re: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <wklme1z09j.fsf@attbi.com>
Paolo Amoroso <·······@mclink.it> writes:
> On 10 Feb 2002 11:36:17 +0100, ······@math.bme.hu (Simon Andr�s) wrote:
>
> > While we are at it: how does one debug C called from lisp on Unix? I'm
> > especially interested in C called from ACL and CMU CL on Linux.
>
> The paper "Instruction-Level Breakpoint Stepping in the Current Process" by
> Duane Rettig, presented at LUGM '99 and published in the conference
> proceedings, may be relevant.
Google doesn't return any relevant hits on either lugm 99 or the paper
name. Were the proceedings published in hard-copy format only, or is there an
on-line version available somewhere? If the former, is there any way to get
hold of copies of the proceedings?
--
Bill Clementson
From: Duane Rettig
Subject: Re: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <48za0om61.fsf@beta.franz.com>
Bill Clementson <·······@attbi.com> writes:
> Paolo Amoroso <·······@mclink.it> writes:
>
> > On 10 Feb 2002 11:36:17 +0100, ······@math.bme.hu (Simon Andr�s) wrote:
> >
> > > While we are at it: how does one debug C called from lisp on Unix? I'm
> > > especially interested in C called from ACL and CMU CL on Linux.
> >
> > The paper "Instruction-Level Breakpoint Stepping in the Current Process" by
> > Duane Rettig, presented at LUGM '99 and published in the conference
> > proceedings, may be relevant.
>
> Google doesn't return any relevant hits on either lugm 99 or the paper
> name. Were the proceedings published in hard-copy format only, or is there an
> on-line version available somewhere? If the former, is there any way to get
> hold of copies of the proceedings?
I've made my paper available at
ftp://ftp.franz.com/pub/duane/break.ps
Unfortunately, it is only relevant to debugging lisp-compiled code. Note
that part of the name is "in the Current Process". Since C code tends to
be placed in read-only/shared space, there is no way for a self-debugger
like the one I present to apply to C code. One must use a separate
program which gets help from the kernel (such as gdb, using ptrace()).
Just another case where Lisp treats its code as data - not only source
code or its list representation, but also in its compiled form in
machine code objects!
I have theorized (but haven't pursued) the possibility that since
read-only code is readable, it can be copied out of its read-only
space, relinked at the new location, and breakpoints could then be
set. This presumes many things, including that the code is generated
in a PIC style, and that the linkage is well-known. I was going to
start such a study on our own "runtime-system" functions [lisp code
that has been compiled to assembler source and then linked in as part
of the Allegro shared-library - these are functions you disassemble
by entry-point name, like (disassemble "qcons") for example] because
these functions are under our control and they can understand both
CL and C calling sequences. And who knows? Maybe in my infinite
spare time I'll get around to doing just that... :-)
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: Bill Clementson
Subject: Re: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <wk7kpjb60e.fsf@attbi.com>
Duane Rettig <·····@franz.com> writes:
> I've made my paper available at
>
> ftp://ftp.franz.com/pub/duane/break.ps
Thank you.
--
Bill Clementson
From: Simon Andr�s
Subject: Re: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <vcd8za15faf.fsf@russell.math.bme.hu>
Paolo Amoroso <·······@mclink.it> writes:
> On 10 Feb 2002 11:36:17 +0100, ······@math.bme.hu (Simon Andr�s) wrote:
>
> > While we are at it: how does one debug C called from lisp on Unix? I'm
> > especially interested in C called from ACL and CMU CL on Linux.
>
> The paper "Instruction-Level Breakpoint Stepping in the Current Process" by
> Duane Rettig, presented at LUGM '99 and published in the conference
> proceedings, may be relevant.
Thanks!
Andras
>
>
> Paolo
> --
> EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
> http://www.paoloamoroso.it/ency/README
> [http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/]
From: Martin Cracauer
Subject: Re: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <a48vhc$ebn$1@counter.bik-gmbh.de>
I missed the beginning of the discussion, so hopefully this is not
redundant:
If you segfault (without or with intention) in C code running as CMUCL
alien code, you can attach gdb to the prosess with no problems and get
a meaningful stack trace for the levels of C code you are in. You get
the backtrace for the Lisp part above the C code anyway since the
segfault drops you into the CMUCL debugger. I am not sure what
happens when you have several layers of C and Lisp intermixed, I guess
hexdump is your friend then :-)
Martin
--
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <········@bik-gmbh.de> http://www.bik-gmbh.de/~cracauer/
FreeBSD - where you want to go. Today. http://www.freebsd.org/
From: ·······@inetmi.com
Subject: Re: Debugging C called from Lisp. was: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <ulmdx9jgd.fsf@chicago.inetmi.com>
······@math.bme.hu (Simon Andr�s) writes:
> Greg Menke <···········@mindspring.com> writes:
>
> While we are at it: how does one debug C called from lisp on Unix?
> I'm especially interested in C called from ACL and CMU CL on Linux.
> (There's a curious M-x fi:gdb thing in eli that looks like a
> constant function alwas returning
I've also successfully used gdb with cmucl, attaching it to the
running lisp process. This was always when the lisp process appeared
hung, and I wanted to find out what code was running.
E.g.,
# gdb /usr/bin/lisp 11911
[... gdb startup stuff ...]
0x4013ba92 in __libc_connect () from /lib/i686/libc.so.6
(gdb) bt
#0 0x4013ba92 in __libc_connect () from /lib/i686/libc.so.6
#1 0x10312201 in ?? () at eval.c:41
#2 0x3fffee5c in ?? () at eval.c:41
#3 0x00280000 in ?? () at eval.c:41
A cool thing about MCL is that Digitool provides some "dcmds"
(plugins) for the low-level Macsbug debugger that give Macsbug some
knowledge of how to display and manipulate lisp objects. One cool
thing was that you could get a low-level backtrace that showed both
Lisp functions and C functions. E.g., the macsbug screenshot at
<http://lemonodor.com/images/macsbug.gif>.
I've been curious how one would debug foreign code in ACL under
windows. I'm guessing you can attach a debugger like the one that
comes with Visual C++ to the process similar to the way one uses gdb.
I don't know of a similar tool that would work well with COM, though.
John Wiseman
Paul Tarvydas <·········@tscontrols.com> writes:
> Executive Summary
> -----------------
> - I wonder if really know what the minimum set of IDE operations to
> make a source code change, to save it, to re-compile and test it,
> are.
Using emacs+ilisp, it's C-z c for a single defun, C-z l for a file. I
assume LW has similar functions.
> - I don't get defsystem. I want to view it as a makefile, but certain
> things don't seem to fit this mental model.
Right, it's not quite a makefile. It describes a system of lisp source
files and their interdependencies. (see later for more)
> - I wonder if I've been polluted by the C / Unix model. I vaguely
> think that compile generally produces .fsl files and load loads .fsl
> files into the ide environment and that you have to do things in
> this order (compile, then (re)load). If I do misunderstand this,
> then maybe I'm using the ide in an overly-klunky way.
1. You don't have to compile code.
2. You don't have to load whole files.
3. You can run code that's not even in a file.
> - I keep wishing that there was a way to single-step lisp at the
> source level (this possibly comes from using C ide's).
There is one in CMUCL, and I'm quite sure there should be one in LW.
> - I tend to debug by inserting (break) sexprs into the code, then
> recompiling. When debugging, I often want to grab intermediate
> results off the stack, and try experiments with them in the
> Listener. Right-clicking and turning on Trace doesn't give me this
> effect - trace blows by the place I want to stop at.
Right, trace just traces the execution. I think you're doing this
correctly.
> - I want the compiler to pessimize maximally - I want as much debug
> info as possible and don't care a whit about performance. At the
> time I start up LWW, I immediately cut/paste:
>
> (proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
>
> into the listener and hit RETURN. This line doesn't seem to have
> any effect if I stick it into a .lisp file - I have to do it
> manually in the Listener.
In a lisp file, stick
(declaim (optimize (debug 3) (safety 3) (speed 0) (space 0)))
at the top. I'm not totally clear on the difference, but here goes:
PROCLAIM is more for global declarations. An INLINE declaration that
you want to be visible to all code, e.g. DECLAIM is for file-local
declarations, like specific optimization settings for that one
file. Or am I completely misunderstanding the difference?
> - None of the code I've written uses packages or namespaces. Hmmm.
That's not TOO important until you start dealing with lots of
different libraries/applications in the same image, but it's good to
learn how it works even if you don't use them yet just so that you
understand how LW itself uses them.
> - I don't save workspace images. I generally start from scratch every
> morning. Hmmm.
Defsystem makes loading a full system pretty easy, and that's how I
work.
--
-> -/- - Rahul Jain - -\- <-
-> -\- http://linux.rice.edu/~rahul -=- ············@techie.com -/- <-
-> -/- "I never could get the hang of Thursdays." - HHGTTG by DNA -\- <-
|--|--------|--------------|----|-------------|------|---------|-----|-|
Version 11.423.999.221020101.23.50110101.042
(c)1996-2002, All rights reserved. Disclaimer available upon request.
Rahul Jain <·····@sid-1129.sid.rice.edu> writes:
> In a lisp file, stick
>
> (declaim (optimize (debug 3) (safety 3) (speed 0) (space 0)))
>
> at the top. I'm not totally clear on the difference, but here goes:
> PROCLAIM is more for global declarations. An INLINE declaration that
> you want to be visible to all code, e.g. DECLAIM is for file-local
> declarations, like specific optimization settings for that one
> file. Or am I completely misunderstanding the difference?
Well, for one thing, as far as I can tell declaim is not guaranteed to
be file-local:
As with other defining macros, it is unspecified whether or not the
compile-time side-effects of a declaim persist after the file has
been compiled.
(from <http://www.xanalys.com/software_tools/reference/HyperSpec/Body/m_declai.htm#declaim>)
If that's true, then it seems that declaim should probably never be
used to make optimize declarations, as you risk contaminating
someone's lisp image with your declarations (which could be very
annoying).
To guarantee file-local declarations, I think you'd have to wrap the
whole file in (locally (declare (optimize ...)) ...)
John Wiseman
Others will pitch in on how to compile/eval single definitions and
regions from the editor. I will say that you *really* need to spend
some time with the online editor and debugger manuals -- the environment
really is excellent, very smooth, quick edit-debug turnaround. Very
different development model than the batch-compile model you're using.
As for debugging the C code running inside LWW: Since you're using
MSVC,
it's pretty easy. Go into your project for the dll, and set the
lispworks executable as the executable to be debugged, and set up the
working directory to whatever is convenient. Then go to the Debug tab,
click "additional dll's", and enter all the dlls you want him to load
the symbols for.
Set the breakpoints you want, then hit "debug". MSVC will complain
about
lispworks not having debug symbols; just uncheck the "Tell me every
time"
checkbox and hit "ok".
When you get to your breakpoint, you'll crack into the debugger.
Never actually done this with LWW, but I've debugged ActiveX controls
running inside IE and other apps this way, and it works just fine.
You may need to become familiar with the exceptions list, depending on
if LWW is generating OS-level exceptions internally (sometimes the
debugger sees them and pops up).
Paul Tarvydas wrote:
>
> I'm working alone on an LWW project and I'm convinced that I'm not
> using the IDE to its fullest extent. If I were in a different
> environment, I would be able to look over somebody's shoulder or they
> would look over mine and tell me - "hey, it would be better if you did
> that this way". It's the same with Emacs - one gets stuck in a rut
> using the same set of operations one is familiar with until someone
> else gives you an out-of-the-blue comment.
>
> I'm looking for comments on how I could use the LWW (LispWorks under
> Windows 4.2, soon to be under Linux) more productively. I wonder if I
> can impose on some of you to look over my shoulder as I describe my
> actions in ascii...
>
> Sorry for the length of this posting - I'm trying to be explicit about
> how I do this and am trying to avoid describing my operations in a way
> that is biased on my (possibly incorrect) assumptions. At least I've
> chopped the posting up into a set of short (possibly biased)
> questions, followed by a set of descriptions of how I abuse the
> environment. What's really needed is a small video clip of me doing
> what I do or a webcam session with someone :-).
>
> Thanx in advace!
>
> Executive Summary
> -----------------
> - I wonder if really know what the minimum set of IDE operations to
> make a source code change, to save it, to re-compile and test it,
> are.
>
> - I don't get defsystem. I want to view it as a makefile, but certain
> things don't seem to fit this mental model.
>
> - I wonder if I've been polluted by the C / Unix model. I vaguely
> think that compile generally produces .fsl files and load loads .fsl
> files into the ide environment and that you have to do things in
> this order (compile, then (re)load). If I do misunderstand this,
> then maybe I'm using the ide in an overly-klunky way.
>
> - I keep wishing that there was a way to single-step lisp at the
> source level (this possibly comes from using C ide's).
>
> - I tend to debug by inserting (break) sexprs into the code, then
> recompiling. When debugging, I often want to grab intermediate
> results off the stack, and try experiments with them in the
> Listener. Right-clicking and turning on Trace doesn't give me this
> effect - trace blows by the place I want to stop at.
>
> - I want the compiler to pessimize maximally - I want as much debug
> info as possible and don't care a whit about performance. At the
> time I start up LWW, I immediately cut/paste:
>
> (proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
>
> into the listener and hit RETURN. This line doesn't seem to have
> any effect if I stick it into a .lisp file - I have to do it
> manually in the Listener.
>
> - None of the code I've written uses packages or namespaces. Hmmm.
>
> - I don't save workspace images. I generally start from scratch every
> morning. Hmmm.
>
> Intro
> -----
>
> For background, my project consists of a C-based compiler for a
> java-subset language, and a whole whack of lisp code that implements a
> graphical programming language (picture compiler) and windows-based
> simulator. The final target is a byte-coded machine running on a
> small embedded processor. (The use of C is political, not technical).
>
> The project has now grown to some 70 .lisp files arranged into 11
> sub-directories.
>
> Initial Startup
> ---------------
>
> I start up a stock LWW.
>
> Then open the file make.lisp at the top level directory. It contains
> a series of simple lines like:
>
> (compile-file (current-pathname "util.lisp") :load t)
>
> one, for each .lisp file in the directory and one for each make.lisp
> in each sub-directory. There is a make.lisp in each subdirectory,
> constructed in a similar manner. (I didn't understand defsystem when
> I started, and I still don't really get it. I don't use the System
> Browser at all - hmmm).
>
> The top level make.lisp contains one extra line
>
> (proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
>
> which I manually cut, then paste into the Listener and execute it (I
> want all compilations for all time to be done with the maximum level
> of debuggability. Performance seems not to be an issue at all).
>
> Then, I compile and load the *whole* shebang by pulling down the
> File>>Compile and Load menu. This takes half a minute or so.
>
> Making Changes
> --------------
>
> Every time I make a set of changes to a file, I save them (^X^S) and
> perform FILE>>Compile and Load.
>
> Sometimes, when I think that the change is very temporary, I
> right-click on the definition and click Compile. I'll run a test or
> two with this, but soon, I will File>>Compile and Load the whole file
> "just to be sure".
>
> Running / Debugging
> -------------------
>
> I run the test program by typing a simple sexpr into the Listener.
> The program runs until a bug (or assertion failure) is found. I poke
> around with the visual debugger.
>
> I double-click on items in the stack frame which pops me into the
> editor highlighting source code at that point. I found that if I use
> macros heavily or forget to proclaim pessimization, the editor
> sometimes shows me the wrong spot.
>
> I examine and test variables by clicking on a stack frame and playing
> with them in the Listener.
>
> In LWW4.1, I often got :DONT-KNOW as the value of many variables (I
> haven't seen this yet in my one week of 4.2 use). I assume that the
> compiler has optimized the variable away, even though I asked it
> nicely to not do so.
>
> I often find that I want to stop at a certain spot in the code, so
> that I may examine the local environment (it is a simple way of
> testing - use the debugger to confirm that a data structure conforms
> with what I think it should be; it is also a way of debugging
> confusion - I took the car of xyz here and it didn't give me the
> result I had expected, let me re-discover what the structure of the
> data really is at this point).
>
> To do this, I insert "(break)" and recompile/load. Later, when
> satisfied, I delete the "(break)" and recompile/load.
>
> Code Structure
> --------------
> Everything I've written is in the same namespace (i.e. I haven't used
> any in-package's or the like). My experience says that it is time to
> reconsider the organization of this project. I'm willing to do this,
> but I'm not sure that I understand how best to do this in CL.
>
> C DLL's
> -------
> The C portion is in its own subdirectory containing an MSVC project.
> It produces a .dll which I move to a distinguished place on my disk
> when I'm satisfied that it works.
>
> Debugging C vs. Lisp
> --------------------
> Other than debugging the C in the MSVC environment as much as
> possible, there seems to be no way to debug C while in the lisp
> environment. When an error occurs in the .dll while running lisp, I
> get a dialog (usually a program fault) and then try to deduce what
> part of the C it was in by looking at the lisp stack trace. I then
> unload the .dll and switch to the MSVC environment to find / fix the
> bug. I don't believe that it is possible to invoke the MS debugger
> while running the .dll in lisp.
>
> Delivering
> ----------
> I've got a .bat file that starts up LWW from the command line to
> produce an executable. This is pretty much like the manual says.
>
> Thanks
> Paul Tarvydas
Paul Tarvydas <·········@tscontrols.com> writes:
> - I want the compiler to pessimize maximally - I want as much debug
> info as possible and don't care a whit about performance.
>
> [...]
>
> Making Changes
> --------------
>
> Every time I make a set of changes to a file, I save them (^X^S) and
> perform FILE>>Compile and Load.
>
> Sometimes, when I think that the change is very temporary, I
> right-click on the definition and click Compile. I'll run a test or
> two with this, but soon, I will File>>Compile and Load the whole
> file "just to be sure".
I wonder why you compile at all if you don't care about
performance. Wouldn't it be easier to debug if you were working with
interpreted code?
Edi.
--
Dr. Edmund Weitz
Hamburg
Germany
The Common Lisp Cookbook
<http://agharta.de/cookbook/>
···@agharta.de (Dr. Edmund Weitz) writes:
> I wonder why you compile at all if you don't care about
> performance. Wouldn't it be easier to debug if you were working with
> interpreted code?
Not really, only Lisp environments with very weak debuggers tend to do
a better job debugging interpreted code. When examining stack frames
some debuggers will only print out the pretty names of local variables
when you are working with interpreted code. With those same debuggers
the stack frames of compiled code often still show the variables, but
they will have meaningless names like "Local 1".
The Lispworks debugger is actually quite good. You should have no
trouble debugging compiled code in that environment.
Carl Shapiro <·············@panix.com> writes:
> The Lispworks debugger is actually quite good. You should have no
> trouble debugging compiled code in that environment.
Furthermore there are even implementations, like CMU CL, which can
actually do a much better job debugging compiled code, than debugging
interpreted code.
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
···@agharta.de (Dr. Edmund Weitz) wrote in
···················@bird.agharta.de:
> I wonder why you compile at all if you don't care about
> performance. Wouldn't it be easier to debug if you were working with
> interpreted code?
>
> Edi.
This may sound really dense - but, I didn't know I had the choice!
How does one do this?
I came back to lisp after an extended absence. Something, somewhere,
convinced me that, these days, lisp compiles sexprs before evaluating them.
This might be a root problem in my abuse of the ide :-). I just searched
the online manuals for "interpret" and nothing suggestive came up.
How do you ingest a 70-file project so that it is interpreted instead of
compiled?
[I may, also, find that I have no issue with performance, because I've been
inadvertantly compiling everything - we'll see ;-].
Thanks!
pt
Paul Tarvydas <·········@tscontrols.com> writes:
> ···@agharta.de (Dr. Edmund Weitz) wrote in
> ···················@bird.agharta.de:
>
> > I wonder why you compile at all if you don't care about
> > performance. Wouldn't it be easier to debug if you were working with
> > interpreted code?
> >
> > Edi.
>
> This may sound really dense - but, I didn't know I had the choice!
>
> How does one do this?
If you just type stuff at the toplevel (or ask your editor to evaluate
a form), it'll probably be interpreted. If you have a file full of
code you want to load into the lisp image, you can just (load "foo.lisp").
I tend to do this unless it's vital that a file's compiled, or once
it's settled down and I'm not changing it much anymore. Most of the
time, if I need to change something in a compiled file, pull it up in
Emacs, make the change, and eval the current defun. That changed
function will now be interpreted.
> I came back to lisp after an extended absence. Something, somewhere,
> convinced me that, these days, lisp compiles sexprs before evaluating them.
> This might be a root problem in my abuse of the ide :-). I just searched
> the online manuals for "interpret" and nothing suggestive came up.
Nothing requires a modern lisp to have an interpreter, but it has to
kind of look like it. If you just type something into the toplevel of
an implementation that compiles everything, it'll probably run the
compiler as fast as it can (producing slow-to-execute but fast-to-emit
code). I think LWW has an interpreter, so I'd imagine if you type
forms into the toplevel, they're interpreted.
> How do you ingest a 70-file project so that it is interpreted instead of
> compiled?
You just load the .lisp files.
> [I may, also, find that I have no issue with performance, because I've been
> inadvertantly compiling everything - we'll see ;-].
Possibly. Even so, you don't need to compile entire files. Say I've
got a file that defines functions TITI, TATA, TOTO, and TUTU. I go
edit the definition of TATA and TUTU and tell my editor to have the
lisp evaluate the new definitions. Now those are probably
interpreted. Say TUTU is called from an inner loop, so having it be
interpreted slows things down like crazy. Well, I go to the toplevel, and:
* (compile 'tutu)
Compiling LAMBDA NIL:
Compiling Top-Level Form:
TUTU
NIL
NIL
Now TUTU is compiled. Compiling the entire file at once gives the
compiler more opportunities to optimize things, but if I'm actively
developing the application, convenience is probably more important.
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
In an attempt to throw the authorities off his trail, Paul Tarvydas <·········@tscontrols.com> transmitted:
> ···@agharta.de (Dr. Edmund Weitz) wrote in
> ···················@bird.agharta.de:
>> I wonder why you compile at all if you don't care about
>> performance. Wouldn't it be easier to debug if you were working
>> with interpreted code?
>> Edi.
> This may sound really dense - but, I didn't know I had the choice!
> How does one do this?
It certainly varies by implementation.
-> CLISP seems to do some sort of "intepretive thing" when you throw
code at it at runtime, whereas if you run (compile-file
"whatever.lisp"), it generates bytecode that, once (load
"whatever")ed, is often quite a lot faster.
-> CMU/CL offers the option compiling to object code or to bytecode.
And it looks (Pierre, feel free to give a much-less-BS explanation)
like functions thrown straight at the reader are 'less compiled'
than either of those options.
-> GCL compiles everything to object code (via C).
> I came back to lisp after an extended absence. Something,
> somewhere, convinced me that, these days, lisp compiles sexprs
> before evaluating them. This might be a root problem in my abuse of
> the ide :-). I just searched the online manuals for "interpret" and
> nothing suggestive came up.
You're not typically supposed to be _too_ worried about it 'til you
start trying to performance tune things :-).
--
(concatenate 'string "cbbrowne" ·@canada.com")
http://www3.sympatico.ca/cbbrowne/multiplexor.html
Rules of the Evil Overlord #210. "All guest-quarters will be bugged
and monitored so that I can keep track of what the visitors I have for
some reason allowed to roam about my fortress are actually plotting."
<http://www.eviloverlord.com/>
Christopher Browne wrote:
>
> > I came back to lisp after an extended absence. Something,
> > somewhere, convinced me that, these days, lisp compiles sexprs
> > before evaluating them. This might be a root problem in my abuse of
> > the ide :-). I just searched the online manuals for "interpret" and
> > nothing suggestive came up.
>
> You're not typically supposed to be _too_ worried about it 'til you
> start trying to performance tune things :-).
One caveat here might be that in ACL at least the compiler picks up
stuff the interpreter lets slide. Pardon the over-precise tech jargon.
:)
--
kenny tilton
clinisys, inc
---------------------------------------------------------------
"We have a pond and a pool. The pond would be good for you."
- Ty to Carl, Caddy Shack
Christopher Browne <········@acm.org> writes:
> -> CMU/CL offers the option compiling to object code or to bytecode.
> And it looks (Pierre, feel free to give a much-less-BS explanation)
> like functions thrown straight at the reader are 'less compiled'
> than either of those options.
This is correct, see also my response to Vladimir Zolotykh (Message-ID
<··············@orion.bln.pmsf.de>). To summarize, CMU CL has:
a) A trivial "classic" interpreter, that handles things like function
application, etc. and which is present in all CMU CL cores, even
those without a compiler present.
b) A complete, "non-classical" interpreter, which can handle the full
language. It works by using the first stages of the compiler, to
convert the source code to the IR1 internal representation (which
is a flow-graph based representation, with implicit control-flow
information; also called ICR: Implicit Continuation
Representation), which is then interpreted. This interpreter
depends on the compiler being present. Since it converts before
interpreting, it behaves like a compiler when it comes to things
like macro-expansion, or handling of circular source forms.
c) A byte-code compiler. This also works off the IR1 internal
representation, but instead of interpreting that directly, it
transforms it to byte-codes for a stack-based VM, which can be
written to a processor-independent byte-fasl file. The byte-code
is finally interpreted by a byte-code interpreter.
The main advantage of the byte-code compiler are space-savings,
since the byte-coded representation is fairly compact (the cmu-user
manual gives a factor of 6), yet still faster (by an order of
magnitude, says the manual) than interpretation. It also gives you
processor-independence, though not endianness-independence AFAIK.
You can byte-compile whole files, or you can tell the file compiler
to only byte-compile certain parts, like e.g. top-level forms,
which aren't usually time-critical (this is the default).
d) The native compiler. This also runs through IR1, which is -- after
extensive optimizations -- transformed into IR2, the virtual
machine representation (VMR). The virtual machine that the VMR is
based-on is defined by the processor-specific backends of the
compiler, so that it can be tailored to the target processor as
needed, yet still allows things like register allocation
algorithms, etc. to be shared between backends. Finally the VMR is
converted to assembly code, through the VOP's code-generators. The
assembly code is slightly optimized, then assembled to machine
code, and finally emitted to the FASL file.
Further details can be gotten from the CMU User Manual, and the
"Design of CMU Common Lisp" document, which is available in the source
code of CMU. Both should also be included in the EncyCMUCLopedia.
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
"Pierre R. Mai" <····@acm.org> writes:
> d) The native compiler. This also runs through IR1, which is -- after
[...]
> assembly code is slightly optimized, then assembled to machine
> code, and finally emitted to the FASL file.
Everything Pierre says is true, but I wanted to pick up on this last
bit. The native compiler isn't _just_ for compilation to FASL files;
it also works for in-memory compilation of individual functions. You
can see this by using DISASSEMBLE
(Indeed SBCL - which forked from CMUCL a couple of years ago - has
dropped the IR1 interpreter and the byte compiler, and now compiles
pretty much everything to native code)
-dan
--
http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources
From: Pierre R. Mai
Subject: Re: Compilation in Python (was Re: LWW IDE, Productivity - Help Wanted)
Date:
Message-ID: <87lmdwm1zr.fsf@orion.bln.pmsf.de>
Daniel Barlow <···@telent.net> writes:
> "Pierre R. Mai" <····@acm.org> writes:
>
> > d) The native compiler. This also runs through IR1, which is -- after
> [...]
> > assembly code is slightly optimized, then assembled to machine
> > code, and finally emitted to the FASL file.
>
> Everything Pierre says is true, but I wanted to pick up on this last
> bit. The native compiler isn't _just_ for compilation to FASL files;
> it also works for in-memory compilation of individual functions. You
> can see this by using DISASSEMBLE
Indeed. All compilers can be used for either in-core or
file-compilation. Furthermore CMU CL offers the ability to directly
compile source code from a stream into the core (in effect combining
the compile and load phases, but bypassing the production of a file),
which gives you the equivalent of a compiling-load.
Regs, Pierre.
--
Pierre R. Mai <····@acm.org> http://www.pmsf.de/pmai/
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents. -- Nathaniel Borenstein
Paul Tarvydas <·········@tscontrols.com> writes:
> ···@agharta.de (Dr. Edmund Weitz) wrote in
> ···················@bird.agharta.de:
>
> > I wonder why you compile at all if you don't care about
> > performance. Wouldn't it be easier to debug if you were working
> > with interpreted code?
> >
> > Edi.
>
> This may sound really dense - but, I didn't know I had the choice!
>
> How does one do this?
Others have already explained the most important things, but I'll add
my two cents anyway... :)
1. In the glossary of the CLHS you will find: "interpreted function
n. a function that is not a compiled function. (It is possible for
there to be a conforming implementation which has no interpreted
functions, but a conforming program must not assume that all
functions are compiled functions.)" MCL, e.g., is such an
implementation AFAIK - LispWorks isn't. Here's an example from LW:
CL-USER 12 > (defun foo (n) (let (x) (dotimes (i n) (setq x (* i i))) x))
FOO
CL-USER 13 > (functionp #'foo)
T
CL-USER 14 > (compiled-function-p #'foo)
NIL
CL-USER 15 > (time (foo 1000000))
Timing the evaluation of (FOO 1000000)
user time = 1.720
system time = 0.000
Elapsed time = 0:00:01
Allocation = 23583560 bytes standard / 11002068 bytes fixlen
0 Page faults
999998000001
CL-USER 16 > (compile 'foo)
FOO
NIL
NIL
CL-USER 17 > (compiled-function-p #'foo)
T
CL-USER 18 > (time (foo 1000000))
Timing the evaluation of (FOO 1000000)
user time = 0.700
system time = 0.000
Elapsed time = 0:00:01
Allocation = 23583464 bytes standard / 2013 bytes fixlen
0 Page faults
999998000001
CL-USER 19 > (defun foo (n) n)
FOO
CL-USER 20 > (functionp #'foo)
T
CL-USER 21 > (compiled-function-p #'foo)
NIL
2. The main thing to keep in mind here probably is that you don't have
to run the full edit-compile-run cycle in Lisp as you have to in C
or Java. Just type a function definition into the listener or
evaluate it from the editor and it will be in effect - it'll
probably not be compiled but it will be there. You can also compile
it - just this single function - from the listener if you wish (see
above). If there has been another definition of the same function
before (whether compiled or not) it will be replaced.
3. If you want to use a file full of source code, you should just use
the LOAD function. It'll evaluate the source code in the file more
or less as if you had typed it into the listener - no need to
compile the file first. There's one little caveat, though. If your
source file is 'foo.lisp' and your implementation decides to name
the compiled file 'foo.fasl' it may choose to load 'foo.fasl' the
next time you issue (LOAD "FOO") even if you've changed the
contents of 'foo.lisp' in the meantime (it'll usually warn you,
though). While you're still debugging your app you should probably
make sure that you don't have compiled files around anymore so you
can be sure to always load the most recent version of your
code. This should result in faster turnaround cycles (although
execution speed might be slower).
HTH,
Edi.
--
Dr. Edmund Weitz
Hamburg
Germany
The Common Lisp Cookbook
<http://agharta.de/cookbook/>
On 09 Feb 2002 20:26:52 +0100, ···@agharta.de (Dr. Edmund Weitz)
wrote:
>
>I wonder why you compile at all if you don't care about
>performance. Wouldn't it be easier to debug if you were working with
>interpreted code?
Compiling the code has the significant advantage that when the system
encounters a bug, it can give you (via the show source button) the
exact code line that corresponds to the error. In interpreted mode
you are only brought to the function, not the specific line in it.
Paul Tarvydas <·········@tscontrols.com> writes:
Dear Paul,
Here are a few hints.
If you have 70 lisp files, you *really* need to learn how
to use defsystem. LispWorks defsystem is somewhat bizarre,
but it does work. Otherwise get the MK defsystem from cclan
and learn that instead. Once you've defined a system,
your morning ritual would be:
Open a new lisp, and type
(compile-system 'my-project :load t)
Regarding making changes, recompiling and loading the
entire file is completely unnecessary. If you're changing
a function, editing a function in the editor, and hitting
C-M-x is enough to send the new function to lisp.
(Actually, I'm not sure of the keystroke, as I use emacs/ILISP,
and there I just hit C-c C-c to recompile the function).
You should only need to reload a lot of file when you change
a low level macro which you know is used in a lot of places.
If you've written your defsystem properly, the line above
should recompile all the necessary files.
Learn how to make use of logical pathnames, so your systems
can be anchored anywhere, by changing just one value,
say, *project-root*, which you can set in your .lispworks.
By default, LispWorks will compile things with as much debugging
information as possible. Don't change the compilation flags
until you're _FINISHED_ with a module, and you know it works.
Have all the unit tests running to back up the "it works" statement. :-)
> To do this, I insert "(break)" and recompile/load. Later, when
> satisfied, I delete the "(break)" and recompile/load.
This is a fine thing to do. Sometimes, just (print myvar) is sufficient
as well instead of the (break). Especially good when you don't need
to re-load the entire file, as above.
> When an error occurs in the .dll while running lisp, I
> get a dialog (usually a program fault) and then try to deduce what
> part of the C it was in by looking at the lisp stack trace. I then
Write an run C unit tests _before_ you try to do anything in C.
Establish a clean, documented protocol between Lisp and C, and
prove that it works first. Once Lisp calls C, you're at the mercy
of that DLL. If it crashes, it can take down the entire world,
and you have no way of tracing what happens. (But you knew that. :)
> - I don't save workspace images. I generally start from scratch every
> morning.
If those 70 files represent more than 1 "logical" layer (which I hope
for you that they do) then, when layer zero is debugged working, there's
no reason to keep recompiling it. Save an image having loaded that layer.
Then keep going up to the next layer.
> - None of the code I've written uses packages or namespaces.
That's a hard one. I think you have to be quite good at designing
clean protocols to make packages "worth it"; otherwise you keep
trashing at what's in what package and what should be exported.
Once a layer stabilizes, it normally becomes apparent how to "clean it up"
to become self-contained, and then you can place it in its own package
and directory, and use it as a black box in the rest of the project.
I hope this is useful to you.
--
It would be difficult to construe Larry Wall, in article
this as a feature. <·····················@netlabs.com>
From: Brian P Templeton
Subject: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <87k7tmx9nl.fsf@tunes.org>
Paul Tarvydas <·········@tscontrols.com> writes:
> I'm working alone on an LWW project and I'm convinced that I'm not
> using the IDE to its fullest extent. If I were in a different
> environment, I would be able to look over somebody's shoulder or they
> would look over mine and tell me - "hey, it would be better if you did
> that this way". It's the same with Emacs - one gets stuck in a rut
> using the same set of operations one is familiar with until someone
> else gives you an out-of-the-blue comment.
>
> I'm looking for comments on how I could use the LWW (LispWorks under
> Windows 4.2, soon to be under Linux) more productively. I wonder if I
> can impose on some of you to look over my shoulder as I describe my
> actions in ascii...
>
I use Emacs (with ILISP, of course) on Unix (specifically GNU/Linux),
but I'll try to provide some recommendations...
> Sorry for the length of this posting - I'm trying to be explicit about
> how I do this and am trying to avoid describing my operations in a way
> that is biased on my (possibly incorrect) assumptions. At least I've
> chopped the posting up into a set of short (possibly biased)
> questions, followed by a set of descriptions of how I abuse the
> environment. What's really needed is a small video clip of me doing
> what I do or a webcam session with someone :-).
>
> Thanx in advace!
>
> Executive Summary
> -----------------
> - I wonder if really know what the minimum set of IDE operations to
> make a source code change, to save it, to re-compile and test it,
> are.
>
> - I don't get defsystem. I want to view it as a makefile, but certain
> things don't seem to fit this mental model.
>
Defsystem provides a method of compiling and loading libraries.
Defsystem can allow one to easily compile and load systems. For
example,
(make:defsystem soil-parser
:source-pathname "soil:parser;"
:source-extension "lisp"
:binary-pathname "soil:parser;"
:binary-extension "x86f"
:components ((:file "reader")
(:file "readtable")
(:file "sharpmacros")
(:file "flatmacros")))
could be an example of a defsystem file for a hypothetical future
version of SOIL. Once you load this file, you can evaluate
(make:operate-on-system :soil-parser :compile)
. Defsystem will look for the specified components in the directory
soil:parser (file components without suffixes will have the
source-extension added on after a period if they are not found). It
will compile each one and place the binaries in the directory
soil:parser. Then one can evaluate
(make:operate-on-system :soil-parser :load)
Defsystem will look for each of the listed components, and prefer
binaries to sources (as long as the binaries are not older than the
sources). It will also prompt to compile the sources if the binaries
are not present and you don't use the :load-source-instead-of-binaries
keyword argument (as a non-nil value).
> - I wonder if I've been polluted by the C / Unix model. I vaguely
> think that compile generally produces .fsl files and load loads .fsl
> files into the ide environment and that you have to do things in
> this order (compile, then (re)load). If I do misunderstand this,
> then maybe I'm using the ide in an overly-klunky way.
>
You seem to have been polluted by the C model. There's usually no
reason to compile when testing, and it can make debugging harder.
> - I keep wishing that there was a way to single-step lisp at the
> source level (this possibly comes from using C ide's).
>
Me, too, except for me this comes from Edebug. I have heard that
someone is working on something like Edebug for CL. (I'm not sure how
I should singe-step through a function with the SBCL debugger; maybe I
haven't read the (sparse) documentation thoroughly enough.)
> - I tend to debug by inserting (break) sexprs into the code, then
> recompiling. When debugging, I often want to grab intermediate
> results off the stack, and try experiments with them in the
> Listener. Right-clicking and turning on Trace doesn't give me this
> effect - trace blows by the place I want to stop at.
>
Hrm. Does LW provide a way of setting breakpoints? Perhaps you should
debug the program, and use breakpoints and the equivalent of edebug's
`Go' command to continue until you reach a breakpoint.
> - I want the compiler to pessimize maximally - I want as much debug
> info as possible and don't care a whit about performance. At the
> time I start up LWW, I immediately cut/paste:
>
> (proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
>
> into the listener and hit RETURN. This line doesn't seem to have
> any effect if I stick it into a .lisp file - I have to do it
> manually in the Listener.
>
I'm not sure why it doesn't work in the source file (I don't use LW),
but *don't compile when testing*!!! Compiling in general makes
programs more difficult to debug. This isn't C - you aren't restricted
to a compiler.
> - None of the code I've written uses packages or namespaces. Hmmm.
>
I wouldn't bother for tiny, one-file programs in most cases, but for
Real Work, I recommend that you try to use packages. This should
ensure that you don't accidentally munge things from other libraries
that are USE-PACKAGE'd from the CL-USER package, and probably make the
program generally more comprehensible.
> - I don't save workspace images. I generally start from scratch every
> morning. Hmmm.
>
I also need to do something like this (assuming I'm interpreting
`workspace image' correctly) with Emacs. I don't think it is a huge
problem, especially if you use some Unix such as GNU/Linux or *BSD and
can rely on your machine staying up for several years at a time, but
it could save you some time and make your environment more convenient
to use.
>
> Intro
> -----
>
> For background, my project consists of a C-based compiler for a
> java-subset language, and a whole whack of lisp code that implements a
> graphical programming language (picture compiler) and windows-based
> simulator. The final target is a byte-coded machine running on a
> small embedded processor. (The use of C is political, not technical).
>
If you wanted to use CL instead of C, perhaps you could use ECL and
convince them that it is still C-based :).
> The project has now grown to some 70 .lisp files arranged into 11
> sub-directories.
>
Though I don't know how the files are organized, the subdirectories
may be a good start for adding packages to your program.
> Initial Startup
> ---------------
>
> I start up a stock LWW.
>
> Then open the file make.lisp at the top level directory. It contains
> a series of simple lines like:
>
> (compile-file (current-pathname "util.lisp") :load t)
>
> one, for each .lisp file in the directory and one for each make.lisp
> in each sub-directory. There is a make.lisp in each subdirectory,
> constructed in a similar manner. (I didn't understand defsystem when
> I started, and I still don't really get it. I don't use the System
> Browser at all - hmmm).
>
Could you start a new thread with questions you have about defsystem?
You're replicating part defsystem's purpose here, and with defsystem,
since you're developing the program, you could make creating it
(debugging it, at least) much easier by setting
MAKE:*LOAD-SOURCE-INSTEAD-OF-BINARY* to a non-nil value in order to
*NOT COMPILE* - compiling is the Wrong Thing to do when developing a
program.
> The top level make.lisp contains one extra line
>
> (proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
>
> which I manually cut, then paste into the Listener and execute it (I
> want all compilations for all time to be done with the maximum level
> of debuggability. Performance seems not to be an issue at all).
>
> Then, I compile and load the *whole* shebang by pulling down the
> File>>Compile and Load menu. This takes half a minute or so.
>
Don't compile!
> Making Changes
> --------------
>
> Every time I make a set of changes to a file, I save them (^X^S) and
> perform FILE>>Compile and Load.
>
Don't compile!
> Sometimes, when I think that the change is very temporary, I
> right-click on the definition and click Compile. I'll run a test or
> two with this, but soon, I will File>>Compile and Load the whole file
> "just to be sure".
>
Don't compile!
> Running / Debugging
> -------------------
>
> I run the test program by typing a simple sexpr into the Listener.
> The program runs until a bug (or assertion failure) is found. I poke
> around with the visual debugger.
>
> I double-click on items in the stack frame which pops me into the
> editor highlighting source code at that point. I found that if I use
> macros heavily or forget to proclaim pessimization, the editor
> sometimes shows me the wrong spot.
>
> I examine and test variables by clicking on a stack frame and playing
> with them in the Listener.
>
> In LWW4.1, I often got :DONT-KNOW as the value of many variables (I
> haven't seen this yet in my one week of 4.2 use). I assume that the
> compiler has optimized the variable away, even though I asked it
> nicely to not do so.
>
Don't compile!
> I often find that I want to stop at a certain spot in the code, so
> that I may examine the local environment (it is a simple way of
> testing - use the debugger to confirm that a data structure conforms
> with what I think it should be; it is also a way of debugging
> confusion - I took the car of xyz here and it didn't give me the
> result I had expected, let me re-discover what the structure of the
> data really is at this point).
>
> To do this, I insert "(break)" and recompile/load. Later, when
> satisfied, I delete the "(break)" and recompile/load.
>
Don't compile!
> Code Structure
> --------------
> Everything I've written is in the same namespace (i.e. I haven't used
> any in-package's or the like). My experience says that it is time to
> reconsider the organization of this project. I'm willing to do this,
> but I'm not sure that I understand how best to do this in CL.
>
Look at the structure of your subdirectories to guide the
packagification process. Add a package.lisp file to each directory
containing the DEFPACKAGE form, and (IN-PACKAGE :WHATEVER) to each
source file in that directory, and then use DEFSYSTEM's :SERIAL
component (ensuring that the files are loaded serially so that
package.lisp is loaded first).
> C DLL's
> -------
> The C portion is in its own subdirectory containing an MSVC project.
> It produces a .dll which I move to a distinguished place on my disk
> when I'm satisfied that it works.
>
> Debugging C vs. Lisp
> --------------------
> Other than debugging the C in the MSVC environment as much as
> possible, there seems to be no way to debug C while in the lisp
> environment. When an error occurs in the .dll while running lisp, I
> get a dialog (usually a program fault) and then try to deduce what
> part of the C it was in by looking at the lisp stack trace. I then
> unload the .dll and switch to the MSVC environment to find / fix the
> bug. I don't believe that it is possible to invoke the MS debugger
> while running the .dll in lisp.
>
> Delivering
> ----------
> I've got a .bat file that starts up LWW from the command line to
> produce an executable. This is pretty much like the manual says.
>
I don't use MS-Windows so I don't have many suggestions for these
(except to consider Emacs - I don't write in C any more, but I used
to, and I've been forced to use MSVC for a project - Emacs was a *lot*
easier to use; I'm not sure how Emacsy LW is).
>
>
> Thanks
> Paul Tarvydas
In general, I suggest that you try defsystem once more (and ask here
if you run into problems), and that you not compile the project while
developing it.
By the way... I'm not sure if I mentioned this or not, but *DON'T
COMPILE*! :)
hth,
--
BPT <···@tunes.org> /"\ ASCII Ribbon Campaign
backronym for Linux: \ / No HTML or RTF in mail
Linux Is Not Unix X No MS-Word in mail
Meme plague ;) ---------> / \ Respect Open Standards
"Brian P Templeton" <···@tunes.org> ha scritto nel messaggio
···················@tunes.org...
> Paul Tarvydas <·········@tscontrols.com> writes:
> > - I want the compiler to pessimize maximally - I want as much debug
> > info as possible and don't care a whit about performance. At the
> > time I start up LWW, I immediately cut/paste:
> >
> > (proclaim '(optimize (debug 3) (safety 3) (speed 0) (space 0)))
> >
> > into the listener and hit RETURN. This line doesn't seem to have
> > any effect if I stick it into a .lisp file - I have to do it
> > manually in the Listener.
> >
> I'm not sure why it doesn't work in the source file (I don't use LW),
Of course it doesn't work. If in the file you are compiling
you put (PRINT 'FOO) do you expect to see FOO printed
_while_compiling_?
The OP should use DECLAIM instead.
Try evaluating (MACROEXPAND '(DECLAIM (OPTIMIZE DEBUG)))
to see the reason.
P.
From: Erik Winkels
Subject: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <87r8nrvr4v.fsf@xs4all.nl>
Brian P Templeton <···@tunes.org> writes:
> In general, I suggest that you try defsystem once more (and ask here
> if you run into problems), and that you not compile the project
> while developing it.
Let me put in a vote for defsystem too. I started using it for the
first time this weekend (thanks Matthew Danish). Right away 20% of
the project code --which consisted of defsystem-like functionality--
could be removed.
Just put in half a day to learn and play with it. The invested time
will pay itself back rather quick.
I can also send you the defsystem I've worked with, or a tarball of
the whole archive it's part of.
Documentation for mk:defsystem I've used:
http://ww.telent.net/cliki/mk-defsystem
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/clocc/clocc/src/defsystem-4/docs/
(also the dot-system files in the rest of the CLOCC
( http://clocc.sf.net/ ))
http://alpha.onshored.com/lisp-software/uncommonsql/MaiSQL.system
http://iris.usc.edu/home/raycharles/price/lisp/doc/lisp-utilities.ps
And a bunch of Google Groups posts which I didn't bookmark.
Erik Winkels <·······@xs4all.nl> writes:
> Brian P Templeton <···@tunes.org> writes:
> > In general, I suggest that you try defsystem once more (and ask here
> > if you run into problems), and that you not compile the project
> > while developing it.
>
> Let me put in a vote for defsystem too. I started using it for the
> first time this weekend (thanks Matthew Danish). Right away 20% of
> the project code --which consisted of defsystem-like functionality--
> could be removed.
>
> Just put in half a day to learn and play with it. The invested time
> will pay itself back rather quick.
>
> I can also send you the defsystem I've worked with, or a tarball of
> the whole archive it's part of.
The latest version of MK:DEFSYSTEM is available in the CLOCC CVS.
Please use that one or submit any changes to the clocc-devel mailing
list or to me directly.
Thanks
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
Marco Antoniotti <·······@cs.nyu.edu> writes:
> Erik Winkels <·······@xs4all.nl> writes:
>
> > I can also send you the defsystem I've worked with, or a tarball of
> > the whole archive it's part of.
>
> The latest version of MK:DEFSYSTEM is available in the CLOCC CVS.
> Please use that one or submit any changes to the clocc-devel mailing
> list or to me directly.
I think Erik was talking about the system definition he had written,
not the defsystem utility that parses it ;-)
-dan
--
http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources
From: Erik Winkels
Subject: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <87bsevvdg5.fsf@xs4all.nl>
Marco Antoniotti <·······@cs.nyu.edu> writes:
> > I can also send you the defsystem I've worked with, or a tarball
> > of the whole archive it's part of.
>
> The latest version of MK:DEFSYSTEM is available in the CLOCC CVS.
> Please use that one or submit any changes to the clocc-devel mailing
> list or to me directly.
My mistake, I meant the dot-system file I've worked on. Not
MK:DEFSYSTEM itself.
Erik Winkels <·······@xs4all.nl> writes:
> Marco Antoniotti <·······@cs.nyu.edu> writes:
> > > I can also send you the defsystem I've worked with, or a tarball
> > > of the whole archive it's part of.
> >
> > The latest version of MK:DEFSYSTEM is available in the CLOCC CVS.
> > Please use that one or submit any changes to the clocc-devel mailing
> > list or to me directly.
>
> My mistake, I meant the dot-system file I've worked on. Not
> MK:DEFSYSTEM itself.
Ooops. Sorry.
Cheers
--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.
From: Brian P Templeton
Subject: Re: LWW IDE, Productivity - Help Wanted
Date:
Message-ID: <87k7tjr3bp.fsf@tunes.org>
Brian P Templeton <···@tunes.org> writes:
> Paul Tarvydas <·········@tscontrols.com> writes:
>
[...]
>> Code Structure
>> --------------
>> Everything I've written is in the same namespace (i.e. I haven't used
>> any in-package's or the like). My experience says that it is time to
>> reconsider the organization of this project. I'm willing to do this,
>> but I'm not sure that I understand how best to do this in CL.
>>
> Look at the structure of your subdirectories to guide the
> packagification process. Add a package.lisp file to each directory
> containing the DEFPACKAGE form, and (IN-PACKAGE :WHATEVER) to each
> source file in that directory, and then use DEFSYSTEM's :SERIAL
> component (ensuring that the files are loaded serially so that
> package.lisp is loaded first).
>
Er, or better yet, make *one* package.lisp containing all of the
defpackage forms - that's a much simpler solution... don't know why I
missed that possibility the first time; sorry :/.
[...]
--
BPT <···@tunes.org> /"\ ASCII Ribbon Campaign
backronym for Linux: \ / No HTML or RTF in mail
Linux Is Not Unix X No MS-Word in mail
Meme plague ;) ---------> / \ Respect Open Standards
Paul Tarvydas <·········@tscontrols.com> writes:
> - I wonder if really know what the minimum set of IDE operations to
> make a source code change, to save it, to re-compile and test it,
> are.
I might have browsed the answers to quickly, so forgive me if you
got this answer already: keystroke control-shift-C will compile
the top-level form around your cursor.
> - I wonder if I've been polluted by the C / Unix model. I vaguely
> think that compile generally produces .fsl files and load loads .fsl
> files into the ide environment and that you have to do things in
> this order (compile, then (re)load). If I do misunderstand this,
> then maybe I'm using the ide in an overly-klunky way.
See the above. The right way to work with lisp is _incrementally_,
i.e. usually you're more effecient if you redefine in small chunks.
> - I don't save workspace images. I generally start from scratch every
> morning. Hmmm.
Do you actually *turn off* your workstation?? Leave it on and just
lock the screen...
--
(espen)
You've gotten answers to most of your questions and issues. But, just in
case it was overlooked:
In LWW if the cursor is inside a function you can right click the mouse to
bring up a menu for operations on the Buffer, Definition, Expression, and
File. This menu is a handy shortcut if you don't want to learn all the
Emacs equivalencies.
My favorite is Definition->Compile. That way you can compile just that
function, and not the whole file.
Also, in LWW there is under the Debug->Listener menu a "GUI Debugger", which
is often more convenient than the Listener when looking at lots of data,
moving through the stack frames, etc. Though, it does not support
stepping.
Consider wrapping your break points / debug messages in a macro so that
they can be left in your code and turned on and off as needed.
I use the listener a lot now for debugging. As soon as a function is
written I test it from the listener, and when one is causing trouble I will
sometimes debug it by typing each line of the function in turn into the
listener and evaluating it. Sort of a poor man's stepping.
My apologies if any of this info is redundant from the other posts or
obvious.
Good Luck,
Chris Perkins
Media Lab, Inc.
> Also, in LWW there is under the Debug->Listener menu a "GUI Debugger", which
> is often more convenient than the Listener when looking at lots of data,
> moving through the stack frames, etc. Though, it does not support
> stepping.
Whenever I've tried to use the debugger, I've found it extremely
difficult to track down which frame corresponds to a particular
function call and essentially impossible to identify parameters.
There are considerable numbers of additional frames that the runtime
environment adds and my local function/variable names are not shown.
The debugger seems to work OK, but the information it provides is very
hard to make any sense of.
I'm sure its due to a lack of configuration or incorrect use on my
part, could you be explicit about how to arrange things so the
debugger is more helpful?
Thanks,
Gregm
Greg Menke <···········@mindspring.com> writes:
> Whenever I've tried to use the debugger, I've found it extremely
> difficult to track down which frame corresponds to a particular
> function call and essentially impossible to identify parameters.
> There are considerable numbers of additional frames that the runtime
> environment adds and my local function/variable names are not shown.
> The debugger seems to work OK, but the information it provides is
> very hard to make any sense of.
I had the same problem but I think that the debugger's output has
considerably improved in version 4.2. If you're still using 4.1 you
should consider updating.
Edi.
--
Dr. Edmund Weitz
Hamburg
Germany
The Common Lisp Cookbook
<http://agharta.de/cookbook/>
"Greg Menke" <···········@mindspring.com> wrote in message
···················@mindspring.com...
>
> > Also, in LWW there is under the Debug->Listener menu a "GUI Debugger", which
> > is often more convenient than the Listener when looking at lots of data,
> > moving through the stack frames, etc. Though, it does not support
> > stepping.
>
> Whenever I've tried to use the debugger, I've found it extremely
> difficult to track down which frame corresponds to a particular
> function call and essentially impossible to identify parameters.
> There are considerable numbers of additional frames that the runtime
> environment adds and my local function/variable names are not shown.
> The debugger seems to work OK, but the information it provides is very
> hard to make any sense of.
>
> I'm sure its due to a lack of configuration or incorrect use on my
> part, could you be explicit about how to arrange things so the
> debugger is more helpful?
Was this compiled or interpreted code? In some cases, the debugger doesn't
manage to hide the details of the interpreter. When debugging compiled code,
you probably want to check that (optimize (debug 3)) was in effect during
compilation. Also, check the Tools > Preferences dialog in the debugger to
ensure that none of the "View Frame" options are selected, since they can add
frames that are not in your code.
--
Martin Simmons, Xanalys Software Tools
······@xanalys.com
rot13 to reply
> > I'm sure its due to a lack of configuration or incorrect use on my
> > part, could you be explicit about how to arrange things so the
> > debugger is more helpful?
>
> Was this compiled or interpreted code? In some cases, the debugger doesn't
> manage to hide the details of the interpreter. When debugging compiled code,
> you probably want to check that (optimize (debug 3)) was in effect during
> compilation. Also, check the Tools > Preferences dialog in the debugger to
> ensure that none of the "View Frame" options are selected, since they can add
> frames that are not in your code.
THANK YOU! I've been stuck without a debugger for months! I sure
wish it had been in the docs before this... though I do appreciate the
content they do have. Putting in debug 3 did the trick nicely.
My 4.2 upgrade is in the works, but man does this help out-- thanks
again.
Gregm
Using 4.1, if you double-click on a reasonable-looking frame, LWW should
jump to the editor and highlight the source code. If you're sure that it's
highlighting the wrong code, then you've probably got too much optimization
enabled (see proclaim, declaim, my original question). If the parameter
values are showing up as :DONT-KNOW, I think it's because there's too much
optimization turned on (the optimizer appears to figure out that certain
variables are not used within a particular frame and gets rid of them).
Over-stating the obvious: if you want to look at variables that don't show
up in the auto-magic debugger display, you can always use the Listener -
single-click on a frame, switch to the Listener, type an expression and hit
CR (i.e. if you want to see the value of "x", hit "x<CR>"). If the variable
does't exist within the frame that you clicked on, you will get an error.
Also, you can pull open an LispWorks>>Tools>>Inspector and Edit>>Link it to
the Listener. The Inspector will then display/inspect the last value that
appears in the Listener. Double-clicking, etc, within the Inspector then
lets you traverse the values.
4.2 is better in at least two ways: the gui debugger comes up by default
(you don't have to pull down the menu), the stack frames are displayed in
tree view - you can "expand" more than one frame at a time and see the
variables within those frames all at once.
pt