Hi all,
I'm still absorbing all of the ideas from ILC and feel energized by the
event and the discussions that followed. Thanks everybody!
Something that occurred to me shortly afterward was the possibility of
doing queries on my runtime data structures. This would be at a layer
above an inspector, analogous to how Mac OS X 10.4's Spotlight works at
a level above the file browser.
Here's the problem I ran into. I ran my program and noticed that
something that should have been output once was instead output eight
times. I wanted to search back through the execution to find the
_first_ time something in my code was run eight times. It would have
been useful to have a program slice so that I could explore the history
of how the data was generated. Even then I'd have to walk through the
execution myself, however. It'd be really nice to be able to do a query
like "show me the function call that generated the earliest created
list of length = 8 containing as a sub-element somewhere the string:
'me too but not me three'".
This seems like a feature perfectly suited for Lisp for two reasons.
First, we already live in the world of the REPL rather than batch
model, so interacting with runtime structures directly isn't too far
out. Second, and here's where I'm going out on a limb, but if Lisp's
data is already tagged, then it seems like it should be feasible to
traverse and analyze data structures independent of the contexts that
generated and used them.
Is anyone familiar with research or, even better, implementations of
ideas like this? It seems like an interesting problem from an
information retrieval perspective in terms of figuring out ways to
search the current data structures without indices, or only using the
compilers internal indices. On the other hand, I can see it giving CL
compiler writers heartburn and/or fits.
Thanks for any feedback!
-Earl
·······@gmail.com writes:
> Hi all,
>
> I'm still absorbing all of the ideas from ILC and feel energized by the
> event and the discussions that followed. Thanks everybody!
>
> Something that occurred to me shortly afterward was the possibility of
> doing queries on my runtime data structures. This would be at a layer
> above an inspector, analogous to how Mac OS X 10.4's Spotlight works at
> a level above the file browser.
>
> Here's the problem I ran into. I ran my program and noticed that
> something that should have been output once was instead output eight
> times. I wanted to search back through the execution to find the
> _first_ time something in my code was run eight times. It would have
> been useful to have a program slice so that I could explore the history
> of how the data was generated. Even then I'd have to walk through the
> execution myself, however. It'd be really nice to be able to do a query
> like "show me the function call that generated the earliest created
> list of length = 8 containing as a sub-element somewhere the string:
> 'me too but not me three'".
>
> This seems like a feature perfectly suited for Lisp for two reasons.
> First, we already live in the world of the REPL rather than batch
> model, so interacting with runtime structures directly isn't too far
> out. Second, and here's where I'm going out on a limb, but if Lisp's
> data is already tagged, then it seems like it should be feasible to
> traverse and analyze data structures independent of the contexts that
> generated and used them.
>
> Is anyone familiar with research or, even better, implementations of
> ideas like this? It seems like an interesting problem from an
> information retrieval perspective in terms of figuring out ways to
> search the current data structures without indices, or only using the
> compilers internal indices. On the other hand, I can see it giving CL
> compiler writers heartburn and/or fits.
You could start with:
(defmacro trace-all (&optional (package *package*))
(let ((all-fun (do-symbols (s package)
(when (fboundp s) (push s all-fun)))))
`(trace ,@all-fun)))
(defmacro untrace-all (&optional (package *package*))
(let ((all-fun (do-symbols (s package)
(when (fboundp s) (push s all-fun)))))
`(untrace ,@all-fun)))
(with-open-file (*trace-output* "/tmp/trace" :direction :output
:if-exists :supersede :if-does-not-exist :create)
(trace-all)
(unwind-protect (main)
(untrace-all)))
--
__Pascal Bourguignon__ http://www.informatimago.com/
Nobody can fix the economy. Nobody can be trusted with their finger
on the button. Nobody's perfect. VOTE FOR NOBODY.
"Earl J. Wagner" <·······@gmail.com> writes:
> Thanks for the tip; I wasn't aware of do-symbols. I'm not just
> interested in tracing functions, however. See my other response.
Ah but you don't realize that everything's a function(*) in lisp:
[55]> (ext:without-package-lock ("COMMON-LISP") (trace + - * / sin))
WARNING: TRACE: redefining function + in top-level, was defined in C
;; Tracing function +.
WARNING: TRACE: redefining function - in top-level, was defined in C
;; Tracing function -.
WARNING: TRACE: redefining function * in top-level, was defined in C
;; Tracing function *.
WARNING: TRACE: redefining function / in top-level, was defined in C
;; Tracing function /.
WARNING: TRACE: redefining function SIN in top-level, was defined in C
;; Tracing function SIN.
(+ - * / SIN)
[56]> (SIN (/ (* 2 PI) 3))
1. Trace: (* '2 '3.1415926535897932385L0)
1. Trace: * ==> 6.283185307179586477L0
1. Trace: (/ '6.283185307179586477L0 '3)
1. Trace: / ==> 2.0943951023931954923L0
1. Trace: (SIN '2.0943951023931954923L0)
1. Trace: SIN ==> 0.8660254037844386468L0
0.8660254037844386468L0
(*) but special operators. And for macros, you can use
*macroexpansion-hook* if your interested in them.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Small brave carnivores
Kill pine cones and mosquitoes
Fear vacuum cleaner
In article <·······················@g47g2000cwa.googlegroups.com>,
·······@gmail.com wrote:
> Hi all,
>
> I'm still absorbing all of the ideas from ILC and feel energized by the
> event and the discussions that followed. Thanks everybody!
>
> Something that occurred to me shortly afterward was the possibility of
> doing queries on my runtime data structures. This would be at a layer
> above an inspector, analogous to how Mac OS X 10.4's Spotlight works at
> a level above the file browser.
>
> Here's the problem I ran into. I ran my program and noticed that
> something that should have been output once was instead output eight
> times. I wanted to search back through the execution to find the
> _first_ time something in my code was run eight times. It would have
> been useful to have a program slice so that I could explore the history
> of how the data was generated. Even then I'd have to walk through the
> execution myself, however. It'd be really nice to be able to do a query
> like "show me the function call that generated the earliest created
> list of length = 8 containing as a sub-element somewhere the string:
> 'me too but not me three'".
>
> This seems like a feature perfectly suited for Lisp for two reasons.
> First, we already live in the world of the REPL rather than batch
> model, so interacting with runtime structures directly isn't too far
> out. Second, and here's where I'm going out on a limb, but if Lisp's
> data is already tagged, then it seems like it should be feasible to
The only "tag" that's already there is a type code. You need a much
more invasive infrastructure to do what you're asking.
> traverse and analyze data structures independent of the contexts that
> generated and used them.
>
> Is anyone familiar with research or, even better, implementations of
> ideas like this? It seems like an interesting problem from an
> information retrieval perspective in terms of figuring out ways to
> search the current data structures without indices, or only using the
> compilers internal indices. On the other hand, I can see it giving CL
> compiler writers heartburn and/or fits.
I remember seeing research papers years ago about execution environments
that would keep track of stuff like this. They had features like
letting you run the execution backwards, to see when a data structure
was created. I saw these papers coming out of the MIT LCS, and in
proceedings for various ACM and IEEE programming conferences.
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: Roland Kaufmann
Subject: Re: querying runtime data structures
Date:
Message-ID: <tl2hdexisxd.fsf@space.at>
>>>>> ewagner <·······@gmail.com> writes:
> Hi all,
> I'm still absorbing all of the ideas from ILC and feel energized by the
> event and the discussions that followed. Thanks everybody!
> Something that occurred to me shortly afterward was the possibility of
> doing queries on my runtime data structures. This would be at a layer
> above an inspector, analogous to how Mac OS X 10.4's Spotlight works at
> a level above the file browser.
> Here's the problem I ran into. I ran my program and noticed that
> something that should have been output once was instead output eight
> times. I wanted to search back through the execution to find the
> _first_ time something in my code was run eight times. It would have
> been useful to have a program slice so that I could explore the history
> of how the data was generated. Even then I'd have to walk through the
> execution myself, however. It'd be really nice to be able to do a query
> like "show me the function call that generated the earliest created
> list of length = 8 containing as a sub-element somewhere the string:
> 'me too but not me three'".
> This seems like a feature perfectly suited for Lisp for two reasons.
> First, we already live in the world of the REPL rather than batch
> model, so interacting with runtime structures directly isn't too far
> out. Second, and here's where I'm going out on a limb, but if Lisp's
> data is already tagged, then it seems like it should be feasible to
> traverse and analyze data structures independent of the contexts that
> generated and used them.
> Is anyone familiar with research or, even better, implementations of
> ideas like this? It seems like an interesting problem from an
> information retrieval perspective in terms of figuring out ways to
> search the current data structures without indices, or only using the
> compilers internal indices. On the other hand, I can see it giving CL
> compiler writers heartburn and/or fits.
David Ungar, Henry Lieberman, Christopher Fry
Debugging and the Experience of Immediacy
Communications of the ACM, April 1997, Vol. 40, No. 4, pp. 38--43
they describe ZStep 95, a debugging environment which allows (among
other things) stepping back in time. The examples are all written in
Common Lisp. See also
http://web.media.mit.edu/~lieber/Lieberary/ZStep/ZStep.html
Hope this helps.
regards
Roland
Thanks guys,
I've seen this and it's good work. In fact, I studied with Henry and it
was a mind-opening experience. Just having an open-source reversible
debugger or program slicer for Lisp would be a great first step. I was
thinking about going beyond that, however, to do queries on a record of
execution itself.
Here's a concrete scenario. Suppose you see that the final value for x
is garbled. The var x is computed from y and z and those are computed
from a lot of other variables. It'd be nice to be able to generate a
tree of structs for all of the intermediate variables such that at the
top level is the struct with result = x and value =(fn y z). You could
run a function eval-tree to eval all the structs' value slot sexps and
regenerate the broken value for x. You could also traverse the tree
using the standard list operations like find and, of course, write
functions to do tree traversal, another strength of lisp. Finally, as
with standard slicing, the structs would store refs to particular sexps
in the original code. With support from the environment, you could jump
to the code corresponding to (fn y z) and play the execution in
reverse, for example.
One way to think about this is that with macros we use Lisp to do
operations on code at compile time. Code refactoring involves operating
on code at edit time. The tools I'm talking about here operate on
execution-time representations of code. To me, these directions
represent the fulfillment of the potential Lisp offers with code being
data.
Let me know if you know of anyone doing work along these lines.
Thanks!
-Earl