Firstly, a terminoligy question:
(let ((what-is-this-called? 'something))
(defun closure () (print what-is-this-called?)))
Is there a pithy term to describe "what-is-this-called?"? ... in other
words, to describe variables captured(?)/enclosed(?) by a closure.
Let me refer to such variables as "enclosed variables" for the time
being. Now, suppose I have a number of closures sharing enclosed
variables, such as
(let (ev1 ev2 ev3 .... evn)
(defun closure-1 ...)
...
(defun closure-n ...))
Is there a way of getting at the values of the enclosed variables? I
thought I might do this by including a closure which takes symbols as
arguments, and returns the values of the enclosed variables named by
those symbols ... but I don't see how to make this work (eval uses the
null lexical environment, a variable capturing macro would be expanded
too soon ...).
Thanks,
Jacek
Jacek Generowicz <················@cern.ch> wrote in message
> Is there a way of getting at the values of the enclosed variables? I
No. They are completely private. Only the closure's code component,
the function, can access these variables. You can create several
closures that capture the same bindings; these can become accessors. A
vector of these closures then behaves like an object in a primitive
object-system like that of C++ or Java.
(defstruct bank-account-operations
(deposit)
(withdraw)
(obtain-balance))
(defun make-bank-account (initial-balance)
(let ((balance initial-balance))
(make-bank-account-operations
:deposit (lambda (amount) (incf balance amount))
:withdraw (lambda (amount) (decf balance amount))
:obtain-balance (lambda () balance))))
(defvar *b* (make-bank-account 42))
(funcall (bank-account-operations-obtain-balance *b*))
==> 42
etc. Basic polymorphism is easy to do. What the operations do is
entirely up to the site which creates the bank-account-operations
object. So you could make another version of make-bank-account whose
closures talk to a remote database via SQL.
> thought I might do this by including a closure which takes symbols as
> arguments, and returns the values of the enclosed variables named by
> those symbols ... but I don't see how to make this work (eval uses the
> null lexical environment, a variable capturing macro would be expanded
> too soon ...).
Your observation is right; there is no way to do this. And the reason
for that is that these symbols don't exist! Lexical variables in Lisp
are just like lexical variables in a language like C; the compiler can
strip away the symbols, and produce some low-level references to the
variables---or even cache them in machine registers, if possible. This
is a very good thing, because it means that lexical scoping can be
very efficient!
For example, Corman Lisp is an implementation that always compiles
everything you put into it. Here is some sample output:
(disassemble (let ((counter 0)) (lambda () (incf counter))))
;#x0: 55 push ebp
;#x1: 8BEC mov ebp,esp
;#x3: 57 push edi
;#x4: 85C9 test ecx,ecx
;#x6: 7406 je 000E
;#x8: FF96BC100000 call near dword ptr [esi+0000010BCh]
;#xE: 8B7DFC mov edi,[ebp-00004h]
;#x11: 8B7F03 mov edi,[edi+00003h]
;#x14: 8B47FC mov eax,[edi-00004h]
;#x17: BA08000000 mov edx,0008
;#x1C: A807 test al,007h
;#x1E: 7506 jne 0026
;#x20: 03C2 add eax,edx
;#x22: 7108 jno 002C
;#x24: 2BC2 sub eax,edx
;#x26: FF96FC170000 call near dword ptr [esi+0000017FCh]
;#x2C: 8B7DFC mov edi,[ebp-00004h]
;#x2F: 8B7F03 mov edi,[edi+00003h]
;#x32: 8947FC mov [edi-00004h],eax
;#x35: B901000000 mov ecx,0001
;#x3A: 8BE5 mov esp,ebp
;#x3C: 5D pop ebp
;#x3D: C3 ret
The symbol ``counter'' is nowhere in sight; the machine language uses
indirect index addressing to locate things.
In the language C, you can take the address of a local variable and
pass the pointer down to some function (and in principle you could
return it, if C had garbage collected locals). In Lisp, we don't have
such a low level construct; only closures. Closures effectively ``take
the address'' of local variables, and package them into a high level
object that has a well-defined functional interface to the entire
collection of them. So all remote accesses to the locals are
disciplined---like in the bank account example.
* Jacek Generowicz
| Is there a pithy term to describe "what-is-this-called?"? ... in other
| words, to describe variables captured(?)/enclosed(?) by a closure.
Binding. A closure is closed over bindings, hence the name.
| Is there a way of getting at the values of the enclosed variables?
Just reference them. If you want to have a mapping from symbol to value,
you have to the exact same thing you would have to do with any other
lexical binding: Do it yourself.
(let ((a 2) (b 3) (c 5) (d 7))
(case <symbol> (a a) (b b) (c c) (d d)))
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
Erik Naggum <····@naggum.no> writes:
> * Jacek Generowicz
> | Is there a way of getting at the values of the enclosed variables?
>
> Just reference them. If you want to have a mapping from symbol to value,
> you have to the exact same thing you would have to do with any other
> lexical binding: Do it yourself.
To clarify further, one can only "just reference them" when the source
code can be modified in such a way that the reference is lexical.
To get at the value of a closed-over variable in a dynamic situation,
you have to count on a debugger, just as you would for any
non-closed-over lexical variable.
That said, I just tried Jacek Generowicz' example in Allegro CL and
am embarrassed to say that the debugger gets the count of locals
wrong in a frame with closed-over variables. So the closed-over
variable doesn't show up. I've fixed this, and I may put out a patch
if and when I am confident I didn't break anything.
After having fixed the bug, a modified version of the original
code is correctly handled by the debugger:
CL-USER(2): (shell "cat foo.cl")
(let ((what-is-this-called? 'something))
(defun closure ()(let ((a (print what-is-this-called?))) (break) a)))
0
CL-USER(3): :cl foo
;;; Compiling file foo.cl
;;; Writing fasl file foo.fasl
;;; Fasl write complete
; Fast loading /acl/duane/acl70/src/foo.fasl
CL-USER(4): (closure)
SOMETHING
Break: call to the `break' function.
Restart actions (select using :continue):
0: return from break.
1: Return to Top Level (an "abort" restart).
2: Abort entirely from this process.
[1c] CL-USER(5): :loc
Compiled lexical environment:
0(LOCAL): A: SOMETHING
1(LOCAL): WHAT-IS-THIS-CALLED?: SOMETHING
[1c] CL-USER(6):
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Erik Naggum <····@naggum.no> writes:
> * Jacek Generowicz
> | Is there a pithy term to describe "what-is-this-called?"? ... in other
> | words, to describe variables captured(?)/enclosed(?) by a closure.
>
> Binding. A closure is closed over bindings, hence the name.
Consider
(let ((a 1))
(defun af () (print 'yawn)))
(let ((b 2))
(defun bf () (print b)))
(defun cf ()
(let ((c 3))
(print c)))
A, b and c (or rather their associations with the values 1, 2 and 3
respectively ?) are all bindings, are they not? Is b not
significantly different from a, in that its extent is different?
Is there a pithy name to describe the binding b, which highlights its
difference from a ?
> | Is there a way of getting at the values of the enclosed variables?
>
> Just reference them. If you want to have a mapping from symbol to
> value, you have to the exact same thing you would have to do with
> any other lexical binding: Do it yourself.
>
> (let ((a 2) (b 3) (c 5) (d 7))
> (case <symbol> (a a) (b b) (c c) (d d)))
Indeed. Thank you.
* Jacek Generowicz
| Is there a pithy name to describe the binding b, which highlights its
| difference from a?
I see no difference in the binding and no need to name it differently.
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
Erik Naggum <····@naggum.no> writes:
> * Jacek Generowicz
> | Is there a pithy name to describe the binding b, which highlights its
> | difference from a?
>
> I see no difference in the binding and no need to name it differently.
OK, last try ... could you suggest a better subject line for my
original message ?
* Jacek Generowicz
| OK, last try ... could you suggest a better subject line for my
| original message?
Well, no. What you try to do is almost trivial, as the snippet of code I
wrote to handle it shows. But I think I understand why you think this is
a big deal and I do not. You want to create a mini-environment in which
you can refer to bindings in a slightly different way than normal lexical
scope, and not quite in dynamic scope, either. However, I see no need to
go outside the normal lexical scope as you have explained things so far
and I am sort of waiting to hear more from you why you think so.
Perhaps things could be eased for you with symbol macros? You can then
set up an environment that will map a symbol to a function call that may
well be a closure that captures the symbol's value from an environment
other than your lexical environment. Symbol macros can be defined both
globally and locally. See `define-symbol-macro� and `symbol-macrolet�.
Perhaps you really need a dynamic environment for your mini-"language"?
Setting it up with `progv� is easy and straightforward and if you make no
internal bindings in your mini-language, will effectively be identical to
a lexical environment created on the fly. You can then use `eval�, too.
If I understand your problem description correctly, you have ventured
down a blind alley and ask us to help you break through one of the walls
around you. I try to help you turn back to where you came from and make
a different choice. It is not unlike rats in mazes. Some quickly learn
that scaling the walls does not work, while others do not. Some act like
believers in their prior understanding of the concept and curse the walls
(apologies for the anthropomorphism) and try to claw their way through
them. Some figure out that blind alleys are no good and learn to spot
them early enough that they do not have to back up very often. In human
terms, the stronger you believe that you have found a good solution the
more effort and setbacks you are willing to accept. You find people who
go "this should do it! <pause> shit!" over and over. Sometimes, less
faith in one's ability to intuit solutions before knowing the subject at
hand can do wonders for the ability to find really good solutions. Also
see my current and long-standing .signature.
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
Erik Naggum <····@naggum.no> writes:
> * Jacek Generowicz
> | OK, last try ... could you suggest a better subject line for my
> | original message?
>
> Well, no. What you try to do is almost trivial,
I would even be tempted to drop "almost".
> as the snippet
> of code I wrote to handle it shows. But I think I understand why
> you think this is a big deal and I do not. You want to create a
> mini-environment in which you can refer to bindings in a slightly
> different way than normal lexical scope, and not quite in dynamic
> scope, either. However, I see no need to go outside the normal
> lexical scope as you have explained things so far and I am sort of
> waiting to hear more from you why you think so.
What I want is to cache function evaluations. I have done this. It
gave me a speedup factor of around 17. Given the time I invested in
the task, and the speed at which the program now runs, I am (and more
importantly, my "client" is) satisfied with the result. Later, it
turned out that my client is interested in a subset of the cached
values, as well as the "final" answer in the calculation of which they
are used. That's fine, this was already possible with the initial
implementation. However, I wanted to make my client's life a little
easier (and give a small performance benefit) by letting him write
(get-cache '(a b c d e))
instead of
(list (a-opt) (b-opt) (c-opt) (d-opt) (e-opt))
You, and others, have addressed this to my satisfaction.
There was also a simple terminology question I have been trying to
ask. That question is: What is the shortest way of expressing "those
bindings over which a closure has been formed" in a way which will be
understood by the majority of clued-up Common Lisp users? Some people
have made suggestions; you seem to imply that the question has no
substance.
You are mistaken when you suggest that I think that any of this is a
big deal.
While the proportion of my time I can afford to spend on this is very
small indeed, I am always keen to learn ... particularly when it comes
to Common Lisp, so I will try to investigate various suggestions
people have made.
> Perhaps things could be eased for you with symbol macros?
Given my current understanding of symbol macros, I don't see how they
could help, but I'll look into it if I find the time.
> Perhaps you really need a dynamic environment for your
> mini-"language"? Setting it up with `progv� is easy and
> straightforward and if you make no internal bindings in your
> mini-language, will effectively be identical to a lexical
> environment created on the fly. You can then use `eval�, too.
Maybe you could expand a little on this.
> If I understand your problem description correctly, you have
> ventured down a blind alley and ask us to help you break through
> one of the walls around you.
> You find people who go "this should do it! <pause> shit!" over
> and over.
The way I experienced it was:
I wonder whether this will work? - It does!
Now, can this be made a bit prettier? - Yes, that'll do.
Then I moved on. Maybe there are better ways to do it overall, but
what I've got is good enough for my current needs.
The only thing about which I am (slightly) unsatisfied is the
terminology issue.
Jacek.
* Jacek Generowicz
| You are mistaken when you suggest that I think that any of this is a big
| deal.
OK.
| Maybe you could expand a little on this.
No.
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
Jacek Generowicz wrote in message ...
>Erik Naggum <····@naggum.no> writes:
>
>> * Jacek Generowicz
>> | Is there a pithy term to describe "what-is-this-called?"? ... in other
>> | words, to describe variables captured(?)/enclosed(?) by a closure.
>>
>> Binding. A closure is closed over bindings, hence the name.
>
>Consider
>
>(let ((a 1))
> (defun af () (print 'yawn)))
>
>(let ((b 2))
> (defun bf () (print b)))
>
...
>
>Is there a pithy name to describe the binding b, which highlights its
>difference from a ?
How about "captured"? I have seen that used in re macros, might not be the
right word for closed-over bindings.
kenny
clinisys
Jacek Generowicz wrote in message ...
>Firstly, a terminoligy question:
>
> (let ((what-is-this-called? 'something))
> (defun closure () (print what-is-this-called?)))
>
>Is there a pithy term to describe "what-is-this-called?"? ... in other
>words, to describe variables captured(?)/enclosed(?) by a closure.
>
>Let me refer to such variables as "enclosed variables" for the time
>being. Now, suppose I have a number of closures sharing enclosed
>variables, such as
>
> (let (ev1 ev2 ev3 .... evn)
> (defun closure-1 ...)
> ...
> (defun closure-n ...))
>
>Is there a way of getting at the values of the enclosed variables? I
>thought I might do this by including a closure which takes symbols as
>arguments, and returns the values of the enclosed variables named by
>those symbols ... but I don't see how to make this work ...
Don't use eval, use a case statement hard-coded for each evn.
Or go crazy and add getter and setter closures for each evn:
(defun get-ev1 () ev1)
..etc...
kenny
clinisys
"Kenny Tilton" <·······@nyc.rr.com> writes:
> Or go crazy and add getter and setter closures for each evn:
>
> (defun get-ev1 () ev1)
> ..etc...
This was my first thought, and given that the closed-over bindings in
my closure are created by a macro arleady it would be trivial to have
it generate these too, but I thought that
(get-bindings `(a b c d e))
would be easier on the user than
(list (get-a) (get-b) (get-c) (get-d) (get-e))
which is sort of usage that seems useful in my case.
Apart from that I was just curious whether I was missing any other
possibilities.
Thanks,
In article <···············@pcitapi22.cern.ch>,
Jacek Generowicz <················@cern.ch> wrote:
>Firstly, a terminoligy question:
>
> (let ((what-is-this-called? 'something))
> (defun closure () (print what-is-this-called?)))
>
>Is there a pithy term to describe "what-is-this-called?"? ... in other
>words, to describe variables captured(?)/enclosed(?) by a closure.
I've heard it called a "closed-over variable".
>Let me refer to such variables as "enclosed variables" for the time
>being. Now, suppose I have a number of closures sharing enclosed
>variables, such as
>
> (let (ev1 ev2 ev3 .... evn)
> (defun closure-1 ...)
> ...
> (defun closure-n ...))
>
>Is there a way of getting at the values of the enclosed variables? I
>thought I might do this by including a closure which takes symbols as
>arguments, and returns the values of the enclosed variables named by
>those symbols ... but I don't see how to make this work (eval uses the
>null lexical environment, a variable capturing macro would be expanded
>too soon ...).
(let ((ev1 val1) (ev2 val2) ...)
(defun get-ev-value (var-name)
(ecase var-name
(ev1 ev1)
(ev2 ev2)
...))
...)
If you need to do this often, you could package it up in a macro.
Why do you need to do this in the first place? Maybe you should be using
something other than closures, such as hash tables or CLOS.
--
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> (let ((ev1 val1) (ev2 val2) ...)
> (defun get-ev-value (var-name)
> (ecase var-name
> (ev1 ev1)
> (ev2 ev2)
> ...))
> ...)
Thanks.
> If you need to do this often, you could package it up in a macro.
>
> Why do you need to do this in the first place?
I've got some (mathematical) functions which are built up of lots of
calls to other functions; evaluating the one at the root of the call
tree results in calling some of the others many times with the same
arguments. It seemed a fairly simple optimization to wrap their
definitions in a macro which creates parallel definitions (for every
(defun fn ...) it creates a (defun fn-opt ...)) in which the calls
are substituted by references to cached values (as closed-over
variables). Now I can call the closure (calculate-values x y z ...)
with appropriate values (x y z ...), and it will calculate the values
of all the functions by evaluating each one no more than once. This
speeded up my program by a factor of 15-20. Initially I was interested
only in the function at the root of the tree, but now it would be
useful to have a look at what some of its components are up to. I
already have all the fn-opt closures which could get me this
information, but they all do unnecessary work; I could (as Kenny
pointed out) add getter closures for each binding ...
> Maybe you should be using something other than closures, such as
> hash tables or CLOS.
This had crossed my mind too. But the approach I tried seemed to be
the natural next step from where I was, would almost certainly be more
efficient than CLOS ... and, besides, I was curious what the
possibilities are.
Jacek Generowicz <················@cern.ch> writes:
> I've got some (mathematical) functions which are built up of lots of
> calls to other functions; evaluating the one at the root of the call
> tree results in calling some of the others many times with the same
> arguments.
This sounds like a good fit for a technique known a memoization. The
idea is exactly to avoid computing values more than once, by caching
the results for a given set of arguments. The caching version of a
function is constructed automatically, to avoid cluttering up the
logic in your initial function.
This technique is discussed to some extent in one of the books by Paul
Graham, but unfortunately I don't have either of them available here,
and so cannot check if it's in "ANSI Common Lisp" or "On Lisp".
A quick search on google for "memoization common lisp" seemed to turn
up quite a few ready made packages for doing memoization.
> Initially I was interested only in the function at the root of the
> tree, but now it would be useful to have a look at what some of its
> components are up to. I already have all the fn-opt closures which
> could get me this information, but they all do unnecessary work;
If you memoized each function, you could just call it and receive a
cached result if there was one.
> > Maybe you should be using something other than closures, such as
> > hash tables or CLOS.
I believe it is common for memoization packages to use hash tables to
store the cached results, but I have not seen enough implementations
to conclude.
...Peder...
--
Cogito ergo panta rei.
Peder O. Klingenberg wrote:
> This sounds like a good fit for a technique known a memoization.
[...elided details of memoization...]
I agree.
> This technique is discussed to some extent in one of the books by Paul
> Graham, but unfortunately I don't have either of them available here,
> and so cannot check if it's in "ANSI Common Lisp" or "On Lisp".
FWIW I think that PAIP[1] might be a better place to look for examples.
:)w
[1] www.norvig.com/paip.html
Jacek Generowicz wrote:
>
> I've got some (mathematical) functions which are built up of lots of
> calls to other functions; evaluating the one at the root of the call
> tree results in calling some of the others many times with the same
> arguments. It seemed a fairly simple optimization to wrap their
> definitions in a macro which creates parallel definitions (for every
> (defun fn ...) it creates a (defun fn-opt ...)) in which the calls
> are substituted by references to cached values (as closed-over
> variables). Now I can call the closure (calculate-values x y z ...)
> with appropriate values (x y z ...), and it will calculate the values
> of all the functions by evaluating each one no more than once. This
> speeded up my program by a factor of 15-20. Initially I was interested
> only in the function at the root of the tree, but now it would be
> useful to have a look at what some of its components are up to. I
> already have all the fn-opt closures which could get me this
> information, but they all do unnecessary work; I could (as Kenny
> pointed out) add getter closures for each binding ...
>
> > Maybe you should be using something other than closures, such as
> > hash tables or CLOS.
>
> This had crossed my mind too. But the approach I tried seemed to be
> the natural next step from where I was, would almost certainly be more
> efficient than CLOS ... and, besides, I was curious what the
> possibilities are.
Have you looked at how Norvig does memoizing in PAIP? Seems to be
somewhat simpler to do with hash tables.
--
Martti Halminen <···············@kolumbus.fi> writes:
> Have you looked at how Norvig does memoizing in PAIP? Seems to be
> somewhat simpler to do with hash tables.
·····@news.klingenberg.no (Peder O. Klingenberg) writes:
> This sounds like a good fit for a technique known a memoization.
Yes, I am familiar with memoization, and have used it before.
It this case, the raison d'etre of all but one of the functions is to
help compute the value of THE ONE BIG FUNCTION (well, almost). The
significance of this is that I _know_ that I will get a squillion
calls to the the smaller functions (particularly the ones at the
leaves of the call structure) with exactly the same arguments. Then I
do whatever I want with the values. Then I lose all interest in these
values. Then another squillion calls with identical values follow
... and so on.
Memoization would eat up memory on storing values I don't care about
any more, and would do a squillion hash lookups where the technique I
use references a (closed-over) binding a squillion times.
> I believe it is common for memoization packages to use hash tables to
> store the cached results, but I have not seen enough implementations
> to conclude.
The two or three implementations I have seen (and the few I have
written) all used hash tables.
As others have pointed out, there is not a portable way to do this.
However, with Corman Lisp you can get access, through a non-portable
method, to all closed over bindings accessible from a function.
(ccl:function-environment #'closure)
will return a pseudo-structure containing the bindings. Although the
returned structure is not a real type, you can access the elements of
it using the low-level UREF operator
eg:
(uref (ccl:function-environment #'closure) 2)
returns the first binding, the second is in slot 3, etc.
You can also use the debugger. While in the debugger you can set the
values of the bindings, read them, or even use them in arbitrary
expressions.
Roger
-----------------------------------------------------------
On 17 Oct 2002 15:13:08 +0200, Jacek Generowicz
<················@cern.ch> wrote:
>
>Is there a way of getting at the values of the enclosed variables? I
>thought I might do this by including a closure which takes symbols as
>arguments, and returns the values of the enclosed variables named by
>those symbols ... but I don't see how to make this work (eval uses the
>null lexical environment, a variable capturing macro would be expanded
>too soon ...).
>
>
>Thanks,
>
>Jacek