From: Tom Evans
Subject: beginner closures
Date: 
Message-ID: <3F857177.2F4BC9CA@eng.cam.ac.uk>
I've just had the session below with lisp, and didn't really understand 
the results.  Was kind of hoping one of you clever people could explain 
it to me nice and simply, please.  :-)

[15]> (setq closurelist nil)
NIL
[16]> (dolist (i '(a b))
              (push #'(lambda () (format t "~a" i)) closurelist))
NIL
[17]> (funcall (first closurelist))
B
NIL
[18]> (funcall (second closurelist))
B
NIL

I sort of expected the result of calling the first closure to be A, and 
the second B.  Now, it would appear that both in fact hold on to the 
(symbol?) i, and show i's value at the time of the call to the closure.

Is that understanding remotely correct?

How can I do what I want to do?  (create stored functions where the value 
of i can be put separately into each one)

thanks,
Tom

From: Kaz Kylheku
Subject: Re: beginner closures
Date: 
Message-ID: <cf333042.0310091048.56899a27@posting.google.com>
Tom Evans <···········@uk.co.yahoo.reverse> wrote in message news:<·················@eng.cam.ac.uk>...
> I sort of expected the result of calling the first closure to be A, and 
> the second B.  Now, it would appear that both in fact hold on to the 
> (symbol?) i, and show i's value at the time of the call to the closure.

To answer your embedded question, they hold on to the same variable
binding. A lexical variable is a binding: a storage location that
holds a value. It is these bindings that are captured by a closure.
Your closure does not retain the symbol i, but the binding to which
that i refers in the given context.

Lexical variables are denoted by a symbols in the source code. These
symbols may disappear in compilation, unless they need to be retained
for purposes like more user-friendly interactive debugging, which
allows the user to interrupt the evaluation of some closure body and
ask for the value of i in that context, rather than, say, the third
captured binding in the closure vector.
From: Tom Evans
Subject: Re: beginner closures
Date: 
Message-ID: <3F867098.C4985435@eng.cam.ac.uk>
Kaz Kylheku wrote:
> 
> Tom Evans <···········@uk.co.yahoo.reverse> wrote in message news:<·················@eng.cam.ac.uk>...
> > I sort of expected the result of calling the first closure to be A, and
> > the second B.  Now, it would appear that both in fact hold on to the
> > (symbol?) i, and show i's value at the time of the call to the closure.
> 
> To answer your embedded question, they hold on to the same variable
> binding. A lexical variable is a binding: a storage location that
> holds a value. It is these bindings that are captured by a closure.
> Your closure does not retain the symbol i, but the binding to which
> that i refers in the given context.

Ok.  Would I be right in thinking there is no way any other function 
can get at that binding (a lexical one), _except_ via a closure?  
i.e. you could pass them the symbol, and you could pass them the value 
the binding has at the moment, but you couldn't pass the binding itself or 
anything which would refer to it.

> Lexical variables are denoted by a symbols in the source code. These
> symbols may disappear in compilation, unless they need to be retained
> for purposes like more user-friendly interactive debugging, which
> allows the user to interrupt the evaluation of some closure body and
> ask for the value of i in that context, rather than, say, the third
> captured binding in the closure vector.

Fair enough, I'd expect the names of my local variables to be thoroughly 
destroyed by any c or c++ compiler too.  I'm just trying to make sure I 
understand the details of the whole symbol/variable/binding/value interaction 
in lisp at the moment.

Tom
From: Kaz Kylheku
Subject: Re: beginner closures
Date: 
Message-ID: <cf333042.0310100835.146037bb@posting.google.com>
Tom Evans <···········@uk.co.yahoo.reverse> wrote in message news:<·················@eng.cam.ac.uk>...
> Kaz Kylheku wrote:
> > 
> > Tom Evans <···········@uk.co.yahoo.reverse> wrote in message news:<·················@eng.cam.ac.uk>...
> > > I sort of expected the result of calling the first closure to be A, and
> > > the second B.  Now, it would appear that both in fact hold on to the
> > > (symbol?) i, and show i's value at the time of the call to the closure.
> > 
> > To answer your embedded question, they hold on to the same variable
> > binding. A lexical variable is a binding: a storage location that
> > holds a value. It is these bindings that are captured by a closure.
> > Your closure does not retain the symbol i, but the binding to which
> > that i refers in the given context.
> 
> Ok.  Would I be right in thinking there is no way any other function 
> can get at that binding (a lexical one), _except_ via a closure? 

No. The closure is the only (portable) representation of the lexical
binding as an object. And only the piece of code associated with a
closure can access the bindings. To the outside, it's an opaque object
that can merely be invoked as a function with arguments.

You can provide closures that behave like accessors (set and get
functions) for a captured environment.

> i.e. you could pass them the symbol, and you could pass them the value 
> the binding has at the moment, but you couldn't pass the binding itself or 
> anything which would refer to it.

No; the only portable way to pass the binding would be to make a
closure and pass that. Whatever you want to do with those variables
would have to be done inside the function.

There is no ``address of'' operator in Lisp like unary & in C or
reference parameters in some other languages; making a closure is the
closest thing: it takes the ``address of'' an entire frame of
variables. That frame has indefinite extent, so it can be safely
returned.

Of course, a Lisp implementation could provide some proprietary escape
hatch that would let you treat the closure as, say, a vector of
bindings that you could read and write directly.
From: Tom Evans
Subject: Re: beginner closures
Date: 
Message-ID: <3F86E7BE.459434B2@eng.cam.ac.uk>
Kaz Kylheku wrote:
> 
> Tom Evans <···········@uk.co.yahoo.reverse> wrote in message news:<·················@eng.cam.ac.uk>...
> > Ok.  Would I be right in thinking there is no way any other function
> > can get at that binding (a lexical one), _except_ via a closure?
> 
> No. The closure is the only (portable) representation of the lexical
> binding as an object. And only the piece of code associated with a
> closure can access the bindings. To the outside, it's an opaque object
> that can merely be invoked as a function with arguments.

Unless I'm misunderstanding, I think you're agreeing with what I intended.
Did you mean "Yes." rather than "No."?

Tom
From: Peter Seibel
Subject: Re: beginner closures
Date: 
Message-ID: <m3pth5hxv5.fsf@javamonkey.com>
Tom Evans <···········@uk.co.yahoo.reverse> writes:

> Kaz Kylheku wrote:
> > 
> > Tom Evans <···········@uk.co.yahoo.reverse> wrote in message news:<·················@eng.cam.ac.uk>...
> > > I sort of expected the result of calling the first closure to be A, and
> > > the second B.  Now, it would appear that both in fact hold on to the
> > > (symbol?) i, and show i's value at the time of the call to the closure.
> > 
> > To answer your embedded question, they hold on to the same variable
> > binding. A lexical variable is a binding: a storage location that
> > holds a value. It is these bindings that are captured by a closure.
> > Your closure does not retain the symbol i, but the binding to which
> > that i refers in the given context.
> 
> Ok.  Would I be right in thinking there is no way any other function 
> can get at that binding (a lexical one), _except_ via a closure?  
> i.e. you could pass them the symbol, and you could pass them the value 
> the binding has at the moment, but you couldn't pass the binding itself or 
> anything which would refer to it.

That is correct.

> > Lexical variables are denoted by a symbols in the source code. These
> > symbols may disappear in compilation, unless they need to be retained
> > for purposes like more user-friendly interactive debugging, which
> > allows the user to interrupt the evaluation of some closure body and
> > ask for the value of i in that context, rather than, say, the third
> > captured binding in the closure vector.
> 
> Fair enough, I'd expect the names of my local variables to be
> thoroughly destroyed by any c or c++ compiler too. I'm just trying
> to make sure I understand the details of the whole
> symbol/variable/binding/value interaction in lisp at the moment.

I haven't officially marked this chapter as being ready for review but
if you wanted to take a look at the chapter from my upcoming Common
Lisp book that covers these topics , I'd be interested to hear whether
you find it's explanation useful:

  <http://www.gigamonkeys.com/book/variables-and-assignment.html>

To see other what other chapters are available, in various states of
doneness, check out:

  <http://www.gigamonkeys.com/book/>

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Tom Evans
Subject: Re: beginner closures
Date: 
Message-ID: <3F86E6AF.3EB7DB3F@eng.cam.ac.uk>
Peter Seibel wrote:
> I haven't officially marked this chapter as being ready for review but
> if you wanted to take a look at the chapter from my upcoming Common
> Lisp book that covers these topics , I'd be interested to hear whether
> you find it's explanation useful:
> 
>   <http://www.gigamonkeys.com/book/variables-and-assignment.html>

Thanks, I like it.  I would have benefited from the explanation below 
when I tried to understand Python, coming from C++.  I was used to the 
idea that assignment modified one object to make it equal to the other.  
It would appear a lot of languages don't work that way...

"Whereas, SETFing a variable simply changes the variable to refer to a 
 different object, leaving whatever object the variable originally 
 referenced, untouched. This distinction is also similar to how = behaves 
 in C, Java, Perl, and Python."

> To see other what other chapters are available, in various states of
> doneness, check out:
> 
>   <http://www.gigamonkeys.com/book/>

I have, I just kept getting annoyed when I was just getting into a juicy bit 
and it went back into note form where you hadn't written it yet.  ;-)

Tom
From: Aurélien Campéas
Subject: Re: beginner closures
Date: 
Message-ID: <pan.2003.10.10.19.02.12.546959@wanadoo.fr>
On Fri, 10 Oct 2003 18:04:47 +0100, Tom Evans wrote:

> Thanks, I like it.  I would have benefited from the explanation below 
> when I tried to understand Python, coming from C++.  I was used to the 
> idea that assignment modified one object to make it equal to the other.  
> It would appear a lot of languages don't work that way...
> 

As I understand it, that's a specific C++ abomination. Correct OO
semantics can be gained at the cost of using systematically the
'virtual' keyword + usage of object pointers, which make object
'assignation' behave like in most other oo languages (reference
substitution).

Unless you do this, so-called C++ assignation copies the object's content
into the target object.

Aur�lien.
From: Peter Seibel
Subject: Re: beginner closures
Date: 
Message-ID: <m3ad8ampt2.fsf@javamonkey.com>
Tom Evans <···········@uk.co.yahoo.reverse> writes:

> I've just had the session below with lisp, and didn't really understand 
> the results.  Was kind of hoping one of you clever people could explain 
> it to me nice and simply, please.  :-)
> 
> [15]> (setq closurelist nil)
> NIL
> [16]> (dolist (i '(a b))
>               (push #'(lambda () (format t "~a" i)) closurelist))
> NIL
> [17]> (funcall (first closurelist))
> B
> NIL
> [18]> (funcall (second closurelist))
> B
> NIL
> 
> I sort of expected the result of calling the first closure to be A,
> and the second B. Now, it would appear that both in fact hold on to
> the (symbol?) i, and show i's value at the time of the call to the
> closure.
> 
> Is that understanding remotely correct?

Pretty much. (Well, you should probably expect the first closure to
print B and the second to print A since PUSH pushes on the front. But
that wasn't really your question, I assume.) The problem is that
DOLIST is allowed to either create a new binding for i or to assign to
the same binding. Your implementation is clearly doing the later. Thus
there is one binding which is captured in both closures, more or less
equivalent to doing this:

  > (defvar *closures*) ;; you should define global variables with DEFVAR
                      ;; and name them with *'s at both ends of the name
                      ;; (but again that wasn't your point)

  > (setq *closures* nil)

  > (let (i)                                            ; creates a variable binding with the name i.
     (setf i 'a)                                        ; assigns a value to the variable binding
     (push #'(lambda () (format t "~a" i)) *closures*)  ; captures the *binding*
     (setf i 'b)                                        ; assigns a new value to the variable binding
     (push #'(lambda () (format t "~a" i)) *closures*)) ; captures the same *binding*

  > (funcall (first *closures*))
  B
  NIL
  > (funcall (second *closures*))
  B
  NIL

Since there's only one variable binding, both closures capture the
same one. At different times it has different values but they share
the one binding.

> How can I do what I want to do?  (create stored functions where the value 
> of i can be put separately into each one)

What you want is something like this:

  > (let ((i 'a)) (push #'(lambda () (format t "~a" i)) *closures*))
  (#<Interpreted Closure (unnamed) @ #x7703e702>)
  > (let ((i 'b)) (push #'(lambda () (format t "~a" i)) *closures*))
  (#<Interpreted Closure (unnamed) @ #x7703f0b2> #<Interpreted Closure (unnamed) @ #x7703e702>)
  > (funcall (first *closures*))
  B
  NIL
  > (funcall (second *closures*))
  A
  NIL

In this case, each LET creates a new variable binding which is
captured by the closure. Thus they are completely separate.

You can achieve the desired effect in a DOLIST loop by explicitly
creating a new binding for each closure:

  > (setq *closures* nil)
  NIL
  > (dolist (i '(a b))
      (let ((x i))
        (push #'(lambda () (format t "~a" x)) *closures*)))
  NIL
  > (funcall (first *closures*))
  B
  NIL
  > (funcall (second *closures*))
  A
  NIL


-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Tom Evans
Subject: Re: beginner closures
Date: 
Message-ID: <3F857991.6F8539E6@eng.cam.ac.uk>
Peter Seibel wrote:
> 
> Tom Evans <···········@uk.co.yahoo.reverse> writes:
> 
> > Is that understanding remotely correct?
> 
> Pretty much. (Well, you should probably expect the first closure to
> print B and the second to print A since PUSH pushes on the front. But
> that wasn't really your question, I assume.) The problem is that

ah, good point..

> DOLIST is allowed to either create a new binding for i or to assign to
> the same binding. Your implementation is clearly doing the later. Thus
> there is one binding which is captured in both closures, more or less
> equivalent to doing this:

<snip helpful explanations>

Thanks, I follow it now.  

Thanks to the others who replied too; I think I like this place.

Tom

p.s.  Hurry up and finish that book, I want to go buy a copy.
From: Pascal Costanza
Subject: Re: beginner closures
Date: 
Message-ID: <bm3sij$phi$1@f1node01.rhrz.uni-bonn.de>
Tom Evans wrote:
> I've just had the session below with lisp, and didn't really understand 
> the results.  Was kind of hoping one of you clever people could explain 
> it to me nice and simply, please.  :-)
> 
> [15]> (setq closurelist nil)
> NIL
> [16]> (dolist (i '(a b))
>               (push #'(lambda () (format t "~a" i)) closurelist))
> NIL
> [17]> (funcall (first closurelist))
> B
> NIL
> [18]> (funcall (second closurelist))
> B
> NIL
> 
> I sort of expected the result of calling the first closure to be A, and 
> the second B.  Now, it would appear that both in fact hold on to the 
> (symbol?) i, and show i's value at the time of the call to the closure.
> 
> Is that understanding remotely correct?

Yes, and you can use that to make closures communicate with each other. 
See the following example (not the best way to do this, but anyway):

(setq cl nil)

(dolist (i '(a b))
   (push #'(lambda () (format t "~A" i)) cl)
   (defun change-i (j)
     (setq i j)))

 > (change-i 5)
5

 > (funcall (first cl))
5
NIL

> How can I do what I want to do?  (create stored functions where the value 
> of i can be put separately into each one)

You need to introduce a new unique binding each time, as follows:

(dolist (i '(a b))
   (let ((j i))
      (push #'(lambda () (format t "~A~ j)) cl)))


http://www.flownet.com/gat/specials.pdf provides some more details.


Pascal

-- 
Pascal Costanza               University of Bonn
···············@web.de        Institute of Computer Science III
http://www.pascalcostanza.de  R�merstr. 164, D-53117 Bonn (Germany)
From: Peter Seibel
Subject: Re: beginner closures
Date: 
Message-ID: <m31xtmmoxh.fsf@javamonkey.com>
Pascal Costanza <········@web.de> writes:

> Tom Evans wrote:
> > I've just had the session below with lisp, and didn't really
> > understand the results.  Was kind of hoping one of you clever people
> > could explain it to me nice and simply, please.  :-)
> > [15]> (setq closurelist nil)
> > NIL
> > [16]> (dolist (i '(a b))
> >               (push #'(lambda () (format t "~a" i)) closurelist))
> > NIL
> > [17]> (funcall (first closurelist))
> > B
> > NIL
> > [18]> (funcall (second closurelist))
> > B
> > NIL
> > I sort of expected the result of calling the first closure to be A,
> > and the second B.  Now, it would appear that both in fact hold on to
> > the (symbol?) i, and show i's value at the time of the call to the
> > closure.
> > Is that understanding remotely correct?
> 
> Yes, and you can use that to make closures communicate with each
> other. See the following example (not the best way to do this, but
> anyway):
> 
> (setq cl nil)
> 
> (dolist (i '(a b))
>    (push #'(lambda () (format t "~A" i)) cl)
>    (defun change-i (j)
>      (setq i j)))
> 
>  > (change-i 5)
> 5
> 
>  > (funcall (first cl))
> 5
> NIL

Well, he can't rely on DOLIST creating only one binding. From the
DOLIST dictionary entry:

  It is implementation-dependent whether dolist establishes a new
  binding of var on each iteration or whether it establishes a binding
  for var once at the beginning and then assigns it on any subsequent
  iterations.

-Peter

-- 
Peter Seibel                                      ·····@javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Pascal Costanza
Subject: Re: beginner closures
Date: 
Message-ID: <bm3tr8$un0$1@f1node01.rhrz.uni-bonn.de>
Peter Seibel wrote:

> Well, he can't rely on DOLIST creating only one binding. From the
> DOLIST dictionary entry:
> 
>   It is implementation-dependent whether dolist establishes a new
>   binding of var on each iteration or whether it establishes a binding
>   for var once at the beginning and then assigns it on any subsequent
>   iterations.
> 
> -Peter

Yes, I have missed that bit. Thanks for pointing this out!


Pascal


-- 
Pascal Costanza               University of Bonn
···············@web.de        Institute of Computer Science III
http://www.pascalcostanza.de  R�merstr. 164, D-53117 Bonn (Germany)
From: Frode Vatvedt Fjeld
Subject: Re: beginner closures
Date: 
Message-ID: <2h1xtm4g7c.fsf@vserver.cs.uit.no>
Tom Evans <···········@uk.co.yahoo.reverse> writes:

> [15]> (setq closurelist nil)
> NIL
> [16]> (dolist (i '(a b))
>               (push #'(lambda () (format t "~a" i)) closurelist))
> NIL
> [17]> (funcall (first closurelist))
> B
> NIL
> [18]> (funcall (second closurelist))
> B
> NIL

Now both closures hold on to the same variable, namely the one named
"i" with the dotimes operator. Actually, it is unspecified whether
dotimes re-uses the same variable for each iteration, or if it sets up
a new one each time. Your particular implementation obviously re-uses
the same variable (as I expect most implementations do).

> How can I do what I want to do?  (create stored functions where the
> value of i can be put separately into each one)

Just explicitly specify that you want a new variable for each
iteration:

  (dolist (i '(a b))
    (let ((local-i i))
      (push (lambda () (format t "~A" local-i))
             closurelist)))

-- 
Frode Vatvedt Fjeld