From: Karol Skocik
Subject: understanding closure
Date: 
Message-ID: <1121636220.435512.36400@g49g2000cwa.googlegroups.com>
Hi,
  i would like to know one thing about closures - when i make something
like this :

...
    (push #'(lambda () (... some undo function code ...))
          undo-funcs)
...

to make later something like this :

    (dolist (undo undo-funcs) (funcall undo))

to restore original state of some structure or array.

the variable bindings which make-up closure - are they created when the
code runs on heap or is it solved with increasing the value of "shared
count" in garbage collector for specified value of variable which is
out of closure?

how is this implemented?

thanks,
  karol skocik

From: Tron3k
Subject: Re: understanding closure
Date: 
Message-ID: <1121639797.531272.300850@g47g2000cwa.googlegroups.com>
I guess you might think of it as 'increasing the refcount' (i.e.
grabbing a handle) of some object that is already there. Don't take
this as a definitive answer. :-)

Here is something you might want to be aware of:

(defvar *closures* '())

(dotimes (i 10)
   (push #'(lambda () i) *closures*))

(mapcar #'funcall *closures*) => (10 10 10 10 10 10 10 10 10 10)

So, yeah, there's really only one 'i' that is repeatedly assigned to by
dotimes, and you thus get that result. You can fix it by doing:

(dotimes (i 10)
   (let ((i i))
       (push #'(lambda () i) *closures*)))

Maybe that will help you understand how closures work with respect to
variables.

Karol Skocik wrote:
> Hi,
>   i would like to know one thing about closures - when i make something
> like this :
>
> ...
>     (push #'(lambda () (... some undo function code ...))
>           undo-funcs)
> ...
>
> to make later something like this :
>
>     (dolist (undo undo-funcs) (funcall undo))
>
> to restore original state of some structure or array.
>
> the variable bindings which make-up closure - are they created when the
> code runs on heap or is it solved with increasing the value of "shared
> count" in garbage collector for specified value of variable which is
> out of closure?
> 
> how is this implemented?
> 
> thanks,
>   karol skocik
From: Peter Seibel
Subject: Re: understanding closure
Date: 
Message-ID: <m2oe91exrg.fsf@gigamonkeys.com>
"Tron3k" <······@gmail.com> writes:

> I guess you might think of it as 'increasing the refcount' (i.e.
> grabbing a handle) of some object that is already there. Don't take
> this as a definitive answer. :-)
>
> Here is something you might want to be aware of:
>
> (defvar *closures* '())
>
> (dotimes (i 10)
>    (push #'(lambda () i) *closures*))
>
> (mapcar #'funcall *closures*) => (10 10 10 10 10 10 10 10 10 10)
>
> So, yeah, there's really only one 'i' that is repeatedly assigned to by
> dotimes, and you thus get that result. You can fix it by doing:

In your implementation. That's not required behavior of DOTIMES; it
could also make a fresh binding each time through the loop.

-Peter

-- 
Peter Seibel           * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp  * http://www.gigamonkeys.com/book/
From: Tron3k
Subject: Re: understanding closure
Date: 
Message-ID: <1121649510.786111.294920@g43g2000cwa.googlegroups.com>
Peter Seibel wrote:
> "Tron3k" <······@gmail.com> writes:
>
> > I guess you might think of it as 'increasing the refcount' (i.e.
> > grabbing a handle) of some object that is already there. Don't take
> > this as a definitive answer. :-)
> >
> > Here is something you might want to be aware of:
> >
> > (defvar *closures* '())
> >
> > (dotimes (i 10)
> >    (push #'(lambda () i) *closures*))
> >
> > (mapcar #'funcall *closures*) => (10 10 10 10 10 10 10 10 10 10)
> >
> > So, yeah, there's really only one 'i' that is repeatedly assigned to by
> > dotimes, and you thus get that result. You can fix it by doing:
>
> In your implementation. That's not required behavior of DOTIMES; it
> could also make a fresh binding each time through the loop.
>

Interesting. So how would the macro-expansion of that look like? Would
it be a tail-recursive function?

Anyway, it's still important for the original poster to know what
happens with assignment to variables which are being closed over in a
closure.
From: Kent M Pitman
Subject: Re: understanding closure
Date: 
Message-ID: <u8y043e1z.fsf@nhplace.com>
"Tron3k" <······@gmail.com> writes:

> Peter Seibel wrote:
> > "Tron3k" <······@gmail.com> writes:
> >
> > > I guess you might think of it as 'increasing the refcount' (i.e.
> > > grabbing a handle) of some object that is already there. Don't take
> > > this as a definitive answer. :-)
> > >
> > > Here is something you might want to be aware of:
> > >
> > > (defvar *closures* '())
> > >
> > > (dotimes (i 10)
> > >    (push #'(lambda () i) *closures*))
> > >
> > > (mapcar #'funcall *closures*) => (10 10 10 10 10 10 10 10 10 10)
> > >
> > > So, yeah, there's really only one 'i' that is repeatedly assigned to by
> > > dotimes, and you thus get that result. You can fix it by doing:
> >
> > In your implementation. That's not required behavior of DOTIMES; it
> > could also make a fresh binding each time through the loop.
> >
> 
> Interesting. So how would the macro-expansion of that look like? Would
> it be a tail-recursive function?

It might become any of a variety of other things that just update the
binding with SETF or SETQ, INCF, or implicit assignment via DO. e.g.,

 (let ((i 0))
   (loop (if (>= i 10) (return))
         ...
         (incf i)))

OR

 (do ((i 0 (1+ i)))
     ((>= i 10))
   ...)

> Anyway, it's still important for the original poster to know what
> happens with assignment to variables which are being closed over in a
> closure.

It's important for him to know if indeed it's something that the macro
wants to tell him, but it suffices (as CL does) to say that the intent
is that you don't know and must be explicit if you need to rely on it.

That is, a macro needs to explicitly tell you that it's giving you a
new local binding of a variable on each iteration, OR ELSE tell you
that it is not giving you a new local binding, OR ELSE tell you that
you cannot make an assumption one way or another.  CL does option #3.

The preferred way of highlighting that you're needing a binding when
it's not well-defined that it is there would be:

(let ((l '()))
  (dotimes (i 10)
    (push (let ((i i)) ; <-- bind i to itself, assuring a new binding
            #'(lambda () i))
          l))
  (mapcar #'funcall l))
(9 8 7 6 5 4 3 2 1 0)
From: Steven M. Haflich
Subject: Re: understanding closure
Date: 
Message-ID: <2lBCe.386$NU2.279@newssvr13.news.prodigy.com>
Karol Skocik wrote:
>     (push #'(lambda () (... some undo function code ...))
>           undo-funcs)
> ...

> 
> the variable bindings which make-up closure - are they created when the
> code runs on heap or is it solved with increasing the value of "shared
> count" in garbage collector for specified value of variable which is
> out of closure?

Huh?  Lisp garbage collectors generally do not use reference counts. 
There are references on the web which explain why, but the general idea 
is that reference counts cannot cope with circular references.  Anyway, 
you cannot explain the semantics of CL based on the garbage collector 
simply because a garbage collector is not a required part of the 
language.  In other words, the presence, absence, or action of the gc 
can have no effect whatever on the behavior of a correct program.

You should think about your question using different concepts.  Let's 
rewrite your example slightly:

   (let ((x ...))
     ...
     (push (function (lambda () (... x ...)))
           undo-funcs))

Note that I have replaced #' with function, which is different reader 
notation for exactly the same thing, but it emphasizes the point that 
the FUNCTION special form _is_ a form, and it is executed in the normal 
manner of any other form at execution time.  It creates a closure from 
the sub form lambda expression such that the closure captures the 
current bindings of any variables referenced "free" from the closure.

A lexical variable binding in CL is not a first-class object.  There is 
no way to pass a lexical binding to a function.  You can pass the value 
of a lexical binding to a function, but the representation of the 
binding itself is not manipulable.  It is merely something shared by all 
functions defined within its scope, but once the binding is established 
it cannot be manipulated, nor can the set of functions that can access 
it be changed.

In most implementations closure analysis is performed by the compiler. 
An internal function becomes a (non-null) closure if it makes a free 
reference to some surrounding lexical variable binding.  While a 
non-closed-over lexical variable will generally be allocated on the 
stack, typically the compiler will arrange to copy a closed-over 
variable into a cons or vector that is shared by all internal functions 
that reference the variable.  This cons or vector is typically allocated 
when the binding of the variable is first created (and imposes a small 
run-time overhead) but there are various special-case optimizations that 
an implementation may choose to perform.
From: Karol Skocik
Subject: Re: understanding closure
Date: 
Message-ID: <1121670160.000765.124970@f14g2000cwb.googlegroups.com>
Hi,
  thank you all. i am just not getting the last thing :

Steven M. Haflich wrote:

> A lexical variable binding in CL is not a first-class object.  There is
> no way to pass a lexical binding to a function.  You can pass the value
> of a lexical binding to a function, but the representation of the
> binding itself is not manipulable.

how can the value of lexical binding look like? and how can it be
passed to function as and argument? will the invoked function with this
value of lexical binding have the same output? if you post some lisp
code, that will be great :)

cheers, 
  karol skocik
From: Kent M Pitman
Subject: Re: understanding closure
Date: 
Message-ID: <u64v8jici.fsf@nhplace.com>
"Karol Skocik" <············@gmail.com> writes:

> Hi,
>   thank you all. i am just not getting the last thing :
> 
> Steven M. Haflich wrote:
> 
> > A lexical variable binding in CL is not a first-class object.  There is
> > no way to pass a lexical binding to a function.  You can pass the value
> > of a lexical binding to a function, but the representation of the
> > binding itself is not manipulable.

All that Steve says here is true, but I would add that you can pass the
binding indirectly "wrapped" by a closure.  It's just that the binding
isn't "directly" manipulable--indirectly, though, by calling closure code
that refers to it, you can manipulate it at least by reading and writing it.

 (defun bump (n reader setter)
   (funcall setter (+ n (funcall reader))))

 (let ((x 3))
   (bump 5 #'(lambda () x) #'(lambda (val) (setq x val)))
   x) ; return the lexical x
 => 8

> how can the value of lexical binding look like?

The VALUE of any binding is just an object.

 (let ((x (list 'this 'list)))
   (print `(the value of x is ,x))
   'done)
 (THE VALUE OF X IS (THIS LIST))
 => DONE

> and how can it be passed to function as and argument?

Evaluating a local binding yields the object that is its value.
Typically, objects live in the heap and all objects are passed by pointer,
so a pointer to the object (not to the local binding) is passed.

> will the invoked function with this
> value of lexical binding have the same output?

I didn't read upthread so I don't know what "same input" is.
It will get [a pointer to] the object that was the value of the local binding.

> if you post some lisp code, that will be great :)

(defvar *foo* (list 'a 'b 'c))

(defun f (x y)
  (if (eq x y) 'same-object 'different-object))

(let ((foo *foo*))
  (f foo *foo*))
=> SAME-OBJECT