From: charlie
Subject: Equality of closures?
Date: 
Message-ID: <1133581156.790905.77220@g49g2000cwa.googlegroups.com>
Hi,
Is there a reliable way to assess the equality of closures? I've run a
couple of tests but they give contradictory results. I suspect that
this is because one is interpreted and one's compiled but that doesn't
help me much because the problem's in compiled code anyway.
Some background: I have an event loop (sdl) for a display window. I've
added an idle function hook to the loop. Normally this just checks for
events from other parts of the program (ltk) but I want to use it to
show an animation, so I add the closure that blits a stored frame to
the window and then when the user quits the animation I'd like to
remove the closure from the hook with #'remove but by the time it gets
back it is no longer equal let alone eq.

This is the closure before I start the event loop
#<Closure Over Function "DEFMETHOD SHOW-ANIMATION (IFS-SDL ANIMATION)"
  {594619E1}>
This is the closure after it gets back, note the different numbers in
the {}
#<Closure Over Function "DEFMETHOD SHOW-ANIMATION (IFS-SDL ANIMATION)"
  {59461D69}>

Cheers,
Charlie.

From: Barry Margolin
Subject: Re: Equality of closures?
Date: 
Message-ID: <barmar-43D65B.23374802122005@comcast.dca.giganews.com>
In article <·······················@g49g2000cwa.googlegroups.com>,
 "charlie" <···············@gmail.com> wrote:

> Hi,
> Is there a reliable way to assess the equality of closures? I've run a
> couple of tests but they give contradictory results. I suspect that
> this is because one is interpreted and one's compiled but that doesn't
> help me much because the problem's in compiled code anyway.
> Some background: I have an event loop (sdl) for a display window. I've
> added an idle function hook to the loop. Normally this just checks for
> events from other parts of the program (ltk) but I want to use it to
> show an animation, so I add the closure that blits a stored frame to
> the window and then when the user quits the animation I'd like to
> remove the closure from the hook with #'remove but by the time it gets
> back it is no longer equal let alone eq.
> 
> This is the closure before I start the event loop
> #<Closure Over Function "DEFMETHOD SHOW-ANIMATION (IFS-SDL ANIMATION)"
>   {594619E1}>
> This is the closure after it gets back, note the different numbers in
> the {}
> #<Closure Over Function "DEFMETHOD SHOW-ANIMATION (IFS-SDL ANIMATION)"
>   {59461D69}>

Are you assigning the closure to a variable each time through the loop?  
If so, the interpreter is probably creating a new closure each time, 
while the compiler is able to optimize this away because it detects that 
they all close over the same environment.

The simple solution would be to assign the closure outside the loop.  If 
the closure uses any variables bound within the looping form, you'll 
need to let-bind them outside and change the loop to assign to them.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
From: charlie
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133586460.081431.76740@g43g2000cwa.googlegroups.com>
Barry Margolin wrote:

> Are you assigning the closure to a variable each time through the loop?
> If so, the interpreter is probably creating a new closure each time,
> while the compiler is able to optimize this away because it detects that
> they all close over the same environment.
>
> The simple solution would be to assign the closure outside the loop.  If
> the closure uses any variables bound within the looping form, you'll
> need to let-bind them outside and change the loop to assign to them.
>
Here's the function, please excuse the indentation or inaccuracy
thereof:

(defmethod show-animation ((sdl ifs-sdl) (anim ltk-anim:animation))
  (unless (ltk-anim:frames anim) (error "No rendered frames to show"))
  (let* ((rest-frames (ltk-anim:frames anim))
	 (anim-closure (lambda ()
			            (unless rest-frames
    			                 (setf rest-frames (ltk-anim:frames anim)))
			            (when (ifs-anim:surface (car rest-frames))
			               (sdl:blit-surface
			                   (ifs-anim:surface (car rest-frames))
			                   sgum:+null-pointer+
			                   (surface sdl)
			                   sgum:+null-pointer+)
			               (setf rest-frames (cdr rest-frames)))
			           (sdl:flip (surface sdl)))))
    (print anim-closure)
    (push anim-closure (idle-hook sdl))
    (show-ifs sdl)
    (setf (idle-hook sdl) (remove anim-closure (idle-hook sdl) :test
#'equal))
    (print (idle-hook sdl))))

The output I gave above is cut from the result of this function.
(show-ifs sdl) launches the event loop that contains the following
code:
(dolist (fn (idle-hook sdl))
		    (funcall fn))
The animation part of it works fine but I can't remove the closure from
the list and I don't want to just presume to know its location in the
list.
Why the identity of the closure would change, I don't know but change
it does. Unless I've made some obvious mistake, which I am prone to do.

Cheers,
Charlie.
From: Kaz Kylheku
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133589386.602621.228540@g49g2000cwa.googlegroups.com>
charlie wrote:
> Here's the function, please excuse the indentation or inaccuracy
> thereof:

Disclaimer: I don't know anything about the libraries you are working
with. But I understand that somehow the closure you put in is no longer
the same object when you try to remove it.

>     (print anim-closure)
>     (push anim-closure (idle-hook sdl))
>     (show-ifs sdl)
>     (setf (idle-hook sdl) (remove anim-closure (idle-hook sdl) :test
> #'equal))

See, now how can that happen? Nothing changed the value of
ANIM-CLOSURE. Or did it?

What does SHOW-IFS do? It starts a recursive event dispatch loop,
right?

I suspect that this loop somehow dispatches some processing that causes
your function to be re-entered. Your function makes another closure and
pushes that into the list of hooks.

Could you be confusing yourself because of that?

That would be how the lexical variable could appear to change values:
if it's actually a different activation frame with a different instance
of ANIM-CLOSURE.
From: charlie
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133590731.707660.64750@o13g2000cwo.googlegroups.com>
Kaz Kylheku wrote:
> I suspect that this loop somehow dispatches some processing that causes
> your function to be re-entered. Your function makes another closure and
> pushes that into the list of hooks.
>
> Could you be confusing yourself because of that?
>
> That would be how the lexical variable could appear to change values:
> if it's actually a different activation frame with a different instance
> of ANIM-CLOSURE.

As far as I know it just gets funcalled repeatedly. Is is possible that
a closure can change its id just by changing state? It's too late here
to do the whole fine toothed comb bit at the moment. I'll save that fun
for tomorrow. Have a look if the mood takes you. The functions in
question are all in ifs-sdl.lisp. I've just uploaded the new version of
the IFS Designer that this code comes from. It's at
http://freehost03.websamba.com/charlieb/src/Ifs-pre.tar.gz
Also you'll need cl-sdl, ltk and also the matrix and ltk-goodies
libraries at http://websamba.com/charlieb/
The IFS Designer link on that page is not the latest version.
Load them all up and do (ifs-design:test), add a keyframe, generate the
animation, render the frames and show the animation. It's the "show
animation" step where the code is triggered.

Good-night,
Charlie.
From: charlie
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133629124.892759.157910@o13g2000cwo.googlegroups.com>
I have some minimal code that proves that when a compiled closure
changes state it changes it's id i.e. it isn't equal to it's original
instansiation. I don't know if this is legal behaviour or not but when
the code is interpreted the change of state does not change the
closure's id. Some sample output follows the code.

Enjoy!
Charlie.

;; ----- File start ---------
(defpackage "CLOSURE-TEST"
  (:use :cl)
  (:export "TEST1"
	   "TEST0"))

(in-package :closure-test)

(defun test0 ()
  (let* ((var 1)
	 ;; Closure does something but shouldn't change state
	 (closure (lambda () (format nil "~A" var))))
    (print closure)
    (dotimes (i 100) (funcall closure))
    (print closure)
    nil))

(defun test1 ()
  (let* ((var 1)
	 ;; Closure that changes state
	 (closure (lambda () (incf var))))
    (print closure)
    (dotimes (i 100) (funcall closure))
    (print closure)
    nil))
;; -------- File end -------------

Here's the output:

; Loading #P"/home/charlieb/lisp/closure-test.lisp".
CL-USER> (closure-test:test0)

#<Interpreted Function "LAMBDA NIL" {58FB8531}>
#<Interpreted Function "LAMBDA NIL" {58FB8531}>
NIL
CL-USER> (closure-test:test1)

#<Interpreted Function "LAMBDA NIL" {58FD89B1}>
#<Interpreted Function "LAMBDA NIL" {58FD89B1}>
NIL
;;;; Compile file /home/charlieb/lisp/closure-test.lisp ...

; Python version 1.1, VM version Intel x86 on 03 DEC 05 11:52:04 am.
; Compiling: /home/charlieb/lisp/closure-test.lisp 03 DEC 05 11:51:52
am


; /home/charlieb/lisp/closure-test.x86f written.
; Compilation finished in 0:00:00.
; Loading #P"/home/charlieb/lisp/closure-test.x86f".
CL-USER> (closure-test:test0)

#<Function "DEFUN TEST0" {5902BD31}>
#<Function "DEFUN TEST0" {5902BD31}>
NIL
CL-USER> (closure-test:test1)

#<Closure Over Function "DEFUN TEST1" {590459C1}>
#<Closure Over Function "DEFUN TEST1" {59045D01}> 
NIL
From: ········@comail.ru
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133642888.957848.135770@g14g2000cwa.googlegroups.com>
Hello,

charlie wrote:
>    (print anim-closure)
>   (push anim-closure (idle-hook sdl))
>    (show-ifs sdl)
>    (setf (idle-hook sdl) (remove anim-closure (idle-hook sdl) :test #'equal))
>    (print (idle-hook sdl))))
[...]
> ;;;; Compile file /home/charlieb/lisp/closure-test.lisp ...
>
> ; Python version 1.1, VM version Intel x86 on 03 DEC 05 11:52:04 am.

So, you are using CMUCL. Then there is a simple workaround:

(defun test (x lst)
  (let ((l (lambda (i)
             (list x i))))
    (print l)
    (push l lst)
    (print lst)
    (setq lst (remove l lst))
    (print lst)
    (values)))

(defun test1 (x lst)
  (let ((l (car (list (lambda (i)
                        (list x i))))))
    (print l)
    (push l lst)
    (print lst)
    (setq lst (remove l lst))
    (print lst)
    (values)))

CL-USER> (test 4 '(1 2 3))

#<Closure Over Function "DEFUN TEST" {58B3B861}>
(#<Closure Over Function "DEFUN TEST" {58B3BB49}> 1 2 3)
(#<Closure Over Function "DEFUN TEST" {58B3BB49}> 1 2 3)
; No value
CL-USER> (test1 4 '(1 2 3))

#<Closure Over Function "DEFUN TEST1" {58B3CB49}>
(#<Closure Over Function "DEFUN TEST1" {58B3CB49}> 1 2 3)
(1 2 3) 
; No value
From: charlie
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133652487.329241.327480@o13g2000cwo.googlegroups.com>
So it has nothing to do with changing state. How odd, but thank-you.
I'll use that.

Cheers,
Charlie.
From: Pascal Bourguignon
Subject: Re: Equality of closures?
Date: 
Message-ID: <87y832ds15.fsf@thalassa.informatimago.com>
"charlie" <···············@gmail.com> writes:

> I have some minimal code that proves that when a compiled closure
> changes state it changes it's id i.e. it isn't equal to it's original
> instansiation. I don't know if this is legal behaviour or not but when
> the code is interpreted the change of state does not change the
> closure's id. Some sample output follows the code.
>
> Enjoy!
> Charlie.
>
> ;; ----- File start ---------
> (defpackage "CLOSURE-TEST"
>   (:use :cl)
>   (:export "TEST1"
> 	   "TEST0"))
>
> (in-package :closure-test)
>
> (defun test0 ()
>   (let* ((var 1)
> 	 ;; Closure does something but shouldn't change state
> 	 (closure (lambda () (format nil "~A" var))))
>     (print closure)
>     (dotimes (i 100) (funcall closure))
>     (print closure)
>     nil))
>
> (defun test1 ()
>   (let* ((var 1)
> 	 ;; Closure that changes state
> 	 (closure (lambda () (incf var))))
>     (print closure)
>     (dotimes (i 100) (funcall closure))
>     (print closure)
>     nil))
> ;; -------- File end -------------
>
> Here's the output:
>
> ; Loading #P"/home/charlieb/lisp/closure-test.lisp".
> CL-USER> (closure-test:test0)
>
> #<Interpreted Function "LAMBDA NIL" {58FB8531}>
> #<Interpreted Function "LAMBDA NIL" {58FB8531}>
> NIL
> CL-USER> (closure-test:test1)
>
> #<Interpreted Function "LAMBDA NIL" {58FD89B1}>
> #<Interpreted Function "LAMBDA NIL" {58FD89B1}>
> NIL
> ;;;; Compile file /home/charlieb/lisp/closure-test.lisp ...
>
> ; Python version 1.1, VM version Intel x86 on 03 DEC 05 11:52:04 am.
> ; Compiling: /home/charlieb/lisp/closure-test.lisp 03 DEC 05 11:51:52
> am
>
>
> ; /home/charlieb/lisp/closure-test.x86f written.
> ; Compilation finished in 0:00:00.
> ; Loading #P"/home/charlieb/lisp/closure-test.x86f".
> CL-USER> (closure-test:test0)
>
> #<Function "DEFUN TEST0" {5902BD31}>
> #<Function "DEFUN TEST0" {5902BD31}>
> NIL
> CL-USER> (closure-test:test1)
>
> #<Closure Over Function "DEFUN TEST1" {590459C1}>
> #<Closure Over Function "DEFUN TEST1" {59045D01}> 
> NIL

This doesn't prove anything: the same object can have different
unreadable (#<) print representations at different time for garbage
collection or other implementation dependant reasons.


(defpackage "CLOSURE-TEST"
  (:use :cl)
  (:export "TEST1"
           "TEST0"))

(in-package :closure-test)

(defun test0 ()
  (let* ((var 1)
         ;; Closure does something but shouldn't change state
         (closure (lambda () (format nil "~A" var))))
    (let ((saved closure))
      (print closure)
      (dotimes (i 100) (funcall closure))
      (print closure)
      (assert (eq saved closure)))
    nil))

(defun test1 ()
  (let* ((var 1)
         ;; Closure that changes state
         (closure (lambda () (incf var))))
    (let ((saved closure))
      (print closure)
      (dotimes (i 100) (funcall closure))
      (print closure)
      (assert (eq saved closure)))
    nil))



In clisp:

[37]> (load"src/lisp/encours/usenet/closure-test.lisp")
;; Loading file src/lisp/encours/usenet/closure-test.lisp ...
;; Loaded file src/lisp/encours/usenet/closure-test.lisp
T
[38]> (closure-test:test0)

#<FUNCTION :LAMBDA NIL (FORMAT NIL "~A" CLOSURE-TEST::VAR)> 
#<FUNCTION :LAMBDA NIL (FORMAT NIL "~A" CLOSURE-TEST::VAR)> 
NIL
[39]> (closure-test:test1)

#<FUNCTION :LAMBDA NIL (INCF CLOSURE-TEST::VAR)> 
#<FUNCTION :LAMBDA NIL (INCF CLOSURE-TEST::VAR)> 
NIL
[40]> (load (compile-file"src/lisp/encours/usenet/closure-test.lisp"))
;; Compiling file /local/users/pjb/src/lisp/encours/usenet/closure-test.lisp ...
;; Wrote file /local/users/pjb/src/lisp/encours/usenet/closure-test.fas
0 errors, 0 warnings
;; Loading file /local/users/pjb/src/lisp/encours/usenet/closure-test.fas ...
;; Loaded file /local/users/pjb/src/lisp/encours/usenet/closure-test.fas
T
[41]> (closure-test:test0)

#<COMPILED-FUNCTION CLOSURE-TEST::TEST0-1> 
#<COMPILED-FUNCTION CLOSURE-TEST::TEST0-1> 
NIL
[42]> (closure-test:test1)

#<COMPILED-FUNCTION CLOSURE-TEST::TEST1-1> 
#<COMPILED-FUNCTION CLOSURE-TEST::TEST1-1> 
NIL
[43]> 

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

"By filing this bug report you have challenged the honor of my
family. Prepare to die!"
From: charlie
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133640203.958377.214930@o13g2000cwo.googlegroups.com>
Fair enough, print was a bad choice as was a less than perfectly
knowledgable use of the word "prove". I believe that this chunk of code
better demonstrates my point under cmucl. Under clisp the closures
appear to be eq so is no problem.

Enjoy!
Charlie.

(defpackage "CLOSURE-TEST"
  (:use :cl)
  (:export "TEST1"
	   "TEST0"))

(in-package :closure-test)

(defparameter *tester* nil)

(defun test0 ()
  (setq *tester* nil)
  (let* ((var 1)
	 ;; Closure does something but shouldn't change state
	 (closure (lambda () (format nil "~A" var))))
    (push closure *tester*)
    (dotimes (i 100)
      (dolist (test *tester*)
	(funcall test)))
    (member closure *tester*)))

(defun test1 ()
  (setq *tester* nil)
  (let* ((var 1)
	 ;; Closure that changes state
	 (closure (lambda () (incf var))))
    (push closure *tester*)
    (dotimes (i 100)
      (dolist (test *tester*)
	(funcall test)))
    (member closure *tester* :test #'equalp)))
From: Coby Beck
Subject: Re: Equality of closures?
Date: 
Message-ID: <x0mkf.232659$ir4.150284@edtnps90>
"charlie" <···············@gmail.com> wrote in message 
····························@g43g2000cwa.googlegroups.com...
>
> Barry Margolin wrote:
>
>> Are you assigning the closure to a variable each time through the loop?
>> If so, the interpreter is probably creating a new closure each time,
>> while the compiler is able to optimize this away because it detects that
>> they all close over the same environment.
>>
>> The simple solution would be to assign the closure outside the loop.  If
>> the closure uses any variables bound within the looping form, you'll
>> need to let-bind them outside and change the loop to assign to them.
>>
> Here's the function, please excuse the indentation or inaccuracy
> thereof:
>
> (defmethod show-animation ((sdl ifs-sdl) (anim ltk-anim:animation))
>  (unless (ltk-anim:frames anim) (error "No rendered frames to show"))
>  (let* ((rest-frames (ltk-anim:frames anim))
> (anim-closure (lambda ()
>             (unless rest-frames
>                     (setf rest-frames (ltk-anim:frames anim)))
>             (when (ifs-anim:surface (car rest-frames))
>                (sdl:blit-surface
>                    (ifs-anim:surface (car rest-frames))
>                    sgum:+null-pointer+
>                    (surface sdl)
>                    sgum:+null-pointer+)
>                (setf rest-frames (cdr rest-frames)))
>            (sdl:flip (surface sdl)))))
>    (print anim-closure)
>    (push anim-closure (idle-hook sdl))
>    (show-ifs sdl)
>    (setf (idle-hook sdl) (remove anim-closure (idle-hook sdl) :test
> #'equal))
>    (print (idle-hook sdl))))
>
> The output I gave above is cut from the result of this function.
> (show-ifs sdl) launches the event loop that contains the following
> code:
> (dolist (fn (idle-hook sdl))
>     (funcall fn))

You will probably have to change this code here.  Maybe put your idle-hook 
closures in a hash table or alist instead of a simple list.  Then instead of 
pushing and removing, write an add-hook and remove-hook method.  The 
show-animation method can use a gensym as the key.

Another option would be to save the original list of closures before pushing 
the new one one and then setf the original back afterwards, then only the 
show-animation method needs changing.  Consider if you need to 
unwind-protect this.

-- 
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
From: charlie
Subject: Re: Equality of closures?
Date: 
Message-ID: <1133640985.110328.263850@o13g2000cwo.googlegroups.com>
Thank-you for the suggestions, they are very helpful. I like the
storing a copy of the list especially.
Cheers,
Charlie.