From: Alex Mizrahi
Subject: is this a portable CL? (rebind)
Date: 
Message-ID: <45ec2bac$0$90263$14726298@news.sunsite.dk>
Hello, All!

suppose i want to collect a list of closures i want to evaluate later:

(loop for obj in objects
    collect (lambda () (process obj)))

however, this won't work correctly -- all closures will refer to one 
variable OBJ in LOOP code, so with typical LOOP implementation they will all 
have last OBJ value.

[3]> (loop for fun in
    (loop for obj in '(1 2 3 4 5)
        collect (lambda () obj))
    collect (funcall fun))

(5 5 5 5 5)

to avoid this behaviour, i rebind variable with LET:

(loop for obj in objects
    collect (let ((obj obj))
                collect (lambda () (process obj))))

this really helps:

CL-USER> (loop for fun in
        (loop for obj in '(1 2 3 4 5)
       collect (let ((obj obj)) (lambda () obj)))
        collect (funcall fun))
(1 2 3 4 5)

however, i'm not sure that this fix works according to Common Lisp standard.
i think it is, as LET creates new variable binding (i could call it other 
name), and each closure points to it's own binding.

however at least in one implemenation (ABCL) in compiled mode it doesn't 
work:

CL-USER> (funcall (compile nil (lambda () (loop for fun in
    (loop for o in '(1 2 3 4 5)
          collect (let ((o2 o)) (lambda () o2)))
    collect (funcall fun)))))

(5 5 5 5 5)

while all other implementations i've tested (CLISP, SBCL, ACL, Lispworks) 
produce expected output
(1 2 3 4 5), at least with default compiler settings.

ABCL implementation maintainer (Peter Graves) suggests to use backquote to 
form a list instead of closure, which i find a bit ugly. he says it's a 
question if compiler has to propagate constants.
while i believe my version should be correct according to the semantics 
about how bindings and closures work.

i appreciate if somebody can clarify this, pointing to paragraphs of 
standard specifying which behaviour is correct.

With best regards, Alex Mizrahi. 

From: Tim Bradshaw
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <1173108109.141765.321770@s48g2000cws.googlegroups.com>
On Mar 5, 2:39 pm, "Alex Mizrahi" <········@users.sourceforge.net>
wrote:

>
> (loop for obj in objects
>     collect (lambda () (process obj)))
>
> however, this won't work correctly -- all closures will refer to one
> variable OBJ in LOOP code, so with typical LOOP implementation they will all
> have last OBJ value.

yes.  I think there is a note somewhere in the spec about exactly this
issue.

> CL-USER> (loop for fun in
>         (loop for obj in '(1 2 3 4 5)
>        collect (let ((obj obj)) (lambda () obj)))
>         collect (funcall fun))
> (1 2 3 4 5)
>
> however, i'm not sure that this fix works according to Common Lisp standard.

Yes, this is the right way of doing this.

--tim
From: Vassil Nikolov
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <yy8vfy8jufps.fsf@eskimo.com>
On 5 Mar 2007 07:21:49 -0800, "Tim Bradshaw" <··········@tfeb.org> said:

| On Mar 5, 2:39 pm, "Alex Mizrahi" <········@users.sourceforge.net>
| wrote:

|| 
|| (loop for obj in objects
|| collect (lambda () (process obj)))
|| 
|| however, this won't work correctly -- all closures will refer to one
|| variable OBJ in LOOP code, so with typical LOOP implementation they will all
|| have last OBJ value.

| yes.  I think there is a note somewhere in the spec about exactly this
| issue.

  ...and the one-line summary is that a closure is over bindings, rather
  than values.  Performance considerations apart, each iteration can
  rebind the control variable or it can reuse the same binding at the
  discretion of the implementation, and obviosly the difference becomes
  visible when closures are made.

  ---Vassil.


-- 
Is your code free of side defects?
From: Rob Warnock
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <3KmdnbiVM9VTfnLYnZ2dnUVZ_ozinZ2d@speakeasy.net>
Tim Bradshaw <··········@tfeb.org> wrote:
+---------------
| "Alex Mizrahi" <········@users.sourceforge.net> wrote:
| > CL-USER> (loop for fun in
| >         (loop for obj in '(1 2 3 4 5)
| >        collect (let ((obj obj)) (lambda () obj)))
| >         collect (funcall fun))
| > (1 2 3 4 5)
| >
| > however, i'm not sure that this fix works according to
| > Common Lisp standard.
| 
| Yes, this is the right way of doing this.
+---------------

Though it would be a *lot* easier to read if written this way:

    > (loop for item in '(1 2 3 4 5)
	collect (let ((obj item)) (lambda () obj)))

    (#<Interpreted Function "LOOP FOR" {489410B1}>
     #<Interpreted Function "LOOP FOR" {48941141}>
     #<Interpreted Function "LOOP FOR" {489411D1}>
     #<Interpreted Function "LOOP FOR" {48941261}>
     #<Interpreted Function "LOOP FOR" {489412F1}>)
    > (mapcar 'funcall *)

    (1 2 3 4 5)
    > 


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Rob Warnock
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <3KmdnbuVM9WueHLYnZ2dnUVZ_ozinZ2d@speakeasy.net>
I just wrote:
+---------------
| Though it would be a *lot* easier to read if written this way:
| 
|     > (loop for item in '(1 2 3 4 5)
| 	collect (let ((obj item)) (lambda () obj)))
| 
|     (#<Interpreted Function "LOOP FOR" {489410B1}>
|      #<Interpreted Function "LOOP FOR" {48941141}>
|      #<Interpreted Function "LOOP FOR" {489411D1}>
|      #<Interpreted Function "LOOP FOR" {48941261}>
|      #<Interpreted Function "LOOP FOR" {489412F1}>)
|     > (mapcar 'funcall *)
| 
|     (1 2 3 4 5)
|     > 
+---------------

It also works [in CMUCL, at least] when compiled:

    > (funcall
       (compile nil
	        (lambda ()
		  (loop for item in '(1 2 3 4 5)
		    collect (let ((obj item)) (lambda () obj))))))

    ; Compiling LAMBDA NIL: 
    ; Compiling Top-Level Form: 

    (#<Closure Over Function "LAMBDA NIL" {4895F0D1}>
     #<Closure Over Function "LAMBDA NIL" {4895F0E9}>
     #<Closure Over Function "LAMBDA NIL" {4895F101}>
     #<Closure Over Function "LAMBDA NIL" {4895F119}>
     #<Closure Over Function "LAMBDA NIL" {4895F131}>)
    > (mapcar 'funcall *)

    (1 2 3 4 5)
    > 


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Wolfram Fenske
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <1173145359.346830.199730@n33g2000cwc.googlegroups.com>
"Alex Mizrahi" <········@users.sourceforge.net> writes:

> [3]> (loop for fun in
>     (loop for obj in '(1 2 3 4 5)
>         collect (lambda () obj))
>     collect (funcall fun))
>
> (5 5 5 5 5)
>
> to avoid this behaviour, i rebind variable with LET:

[...]

> this really helps:
>
> CL-USER> (loop for fun in
>         (loop for obj in '(1 2 3 4 5)
>        collect (let ((obj obj)) (lambda () obj)))
>         collect (funcall fun))
> (1 2 3 4 5)
>
> however, i'm not sure that this fix works according to Common Lisp standard.
> i think it is, as LET creates new variable binding (i could call it other
> name), and each closure points to it's own binding.
>
> however at least in one implemenation (ABCL) in compiled mode it doesn't
> work:
>
> CL-USER> (funcall (compile nil (lambda () (loop for fun in
>     (loop for o in '(1 2 3 4 5)
>           collect (let ((o2 o)) (lambda () o2)))
>     collect (funcall fun)))))
>
> (5 5 5 5 5)
>
> while all other implementations i've tested (CLISP, SBCL, ACL, Lispworks)
> produce expected output
> (1 2 3 4 5), at least with default compiler settings.
>
> ABCL implementation maintainer (Peter Graves) suggests to use backquote to
> form a list instead of closure, which i find a bit ugly. he says it's a
> question if compiler has to propagate constants.

If that is the case then ABCL's constant propagation is broken.  (BTW,
I think this should be "copy propagation".  I don't see how constant
propagation fits in this example.  My guess is that ABCL "optimizes"
(let ((o2 o)) (lambda () o2)) into (lambda () o), which is exactly
what you tried to avoid.)

> while i believe my version should be correct according to the semantics
> about how bindings and closures work.

I agree.  By wrapping the LAMBDA in a LET you create new bindings,
which are captured by the closures.  In your example, these bindings
cannot
be modified after each closure has been created.  ABCL must generate
wrong code at some point.

--
Wolfram Fenske

A: Yes.
>Q: Are you sure?
>>A: Because it reverses the logical flow of conversation.
>>>Q: Why is top posting frowned upon?
From: Vassil Nikolov
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <yy8vejo3ueuo.fsf@eskimo.com>
On Mon, 5 Mar 2007 16:39:37 +0200, "Alex Mizrahi" <········@users.sourceforge.net> said:
| ...
| however at least in one implemenation (ABCL) in compiled mode it doesn't 
| work:
|
| CL-USER> (funcall (compile nil (lambda () (loop for fun in
|     (loop for o in '(1 2 3 4 5)
|           collect (let ((o2 o)) (lambda () o2)))
|     collect (funcall fun)))))
|
| (5 5 5 5 5)

  Hmmm...  You write "in compiled mode"---are we to think that it works
  differently without the call to COMPILE (which would be another bug)?

  ---Vassil.


-- 
Is your code free of side defects?
From: Alex Mizrahi
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <45ed3a94$0$90272$14726298@news.sunsite.dk>
(message (Hello 'Vassil)
(you :wrote  :on '(05 Mar 2007 21:21:19 -0800))
(

 VN> | CL-USER> (funcall (compile nil (lambda () (loop for fun in
 VN> |     (loop for o in '(1 2 3 4 5)
 VN> |           collect (let ((o2 o)) (lambda () o2)))
 VN> |     collect (funcall fun)))))
 VN> |
 VN> | (5 5 5 5 5)

 VN>   Hmmm...  You write "in compiled mode"---are we to think that it works
 VN>   differently without the call to COMPILE

yes, without COMPILE all implementations i've tested return (1 2 3 4 5).
with COMPILE ABCL returns (5 5 5 5 5), while other implementations still 
return (1 2 3 4 5).

i agree in guess with Wolfram Fenske -- looks like compiler over-optimizes, 
removing LET.
but maybe it's something with closure creation broken..

 VN> (which would be another bug)?

why that would be another bug?

)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"�� ���� ������� �����") 
From: Vassil Nikolov
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <yy8vmz2p7qnr.fsf@eskimo.com>
On Tue, 6 Mar 2007 11:55:28 +0200, "Alex Mizrahi" <········@users.sourceforge.net> said:
| yes, without COMPILE all implementations i've tested return (1 2 3 4 5).
| with COMPILE ABCL returns (5 5 5 5 5), while other implementations still 
| return (1 2 3 4 5).
| ...
| why that would be another bug?

  Compiled code behaving differently from interpreted code.  (Well, the
  underlying reason for that may be the same as for the ``let ((o2 o))''
  test case producing an incorrect result, but I am not looking inside
  the black box like a good QA tester.)

  ---Vassil.


-- 
Is your code free of side defects?
From: Pascal Bourguignon
Subject: Re: is this a portable CL? (rebind)
Date: 
Message-ID: <87bqj6adyf.fsf@voyager.informatimago.com>
"Alex Mizrahi" <········@users.sourceforge.net> writes:

> Hello, All!
>
> suppose i want to collect a list of closures i want to evaluate later:
>
> (loop for obj in objects
>     collect (lambda () (process obj)))
>
> however, this won't work correctly -- all closures will refer to one 
> variable OBJ in LOOP code, so with typical LOOP implementation they will all 
> have last OBJ value.

Or may be not.  Depends on the implementation. See section 6.1.9 Notes about Loop


> [3]> (loop for fun in
>     (loop for obj in '(1 2 3 4 5)
>         collect (lambda () obj))
>     collect (funcall fun))
>
> (5 5 5 5 5)
>
> to avoid this behaviour, i rebind variable with LET:
>
> (loop for obj in objects
>     collect (let ((obj obj))
>                 collect (lambda () (process obj))))
>
> this really helps:
>
> CL-USER> (loop for fun in
>         (loop for obj in '(1 2 3 4 5)
>        collect (let ((obj obj)) (lambda () obj)))
>         collect (funcall fun))
> (1 2 3 4 5)
>
> however, i'm not sure that this fix works according to Common Lisp standard.

Yes, that's how it should be done.


> i think it is, as LET creates new variable binding (i could call it other 
> name), and each closure points to it's own binding.

Indeed.


> however at least in one implemenation (ABCL) in compiled mode it doesn't 
> work:
>
> CL-USER> (funcall (compile nil (lambda () (loop for fun in
>     (loop for o in '(1 2 3 4 5)
>           collect (let ((o2 o)) (lambda () o2)))
>     collect (funcall fun)))))
>
> (5 5 5 5 5)

This is a bug.


> while all other implementations i've tested (CLISP, SBCL, ACL, Lispworks) 
> produce expected output
> (1 2 3 4 5), at least with default compiler settings.
>
> ABCL implementation maintainer (Peter Graves) suggests to use backquote to 
> form a list instead of closure, which i find a bit ugly. he says it's a 
> question if compiler has to propagate constants.

Well, indeed, compiling explicitely a new form is a workaround:

 ...  collect #+abcl(compile nil `(let ((o2 ,o)) (lambda () o2)))
              #-abcl(let ((o2 o)) (lambda () o2) 


> while i believe my version should be correct according to the semantics 
> about how bindings and closures work.

Yes, it should.


> i appreciate if somebody can clarify this, pointing to paragraphs of 
> standard specifying which behaviour is correct.
>
> With best regards, Alex Mizrahi. 

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
        Un chat errant
se soulage
        dans le jardin d'hiver
                                        Shiki