I'm a beginner to Lisp. I was writing a function which uses an array
for its internal calculations, and noticed behavior that was unexpected
(to me). Here's an illustration:
(defun test1 ()
(let ((x 0)) ; initialize x to 0
(setf x (1+ x)) ; increment x
x)) ; return x
(defun test2 ()
(let ((x #(0))) ; initialize x as a single-cell
array, containing 0
(setf (aref x 0) (1+ (aref x 0))) ; increment the first cell of x
(aref x 0))) ; return the first cell of x
Repeated evaluation of (test1) produces 1, then 1 again, etc. as
expected. However, repeated evaluation of (test2) produces 1, then 2,
then 3, etc. It seems that the array is not being re-initialized to 0
on subsequent calls. What's going on here? I've tried it in CLISP and
CMUCL with the same result.
micromoog wrote:
> I'm a beginner to Lisp. I was writing a function which uses an array
> for its internal calculations, and noticed behavior that was unexpected
> (to me). Here's an illustration:
>
> (defun test1 ()
> (let ((x 0)) ; initialize x to 0
> (setf x (1+ x)) ; increment x
> x)) ; return x
>
> (defun test2 ()
> (let ((x #(0))) ; initialize x as a single-cell
> array, containing 0
> (setf (aref x 0) (1+ (aref x 0))) ; increment the first cell of x
> (aref x 0))) ; return the first cell of x
>
> Repeated evaluation of (test1) produces 1, then 1 again, etc. as
> expected. However, repeated evaluation of (test2) produces 1, then 2,
> then 3, etc. It seems that the array is not being re-initialized to 0
> on subsequent calls. What's going on here? I've tried it in CLISP and
> CMUCL with the same result.
#( ... ) creates a literal object at read-time. It's similar to creating
strings at read-time by using double quotes - e.g., "Hello, World!") -
or creating literal objects at read-time with single quotes - e.g., '(1
2 3).
In other words, it _is_ always the same object.
If you want to ensure that a fresh array is created at runtime, say
(vector 0) instead.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Pascal Costanza wrote:
> micromoog wrote:
> > I'm a beginner to Lisp. I was writing a function which uses an array
> > for its internal calculations, and noticed behavior that was unexpected
> > (to me). Here's an illustration:
> >
> > (defun test1 ()
> > (let ((x 0)) ; initialize x to 0
> > (setf x (1+ x)) ; increment x
> > x)) ; return x
> >
> > (defun test2 ()
> > (let ((x #(0))) ; initialize x as a single-cell
> > array, containing 0
> > (setf (aref x 0) (1+ (aref x 0))) ; increment the first cell of x
> > (aref x 0))) ; return the first cell of x
> >
> > Repeated evaluation of (test1) produces 1, then 1 again, etc. as
> > expected. However, repeated evaluation of (test2) produces 1, then 2,
> > then 3, etc. It seems that the array is not being re-initialized to 0
> > on subsequent calls. What's going on here? I've tried it in CLISP and
> > CMUCL with the same result.
>
> #( ... ) creates a literal object at read-time. It's similar to creating
> strings at read-time by using double quotes - e.g., "Hello, World!") -
> or creating literal objects at read-time with single quotes - e.g., '(1
> 2 3).
>
> In other words, it _is_ always the same object.
>
> If you want to ensure that a fresh array is created at runtime, say
> (vector 0) instead.
>
>
Ahhh, "vector" seems to be the keyword I'm looking for. Works as
expected, thanks!
So, in my original code, I'm creating an array object during the first
evaluation, which then sits around with no references to it until I
re-attach to the same object on successive evaluations? Does this mean
that the object, between evaluations, has no references and would
therefore be a candidate for the GC? In other words, if I evaluated it
three times and got 1, 2, 3, then the GC runs, the next time I evaluate
it I'd get 1 again?
I'm just trying to understand Lisp memory, and not quite getting where
that array "goes" between evaluations, since it's scoped locally to the
function.
micromoog wrote:
> Pascal Costanza wrote:
> > micromoog wrote:
>
> Ahhh, "vector" seems to be the keyword I'm looking for. Works as
> expected, thanks!
Just to be clear, or pedantic, vector is not a "keyword", it is an
ordinary Common Lisp function, named VECTOR. You can find it documented
in the Common Lisp Hyperspec, at
http://www.lispworks.com/documentation/HyperSpec/Body/f_vector.htm
A more general version of the function is MAKE-ARRAY,
http://www.lispworks.com/documentation/HyperSpec/Body/f_mk_ar.htm
> I'm just trying to understand Lisp memory, and not quite getting where
> that array "goes" between evaluations, since it's scoped locally to the
> function.
Pascal gave a good explanation. One way to think about it is that the
reading of the #( syntax caused the Lisp reader (which is the first
step to executing or compiling your code) to make a call to VECTOR,
resulting in a literal vector which became part of the function
definition. A compiler could, in principle, choose to put that vector
in read-only memory, resulting in a memory fault when your code tries
to modify the contents.
As long as the function definition is unchanged, the vector itself
stays around as part of the definition.
··············@hotmail.com wrote:
> micromoog wrote:
> > Pascal Costanza wrote:
> > > micromoog wrote:
>
> >
> > Ahhh, "vector" seems to be the keyword I'm looking for. Works as
> > expected, thanks!
>
> Just to be clear, or pedantic, vector is not a "keyword", it is an
> ordinary Common Lisp function, named VECTOR. You can find it documented
> in the Common Lisp Hyperspec, at
> http://www.lispworks.com/documentation/HyperSpec/Body/f_vector.htm
>
> A more general version of the function is MAKE-ARRAY,
> http://www.lispworks.com/documentation/HyperSpec/Body/f_mk_ar.htm
>
> > I'm just trying to understand Lisp memory, and not quite getting where
> > that array "goes" between evaluations, since it's scoped locally to the
> > function.
>
> Pascal gave a good explanation. One way to think about it is that the
> reading of the #( syntax caused the Lisp reader (which is the first
> step to executing or compiling your code) to make a call to VECTOR,
> resulting in a literal vector which became part of the function
> definition. A compiler could, in principle, choose to put that vector
> in read-only memory, resulting in a memory fault when your code tries
> to modify the contents.
>
> As long as the function definition is unchanged, the vector itself
> stays around as part of the definition.
That's a good way to think about it . . . I'm imagining that although
my source code looks like this:
(let ((x #(0)))
the function that gets created actually looks something like this:
(let ((x [the nameless array object that was created at read-time] ))
micromoog wrote:
> That's a good way to think about it . . . I'm imagining that although
> my source code looks like this:
>
> (let ((x #(0)))
>
> the function that gets created actually looks something like this:
>
> (let ((x [the nameless array object that was created at read-time] ))
Try (disassemble 'test2), and again after each call. In CLISP, the
constants are held in a special "constant data" portion of the
function, and you can see how they are used. The name 'x' does not
appear in the function, because it is a lexical variable.
··············@hotmail.com wrote:
> micromoog wrote:
> > That's a good way to think about it . . . I'm imagining that although
> > my source code looks like this:
> >
> > (let ((x #(0)))
> >
> > the function that gets created actually looks something like this:
> >
> > (let ((x [the nameless array object that was created at read-time] ))
>
> Try (disassemble 'test2), and again after each call. In CLISP, the
> constants are held in a special "constant data" portion of the
> function, and you can see how they are used. The name 'x' does not
> appear in the function, because it is a lexical variable.
Looking at the disassembly makes it very clear. Here I can visualize
exactly what another poster (Barry Margolin) described: "Every time you
assigned to the array, you were effectively modifying the code of your
function".
Thanks!
"micromoog" <·········@gmail.com> writes:
> That's a good way to think about it . . . I'm imagining that although
> my source code looks like this:
>
> (let ((x #(0)))
>
> the function that gets created actually looks something like this:
>
> (let ((x [the nameless array object that was created at read-time] ))
The various parts of the REPL, the Read-Eval-Print-Loop are
available separately, so you can actually check this out.
First use READ on its own to capture the data structure that
read builds.
CL-USER> (defvar literal (read))
(let ((v #(0)))
(aref v 0))
LITERAL
CL-USER> literal
(LET ((V #(0)))
(AREF V 0))
CL-USER> (second *)
((V #(0)))
CL-USER> (car *)
(V #(0))
CL-USER> (second *)
#(0)
CL-USER> (type-of *)
(SIMPLE-VECTOR 1)
so yes, the data structure that READ builds, which plays the
role of source code as far as EVAL is concerned, contains an
anonymous array object.
Doing the same thing with vector ...
CL-USER> (defvar vector (read))
(let ((v (vector 0)))
(aref v 0))
VECTOR
CL-USER> vector
(LET ((V (VECTOR 0)))
(AREF V 0))
CL-USER> (second *)
((V (VECTOR 0)))
CL-USER> (car *)
(V (VECTOR 0))
CL-USER> (second *)
(VECTOR 0)
CL-USER> (type-of *)
CONS
... you end up with a cons, ie a non-empty list.
It is the form: (vector 0), a list of two items, the operator
`vector' and the operand `0'.
These are instructions that eval can understand and which
instruct it to build a vector of one item, initialised to
zero. However it is just the instructions, ready to be
executed...
Alan Crowe
Edinburgh
Scotland
micromoog wrote:
> Pascal Costanza wrote:
>
>>micromoog wrote:
>>
>>>I'm a beginner to Lisp. I was writing a function which uses an array
>>>for its internal calculations, and noticed behavior that was unexpected
>>>(to me). Here's an illustration:
>>>
>>>(defun test1 ()
>>> (let ((x 0)) ; initialize x to 0
>>> (setf x (1+ x)) ; increment x
>>> x)) ; return x
>>>
>>>(defun test2 ()
>>> (let ((x #(0))) ; initialize x as a single-cell
>>>array, containing 0
>>> (setf (aref x 0) (1+ (aref x 0))) ; increment the first cell of x
>>> (aref x 0))) ; return the first cell of x
>>>
>>>Repeated evaluation of (test1) produces 1, then 1 again, etc. as
>>>expected. However, repeated evaluation of (test2) produces 1, then 2,
>>>then 3, etc. It seems that the array is not being re-initialized to 0
>>>on subsequent calls. What's going on here? I've tried it in CLISP and
>>>CMUCL with the same result.
>>
>>#( ... ) creates a literal object at read-time. It's similar to creating
>>strings at read-time by using double quotes - e.g., "Hello, World!") -
>>or creating literal objects at read-time with single quotes - e.g., '(1
>>2 3).
>>
>>In other words, it _is_ always the same object.
[...]
> So, in my original code, I'm creating an array object during the first
> evaluation, which then sits around with no references to it until I
> re-attach to the same object on successive evaluations? Does this mean
> that the object, between evaluations, has no references and would
> therefore be a candidate for the GC? In other words, if I evaluated it
> three times and got 1, 2, 3, then the GC runs, the next time I evaluate
> it I'd get 1 again?
>
> I'm just trying to understand Lisp memory, and not quite getting where
> that array "goes" between evaluations, since it's scoped locally to the
> function.
No, it's not created the first time the function is executed. It already
exists before the function is ever executed. The #( notation is a read
macro that tells the Lisp reader to create an array at read time. The
Lisp reader corresponds to what is traditionally called a scanner/parser
in other languages. So this happens even before the code is compiled.
It is actually the case that you can produce arbitrary objects at read
time and make them part of your source code. Since the evaluation rules
state that all objects which are neither conses nor symbols evaluate to
themselves, this is well-defined.
Since they are part of the source code, and will eventually become part
of the compiled code, they won't be garbage collected (unless the
function is redefined at runtime).
However, the evaluation rules also state that destructive operations on
literal objects / self-evaluating objects lead to undefined behavior.
The fact that your original code treats the array created at read time
as a regular array that can be modified via side effects is only an
"accidental" behavior (or an extension of Common Lisp, so to speak).
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Pascal Costanza wrote:
> micromoog wrote:
> > Pascal Costanza wrote:
> >
> >>micromoog wrote:
> >>>
> >>>(defun test2 ()
> >>> (let ((x #(0))) ; initialize x as a single-cell
> >>>array, containing 0
> >>> (setf (aref x 0) (1+ (aref x 0))) ; increment the first cell of x
> >>> (aref x 0))) ; return the first cell of x
> >>>
> >>>repeated evaluation of (test2) produces 1, then 2,
> >>>then 3, etc. It seems that the array is not being re-initialized to 0
> >>>on subsequent calls. What's going on here? I've tried it in CLISP and
> >>>CMUCL with the same result.
> >>
> >>#( ... ) creates a literal object at read-time. It's similar to creating
> >>strings at read-time by using double quotes - e.g., "Hello, World!") -
> >>or creating literal objects at read-time with single quotes - e.g., '(1
> >>2 3).
> >>
>
> > So, in my original code, I'm creating an array object during the first
> > evaluation, which then sits around with no references to it until I
> > re-attach to the same object on successive evaluations? Does this mean
> > that the object, between evaluations, has no references and would
> > therefore be a candidate for the GC? In other words, if I evaluated it
> > three times and got 1, 2, 3, then the GC runs, the next time I evaluate
> > it I'd get 1 again?
>
> No, it's not created the first time the function is executed. It already
> exists before the function is ever executed. The #( notation is a read
> macro that tells the Lisp reader to create an array at read time. The
> Lisp reader corresponds to what is traditionally called a scanner/parser
> in other languages. So this happens even before the code is compiled.
>
> It is actually the case that you can produce arbitrary objects at read
> time and make them part of your source code. Since the evaluation rules
> state that all objects which are neither conses nor symbols evaluate to
> themselves, this is well-defined.
>
> Since they are part of the source code, and will eventually become part
> of the compiled code, they won't be garbage collected (unless the
> function is redefined at runtime).
>
> However, the evaluation rules also state that destructive operations on
> literal objects / self-evaluating objects lead to undefined behavior.
> The fact that your original code treats the array created at read time
> as a regular array that can be modified via side effects is only an
> "accidental" behavior (or an extension of Common Lisp, so to speak).
>
Thanks again; I think I've almost got it:
1) At read-time (when the defun is evaluated), the #(0) expression
creates an array object, initialized to contain the value #(0). This
object has no name and is only accessible from within that function (in
fact, only from within the expression which contains it), but will
continue to exist for the lifetime of the REPL.
2) At runtime, during each successive call to the function, a new local
variable x is bound to the persistent object created above. When
destructive edits are made to x, the nameless array object is edited
(but this part is an undefined, implementation-specific detail).
3) I shouldn't be using the #( syntax to create arrays without good
reason, and even though the printed output of the expressions (vector
0) and #(0) are the same, what they actually do isn't the same at all
(the latter being rife with side-effects).
In article <························@g14g2000cwa.googlegroups.com>,
"micromoog" <·········@gmail.com> wrote:
> Thanks again; I think I've almost got it:
>
> 1) At read-time (when the defun is evaluated), the #(0) expression
> creates an array object, initialized to contain the value #(0). This
> object has no name and is only accessible from within that function (in
> fact, only from within the expression which contains it), but will
> continue to exist for the lifetime of the REPL.
>
> 2) At runtime, during each successive call to the function, a new local
> variable x is bound to the persistent object created above. When
> destructive edits are made to x, the nameless array object is edited
> (but this part is an undefined, implementation-specific detail).
>
> 3) I shouldn't be using the #( syntax to create arrays without good
> reason, and even though the printed output of the expressions (vector
> 0) and #(0) are the same, what they actually do isn't the same at all
> (the latter being rife with side-effects).
Yes, you're pretty much got it.
What you created was "self-modifying code". Every time you assigned to
the array, you were effectively modifying the code of your function.
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
micromoog <·········@gmail.com> wrote:
+---------------
| Thanks again; I think I've almost got it:
| 1) At read-time (when the defun is evaluated), the #(0) expression
| creates an array object, initialized to contain the value #(0). This
| object has no name and is only accessible from within that function
| (in fact, only from within the expression which contains it)...
+---------------
Not always!! If you COMPILE-FILE the file containing the function,
and there's another function with the "same"[1] constant in it, the
compiler is permitted to "coalesce" the two constants, which can
cause unwanted coupling between the two functions. Here's an actual
example [though note that I changed your vector #(0) to a list '(0)
and changed AREF to NTH, since the compiler I was using didn't coalesce
the two constant vectors (but did coalesce the two constant lists)]:
$ cat foo.lisp
(defun foo (n)
(let ((x '(0)))
(incf (nth n x))))
(defun bar (m)
(let ((y '(0)))
(incf (nth m y))))
$ my_favorite_common_lisp
...chatter...
> (compile-file "foo")
...more chatter...
#p"/u/rpw3/src/cmd/qdl/foo.x86f"
NIL
NIL
> (load *)
; Loading #p"/u/rpw3/src/cmd/qdl/foo.x86f".
T
> (foo 0)
1
> (foo 0) ; Calling FOO1 twice shows the problem you saw.
2
> (bar 0) ; Now call the *other* function!!
3
>
Clearly, the two functions are sharing the same '(0)!!
Since modifying a constant [as above] is "undefined" behavior,
worse things could happen. YMMV...
-Rob
[1] "Same" here is defined by CLHS 3.2.4 "Literal Objects in Compiled
Files", 3.2.4.2.2 "Definition of Similarity", and 3.2.4.4 "Additional
Constraints on Externalizable Objects", which contains the rules on
coalescing similar objects.
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Rob Warnock wrote:
> micromoog <·········@gmail.com> wrote:
> +---------------
> | Thanks again; I think I've almost got it:
> | 1) At read-time (when the defun is evaluated), the #(0) expression
> | creates an array object, initialized to contain the value #(0). This
> | object has no name and is only accessible from within that function
> | (in fact, only from within the expression which contains it)...
> +---------------
>
> Not always!! If you COMPILE-FILE the file containing the function,
> and there's another function with the "same"[1] constant in it, the
> compiler is permitted to "coalesce" the two constants, which can
> cause unwanted coupling between the two functions. Here's an actual
> example [though note that I changed your vector #(0) to a list '(0)
> and changed AREF to NTH, since the compiler I was using didn't coalesce
> the two constant vectors (but did coalesce the two constant lists)]:
>
> $ cat foo.lisp
> (defun foo (n)
> (let ((x '(0)))
> (incf (nth n x))))
>
> (defun bar (m)
> (let ((y '(0)))
> (incf (nth m y))))
>
> $ my_favorite_common_lisp
> ...chatter...
> > (compile-file "foo")
> ...more chatter...
> #p"/u/rpw3/src/cmd/qdl/foo.x86f"
> NIL
> NIL
> > (load *)
>
> ; Loading #p"/u/rpw3/src/cmd/qdl/foo.x86f".
> T
> > (foo 0)
>
> 1
> > (foo 0) ; Calling FOO1 twice shows the problem you saw.
>
> 2
> > (bar 0) ; Now call the *other* function!!
>
> 3
> >
>
> Clearly, the two functions are sharing the same '(0)!!
>
> Since modifying a constant [as above] is "undefined" behavior,
> worse things could happen. YMMV...
>
Wow, that is truly weird. I think I'd better stay away from modifying
constants for now :)
micromoog <·········@gmail.com> wrote:
+---------------
| Rob Warnock wrote:
| > Clearly, the two functions are sharing the same '(0)!!
| > Since modifying a constant [as above] is "undefined" behavior,
| > worse things could happen. YMMV...
|
| Wow, that is truly weird. I think I'd better stay away from
| modifying constants for now :)
+---------------
Please, not just "for now" -- forever! Remember, "undefined behavior"
means *anything* can happen: a core-dump, a locked-up system, deleting
all your files ("rm -rf /"), launching thermonuclear weapons, etc.
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
micromoog wrote:
> 1) At read-time (when the defun is evaluated), the #(0) expression
> creates an array object, initialized to contain the value #(0). This
> object has no name and is only accessible from within that function (in
> fact, only from within the expression which contains it), but will
> continue to exist for the lifetime of the REPL.
...for the lifetime of a program.
> 2) At runtime, during each successive call to the function, a new local
> variable x is bound to the persistent object created above. When
> destructive edits are made to x, the nameless array object is edited
> (but this part is an undefined, implementation-specific detail).
>
> 3) I shouldn't be using the #( syntax to create arrays without good
> reason, and even though the printed output of the expressions (vector
> 0) and #(0) are the same, what they actually do isn't the same at all
> (the latter being rife with side-effects).
The #( syntax can/should be used for literal constants.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Pascal Costanza wrote:
> micromoog wrote:
> >>>I'm a beginner to Lisp. I was writing a function which uses an array
> >>>for its internal calculations, and noticed behavior that was unexpected
> >>>(to me). Here's an illustration:
> >>>
> >>>(defun test1 ()
> >>> (let ((x 0)) ; initialize x to 0
> >>> (setf x (1+ x)) ; increment x
> >>> x)) ; return x
> >>>
> >>>(defun test2 ()
> >>> (let ((x #(0))) ; initialize x as a single-cell
> >>>array, containing 0
> >>> (setf (aref x 0) (1+ (aref x 0))) ; increment the first cell of x
> >>> (aref x 0))) ; return the first cell of x
> >>>
>
> However, the evaluation rules also state that destructive operations on
> literal objects / self-evaluating objects lead to undefined behavior.
> The fact that your original code treats the array created at read time
> as a regular array that can be modified via side effects is only an
> "accidental" behavior (or an extension of Common Lisp, so to speak).
In fact, this is a big Lisp gotcha.
http://wiki.alu.org/Lisp_Gotchas
Tayssir
Hi.
......
> (defun test2 ()
> (let ((x #(0))) ; initialize x as a single-cell
> array, containing 0
> (setf (aref x 0) (1+ (aref x 0))) ; increment the first cell of x
> (aref x 0))) ; return the first cell of x
......
Do not modify literals (#(0) for example), ansi standard do not specify
behavior of this... You can use LOAD-TIME-VALUE (or closure) for this, try:
(defun test1 () (let ((x (load-time-value (vector 0)))) (incf (aref x 0))))
Regards, Szymon.
ps.
CL has 0-rank arrays:
CL-USER> (array-rank (make-array nil))
0
(defun test1 (&aux (x (load-time-value (make-array nil)))) (incf (aref x)))
;-)