From: Michael J. Conroy
Subject: Newbie variable question
Date: 
Message-ID: <eZ94a.2904$Zc3.323957939@newssvr10.news.prodigy.com>
Hi all...

I just don't understand...

(setq list0 '(b c d))
(setq list1 (cons 'a list0))
(setq list2 (cons 'a list0))

list0
    (B C D)
list1
    (A B C D)
list2
    (A B C D)

(setq list0 (append list0 '(e))
    (B C D E) <---- this is what I was after

list1
    (A B C D) <----ARHGHGHGHG...  I *wanted* (A B C D E)

WHY...  Why is list1 and list2 not referencing list0?  I thought setq was
setting pointers.  If so, shouldn't list1 ALWAYS be (a (list0))?

HELP!

Grrrr...

:(
Mike

From: Paul Dietz
Subject: Re: Newbie variable question
Date: 
Message-ID: <3E513065.8ECC8374@motorola.com>
"Michael J. Conroy" wrote:
> 
> Hi all...
> 
> I just don't understand...
> 
> (setq list0 '(b c d))
> (setq list1 (cons 'a list0))
> (setq list2 (cons 'a list0))
> 
> list0
>     (B C D)
> list1
>     (A B C D)
> list2
>     (A B C D)
> 
> (setq list0 (append list0 '(e))
>     (B C D E) <---- this is what I was after
> 
> list1
>     (A B C D) <----ARHGHGHGHG...  I *wanted* (A B C D E)
> 
> WHY...  Why is list1 and list2 not referencing list0?  I thought setq was
> setting pointers.  If so, shouldn't list1 ALWAYS be (a (list0))?


APPEND is non-destructive.  It does not alter any existing
list structure.

	Paul
From: Nils Goesche
Subject: Re: Newbie variable question
Date: 
Message-ID: <lyptpqhhbv.fsf@cartan.de>
"Michael J. Conroy" <········@snet.net> writes:

> I just don't understand...
> 
> (setq list0 '(b c d))

Do you know that you must not modify list literals?  You are not doing
it here, but you would like to :-)  For safety, make that:

(setq list0 (list 'b 'c 'd))

or

(setq list0 (copy-list '(b c d)))

> (setq list1 (cons 'a list0))
> (setq list2 (cons 'a list0))
> 
> list0
>     (B C D)
> list1
>     (A B C D)

Evaluate (eql list0 (cdr list1)) here.

> list2
>     (A B C D)
> 
> (setq list0 (append list0 '(e))
>     (B C D E) <---- this is what I was after

Evaluate (eql list0 (cdr list1)) again (and beware of the list literal
'(e)).

> list1
>     (A B C D) <----ARHGHGHGHG...  I *wanted* (A B C D E)
> 
> WHY...  Why is list1 and list2 not referencing list0?  I thought
> setq was setting pointers.  If so, shouldn't list1 ALWAYS be (a
> (list0))?

There are no ``pointers�� in Lisp.  Actually, it looks as if your
mental model is correct.  What you don't know, however, is that APPEND
will /copy/ LIST0 before returning the appended list :-) If you don't
want that, and instead want to modify the list in LIST0 destructively,
you should use NCONC instead of APPEND.

P.S.: I used EQL instead of EQ because I think that newbies shouldn't
      use EQ at all.  YMMV.

Regards,
-- 
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x0655CFA0
From: Michael J. Conroy
Subject: Re: Newbie variable question
Date: 
Message-ID: <oMa4a.2915$094.327834685@newssvr10.news.prodigy.com>
Nils,

Thanks for your quick reply.  May I trouble you with just one more question?

If I want to generate the following structures (in sequence):

1  (A)
2  (A B)
3  (A B (C))
4  (A B (C D))
5  (A B (C D) E)

I currently do the following:

1  (setq list0 (list 'A))
    (A)
2  (setq list0 (append list0 (list 'B)))
    (A B)
3  (nconc list0 (setq list1 (list '(C))))
    (A B (C))
4  (nconc (car list0) (list 'D))
    (C D)
4A list0
    (A B (C D))
5  (setq list0 (append list0 (list 'E)))
    (A B (C D) E)

This sequence appears to work OK, however, being a newbie, I'm not sure of
its sanity.  Any additional thoughts and/or helpful hints would be very much
appreciated.

Thanks,
Mike
From: Nils Goesche
Subject: Re: Newbie variable question
Date: 
Message-ID: <lyd6lqhef6.fsf@cartan.de>
"Michael J. Conroy" <········@snet.net> writes:

> If I want to generate the following structures (in sequence):
> 
> 1  (A)
> 2  (A B)
> 3  (A B (C))
> 4  (A B (C D))
> 5  (A B (C D) E)
> 
> I currently do the following:
> 
> 1  (setq list0 (list 'A))
>     (A)
> 2  (setq list0 (append list0 (list 'B)))
>     (A B)
> 3  (nconc list0 (setq list1 (list '(C))))
>     (A B (C))

Beware, you have a list literal '(c) there which you must not modify...

> 4  (nconc (car list0) (list 'D))
>     (C D)

but you do modify it (assuming you really meant (car list1) here).

> 4A list0
>     (A B (C D))
> 5  (setq list0 (append list0 (list 'E)))
>     (A B (C D) E)
> 
> This sequence appears to work OK, however, being a newbie, I'm not
> sure of its sanity.  Any additional thoughts and/or helpful hints
> would be very much appreciated.

One more thing: You should always use the return value of NCONC.  It
is not always necessary, but not using directly the return value of
NCONC can lead to surprises:

CL-USER 43 > (defparameter *foo* (list 1 2 3))
*FOO*

CL-USER 44 > (defparameter *bar* (list 4 5 6))
*BAR*

CL-USER 45 > (defparameter *empty* nil)
*EMPTY*

CL-USER 46 > (nconc *empty* *foo* *bar*)
(1 2 3 4 5 6)

CL-USER 47 > *foo*
(1 2 3 4 5 6)

CL-USER 48 > *bar*
(4 5 6)

CL-USER 49 > *empty*
NIL

Oops!

So, what was intended is

CL-USER 46 > (setq *empty* (nconc *empty* *foo* *bar*))

Regards,
-- 
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x0655CFA0
From: Martti Halminen
Subject: Re: Newbie variable question
Date: 
Message-ID: <3E515F19.2B8A5B28@kolumbus.fi>
"Michael J. Conroy" wrote:
> 
> Nils,
> 
> Thanks for your quick reply.  May I trouble you with just one more question?
> 
> If I want to generate the following structures (in sequence):
> 
> 1  (A)
> 2  (A B)
> 3  (A B (C))
> 4  (A B (C D))
> 5  (A B (C D) E)
> 
> I currently do the following:
> 
> 1  (setq list0 (list 'A))
>     (A)
> 2  (setq list0 (append list0 (list 'B)))
>     (A B)
> 3  (nconc list0 (setq list1 (list '(C))))
>     (A B (C))
> 4  (nconc (car list0) (list 'D))
>     (C D)
> 4A list0
>     (A B (C D))
> 5  (setq list0 (append list0 (list 'E)))
>     (A B (C D) E)
> 
> This sequence appears to work OK, however, being a newbie, I'm not sure of
> its sanity.  Any additional thoughts and/or helpful hints would be very much
> appreciated.

Considering that a list is most easily accessed at the front end, it is
somewhat unnatural Lisp style to add stuff to the end of the list. A
more common idiom would be to collect the stuff in reverse order, and
reverse the final result:

(setq list0 (list 'A))
(push 'B list0)
(push (list 'C 'D) list0)
(push 'E list0)
(setq  list0 (reverse list0))  ;; might use nreverse, if you own list0

--
From: Raymond Laning
Subject: Re: Newbie variable question
Date: 
Message-ID: <3E559140.8B1CEA3E@ix.netcom.com>
It would be a disservice to Mr. Conroy to fail to mention that use of
nconc is not encouraged since mistakes in its use can lead to very subtle
errors that are difficult to debug.  The gain in efficiency can be more
than offset by the dangers of destructive code.  I recommend getting your
code to work with append and then, if you need to squeeze the last few
microseconds out of your code, using nconc.


R Laning

Nils Goesche wrote:

> "Michael J. Conroy" <········@snet.net> writes:
>
> > I just don't understand...
> >
> > (setq list0 '(b c d))
>
> Do you know that you must not modify list literals?  You are not doing
> it here, but you would like to :-)  For safety, make that:
>
> (setq list0 (list 'b 'c 'd))
>
> or
>
> (setq list0 (copy-list '(b c d)))
>
> > (setq list1 (cons 'a list0))
> > (setq list2 (cons 'a list0))
> >
> > list0
> >     (B C D)
> > list1
> >     (A B C D)
>
> Evaluate (eql list0 (cdr list1)) here.
>
> > list2
> >     (A B C D)
> >
> > (setq list0 (append list0 '(e))
> >     (B C D E) <---- this is what I was after
>
> Evaluate (eql list0 (cdr list1)) again (and beware of the list literal
> '(e)).
>
> > list1
> >     (A B C D) <----ARHGHGHGHG...  I *wanted* (A B C D E)
> >
> > WHY...  Why is list1 and list2 not referencing list0?  I thought
> > setq was setting pointers.  If so, shouldn't list1 ALWAYS be (a
> > (list0))?
>
> There are no ``pointers�� in Lisp.  Actually, it looks as if your
> mental model is correct.  What you don't know, however, is that APPEND
> will /copy/ LIST0 before returning the appended list :-) If you don't
> want that, and instead want to modify the list in LIST0 destructively,
> you should use NCONC instead of APPEND.
>
> P.S.: I used EQL instead of EQ because I think that newbies shouldn't
>       use EQ at all.  YMMV.
>
> Regards,
> --
> Nils G�sche
> "Don't ask for whom the <CTRL-G> tolls."
>
> PGP key ID 0x0655CFA0
From: Kenny Tilton
Subject: Re: Newbie variable question
Date: 
Message-ID: <3E55AFA4.2010504@nyc.rr.com>
Raymond Laning wrote:
> It would be a disservice to Mr. Conroy to fail to mention that use of
> nconc is not encouraged since mistakes in its use can lead to very subtle
> errors that are difficult to debug. 

not encouraged? oh my, who teaches Lisp that way, and can you make them 
stop? :)

why not learn more (about which functions are destructive and when 
destructive is OK) instead of use less of the language being learned? 
nconc is in fact strongly recommended when the structure at hand is 
freely whackable. append is discouraged fiercely because when misapplied 
(easy to do since we all suffer from destructophobia) it can quietly 
knock a system to its knees and perpetuate the myth that lisp is slow.

i feel dread whether i am coding nconc or append. either i am, yes, 
creating a devilish bug or bogging my system down with a casual bit of 
code i won't remember when i notice things are slowing down.

always move towards the light is how i see it. learn what nconc does and 
when you can use it and continue to grow in lisp. reminding me...

I recently triggered mass hysteria here on c.l.l. by saying lispniks 
need to learn to think in terms of conses, cars, and cdrs. no hiding 
between syntactic sugar like first and rest. i should add to the list 
knowing when one can destructively modify a list.

cue the hysterics, but they are wrong. a lispnik needs to look at code 
and /see/ nifty will cons diagrams or they are doomed to a very short 
lisplife of nconc bugs and/or excess-consing bogs.

-- 

  kenny tilton
  clinisys, inc
  http://www.tilton-technology.com/
  ---------------------------------------------------------------
"Cells let us walk, talk, think, make love and realize
  the bath water is cold." -- Lorraine Lee Cudmore
From: Thomas F. Burdick
Subject: Re: Newbie variable question
Date: 
Message-ID: <xcv7kbto4je.fsf@monsoon.OCF.Berkeley.EDU>
Kenny Tilton <·······@nyc.rr.com> writes:

> i feel dread whether i am coding nconc or append. either i am, yes, 
> creating a devilish bug or bogging my system down with a casual bit of 
> code i won't remember when i notice things are slowing down.

I'd think the thing to do would be to teach a newbie about append and
nconc, and then tell them that they're almost always the wrong tool.
It only takes about 10 lines of code to implement queues: make-queue,
queue-pop, queue-push, queue-append, queue-list; and it only costs you
a cons cell.

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | No to Imperialist war |                        
     ,--'    _,'   | Wage class war!       |                        
    /       /      `-----------------------'                        
   (   -.  |                               
   |     ) |                               
  (`-.  '--.)                              
   `. )----'                               
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.21.04.10.55.73415@consulting.net.nz>
Hi Nils Goesche,

> There are no ``pointers�� in Lisp.  Actually, it looks as if your
> mental model is correct.  What you don't know, however, is that APPEND
> will /copy/ LIST0 before returning the appended list :-) If you don't
> want that, and instead want to modify the list in LIST0 destructively,
> you should use NCONC instead of APPEND.
> 
> P.S.: I used EQL instead of EQ because I think that newbies shouldn't
>       use EQ at all.  YMMV.

I'd assert that Common Lisp's EQ can only be understood after one comes to
understand pointers. It's one of those cases of being able to feel the
bits between your toes. Instead of trying to work though the HyperSpec's
obtuse definition of EQ one merely needs to have a mental model of
pointers.

Pointers pervade Common Lisp.

Regards,
Adam
From: Jochen Schmidt
Subject: Re: Newbie variable question
Date: 
Message-ID: <b34t9k$ash$03$1@news.t-online.com>
Adam Warner wrote:

> I'd assert that Common Lisp's EQ can only be understood after one comes to
> understand pointers. It's one of those cases of being able to feel the
> bits between your toes. Instead of trying to work though the HyperSpec's
> obtuse definition of EQ one merely needs to have a mental model of
> pointers.

I don't think that is true.

EQ is simply a test for object identity. The interesting bit is that Common 
Lisp allows character objects and numbers to get copied all the time. So 
two numbers who represent the same logical number might not necessarily be 
the identical object.

So we have EQL which has special cases for numbers and characters to compare 
them on the represented value and not their identity. One could probably 
speak of "logical identity" vs. "physical identity" or something like that.

ciao,
Jochen
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.21.11.06.21.217051@consulting.net.nz>
Hi Jochen Schmidt,

> Adam Warner wrote:
> 
>> I'd assert that Common Lisp's EQ can only be understood after one comes
>> to understand pointers. It's one of those cases of being able to feel
>> the bits between your toes. Instead of trying to work though the
>> HyperSpec's obtuse definition of EQ one merely needs to have a mental
>> model of pointers.
> 
> I don't think that is true.
> 
> EQ is simply a test for object identity. The interesting bit is that
> Common Lisp allows character objects and numbers to get copied all the
> time. So two numbers who represent the same logical number might not
> necessarily be the identical object.

Right. So there will be a different pointer to a copy of the content
identical object. So as I have come to understand it, EQ is a fast
low-level comparison that makes it possible to compare two objects without
even looking at their contents. If compared objects are stored in the same
memory location then they must be identical. By analogy to the Unix
filesystem, n files must be the same if one or more is a symbolic link to
a single file. There is no need to compare the contents of the files to
know they are identical.

So EQ is one of Common Lisp's low-level features that allows it to compete
speed-wise with C. Checking if two objects are stored at the same memory
location is faster than comparing the contents of two objects. Its use
makes sense by thinking about quick machine-level pointer comparisons.

While this doesn't describe the standard it explains why an operator such
as EQ exists within the standard.

> So we have EQL which has special cases for numbers and characters to
> compare them on the represented value and not their identity. One could
> probably speak of "logical identity" vs. "physical identity" or
> something like that.

So we have EQL to test for additional cases where the objects point to
data at different memory locations. It may now be necessary to compare the
actual data. If for example a string has been copied then there will be a
different pointer to that data. The contents of the strings must be
compared to determine whether they are identical. I don't see these as
"special cases." Rather I see them as illustrations of what is necessary
to determine whether the contents of two objects are identical.

Regards,
Adam
From: Jochen Schmidt
Subject: Re: Newbie variable question
Date: 
Message-ID: <b3531i$g6f$03$1@news.t-online.com>
Adam Warner wrote:

> Hi Jochen Schmidt,
> 
>> Adam Warner wrote:
>> 
>>> I'd assert that Common Lisp's EQ can only be understood after one comes
>>> to understand pointers. It's one of those cases of being able to feel
>>> the bits between your toes. Instead of trying to work though the
>>> HyperSpec's obtuse definition of EQ one merely needs to have a mental
>>> model of pointers.
>> 
>> I don't think that is true.
>> 
>> EQ is simply a test for object identity. The interesting bit is that
>> Common Lisp allows character objects and numbers to get copied all the
>> time. So two numbers who represent the same logical number might not
>> necessarily be the identical object.
> 
> Right. So there will be a different pointer to a copy of the content
> identical object. So as I have come to understand it, EQ is a fast
> low-level comparison that makes it possible to compare two objects without
> even looking at their contents. If compared objects are stored in the same
> memory location then they must be identical. By analogy to the Unix
> filesystem, n files must be the same if one or more is a symbolic link to
> a single file. There is no need to compare the contents of the files to
> know they are identical.

You need no "pointers" to describe anything here. Even if you go down to the 
machine you have objects at different _addresses_ but not different 
_pointers_ (no this are not synonyms).

> So EQ is one of Common Lisp's low-level features that allows it to compete
> speed-wise with C. Checking if two objects are stored at the same memory
> location is faster than comparing the contents of two objects. Its use
> makes sense by thinking about quick machine-level pointer comparisons.

No - its use makes sense when thinking about identity of objects. There is a 
semantic difference between EQ and EQL. EQ is not just a quick and dirty 
variant of EQL.

> While this doesn't describe the standard it explains why an operator such
> as EQ exists within the standard.

Not really (AFAIU)

>> So we have EQL which has special cases for numbers and characters to
>> compare them on the represented value and not their identity. One could
>> probably speak of "logical identity" vs. "physical identity" or
>> something like that.
> 
> So we have EQL to test for additional cases where the objects point to
> data at different memory locations. It may now be necessary to compare the
> actual data. If for example a string has been copied then there will be a
> different pointer to that data. The contents of the strings must be
> compared to determine whether they are identical. I don't see these as
> "special cases." Rather I see them as illustrations of what is necessary
> to determine whether the contents of two objects are identical.

Please RTFM. EQL does not compare strings or other structured objects any 
different to EQ. EQL does _only_ specially handle /numbers/ and 
/characters/ .

ciao,
Jochen
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.21.12.31.35.690487@consulting.net.nz>
Hi Jochen Schmidt,

> You need no "pointers" to describe anything here. Even if you go down to
> the machine you have objects at different _addresses_ but not different
> _pointers_ (no this are not synonyms).
> 
>> So EQ is one of Common Lisp's low-level features that allows it to
>> compete speed-wise with C. Checking if two objects are stored at the
>> same memory location is faster than comparing the contents of two
>> objects. Its use makes sense by thinking about quick machine-level
>> pointer comparisons.
> 
> No - its use makes sense when thinking about identity of objects. There
> is a semantic difference between EQ and EQL. EQ is not just a quick and
> dirty variant of EQL.

I didn't make this argument.

Thank you for telling me to RTFM. I was attempting to describe a way of
conceptualising the differences at a lower level than the standard. I've
now Googled to see how others have expressed this.

Successful Lisp - Chapter 3
http://www.psg.com/~dlamkins/sl/chapter03-05.html

   A value can have more than one name. That is, more than one symbol can
   share a value. Other languages have pointers that work this way. Lisp
   does not expose pointers to the programmer, but does have shared
   objects. An object is considered identical when it passes the EQ test.

http://www.sandelman.ottawa.on.ca/People/Michael_Richardson/finalReport/finalReport.foot.html

   ...eq
    In Lisp, eq does a direct pointer test. Since all atoms are typically
    pointers to some atom object, an atom is eq to itself. Numbers may or
    may not be eq to themselves.

http://www.cs.rochester.edu/u/myros/classes/cs244/lispTutorial.ps

   eq is true on identical pointers only (implementation dependent whether
   same numbers or chars are same pointers).

http://www.ai.mit.edu/people/phw/Books/LISPTABLE.HTML

   "EQ Checks Pointers Only"

You've been very quick to rubbish my conceptualisation, yet coming to
understand pointers still seems to be an appropriate way to understand
when two objects are EQ.

Regards,
Adam
From: Jochen Schmidt
Subject: Re: Newbie variable question
Date: 
Message-ID: <b35fp4$sj9$02$1@news.t-online.com>
Adam Warner wrote:

> You've been very quick to rubbish my conceptualisation, yet coming to
> understand pointers still seems to be an appropriate way to understand
> when two objects are EQ.

Well you said:

"I'd assert that Common Lisp's EQ can only be understood after one comes to
understand pointers."

Furthermore you claimed the standards description of EQ and EQL to be 
suboptimal. I merely tried to show you that you do not even need the notion 
of a "pointer" to describe and "conceptualize" the behaviour of this 
operators. Describing EQ in terms of pointers is something for people with 
prior experience in languages where pointers (variables that hold 
addresses) are used. CL does not have pointers in the sense C people know 
them.

You then showed that you did not even read the description of EQL by 
claiming it would compare strings structurally and not by identity. This 
were the point were I suggested you to RTFM. Instead of complaining about 
me being destructive to your conceptualisation and further wasting my time 
you should take this as a good advice and simply read it.

ciao,
Jochen
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.21.23.35.26.580315@consulting.net.nz>
Hi Jochen Schmidt,

> Please RTFM. EQL does not compare strings or other structured objects
> any different to EQ. EQL does _only_ specially handle /numbers/ and
> /characters/ .

While I was reading it at the time I now realise I was inappropriately
taking about strings instead of characters in my comment. Please read my
comment in relation to EQUAL instead of EQL.

Regards,
Adam
From: Nils Goesche
Subject: Re: Newbie variable question
Date: 
Message-ID: <lysmuhboib.fsf@cartan.de>
"Adam Warner" <······@consulting.net.nz> writes:

> > There are no ``pointers�� in Lisp.
...
> > P.S.: I used EQL instead of EQ because I think that newbies shouldn't
> >       use EQ at all.  YMMV.
> 
> I'd assert that Common Lisp's EQ can only be understood after one
> comes to understand pointers. It's one of those cases of being able
> to feel the bits between your toes. Instead of trying to work though
> the HyperSpec's obtuse definition of EQ one merely needs to have a
> mental model of pointers.
> 
> Pointers pervade Common Lisp.

Then they should pervade the HyperSpec, too, which doesn't even
mention pointers.

I think the whole concept of pointers doesn't apply here.  To me, a
pointer denotes a memory address, a number.  But the memory address of
a Lisp object is something totally uninteresting.  If a copying
garbage collector moves an object around that doesn't change the
identity of that object at all.  And can you have pointers without
having dangling pointers? ;-)  We don't need any notion of
``pointers�� to explain what's going on.

The semantics of EQ are absolutely clear, however: EQ simply tests if
two objects are the same, identical object.  There is nothing unclear
about that, object identity is the most simple concept there is.  For
instance, it is clear that

(let* ((x 42)
       (y x))
  (eq x y))

will return true, because x and y are bound to the same, identical
object.  We also have a guarantee that

 (eq 'foo 'foo)

will return true because symbols are interned.  Things can get a bit
hairy, however, once you try things like

 (eq 42 42)

Will this be true?  We don't know.  That doesn't mean that EQ has
unclear semantics -- what's unclear is whether those two 42 constants
will be read as the same, identical object or not.  We have no
guarantee about that.

But it is also clear that on a conceptual level, the integers 42 and
42 are the ``same��.  So are the characters #\a and #\a.  So, there is
another equality function EQL that will make the additional guarantee,
that it will measure the same characters and the same numbers as
equal.  As far as EQL is concerned, every number and every character
exists precisely once.  Now we have got what we want, and we call two
objects the ``same�� if they compare equal under EQL.

One might ask now why the standard doesn't simply mandate, as it does
for symbols, that the same numbers and the same characters are even
EQ!  It could easily do that -- this requirement wouldn't break one
thing.  Then, there would be no difference between EQ and EQL at all,
and both would measure object identity.  Note that this doesn't mean
that EQ is defined somewhat arbitrarily -- it isn't!  EQ means object
identity and nothing else.  The arbitrariness is in the decision
whether there are several different objects that are the same as 42 or
not.

So, by adding one or two sentences to the standard, we would have a
language where what is now EQL measures object identity and (EQ 42 42)
will always be true.  This would even have some merit -- it is not a
coincidence that EQL, /not/ EQ is the default equality predicate for
all but very few Common Lisp functions.  CATCH, GETF, THROW, GET,
REMF, GET-PROPERTIES and REMPROP are the only exceptions.  The
language would be easier that way.

The reason this isn't done is that it would slow down the language.
We can't ``intern�� bignums because that would eat too much memory, so
EQ would have to make additional checks even in those cases where we
know we are specifically searching for an object by its identity.  So
we have EQ and EQL different.

Since we can't say that EQL measures object identity when it is
in fact different from EQ which does, we have to invent another name
for the relation it defines.  I like calling it ``conceptual
identity��.

In my opinion newbies are better off if they start thinking in terms
of ``conceptual identity�� and don't worry about the subtleties
introduced by possibly not having 42 and 42 be the same object.

And pointers have nothing to do with it :-)

Regards,
-- 
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x0655CFA0
From: Thomas A. Russ
Subject: Semantics of Pointers [Was Re: Newbie variable question]
Date: 
Message-ID: <ymiu1expkhq.fsf_-_@sevak.isi.edu>
Nils Goesche <······@cartan.de> writes:

> I think the whole concept of pointers doesn't apply here.  To me, a
> pointer denotes a memory address, a number.  But the memory address of
> a Lisp object is something totally uninteresting.  If a copying
> garbage collector moves an object around that doesn't change the
> identity of that object at all.  And can you have pointers without
> having dangling pointers? ;-)  We don't need any notion of
> ``pointers�� to explain what's going on.

Hmmm.  I take a more conceptual view of pointers, one which is divorced
from machine addresses.  (I guess that's one advantage from never really
having been a C programmer :)  I tend to think of them more on the level
of the arrows that are drawn on a board.  Various operations in Common
Lisp change particular pointers to point at different things.

I find it really difficult to think about destructive list operations
without some notion of the parts of cons cells pointing at something
else.


-- 
Thomas A. Russ,  USC/Information Sciences Institute          ···@isi.edu    
From: Barry Margolin
Subject: Re: Semantics of Pointers [Was Re: Newbie variable question]
Date: 
Message-ID: <Fsy5a.25$rO3.630@paloalto-snr1.gtei.net>
In article <··············@ibook.local.>,
Alain Picard  <·······················@optushome.com.au> wrote:
>···@sevak.isi.edu (Thomas A. Russ) writes:
>
>> I find it really difficult to think about destructive list operations
>> without some notion of the parts of cons cells pointing at something
>> else.
>
>Then try to think of "the parts of cons cells" (or, more generally
>_places_) as something which can _hold_ another object, rather
>than point to it.

If you think of it that way, it's hard to contemplate the same object being
"held" in two different places.  It's also hard to picture a list made up
of a chain of conses -- each one has to be smaller than the previous one so
that it can fit inside its cdr.  And how would you draw the situation where
two lists share the same tail, e.g.

(setq common-tail (list 3 4 5))
(setq list1 (list* 1 2 common-tail))
(setq list2 (list* 'a 'b common-tail))

I also learned Lisp using arrows on a blackboard, and I've always thought
it's the best way to understand objects and the references among them.

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.21.23.57.15.838074@consulting.net.nz>
Hi Nils Goesche,

> "Adam Warner" <······@consulting.net.nz> writes:
> 
>> > There are no ``pointers�� in Lisp.
> ...
>> > P.S.: I used EQL instead of EQ because I think that newbies shouldn't
>> >       use EQ at all.  YMMV.
>> 
>> I'd assert that Common Lisp's EQ can only be understood after one comes
>> to understand pointers. It's one of those cases of being able to feel
>> the bits between your toes. Instead of trying to work though the
>> HyperSpec's obtuse definition of EQ one merely needs to have a mental
>> model of pointers.
>> 
>> Pointers pervade Common Lisp.
> 
> Then they should pervade the HyperSpec, too, which doesn't even mention
> pointers.

It's a mere implementation detail, just like garbage collection which the
HyperSpec doesn't mention either :-)

> I think the whole concept of pointers doesn't apply here.  To me, a
> pointer denotes a memory address, a number.  But the memory address of a
> Lisp object is something totally uninteresting.  If a copying garbage
> collector moves an object around that doesn't change the identity of
> that object at all.  And can you have pointers without having dangling
> pointers? ;-)  We don't need any notion of ``pointers�� to explain
> what's going on.

The address isn't interesting. Whether two object share the same address
may be. If they do then a quick machine-level match is possible even
though it may be disguised by the use of symbols.

> The semantics of EQ are absolutely clear, however: EQ simply tests if
> two objects are the same, identical object.  There is nothing unclear
> about that, object identity is the most simple concept there is.  For
> instance, it is clear that
> 
> (let* ((x 42)
>        (y x))
>   (eq x y))
> 
> will return true, because x and y are bound to the same, identical
> object.  We also have a guarantee that
> 
>  (eq 'foo 'foo)
> 
> will return true because symbols are interned.  Things can get a bit
> hairy, however, once you try things like
> 
>  (eq 42 42)
> 
> Will this be true?  We don't know.  That doesn't mean that EQ has
> unclear semantics -- what's unclear is whether those two 42 constants
> will be read as the same, identical object or not.  We have no guarantee
> about that.

We can only accept that (eq 42 42) may not be true because we have a model
of the machine representations of 42. It would be appropriate in a higher
level language to define numbers as always being equal when those numbers
have the same value. In this case Common Lisp is defining a very low level
form of equality to allow very fast implementations of a very particular
comparison.

I wholeheartedly agree that newbies shouldn't use EQ at all. In the past
year of learning Common Lisp I have come to greatly appreciate the years
of collective intelligence that went into defining the specification.
Everything so far has made sense when viewed from the appropriate
perspective. Common Lisp is an extraordinary and seamless mix of high
level and low level operations. EQ just happens to be one of the low level
operations. If speed wasn't a consideration we could throw it away in an
instant and accept that numbers and characters with the same value are
equal.

> But it is also clear that on a conceptual level, the integers 42 and 42
> are the ``same��.  So are the characters #\a and #\a.  So, there is
> another equality function EQL that will make the additional guarantee,
> that it will measure the same characters and the same numbers as equal.
> As far as EQL is concerned, every number and every character exists
> precisely once.  Now we have got what we want, and we call two objects
> the ``same�� if they compare equal under EQL.
> 
> One might ask now why the standard doesn't simply mandate, as it does
> for symbols, that the same numbers and the same characters are even EQ!
> It could easily do that -- this requirement wouldn't break one thing.

Just that implementations would be forced to perform slower comparisons.

> Then, there would be no difference between EQ and EQL at all, and both
> would measure object identity.  Note that this doesn't mean that EQ is
> defined somewhat arbitrarily -- it isn't!  EQ means object identity and
> nothing else.  The arbitrariness is in the decision whether there are
> several different objects that are the same as 42 or not.

It is defined by machine code speed concerns. That's why it isn't
arbitrary. Anything else is just a rationalisation of the decision to
produce fast machine code. There is no other reason to desire that 42 and
42 may not be equal.

> So, by adding one or two sentences to the standard, we would have a
> language where what is now EQL measures object identity and (EQ 42 42)
> will always be true.  This would even have some merit -- it is not a
> coincidence that EQL, /not/ EQ is the default equality predicate for all
> but very few Common Lisp functions.  CATCH, GETF, THROW, GET, REMF,
> GET-PROPERTIES and REMPROP are the only exceptions.  The language would
> be easier that way.
> 
> The reason this isn't done is that it would slow down the language.

I'm reading as I comment. I see we're on the same wavelength, but your
explanation below about bignums is superb.

> We can't ``intern�� bignums because that would eat too much memory, so
> EQ would have to make additional checks even in those cases where we
> know we are specifically searching for an object by its identity.  So we
> have EQ and EQL different.
> 
> Since we can't say that EQL measures object identity when it is in fact
> different from EQ which does, we have to invent another name for the
> relation it defines.  I like calling it ``conceptual identity��.
> 
> In my opinion newbies are better off if they start thinking in terms of
> ``conceptual identity�� and don't worry about the subtleties
> introduced by possibly not having 42 and 42 be the same object.
> 
> And pointers have nothing to do with it :-)

Thank you for this explanation. CLISP provides a good example of this:
http://clisp.sourceforge.net/impnotes.html

   EQ compares CHARACTERs and FIXNUMs as EQL does. No unnecessary copies
   are made of CHARACTERs and NUMBERs. Nevertheless, one should use EQL as
   it is more portable across Common Lisp implementations.

Regards,
Adam
From: Rob Warnock
Subject: Re: Newbie variable question
Date: 
Message-ID: <sjWdne8g_7i1o8qjXTWc-w@speakeasy.net>
Adam Warner <······@consulting.net.nz> wrote:
+---------------
| >  (eq 42 42)
| > 
| > Will this be true?  We don't know.
| 
| We can only accept that (eq 42 42) may not be true because we
| have a model of the machine representations of 42.
+---------------

Well, that (eq 42 42) might not be true is generally hard for newbies --
or even anyone who's looked a bit at implementation -- to understand,
especially since in every Common Lisp implementation they're going to run
into it *is* true (as a happy accident of any reasonable implementation)
that "equal" fixnums are "eq":

    > (eq 42 42)
    T
    > most-positive-fixnum	; some sample implementation
    536870911
    > (eq 536870911 536870911)
    T
    > (let ((x most-positive-fixnum))
        ;; even more confusing
        (eq (+ 41 (floor x 53) -4) (+ 37 (/ (- 536870911 44) 53))))
    T
    >

So when we talk about this issue, we really should be using examples
that make it more obvious that "eq" might *not* compare true, e.g.,
something like (expt 2 100):

    > (eql 1267650600228229401496703205376 1267650600228229401496703205376)
    T
    > (eq 1267650600228229401496703205376 1267650600228229401496703205376)
    NIL
    > 


-Rob

-----
Rob Warnock, PP-ASEL-IA		<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.22.03.46.42.734922@consulting.net.nz>
Hi Nils Goesche,

> One might ask now why the standard doesn't simply mandate, as it does
> for symbols, that the same numbers and the same characters are even
> EQ!  It could easily do that -- this requirement wouldn't break one
> thing.  Then, there would be no difference between EQ and EQL at all,
> and both would measure object identity.  Note that this doesn't mean
> that EQ is defined somewhat arbitrarily -- it isn't!  EQ means object
> identity and nothing else.  The arbitrariness is in the decision
> whether there are several different objects that are the same as 42 or
> not.
> 
> So, by adding one or two sentences to the standard, we would have a
> language where what is now EQL measures object identity and (EQ 42 42)
> will always be true.  This would even have some merit -- it is not a
> coincidence that EQL, /not/ EQ is the default equality predicate for
> all but very few Common Lisp functions.  CATCH, GETF, THROW, GET,
> REMF, GET-PROPERTIES and REMPROP are the only exceptions.  The
> language would be easier that way.
> 
> The reason this isn't done is that it would slow down the language.
> We can't ``intern�� bignums because that would eat too much memory, so
> EQ would have to make additional checks even in those cases where we
> know we are specifically searching for an object by its identity.  So
> we have EQ and EQL different.
> 
> Since we can't say that EQL measures object identity when it is
> in fact different from EQ which does, we have to invent another name
> for the relation it defines.  I like calling it ``conceptual
> identity��.
> 
> In my opinion newbies are better off if they start thinking in terms
> of ``conceptual identity�� and don't worry about the subtleties
> introduced by possibly not having 42 and 42 be the same object.

I've just analysed my months of coding using cat, grep and wc. There is
around 7,000 lines of code including liberal comments and whitespace. Here
are the statistics on my usages of various tests for equality:

Predicate   Approx. usage (times)
   eq                108
   eql                 0
   equal              11
   =                  27
   char=              78
   string=            77

It's obvious why I mucked up my explanation of eql: I haven't used it! I use
eq to test for symbol identity and char= or = depending upon whether I am
testing for character or number equality. As I have come to use eq so
widely perhaps it does have a significant and direct role, even for
newbies? eql has to be taught because it is used so widely indirectly (for
example it explains why I was surprised when case couldn't test for string
matches. I'm self-taught in Common Lisp, so don't go blaming an institution!) 

It is looking like eql is the more arbitrary predicate. eq is described by
machine-like object identity. equal is described by the same content. eql
is the arbitrary mix of the two, ensuring that the same symbols, numbers
and characters are true but being allowed to return nil for exactly the
same string contents, etc.

The problem with any term like ``conceptual identity'' for eql is that
"adam" and "adam" are conceptually equal.

[1]> (defvar x "adam")
x
[2]> (eql x x)
t
[3]> (eql "adam" "adam")
nil

I've got to agree with Jochen's earlier point that eql is just some
special cases tacked onto eq. Does eql ever fail to test whether the
contents of two base objects are the same? That is, can eql be
rationalised as a failsafe test of whether building block objects are the
same? (strings are for example excluded because they are composed of
characters)

Regards,
Adam
From: Coby Beck
Subject: Re: Newbie variable question
Date: 
Message-ID: <yLE5a.53067$jM5.132991@newsfeeds.bigpond.com>
"Adam Warner" <······@consulting.net.nz> wrote in message
···································@consulting.net.nz...
>
> The problem with any term like ``conceptual identity'' for eql is that
> "adam" and "adam" are conceptually equal.

Please note you changed a very critical word in mid argument above.
Conceptually identical is not conceptually equal!  My glass full of apple
juice might be conceptually equal to yours (maybe, depending on our
preferences, even equal to your glass of orange juice) but few sane people
would try to tell you it is the identical (ie EQ) glass in two places at
once.

>
> [1]> (defvar x "adam")
> x
> [2]> (eql x x)
> t
> [3]> (eql "adam" "adam")
> nil

CL-USER 1 > (equal "adam" "adam")
T

CL-USER 2 > (equal '(a d a m) '(a d a m))
T

.....may be more what you are looking for.  There is even

CL-USER 4 > (equalp "ADAM" "adam")
T

Equality is not identity and there have been quite a few interesting
discussions about that and what is equality around here (check out a thread
awhile ago about "deep copy").

> I've got to agree with Jochen's earlier point that eql is just some
> special cases tacked onto eq. Does eql ever fail to test whether the
> contents of two base objects are the same? That is, can eql be
> rationalised as a failsafe test of whether building block objects are the
> same?

"The value of eql is true of two objects, x and y, in the folowing cases:

  1. If x and y are eq.
  2. If x and y are both numbers of the same type and the same value.
  3. If they are both characters that represent the same character.

Otherwise the value of eql is false."

(from the CLHS entry on EQL)

I personally tend to use equal and equalp unless I need to think about it
and have a good reason to be more discriminating.  If it is any kind of
application specific concept, give it a name and write your own.

--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.22.23.48.28.241961@consulting.net.nz>
Hi Coby Beck,

>> The problem with any term like ``conceptual identity'' for eql is that
>> "adam" and "adam" are conceptually equal.
> 
> Please note you changed a very critical word in mid argument above.
> Conceptually identical is not conceptually equal!  My glass full of
> apple juice might be conceptually equal to yours (maybe, depending on
> our preferences, even equal to your glass of orange juice) but few sane
> people would try to tell you it is the identical (ie EQ) glass in two
> places at once.

Ergo, few sane people would say that two characters are identical when
they exist in two different places at once. So if someone is going to say
that #\a and #\a have conceptual identity---even though they exist in two
places at once---then they should be prepared to say that "adam" and "adam"
have conceptual identity.

> There is even
> 
> CL-USER 4 > (equalp "ADAM" "adam")
> T

That's neat. Thanks for the tip.

> Equality is not identity and there have been quite a few interesting
> discussions about that and what is equality around here (check out a
> thread awhile ago about "deep copy").
> 
>> I've got to agree with Jochen's earlier point that eql is just some
>> special cases tacked onto eq. Does eql ever fail to test whether the
>> contents of two base objects are the same? That is, can eql be
>> rationalised as a failsafe test of whether building block objects are
>> the same?
> 
> "The value of eql is true of two objects, x and y, in the folowing
> cases:
> 
>   1. If x and y are eq.
>   2. If x and y are both numbers of the same type and the same value.
>   3. If they are both characters that represent the same character.
> 
> Otherwise the value of eql is false."
> 
> (from the CLHS entry on EQL)

Quoting the definition doesn't help to explain this arbitrariness of being
inbetween identity and equality. What history lead to the idea that much
of Common Lisp should be based on EQL predicate tests to the sacrifice of
a more expansive predicate like EQUAL? I suspect it's because Lisp
developed from a perspective of symbol identity and the tests for equality
are more recent developments. 

> I personally tend to use equal and equalp unless I need to think about
> it and have a good reason to be more discriminating.  If it is any kind
> of application specific concept, give it a name and write your own.

Thanks for the tips.

Regards,
Adam
From: Don Geddis
Subject: Re: Newbie variable question
Date: 
Message-ID: <m33cmej0hu.fsf@maul.geddis.org>
"Adam Warner" <······@consulting.net.nz> writes:
> Quoting the definition doesn't help to explain this arbitrariness of being
> inbetween identity and equality. What history lead to the idea that much
> of Common Lisp should be based on EQL predicate tests to the sacrifice of
> a more expansive predicate like EQUAL?

I suspect that implementation efficiency had a lot to do with the default
choice.

> I suspect it's because Lisp developed from a perspective of symbol identity
> and the tests for equality are more recent developments.

You shouldn't leave out the fact that there's no perfect "equal" predicate
even possible.  Hence any choice is going to be somewhat arbitrary.

You might find it useful to review Kent Pitman's excellent article on the
subject:
        http://world.std.com/~pitman/PS/EQUAL.html

_______________________________________________________________________________
Don Geddis                    http://don.geddis.org              ···@geddis.org
Karate is a form of martial arts in which people who have had years and years
of training can, using only their hands and feet, make some of the worst movies
in the history of the world.  -- Dave Barry
From: Nils Goesche
Subject: Re: Newbie variable question
Date: 
Message-ID: <871y1z70v6.fsf@darkstar.cartan>
"Adam Warner" <······@consulting.net.nz> writes:

> Hi Coby Beck,
> 
> >> The problem with any term like ``conceptual identity'' for
> >> eql is that "adam" and "adam" are conceptually equal.
> > 
> > Please note you changed a very critical word in mid argument
> > above.  Conceptually identical is not conceptually equal!  My
> > glass full of apple juice might be conceptually equal to
> > yours (maybe, depending on our preferences, even equal to
> > your glass of orange juice) but few sane people would try to
> > tell you it is the identical (ie EQ) glass in two places at
> > once.
> 
> Ergo, few sane people would say that two characters are
> identical when they exist in two different places at once. So
> if someone is going to say that #\a and #\a have conceptual
> identity---even though they exist in two places at once---then
> they should be prepared to say that "adam" and "adam" have
> conceptual identity.

No.  The important difference is that strings are mutable whereas
numbers and characters are not.  BTW, in most Lisp
implementations, every #\a will actually /be/ EQ to every other
#\a.  So, there is only one #\a (in those implementations).

Regards,
-- 
Nils G�sche
Ask not for whom the <CONTROL-G> tolls.

PGP key ID #xD26EF2A0
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.23.11.54.18.808378@consulting.net.nz>
Hi Nils Goesche,

>> Ergo, few sane people would say that two characters are identical when
>> they exist in two different places at once. So if someone is going to
>> say that #\a and #\a have conceptual identity---even though they exist
>> in two places at once---then they should be prepared to say that "adam"
>> and "adam" have conceptual identity.
> 
> No.  The important difference is that strings are mutable whereas
> numbers and characters are not.  BTW, in most Lisp implementations,
> every #\a will actually /be/ EQ to every other #\a.  So, there is only
> one #\a (in those implementations).

So does EQL work correctly with all immutable objects? If it does then
that would be a significant and non-arbitrary characteristic.

Can you please expand upon what you mean by immutable. I'm thinking of
objects that cannot be changed without undefined consequences, for example
a quoted list. The fact that even (eql '(1) '(1)) can return nil seems to
put your important difference upon shaky ground. Why is it OK for two
numbers to have conceptual identity but two sets containing those same
numbers to not?

* (eql '() '())                  
t
* (eql '(1) '(1))
nil
* (eql (first '(1)) (first '(1)))
t

We start with two conceptually identical empty sets. We place two
conceptually identical numbers into each set. The results are not
conceptually identical?

Another example:

* (eql '(()) '(()))
nil

Immutable empty lists are conceptually identical yet immutable lists
containing immutable empty lists are not?

Sometimes what appear to be helpful descriptions break down until all one
is left with is learning the arbitrary rules.

Regards,
Adam
From: Nils Goesche
Subject: Re: Newbie variable question
Date: 
Message-ID: <87wujr5en0.fsf@darkstar.cartan>
"Adam Warner" <······@consulting.net.nz> writes:

> >> Ergo, few sane people would say that two characters are
> >> identical when they exist in two different places at
> >> once. So if someone is going to say that #\a and #\a have
> >> conceptual identity---even though they exist in two places
> >> at once---then they should be prepared to say that "adam"
> >> and "adam" have conceptual identity.
> > 
> > No.  The important difference is that strings are mutable
> > whereas numbers and characters are not.  BTW, in most Lisp
> > implementations, every #\a will actually /be/ EQ to every
> > other #\a.  So, there is only one #\a (in those
> > implementations).
> 
> So does EQL work correctly with all immutable objects? If it
> does then that would be a significant and non-arbitrary
> characteristic.

I don't know.  I don't check because I don't care: It is not
important.  What's important is that EQL has a different quality
than EQUAL or EQUALP.  The statement (EQL X Y) => T has a
different quality than (EQUAL X Y) => T.  I am trying to give
this quality a name.  Quite tellingly, the HyperSpec calls
objects that are EQL ``the same object��.  EQL tells you if X and
Y are bound to (conceptually) the same object.  That's simply
it's purpose.  Now, two strings that are EQUAL will generally not
be EQL.  Does this surprise you?  Would you think of such strings
as the same object?  I don't.  Also, consider:

CL-USER 1 > (let* ((x (copy-seq "foobar"))
                   (y x))
              (setf (char x 5) #\z)
              (values x (eql x y)))
"foobaz"
T

So, X is still bound to the same object even after I modified
it.  Does this mean that "foobar" is the same as "foobaz"?  In
contrast:

CL-USER 2 > (let* ((x 42)
                   (y x))
              (incf x)
              (values x (eql x y)))
43
NIL

X is bound to a different object after I incremented it.

> Can you please expand upon what you mean by immutable. I'm
> thinking of objects that cannot be changed without undefined
> consequences, for example a quoted list. The fact that even
> (eql '(1) '(1)) can return nil seems to put your important
> difference upon shaky ground.

No.  A cons cell is a cons cell, there are not two types of cons
cells in Lisp.  I may not be allowed to modify list literals, but
nothing can stop me if I am determined to do so.  But I cannot
modify 42, no matter how hard I try.

> Why is it OK for two numbers to have conceptual identity but
> two sets containing those same numbers to not?
> 
> * (eql '() '())                  
> t
> * (eql '(1) '(1))
> nil
> * (eql (first '(1)) (first '(1)))
> t

> We start with two conceptually identical empty sets. We place
> two conceptually identical numbers into each set. The results
> are not conceptually identical?

Consider, again:

CL-USER 3 > (let* ((x (list 1))
                   (y x))
              (incf (car x))
              (values x (eql x y)))
(2)
T

It returned T, not NIL!  That's because identity is not about the
/content/ of a cons cell.  Unlike EQUAL, EQL is about objects.

> Another example:
> 
> * (eql '(()) '(()))
> nil
> 
> Immutable empty lists are conceptually identical yet immutable
> lists containing immutable empty lists are not?
> 
> Sometimes what appear to be helpful descriptions break down
> until all one is left with is learning the arbitrary rules.

I do not understand what your problem is.  All rules are somewhat
arbitrary.  That's why they are made.  If a rule weren't
arbitrary but instead followed logically from other rules it
would be unnecessary.

EQL has a purpose: We have to understand its purpose first, which
is to measure whether two objects are conceptually the same.
That's it.  /Then/, after that, you find out that, in Lisp, the
character #\a exists precisely once (on a conceptual level),
whereas there can be several different objects STRING= to
"foobar".  These are arbitrary decisions -- whenever you design a
language you will have to decide whether you want it this way or
another.  In Lisp, the implementor is free to decide whether it
will implement character objects in a way that #\a can exist more
than once on implementation level.  You could tell with EQ.
However, they will always be EQL.  And that is all that counts
because you do not use EQ anyway when you think of a character as
a character -- you only use EQ when you are searching for an
arbitrary object by its (physical) identity.  I use both EQ and
EQL all the time and never accidentally use the wrong one --
because these predicates have a meaning for me.  If they don't
for you, you should work on that until they do :-)  EQUAL and
EQUALP are different there -- I often have to look up again what
they exactly do.  /They/ are somewhat arbitrary and without a
clear ``meaning��.  But EQ and EQL are not.

Earlier you claimed that you did not use EQL, but I don't believe
you unless you tell me you always call functions like FIND or
UNION with a :TEST different from EQL.  And that you never use
CASE...  EQL is the default all over the place.  Also, I would
find code like

  (and char (char= char #\a))

rather strange in comparison with

  (eql char #\a)

let alone

  (when (or (char= char #\a) (char= char #\b))
    ...)

when you could simply do

  (case char
    ((#\a #\b) ...))

Regards,
-- 
Nils G�sche
Ask not for whom the <CONTROL-G> tolls.

PGP key ID #xD26EF2A0
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.23.21.56.22.247536@consulting.net.nz>
Hi Nils Goesche,

> "Adam Warner" <······@consulting.net.nz> writes:
> 
>> >> Ergo, few sane people would say that two characters are
>> >> identical when they exist in two different places at
>> >> once. So if someone is going to say that #\a and #\a have
>> >> conceptual identity---even though they exist in two places
>> >> at once---then they should be prepared to say that "adam"
>> >> and "adam" have conceptual identity.
>> > 
>> > No.  The important difference is that strings are mutable
>> > whereas numbers and characters are not.  BTW, in most Lisp
>> > implementations, every #\a will actually /be/ EQ to every
>> > other #\a.  So, there is only one #\a (in those
>> > implementations).
>> 
>> So does EQL work correctly with all immutable objects? If it
>> does then that would be a significant and non-arbitrary
>> characteristic.
> 
> I don't know.  I don't check because I don't care: It is not
> important.  What's important is that EQL has a different quality
> than EQUAL or EQUALP.  The statement (EQL X Y) => T has a
> different quality than (EQUAL X Y) => T.  I am trying to give
> this quality a name.  Quite tellingly, the HyperSpec calls
> objects that are EQL ``the same object��.  EQL tells you if X and
> Y are bound to (conceptually) the same object.  That's simply
> it's purpose.  Now, two strings that are EQUAL will generally not
> be EQL.  Does this surprise you?  Would you think of such strings
> as the same object?  I don't.  Also, consider:
> 
> CL-USER 1 > (let* ((x (copy-seq "foobar"))
>                    (y x))
>               (setf (char x 5) #\z)
>               (values x (eql x y)))
> "foobaz"
> T
> 
> So, X is still bound to the same object even after I modified
> it.  Does this mean that "foobar" is the same as "foobaz"?  In
> contrast:
> 
> CL-USER 2 > (let* ((x 42)
>                    (y x))
>               (incf x)
>               (values x (eql x y)))
> 43
> NIL
> 
> X is bound to a different object after I incremented it.
> 
>> Can you please expand upon what you mean by immutable. I'm
>> thinking of objects that cannot be changed without undefined
>> consequences, for example a quoted list. The fact that even
>> (eql '(1) '(1)) can return nil seems to put your important
>> difference upon shaky ground.
> 
> No.  A cons cell is a cons cell, there are not two types of cons
> cells in Lisp.  I may not be allowed to modify list literals, but
> nothing can stop me if I am determined to do so.  But I cannot
> modify 42, no matter how hard I try.
> 
>> Why is it OK for two numbers to have conceptual identity but
>> two sets containing those same numbers to not?
>> 
>> * (eql '() '())                  
>> t
>> * (eql '(1) '(1))
>> nil
>> * (eql (first '(1)) (first '(1)))
>> t
> 
>> We start with two conceptually identical empty sets. We place
>> two conceptually identical numbers into each set. The results
>> are not conceptually identical?
> 
> Consider, again:
> 
> CL-USER 3 > (let* ((x (list 1))
>                    (y x))
>               (incf (car x))
>               (values x (eql x y)))
> (2)
> T
> 
> It returned T, not NIL!  That's because identity is not about the
> /content/ of a cons cell.  Unlike EQUAL, EQL is about objects.

Since you did not assign a copy of the list to y, y is also modified
when x is modified. You demonstrated nothing special about EQL over EQ.

* (defvar x (list 1))

x
* (defvar y x)

y
* (incf (car x))

2
* x

(2)
* y

(2)
* (eql x y)

t
* (eq x y) 

t



* (setf x (list "oh look a mutable string"))          

("oh look a mutable string")
* (setf y x)                                  

("oh look a mutable string")
* (setf (aref (car x) 0) #\O)                 

#\O
* (eq x y)

t
* x

("Oh look a mutable string")
* y

("Oh look a mutable string")


>> Another example:
>> 
>> * (eql '(()) '(()))
>> nil
>> 
>> Immutable empty lists are conceptually identical yet immutable
>> lists containing immutable empty lists are not?
>> 
>> Sometimes what appear to be helpful descriptions break down
>> until all one is left with is learning the arbitrary rules.
> 
> I do not understand what your problem is.  All rules are somewhat
> arbitrary.  That's why they are made.  If a rule weren't
> arbitrary but instead followed logically from other rules it
> would be unnecessary.
> 
> EQL has a purpose: We have to understand its purpose first, which
> is to measure whether two objects are conceptually the same.
> That's it.  /Then/, after that, you find out that, in Lisp, the
> character #\a exists precisely once (on a conceptual level),
> whereas there can be several different objects STRING= to
> "foobar".  These are arbitrary decisions -- whenever you design a
> language you will have to decide whether you want it this way or
> another.  In Lisp, the implementor is free to decide whether it
> will implement character objects in a way that #\a can exist more
> than once on implementation level.  You could tell with EQ.
> However, they will always be EQL.  And that is all that counts
> because you do not use EQ anyway when you think of a character as
> a character -- you only use EQ when you are searching for an
> arbitrary object by its (physical) identity.  I use both EQ and
> EQL all the time and never accidentally use the wrong one --
> because these predicates have a meaning for me.  If they don't
> for you, you should work on that until they do :-)  EQUAL and
> EQUALP are different there -- I often have to look up again what
> they exactly do.  /They/ are somewhat arbitrary and without a
> clear ``meaning��.  But EQ and EQL are not.
> 
> Earlier you claimed that you did not use EQL, but I don't believe
> you unless you tell me you always call functions like FIND or
> UNION with a :TEST different from EQL.  And that you never use
> CASE...  EQL is the default all over the place.

Please don't put words in my mouth with this straw man. I grepped my
source code to find explicit use of EQL and I stated "it is used so widely
indirectly". You have constructed a fiction of what I claimed merely to
knock it down.

I am well aware of EQL and CASE, and I brought it up explicitly in the same
message: "eql has to be taught because it is used so widely indirectly (for
example it explains why I was surprised when case couldn't test for string
matches."

> Also, I would find code like
> 
>   (and char (char= char #\a))
> 
> rather strange in comparison with
> 
>   (eql char #\a)

I would indeed have written the first code snippet. Thanks for the info.

Regards,
Adam
From: Nils Goesche
Subject: Re: Newbie variable question
Date: 
Message-ID: <ly7kbpbzaa.fsf@cartan.de>
"Adam Warner" <······@consulting.net.nz> writes:

> > "Adam Warner" <······@consulting.net.nz> writes:
> > 
> >> Why is it OK for two numbers to have conceptual identity but
> >> two sets containing those same numbers to not?
> >> 
> >> * (eql '() '())                  
> >> t
> >> * (eql '(1) '(1))
> >> nil
> >> * (eql (first '(1)) (first '(1)))
> >> t
> > 
> >> We start with two conceptually identical empty sets. We place
> >> two conceptually identical numbers into each set. The results
> >> are not conceptually identical?
> > 
> > Consider, again:
> > 
> > CL-USER 3 > (let* ((x (list 1))
> >                    (y x))
> >               (incf (car x))
> >               (values x (eql x y)))
> > (2)
> > T
> > 
> > It returned T, not NIL!  That's because identity is not about the
> > /content/ of a cons cell.  Unlike EQUAL, EQL is about objects.
> 
> You demonstrated nothing special about EQL over EQ.

The point wasn't to demonstrate something special about EQL over EQ,
it was only to show what both EQ and EQL are all about -- identity of
objects; one about physical identity, one about conceptual identity,
as I like calling it.  Identity is not equality.  I keep talking about
this because you seem to have a problem with this concept:

> Since you did not assign a copy of the list to y, y is also modified
> when x is modified.

This is /wrong/!  Y is not modified /at all/ in that example!  And
that's precisely what EQL was telling us (yes, I could have used EQ as
well).

Let's look again closely at the above examples: Your example was

* (eql '() '())                  
t
* (eql '(1) '(1))
nil
* (eql (first '(1)) (first '(1)))
t

and you commented:

> >> We start with two conceptually identical empty sets.

There is only one empty set (or list), as the first test shows.

> >> We place two conceptually identical numbers into each set.

We can't do that.  Once you place an object into the empty list it is
not the same list anymore.  This is not necessarily so, of course.
But such is the case in Lisp and many other languages.  For this, we
would have to do something like

(let* ((x '())
       (y x))
  (do-something-impossible-with-x-and-not-touching-y)
  (values x (eql x y)))

which would have to return (1) and T.  But this cannot be done in
Lisp.

> >> The results are not conceptually identical?

Obviously not, as EQL has told us.  To be otherwise, there would have
to be only /one/ list EQUAL to (1).  Then we would get

(let* ((x (list 2))
       (y x))
  (decf (car x))
  (values x (eql x y)))

=> (1) NIL ; NOT!

because if there was only one list containing 1 and nothing else, X
must have been bound to a /different/ object at first, and (DECF (CAR
X)) would have to change the binding of X.  Since it cannot change the
binding of Y, however, X and Y must be bound to a different object in
this language after that.

(Note that what EQL returns here is determined by our decisions about
 what kind of /objects/ exist in our language.  We do not contemplate
 what EQL should return but what kind of objects we want to have, and
 what EQL returns follows from that.)

Lisp works differently, so, in fact we get

=> (1) T

similar to my example above, and to your example below:

> * (defvar x (list 1))
> 
> x
> * (defvar y x)
> 
> y
> * (incf (car x))

Note that the above line does not change Y at all (Y does not even
occur in that expression.  Y is still bound to the same object as
before).

> 2
> * x
> 
> (2)
> * y
> 
> (2)
> * (eql x y)
> 
> t
> * (eq x y) 
> 
> t

And this shows us that X is still bound to this same object, too.
This object is a (mutable) cons cell, and the identity of a cons cell
does not depend on its content at all.  There are lots of different
cons cells in Lisp.

> * (setf x (list "oh look a mutable string"))          
> 
> ("oh look a mutable string")
> * (setf y x)                                  
> 
> ("oh look a mutable string")
> * (setf (aref (car x) 0) #\O)

Note that you shouldn't modify string literals, either ;-)

> #\O
> * (eq x y)
> 
> t
> * x
> 
> ("Oh look a mutable string")
> * y
> 
> ("Oh look a mutable string")

As you seem to worry about the difference between EQ and EQL, here is
yet another example:

CL-USER 70 > (let* ((x 27345273452364256725)
                    (y x))
               (incf x)
               (values (eql x 27345273452364256726) (eql x y)))
T
NIL

Every integer exists precisely once on the conceptual level, so after
the INCF line, X must be bound to a different object.  Y has not
changed, however, so the second return value is NIL and the first is
T.  So much for the concepts, now for the implementation.  In
LispWorks:

CL-USER 71 > (let* ((x 27345273452364256725)
                    (y x))
               (incf x)
               (values (eq x 27345273452364256726) (eql x y)))
NIL
NIL

which shows that LispWorks, like probably all implementations, decided
to implement bignums in a way that there are, physically, lots of
different bignums equal to 27345273452364256726.  The objects compared
by EQ here are not physically identical.  But they are conceptually
identical, which is why the former example definitely returns T as
its first return value.

Again, all of the above considerations are about /objects/ and their
/identity/, not equality as such.

> > Earlier you claimed that you did not use EQL, but I don't believe
> > you unless you tell me you always call functions like FIND or
> > UNION with a :TEST different from EQL.  And that you never use
> > CASE...  EQL is the default all over the place.
> 
> Please don't put words in my mouth with this straw man. I grepped my
> source code to find explicit use of EQL and I stated "it is used so
> widely indirectly". You have constructed a fiction of what I claimed
> merely to knock it down.

Well, sorry about that.  Actually, I wasn't trying to ``knock��
something ``down��, I only wanted to emphasize how /important/ EQL is
by pointing out again that it is the default test all over the place.
When we do

  (find x foo)

we are testing whether the object in X is contained in the sequence
FOO.  And we are looking for it by its identity as an object, only
that we do /not/ take its physical identity but something else, which
I like to call ``conceptual identity�� because it is ``conceptually��
so close to doing (find x foo :test #'eq), but very, very far from
something like (find x foo :test #'equal).  I am aware that some
people might disagree with my calling it ``identity��, too.  However,
I could say, with HyperSpec terminology, that (find x foo) will return
true iff one of the objects in FOO is the ``same object�� as the
object X is bound to.  That's why I think my terminology is justified.

> I am well aware of EQL and CASE, and I brought it up explicitly in
> the same message: "eql has to be taught because it is used so widely
> indirectly (for example it explains why I was surprised when case
> couldn't test for string matches."

The reason you were surprised is that you didn't expect CASE to
compare for identity.  There is a reason Kent Pitman says ``identity,
identity, identity!�� all the time :-)

> > Also, I would find code like
> > 
> >   (and char (char= char #\a))
> > 
> > rather strange in comparison with
> > 
> >   (eql char #\a)
> 
> I would indeed have written the first code snippet. Thanks for the info.

The two snippets express a different way of thinking.  I hope that has
become a bit clearer now.

Regards,
-- 
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x0655CFA0
From: Tim Bradshaw
Subject: Re: Newbie variable question
Date: 
Message-ID: <ey3adgmjrdo.fsf@cley.com>
* Adam Warner wrote:

> So does EQL work correctly with all immutable objects? If it does then
> that would be a significant and non-arbitrary characteristic.

It works with a really important subset of immutable objects, in
particular a really important subset of objects which are immutable by
virtue of their type rather than by virtue of their happening to be a
literal.

--tim
From: Coby Beck
Subject: Re: Newbie variable question
Date: 
Message-ID: <1vX5a.53616$jM5.135336@newsfeeds.bigpond.com>
"Adam Warner" <······@consulting.net.nz> wrote in message
···································@consulting.net.nz...
> Hi Coby Beck,
>
> >> The problem with any term like ``conceptual identity'' for eql is that
> >> "adam" and "adam" are conceptually equal.
> >
> > Please note you changed a very critical word in mid argument above.
> > Conceptually identical is not conceptually equal!  My glass full of
> > apple juice might be conceptually equal to yours (maybe, depending on
> > our preferences, even equal to your glass of orange juice) but few sane
> > people would try to tell you it is the identical (ie EQ) glass in two
> > places at once.
>
> Ergo, few sane people would say that two characters are identical when
> they exist in two different places at once. So if someone is going to say
> that #\a and #\a have conceptual identity---even though they exist in two
> places at once---then they should be prepared to say that "adam" and
"adam"
> have conceptual identity.

This is not so clear.  It really does depend on your usage, your
application.  If you are just storing arbitrary information in a sequence of
characters, then yes it is the same information.  But if these are peoples
names then "Adam" of the Warner's *is not* "Adam" from the garden of Eden.
(BTW #\a and #\a are not guaranteed to be EQ, to have conceptual identity.)

Characters and numbers are definitely "in between" EQ-ness and application
dependant equality, I think EQL is a Good Thing.  No one has yet, so I will
post a link to an article by Kent Pitman that discusses this issue
(confession: I have skimmed it but not carefully read it but have no doubt
it is well presented and enlightening)
http://world.std.com/~pitman/PS/EQUAL.html

> > "The value of eql is true of two objects, x and y, in the folowing
> > cases:
> >
> >   1. If x and y are eq.
> >   2. If x and y are both numbers of the same type and the same value.
> >   3. If they are both characters that represent the same character.
> >
> > Otherwise the value of eql is false."
> >
> > (from the CLHS entry on EQL)
>
> Quoting the definition doesn't help to explain this arbitrariness of being
> inbetween identity and equality.

(see above)  I think it is merely a reflection of reality, not arbitration.

> > I personally tend to use equal and equalp unless I need to think about
> > it and have a good reason to be more discriminating.  If it is any kind
> > of application specific concept, give it a name and write your own.
>
> Thanks for the tips.

No problem...

--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
From: Pascal Costanza
Subject: Re: Newbie variable question
Date: 
Message-ID: <costanza-3BABBD.13240423022003@news.netcologne.de>
In article <······················@newsfeeds.bigpond.com>,
 "Coby Beck" <·····@mercury.bc.ca> wrote:

> Characters and numbers are definitely "in between" EQ-ness and application
> dependant equality, I think EQL is a Good Thing.  No one has yet, so I will
> post a link to an article by Kent Pitman that discusses this issue
> (confession: I have skimmed it but not carefully read it but have no doubt
> it is well presented and enlightening)
> http://world.std.com/~pitman/PS/EQUAL.html

Another excellent article on the topic of language supplied comparison 
operators is "Copying and Comparing: Problems and Solutions" by Peter 
Grogono and Markku Sakkinen. See http://citeseer.nj.nec.com/grogono00copying.htm


Pascal

-- 
"If I could explain it, I wouldn't be able to do it."
A.M.McKenzie
From: Tim Bradshaw
Subject: Re: Newbie variable question
Date: 
Message-ID: <ey3el5yjrix.fsf@cley.com>
* Adam Warner wrote:
> I've got to agree with Jochen's earlier point that eql is just some
> special cases tacked onto eq. Does eql ever fail to test whether the
> contents of two base objects are the same? That is, can eql be
> rationalised as a failsafe test of whether building block objects are the
> same? (strings are for example excluded because they are composed of
> characters)

But *all* other equality predicates are `special cases tacked onto
EQ'.  CL just happens to provide some useful ones.  EQL is, I think,
the most useful additional one, because it treats some instances of
important immutable types as the same even when in fact they might not
be the same object (the immutability is important because you know
that if (eql x y) now, then always (eql x y)).

--tim
From: Kalle Olavi Niemitalo
Subject: Re: Newbie variable question
Date: 
Message-ID: <87ptphusn6.fsf@Astalo.kon.iki.fi>
Nils Goesche <······@cartan.de> writes:

> For instance, it is clear that
>
> (let* ((x 42)
>        (y x))
>   (eq x y))
>
> will return true, because x and y are bound to the same, identical 
> object.

Common Lisp does not guarantee that.  Not even (eq x x) will 
necessarily be true; see the last example on the EQ page of CLHS.

I wonder... there are some forms, such as (eq (the number x) x), 
where the standard does not specify what value will be returned. 
A compiler is thus free to optimize the comparison out and choose 
any value it likes.  Do compilers always choose the same value, 
or do they look in the context?  I mean, in

  (when (eq (the number x) x)
    (lengthy-computation))

it could be beneficial to make EQ return false, because then we 
can skip the LENGTHY-COMPUTATION call.  But if you substitute 
UNLESS there, then it would be better for EQ to return true.

I suppose such cases don't appear in real programs though, so it 
isn't useful to make compilers optimize them.  And who knows, 
maybe calling LENGTHY-COMPUTATION there would have avoided 
something even worse later on.
From: Nils Goesche
Subject: Re: Newbie variable question
Date: 
Message-ID: <lyvfz8a7yt.fsf@cartan.de>
Kalle Olavi Niemitalo <···@iki.fi> writes:

> Nils Goesche <······@cartan.de> writes:
> 
> > For instance, it is clear that
> >
> > (let* ((x 42)
> >        (y x))
> >   (eq x y))
> >
> > will return true, because x and y are bound to the same, identical
> > object.
> 
> Common Lisp does not guarantee that.  Not even (eq x x) will
> necessarily be true; see the last example on the EQ page of CLHS.

Heh -- remarkable!  Thanks for pointing this out.  And it even makes
sense, somehow: If, say, an EXTENDED-CHAR is not an immediate object,
it might be a good idea to deep-copy it when storing it to, say, a
specialized array.  Or an implementation could stack-allocate space of
some fixed size large enough to hold any character object and
deep-copy on assignment to that local variable, or when calling a
function.  All this would break EQ-ness.  And it makes EQL even more
important.

Regards,
-- 
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x0655CFA0
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.25.01.54.09.544418@consulting.net.nz>
Hi Kalle Olavi Niemitalo,

> Nils Goesche <······@cartan.de> writes:
> 
>> For instance, it is clear that
>>
>> (let* ((x 42)
>>        (y x))
>>   (eq x y))
>>
>> will return true, because x and y are bound to the same, identical
>> object.
> 
> Common Lisp does not guarantee that.  Not even (eq x x) will necessarily
> be true; see the last example on the EQ page of CLHS.

This indicates that an implementation can copy the value of x. If a copy
of x is made then the pointers to the values will be different/the values
will exist at different address locations/the values won't be physically
identical.

I think I can now make sense of this: assignment creates a pointer to a
value (not a pointer to a symbol). If assignments leads to the pointer
values being equal (i.e. the value is held at the same address location)
then EQ will return t.

A slight modification to help me understand the process:
(let* ((x "abc") (y x)) (eq x y))

The above returns t (by good fortune?) So does this:
(let* ((x "abc") (y x)) (eq x (string y)))

This is a shocker to me because I thought string was what one could use to
create a mutable (copy of a) string. But it doesn't: "If x is a string, it
is returned."

I have now discovered that I must use COPY-SEQ:
(let* ((x "abc") (y x)) (eq x (copy-seq y)))

Which now returns nil.

Is this revelation correct: assignment _never copies_ a value. It merely
copies the generalised reference (storage location) of the object. If a
new object is created then the object will have a different generalised
reference/storage location when it is assigned.

For your criticism of Nils to be correct a Common Lisp implementation must
be able to make a copy of y with a different generalised reference before
the EQ test is conducted. Is this your assertion?

Regards,
Adam
From: Jochen Schmidt
Subject: Re: Newbie variable question
Date: 
Message-ID: <b3fhau$f4q$03$1@news.t-online.com>
Adam Warner wrote:

> Is this revelation correct: assignment _never copies_ a value. It merely
> copies the generalised reference (storage location) of the object. If a
> new object is created then the object will have a different generalised
> reference/storage location when it is assigned.

A generalized reference is something different in the context of ANSI-CL.

To put some other terms into the discussion:

Say we have a (lexical) variable x. This variable will then (always) have a 
binding to some object. You can imagine a binding as a storage-cell which 
can hold exactly one lisp-object.

x: [nil]

Think of the above as the variable x with the binding denoted as the 
brackets. As you can see there is the object NIL bound. The way I wrote it 
would imply that NIL is encoded as an immediate object so we can simply 
store the object reference of NIL into this cell and we have all we need.

Now we want to assign the fixnum 3 to the variable x.

x: [3]

We do this by overwriting the object reference to nil with the object 
reference to 3 in the binding cell.

Now we assign the string "Hello" to the variable x

x: [ ]------------->"Hello"

The string "Hello" is stored somewhere else in memory. As an lisp-object it 
is accessed by its object reference which would be the encoded address of 
the string and some tags for type information. This object reference is 
then stored into the binding cell.

Now we introduce another variable y which binds the same object which is 
bound to x at this time.

x: [ ]-------------->"Hello"
                      /
y: [ ]---------------/

Now we mutate the string by replacing the #\e with an #\a.

x: [ ]--------------->"Hallo"
                      /
y: [ ]---------------/


Now we assign the bignum #xFFFFFFFFFFFE to x

x: [ ]--------------->#xFFFFFFFFFFFD

y: [ ]--------------->"Hallo"
 

We increment the bignum in x by one

x: [ ]--------------->#xFFFFFFFFFFFE

                      #xFFFFFFFFFFFD  <-- Garbage

y: [ ]--------------->"Hallo"

We assign the bignum in x to the variable y
(there are two alternatives)
1)
x: [ ]--------------->#xFFFFFFFFFFFE
                      /
y: [ ]---------------/

2)
x: [ ]--------------->#xFFFFFFFFFFFE

y: [ ]--------------->#xFFFFFFFFFFFE


Now we increment x again:
(depending on the above alternative)

1.1)
x: [ ]--------------->#xFFFFFFFFFFFF
                      
y: [ ]--------------->#xFFFFFFFFFFFE

2.1)
x: [ ]--------------->#xFFFFFFFFFFFF

                      #xFFFFFFFFFFFE <-- Garbage

y: [ ]--------------->#xFFFFFFFFFFFE


Now it might seem that regarding to the immutability of bignums the 
alternative 1 might be always the better one because it produces less 
garbage. So we might conclude that having multiple copies of (equal) 
bignums is simply a waste of memory (and GC time). On the other side in the 
above case it was obvious that x and y refer to the same object because we 
directly assigned them. What if such an association is made indirectly by 
incrementing a number up until it reaches the value of another number bound 
to some other variable? We would then have to "intern" all bignums we 
allocate and this would be rather expensive.

Another thing is that you can reuse the storage used for a bignum in some 
operations when you can prove that this bignum-object is not used somewhere 
else. For example you could increment the bignum in x by simply *mutating* 
it internally if it is not referenced somewhere else.

ciao,
Jochen
From: Adam Warner
Subject: Re: Newbie variable question
Date: 
Message-ID: <pan.2003.02.25.12.21.04.277447@consulting.net.nz>
Wonderful exposition thanks Jochen! I think it has all just fallen into
place!

> We increment the bignum in x by one
> 
> x: [ ]--------------->#xFFFFFFFFFFFE
> 
>                       #xFFFFFFFFFFFD  <-- Garbage
> 
> y: [ ]--------------->"Hallo"
> 
> We assign the bignum in x to the variable y
> (there are two alternatives)
> 1)
> x: [ ]--------------->#xFFFFFFFFFFFE
>                       /
> y: [ ]---------------/

1: y points to the single bignum object, or:

> 2)
> x: [ ]--------------->#xFFFFFFFFFFFE
> 
> y: [ ]--------------->#xFFFFFFFFFFFE

2: y points to a copy of the object (NB this is a counterexample that
assignment never copies an object).
 
> Now we increment x again:
> (depending on the above alternative)
> 
> 1.1)
> x: [ ]--------------->#xFFFFFFFFFFFF
>                       
> y: [ ]--------------->#xFFFFFFFFFFFE

A new object is created for x, and y is perhaps left with the pointer to
the original object. So incrementing x breaks the object identity! (I had
started to get the bizarre idea that incrementing x would increment y
because they were pointing to the same object :-)

> 2.1)
> x: [ ]--------------->#xFFFFFFFFFFFF
> 
>                       #xFFFFFFFFFFFE <-- Garbage
> 
> y: [ ]--------------->#xFFFFFFFFFFFE

Since x and y were already separate objects a new object is created for x
and the old one discarded, creating garbage.

> Now it might seem that regarding to the immutability of bignums the 
> alternative 1 might be always the better one because it produces less 
> garbage.

Yes.

> So we might conclude that having multiple copies of (equal) 
> bignums is simply a waste of memory (and GC time).

Yes. It's (naively) compelling.

> On the other side in the 
> above case it was obvious that x and y refer to the same object because we 
> directly assigned them. What if such an association is made indirectly by 
> incrementing a number up until it reaches the value of another number bound 
> to some other variable? We would then have to "intern" all bignums we 
> allocate and this would be rather expensive.

Ouch. One would have to check every object to making sure no two bignums
were the same? Rather expensive indeed. I can now appreciate why it is
unrealistic to expect two bignums are EQ.
 
> Another thing is that you can reuse the storage used for a bignum in some 
> operations when you can prove that this bignum-object is not used somewhere 
> else. For example you could increment the bignum in x by simply *mutating* 
> it internally if it is not referenced somewhere else.

A great point, and one which drives home how two numerically equal bignums
may not be EQ.

Numerically equal fixnums are almost certainly EQ simply because they
are encoded as "immediate objects" that include a bit pattern of their
type and value. The "pointer" is the object and value. I guess what's
so cool is that the type checking that EQ has to perform comes for free:
Two equal "values" of a different type will not be EQ because the lower
three (or so) bits will not match when both "object references" are
compared.

Thank you very much for your help. I've learned a great deal.

Regards,
Adam
From: Tim Bradshaw
Subject: Re: Newbie variable question
Date: 
Message-ID: <ey3smuch4b8.fsf@cley.com>
* Adam Warner wrote:
> Numerically equal fixnums are almost certainly EQ simply because they
> are encoded as "immediate objects" that include a bit pattern of their
> type and value. The "pointer" is the object and value. I guess what's
> so cool is that the type checking that EQ has to perform comes for free:
> Two equal "values" of a different type will not be EQ because the lower
> three (or so) bits will not match when both "object references" are
> compared.

Imagine, though, a system which has two representations for some
fixnum.  Maybe fixnums are stored as <tag bits><sign bit><magnitude>.
There are two zeros in this representation, and EQ, which simply says
`are these two words the same' will say `no, they are not'.  But,
actually, they are the same number.

Another case is systems which support unboxed `small immediate
bignums' (I just invented this term): CMUCL is an example, as are
probably many other systems.  In these systems, a full 32bit (or
64bit) integer can end up in a register.  If EQ is asked to compare
one of these with a non-immediate bignum then it will say they differ.

--tim
From: Timothy Moore
Subject: Re: Newbie variable question
Date: 
Message-ID: <b3g5kg$1uc$0@216.39.145.192>
Tim Bradshaw <···@cley.com> writes:

> * Adam Warner wrote:
> > Numerically equal fixnums are almost certainly EQ simply because they
> > are encoded as "immediate objects" that include a bit pattern of their
> > type and value. The "pointer" is the object and value. I guess what's
> > so cool is that the type checking that EQ has to perform comes for free:
> > Two equal "values" of a different type will not be EQ because the lower
> > three (or so) bits will not match when both "object references" are
> > compared.
> 
> Imagine, though, a system which has two representations for some
> fixnum.  Maybe fixnums are stored as <tag bits><sign bit><magnitude>.
> There are two zeros in this representation, and EQ, which simply says
> `are these two words the same' will say `no, they are not'.  But,
> actually, they are the same number.
> 
> Another case is systems which support unboxed `small immediate
> bignums' (I just invented this term): CMUCL is an example, as are
> probably many other systems.  In these systems, a full 32bit (or
> 64bit) integer can end up in a register.  If EQ is asked to compare
> one of these with a non-immediate bignum then it will say they differ.

Actually, EQ can't compare an unboxed value to a boxed value at all
without bit-twiddling; if it went ahead and did the comparison without
doing anything to the unboxed value it might give a false positive  The
compiler knows the type of the value in the unboxed register; assuming
that it's a number or character, it could return NIL for any
comparison between that value and a value in a boxed register.

Tim
From: Tim Bradshaw
Subject: Re: Newbie variable question
Date: 
Message-ID: <ey3isv8gv0p.fsf@cley.com>
* Timothy Moore wrote:
> Actually, EQ can't compare an unboxed value to a boxed value at all
> without bit-twiddling; if it went ahead and did the comparison without
> doing anything to the unboxed value it might give a false positive  The
> compiler knows the type of the value in the unboxed register; assuming
> that it's a number or character, it could return NIL for any
> comparison between that value and a value in a boxed register.

Good point.

--tim
From: Tim Bradshaw
Subject: Re: Newbie variable question
Date: 
Message-ID: <ey3bs10imm9.fsf@cley.com>
* Adam Warner wrote:

> This indicates that an implementation can copy the value of x. If a copy
> of x is made then the pointers to the values will be different/the values
> will exist at different address locations/the values won't be physically
> identical.

Well, it indicates that numbers and characters are allowed to be weird
as regards EQ, in particular `An implementation is permitted to make
``copies'' of characters and numbers at any time.'.  That liberty is
not granted for other types.

Really, what you have to understand is that characters and numbers are
strange in CL as regards to EQness.

--tim
From: Kalle Olavi Niemitalo
Subject: Re: Newbie variable question
Date: 
Message-ID: <87heasuq62.fsf@Astalo.kon.iki.fi>
"Adam Warner" <······@consulting.net.nz> writes:

> For your criticism of Nils to be correct a Common Lisp implementation must
> be able to make a copy of y with a different generalised reference before
> the EQ test is conducted. Is this your assertion?

I assert that the implementation is allowed to copy 42 before
comparing it, but only because it is a number.  Generalized
references are not relevant here.
From: Thomas A. Russ
Subject: Re: Newbie variable question
Date: 
Message-ID: <ymiwujupjld.fsf@sevak.isi.edu>
"Michael J. Conroy" <········@snet.net> writes:

> 
> Hi all...
> 
> I just don't understand...
> 
> (setq list0 '(b c d))
> (setq list1 (cons 'a list0))
> (setq list2 (cons 'a list0))
> 
> list0
>     (B C D)
> list1
>     (A B C D)
> list2
>     (A B C D)

Initially

> 
> (setq list0 (append list0 '(e))
>     (B C D E) <---- this is what I was after

After Append

> 
> list1
>     (A B C D) <----ARHGHGHGHG...  I *wanted* (A B C D E)
> 
> WHY...  Why is list1 and list2 not referencing list0?  I thought setq was
> setting pointers.  If so, shouldn't list1 ALWAYS be (a (list0))?

No.  That is because pointers are being manipulated.  

This would be much easier to do with a real picture, but I'll try
anyway, using [ ... ]  to represent cons cells.

What you have initially is:


   list0    ---------------------> [ B  :  ]
                                  ^ ^   V
                      +----------/ /    [ C  : ]
                      |            |         V
                      |            |         [ D  NIL]
                      |            |
   list1   --->  [ A  : ]          |
                                   |
                      +------------+
   list2   --->  [ A  : ]


list1 and list2 have separate heads, but share their tails.

After Append you will have the following, because APPEND
copies all but its last argument.  This is required by the
standard since it is NON-DESTRUCTIVE.

                           [ E NIL ]
                           ^
                      [ D  : ]
                      ^
                 [ C  : ]
                 ^    
              [B : ]  
              ^
              |
   list0    --+                    [ B  :  ]
                                  ^ ^   V
                      +----------/ /  [ C  : ]
                      |            |       V
                      |            |     [ D  NIL]
                      |            |
   list1   --->  [ A  : ]          |
                                   |
                      +------------+
   list2   --->  [ A  : ]



You will note, that the tails of list1 and list2 still point to the
original list that was in list0.

To see this in Lisp you could try the following:

(setq list0 '(b c d))
(setq list1 (cons 'a list0))
(setq list2 (cons 'a list0))
(setq list-save list0)

(eq list0 list-save)    ;; => T
(eq list1 list2)        ;; => NIL
(eq (cdr list1) list0)  ;; => T
(eq (cdr list2) list0)  ;; => T

(eq (cdr list1) (cdr list2)) ;; => T

;; Now do the append:

(setq list0 (append list0 '(e)))

(eq list0 list-save)         ;; => NIL because the contents of list0 
                             ;; were copied and a new binding of the
                             ;; variable list0 made
(eq (cdr list1) list-save)   ;;  => T
(eq (cdr list2) list-save)   ;;  => T
(eq (cdr list1) (cdr list2)) ;; => T


-- 
Thomas A. Russ,  USC/Information Sciences Institute          ···@isi.edu