From: Adam Warner
Subject: (eq x x) => false
Date: 
Message-ID: <pan.2003.02.25.03.22.34.277028@consulting.net.nz>
Hi all,

Has any implementation of Lisp every existed where (eq x x) is false? (x
is a symbol name for a variable containing any contents whatsoever) 

The HyperSpec contains this example:

(let ((x 5)) (eq x x))
=>  true
OR=>  false

I can't understand why anyone would implement a Lisp that doesn't return
the same generalised reference for an object with the same symbol name.

To compound the strangeness the HyperSpec also contains this example:

(let ((x "Foo")) (eq x x)) =>  true

How the heck can this be guaranteed to be true but the example above
cannot?

Thanks,
Adam

From: Timothy Moore
Subject: Re: (eq x x) => false
Date: 
Message-ID: <b3f1kr$381$0@216.39.145.192>
"Adam Warner" <······@consulting.net.nz> writes:

> Hi all,
> 
> Has any implementation of Lisp every existed where (eq x x) is false? (x
> is a symbol name for a variable containing any contents whatsoever) 
> 
> The HyperSpec contains this example:
> 
> (let ((x 5)) (eq x x))
> =>  true
> OR=>  false
> 
Note that this is not "any old x", but specifically the case where x
is a number.

This does seem pretty silly, but I think it's a distillation of
various other cases that have been presented.  There have certainly
been Lisps where the boxed representation of fixnums is not unique.
For example in a BIBOP representation with 18 bit pointers and cons
cells, you have a situation where all numbers behave like bignums for
purposes of equality testing. (I was tempted to say "MacLisp", but I'm
not sure that it actually behaves this way.) Alternatively, you could
say that two different boxed fixnums could represent the same fixnum.  The
HyperSpec is being very non-committal as to when an implementation is
allowed to bring a new number into being, even if other numbers that
are "the same" happen to be around.

> I can't understand why anyone would implement a Lisp that doesn't return
> the same generalised reference for an object with the same symbol name.
> 
> To compound the strangeness the HyperSpec also contains this example:
> 
> (let ((x "Foo")) (eq x x)) =>  true
> 
> How the heck can this be guaranteed to be true but the example above
> cannot?

Because "An implementation is permitted to make ``copies'' of
characters and numbers at any time. The effect is that Common Lisp
makes no guarantee that eq is true even when both its arguments are
``the same thing'' if that thing is a character or number. " 

Tim
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwbs10tfcr.fsf@shell01.TheWorld.com>
"Adam Warner" <······@consulting.net.nz> writes:

> Hi all,
> 
> Has any implementation of Lisp every existed where (eq x x) is false? (x
> is a symbol name for a variable containing any contents whatsoever) 
> 
> The HyperSpec contains this example:
> 
> (let ((x 5)) (eq x x))
> =>  true
> OR=>  false
> 
> I can't understand why anyone would implement a Lisp that doesn't return
> the same generalised reference for an object with the same symbol name.
> 
> To compound the strangeness the HyperSpec also contains this example:
> 
> (let ((x "Foo")) (eq x x)) =>  true
> 
> How the heck can this be guaranteed to be true but the example above
> cannot?

If I recall correctly, in Maclisp (a pre-CL Lisp, but one that used 
implementation strageties we wanted to permit in CL), arguments to 
subrs [functions of a fixed number of up to 5 arguments] were passed
in registers.  Moreover, the first 16 memory locations WERE the registers.
That is, address 0 was register 0, address 1 was register 1, etc.
Since all arguments were pointers, this caused no problem.  
(let ((x 3)) (eq x x))
would give you a register with a pointer to 3 and another register with
a pointer to the same 3.  HOWEVER, you could do a thing called "number
compiling" in which case you passed numeric arguments in registers as their
raw values using a different set of registers.  In this case, the numbers 
in the caller might be split, and the register for arg1 and the register
for arg2 would contain the immediate number 3, and the only "pointer"
that could be passed to EQ, especially if this happened across a
function boundary, as in
 (let ((x 3)) (g x x))
 (defun g (x y) (if (eq x x) (+ x x) (- x x)))
would be the register addresses themselves.

Experience with Maclisp suggested that to require that (EQ x x)
be true is to require that the identity of x never be lost, which is
to require that numeric args never be passed by immediate value and 
always be passed heap-consed so that they don't lose their identity in
case it is later needed.  In the design of CL, we took this issue
into account and left implementations the flexibility to register-allocate
numbers.

This issue does not come up with strings because strings are not likely
to be register-allocated immediate quantities.  Representing a string as
a pointer is likely always the most efficient representation anyway, so
no loss of efficiency comes from requiring that (eq x x) be true.

Rather than obsess about whether this makes the langauge inelegant, I
suggest the following better interpretation:

 Always use EQL.  Never use EQ.

 Think of EQ as something we added for the sake of 
 implementation-specific code.  Its presence at all is the irregularity,
 not the aspects of its behavior that you are citing.

If EQ were not flexible in the way you are questioning, there would be
no split between EQ and EQL.  There would be only EQL (though we would
probably have called it EQ), and it would occasionally have been less
efficient than it now is.
From: Roger Corman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <3e5c2869.365036064@nntp.sonic.net>
On 25 Feb 2003 12:55:16 -0500, Kent M Pitman <······@world.std.com>
wrote:

>"Adam Warner" <······@consulting.net.nz> writes:
>
>> Hi all,
>> 
>> Has any implementation of Lisp every existed where (eq x x) is false? (x
>> is a symbol name for a variable containing any contents whatsoever) 
>> 
>> The HyperSpec contains this example:
>> 
>> (let ((x 5)) (eq x x))
>> =>  true
>> OR=>  false
>> 
>> I can't understand why anyone would implement a Lisp that doesn't return
>> the same generalised reference for an object with the same symbol name.

I can't understand this either, nor does Kent's logic explain it to
me. If you said:

(let ((x 5) (y 5)) (eq x y))

then I agree it could be true or false.

If you said:
(let ((x 5) (y x)) (eq x y))

I think I could even agree that might be false in some implementation.

However, I cannot imagine how any implementation would evaluate the
above example to false, or why it should be "supported" in the name of
implementation optimization. Even if you store the binding of x in a
register, you are still talking about comparing the exact same value
to itself. I don't see how it can lose its identity.

Roger
From: Matthew Danish
Subject: Re: (eq x x) => false
Date: 
Message-ID: <20030225224114.A16756@lain.cheme.cmu.edu>
On Wed, Feb 26, 2003 at 02:44:02AM +0000, Roger Corman wrote:
> I can't understand this either, nor does Kent's logic explain it to
> me. If you said:
> 
> (let ((x 5) (y 5)) (eq x y))
> 
> then I agree it could be true or false.
> 
> If you said:
> (let ((x 5) (y x)) (eq x y))
> 
> I think I could even agree that might be false in some implementation.
> 
> However, I cannot imagine how any implementation would evaluate the
> above example to false, or why it should be "supported" in the name of
> implementation optimization. Even if you store the binding of x in a
> register, you are still talking about comparing the exact same value
> to itself. I don't see how it can lose its identity.

In the process of applying the function #'eq, the operations that take
place may or may not copy each argument separately.  I'm not sure
whether this gains any particular efficiency benefit, but it does seem
to be a natural consequence of the statement that numbers and characters
may be copied at /any/ time.

-- 
; Matthew Danish <·······@andrew.cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian.org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfw7kbnrvqo.fsf@shell01.TheWorld.com>
Matthew Danish <·······@andrew.cmu.edu> writes:

> On Wed, Feb 26, 2003 at 02:44:02AM +0000, Roger Corman wrote:
> > I can't understand this either, nor does Kent's logic explain it to
> > me. If you said:
> > 
> > (let ((x 5) (y 5)) (eq x y))
> > 
> > then I agree it could be true or false.
> > 
> > If you said:
> > (let ((x 5) (y x)) (eq x y))
> > 
> > I think I could even agree that might be false in some implementation.
> > 
> > However, I cannot imagine how any implementation would evaluate the
> > above example to false, or why it should be "supported" in the name of
> > implementation optimization. Even if you store the binding of x in a
> > register, you are still talking about comparing the exact same value
> > to itself. I don't see how it can lose its identity.
> 
> In the process of applying the function #'eq, the operations that take
> place may or may not copy each argument separately.  I'm not sure
> whether this gains any particular efficiency benefit, but it does seem
> to be a natural consequence of the statement that numbers and characters
> may be copied at /any/ time.

Right.  The problem may be that you're seeing the function EQ here
(which is probably compiled open and probably does not involve arg
passing) and not the more general case of some function MY-EQ which is
not compiled open, and which might be accomplishing EQ but by a path
that involves separating the identity because of the implementation's
particular strategy for argument passing.
From: Adam Warner
Subject: Re: (eq x x) => false
Date: 
Message-ID: <pan.2003.02.25.06.15.44.492676@consulting.net.nz>
I wrote:

> Has any implementation of Lisp every existed where (eq x x) is false? (x
> is a symbol name for a variable containing any contents whatsoever)
> 
> The HyperSpec contains this example:
> 
> (let ((x 5)) (eq x x))
> =>  true
> OR=>  false
> 
> I can't understand why anyone would implement a Lisp that doesn't return
> the same generalised reference for an object with the same symbol name.
> 
> To compound the strangeness the HyperSpec also contains this example:
> 
> (let ((x "Foo")) (eq x x)) =>  true
> 
> How the heck can this be guaranteed to be true but the example above
> cannot?

Upon further searching the answer appears to be that Common Lisp allows
(and only allows) NUMBERS AND CHARACTERS TO BE COPIED _AT ANY TIME_!

From Common Lisp the Language, 2nd Edition, 6.3. Equality Predicates:

   In Common Lisp, unlike some other Lisp dialects, the implementation is
   permitted to make ``copies'' of characters and numbers at any time.
   (This permission is granted because it allows tremendous performance
   improvements in many common situations.) The net effect is that Common
   Lisp makes no guarantee that eq will be true even when both its
   arguments are ``the same thing'' if that thing is a character or
   number. For example:

   (let ((x 5)) (eq x x)) might be true or false.

So (let ((x "Foo")) (eq x x)) is always true simply because
implementations are not allowed to copy strings at any time (even though a
case could probably be made for "tremendous performance improvements" if
this was allowed).

Following is an implementation note that exonerates my claim that one can
understand EQ as a pointer comparison: "eq simply compares the two given
pointers..."

Regards,
Adam
From: Andy Freeman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <8bbd9ac3.0302250926.4d0ea27f@posting.google.com>
"Adam Warner" <······@consulting.net.nz> wrote in message news:<······························@consulting.net.nz>...
> Upon further searching the answer appears to be that Common Lisp allows
> (and only allows) NUMBERS AND CHARACTERS TO BE COPIED _AT ANY TIME_!

As (IIRC) Kent Pitman points out, sameness and mutablity are intertwined
and that "sameness" isn't all that well defined anyway.

In CL, neither numbers nor characters are mutable.  Most other "base"
CL items are mutable.  This difference makes different sameness definitions
plausible.

I seem to remember that eql has relevant properties.

> So (let ((x "Foo")) (eq x x)) is always true simply because
> implementations are not allowed to copy strings at any time (even though a
> case could probably be made for "tremendous performance improvements" if
> this was allowed).

I'd like to see an actual example, one that accounts for the fact that
CL strings are mutable.  (Note that python strings are not mutable and,
IIRC, its "is" operator, which is much like CL's eq, may be allowed to
give some seemingly strange results with strings.)

-andy
From: Ingvar Mattsson
Subject: Re: (eq x x) => false
Date: 
Message-ID: <87lm04b6he.fsf@gruk.tech.ensign.ftech.net>
······@earthlink.net (Andy Freeman) writes:

> "Adam Warner" <······@consulting.net.nz> wrote in message news:<······························@consulting.net.nz>...
> > Upon further searching the answer appears to be that Common Lisp allows
> > (and only allows) NUMBERS AND CHARACTERS TO BE COPIED _AT ANY TIME_!
> 
> As (IIRC) Kent Pitman points out, sameness and mutablity are intertwined
> and that "sameness" isn't all that well defined anyway.
> 
> In CL, neither numbers nor characters are mutable.  Most other "base"
> CL items are mutable.  This difference makes different sameness definitions
> plausible.
> 
> I seem to remember that eql has relevant properties.
> 
> > So (let ((x "Foo")) (eq x x)) is always true simply because
> > implementations are not allowed to copy strings at any time (even though a
> > case could probably be made for "tremendous performance improvements" if
> > this was allowed).
> 
> I'd like to see an actual example, one that accounts for the fact that
> CL strings are mutable.  (Note that python strings are not mutable and,
> IIRC, its "is" operator, which is much like CL's eq, may be allowed to
> give some seemingly strange results with strings.)

(let ((x "Foo")
      (y "Foo"))
  (eq x y))

should be allowed to return either T or NIL, since the reader is free
to collapse multiple read constant strings to the same string.

Of course, trying to provoke different returns in two random lisps
(CMU CL and Clisp) managed to have CMUCL consistenly return T and
clisp to consistently return nil.

//Ingvar
-- 
Sysadmin is brave
Machine is running for now
Backup on Friday
From: Rob Warnock
Subject: Re: (eq x x) => false
Date: 
Message-ID: <aMKcnTzCtp2DF8GjXTWc-g@speakeasy.net>
Ingvar Mattsson  <······@cathouse.bofh.se> wrote:
+---------------
| (let ((x "Foo")
|       (y "Foo"))
|   (eq x y))
| 
| should be allowed to return either T or NIL, since the reader is free
| to collapse multiple read constant strings to the same string.
| 
| Of course, trying to provoke different returns in two random lisps
| (CMU CL and Clisp) managed to have CMUCL consistenly return T and
| clisp to consistently return nil.
+---------------

You think *that's* fun, then try the following, entirely within CMUCL:

	> (eq '(a . b) '(a . b))

	NIL
	> (progn
	    (eq '(a . b) '(a . b)))

	NIL
	> (let ()
	    (eq '(a . b) '(a . b)))

	T
	> (funcall (compile nil (lambda () (eq '(a . b) '(a . b)))))
	Compiling LAMBDA NIL: 
	Compiling Top-Level Form: 

	T
	> 

[My understanding is that the first two use the "mini-eval" built into
the top-level, while the third uses the byte-compiler, and the last, of
course, uses the full native-code compiler.]


-Rob

-----
Rob Warnock, PP-ASEL-IA		<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Paul F. Dietz
Subject: Re: (eq x x) => false
Date: 
Message-ID: <yr6dnZ5iiYMHCMGjXTWcoA@dls.net>
Rob Warnock wrote:

> You think *that's* fun, then try the following, entirely within CMUCL:
> 
> 	> (eq '(a . b) '(a . b))
> 
> 	NIL
> 	> (progn
> 	    (eq '(a . b) '(a . b)))
> 
> 	NIL
> 	> (let ()
> 	    (eq '(a . b) '(a . b)))
> 
> 	T
> 	> (funcall (compile nil (lambda () (eq '(a . b) '(a . b)))))
> 	Compiling LAMBDA NIL: 
> 	Compiling Top-Level Form: 
> 
> 	T
> 	> 
> 
> [My understanding is that the first two use the "mini-eval" built into
> the top-level, while the third uses the byte-compiler, and the last, of
> course, uses the full native-code compiler.]


This behavior of the compiler is a known ANSI bug, and has (IIRC) been
fixed by the cmucl developers in cvs.  Wait for 18e.

	Paul
From: Rob Warnock
Subject: Re: (eq x x) => false
Date: 
Message-ID: <zP2cnU4rsY0LBcGjXTWc-g@speakeasy.net>
Paul F. Dietz <·····@dls.net> wrote:
+---------------
| Rob Warnock wrote:
| > You think *that's* fun, then try the following, entirely within CMUCL:
| > 	> (let ()
| > 	    (eq '(a . b) '(a . b)))
| > 
| > 	T
| 
| This behavior of the compiler is a known ANSI bug, and has (IIRC) been
| fixed by the cmucl developers in cvs.  Wait for 18e.
+---------------

Do you mean because ANSI CL *forbids* such merging?


-Rob

-----
Rob Warnock, PP-ASEL-IA		<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Paul F. Dietz
Subject: Re: (eq x x) => false
Date: 
Message-ID: <PBycnYi6zJ2CB8GjXTWcpw@dls.net>
Rob Warnock wrote:
> Paul F. Dietz <·····@dls.net> wrote:
> +---------------
> | Rob Warnock wrote:
> | > You think *that's* fun, then try the following, entirely within CMUCL:
> | > 	> (let ()
> | > 	    (eq '(a . b) '(a . b)))
> | > 
> | > 	T
> | 
> | This behavior of the compiler is a known ANSI bug, and has (IIRC) been
> | fixed by the cmucl developers in cvs.  Wait for 18e.
> +---------------
> 
> Do you mean because ANSI CL *forbids* such merging?


The ANSI spec forbids compile and eval (but not the *file*
compiler) from merging literal objects.  It must maintain EQL-ness.
This form:

   (funcall (compile nil (let ((x (cons 'a 'b)) (y (cons 'a 'b)))
				`(lambda () (eq ',x ',y)))))

must return NIL.  Section 3.2.4:

    The functions eval and compile are required to ensure that
    literal objects referenced within the resulting interpreted
    or compiled code objects are the same as the corresponding
    objects in the source code.

where 'same' is defined in the glossary:

    2. (of objects if no predicate is implied by context) indistinguishable
    by eql.

	Paul
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2602031028350001@k-137-79-50-101.jpl.nasa.gov>
In article <······················@dls.net>, "Paul F. Dietz"
<·····@dls.net> wrote:

> The ANSI spec forbids compile and eval (but not the *file*
> compiler) from merging literal objects.

Youch!  That means I can't rely on my file-compiled code to do the same
thing as my interactively compiled code.  That's not good.

E.
From: Barry Margolin
Subject: Re: (eq x x) => false
Date: 
Message-ID: <EL87a.23$BS.1376@paloalto-snr1.gtei.net>
In article <····················@k-137-79-50-101.jpl.nasa.gov>,
Erann Gat <···@jpl.nasa.gov> wrote:
>In article <······················@dls.net>, "Paul F. Dietz"
><·····@dls.net> wrote:
>
>> The ANSI spec forbids compile and eval (but not the *file*
>> compiler) from merging literal objects.
>
>Youch!  That means I can't rely on my file-compiled code to do the same
>thing as my interactively compiled code.  That's not good.

Since the file compiler doesn't have access to the original list
representation, but just a printed representation, it would be pretty
difficult to require it to maintain all those relationships.

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2602031532320001@k-137-79-50-102.jpl.nasa.gov>
In article <················@paloalto-snr1.gtei.net>, Barry Margolin
<··············@level3.com> wrote:

> In article <····················@k-137-79-50-101.jpl.nasa.gov>,
> Erann Gat <···@jpl.nasa.gov> wrote:
> >In article <······················@dls.net>, "Paul F. Dietz"
> ><·····@dls.net> wrote:
> >
> >> The ANSI spec forbids compile and eval (but not the *file*
> >> compiler) from merging literal objects.
> >
> >Youch!  That means I can't rely on my file-compiled code to do the same
> >thing as my interactively compiled code.  That's not good.
> 
> Since the file compiler doesn't have access to the original list
> representation, but just a printed representation, it would be pretty
> difficult to require it to maintain all those relationships.

What "original" list representation?  Most code starts out as text, even
during interactive development.

But this is a red-herring.  The file compiler can be considered to operate
on the data structures resulting from processing the file through READ.  I
don't see why, if EVAL should be forbidden to merge equal constants the
file compiler should not likewise be forbidden from doing so.  The
non-uniformity of this constraint can cause serious problems, and I don't
see that it buys you anything.  Either allow merging everywhere or forbid
it everywhere.  I don't see any advantage to this inconsistency.

E.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwr89u5yn0.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> But this is a red-herring.  The file compiler can be considered to operate
> on the data structures resulting from processing the file through READ.  I
> don't see why, if EVAL should be forbidden to merge equal constants the
> file compiler should not likewise be forbidden from doing so.

Because 

 (1) It was once necessary that at least EVAL or COMPILE not do this merging.
     (It is no longer necessary, but we goofed in updating the language.)
     We at one time needed to require it of at least one.

 (2) It was, even when it was necessary for EVAL and COMPILE not to do the
     merging, still true that file compilation couldn't be forced not to do
     this because there was no sharing of object identity (other than the
     notion of "similarity as a constant") over file compilation.
     We therefore could not have required it of all three.

Therefore, it had to be, and it had to be inconsistent.

> The non-uniformity of this constraint can cause serious problems, 

It's probably best fixed, but the language has survived for years this way
so I dispute this claim if you intend it to mean "will necesssarily cause
serious problems".  Just about anything CAN cause serious problems, but
decades of experience show this has not been an issue.

> and I don't see that it buys you anything.

It used to.  Think appendix.

> Either allow merging everywhere or forbid
> it everywhere.  I don't see any advantage to this inconsistency.

Language stability and unreasonable cost of gratuitous change are still
an advantage that argue for keeping it this way.  At the point that the
language were open in free-for-all mode, which I hope is no time soon,
it surely should be changed.  But not until.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2602031739130001@k-137-79-50-102.jpl.nasa.gov>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

>  (2) It was, even when it was necessary for EVAL and COMPILE not to do the
>      merging, still true that file compilation couldn't be forced not to do
>      this because there was no sharing of object identity (other than the
>      notion of "similarity as a constant") over file compilation.

I don't understand this.  Why does it not suffice to use #n= or #.
notation in a file where object identity is needed?

> > The non-uniformity of this constraint can cause serious problems, 
                                          ^^^
> 
> It's probably best fixed, but the language has survived for years this way
> so I dispute this claim if you intend it to mean "will necesssarily cause
> serious problems".  Just about anything CAN cause serious problems, but
> decades of experience show this has not been an issue.

Look up the meaning of the word "can" and contrast it with the word
"will".  I wrote what I meant.

And it is not at all clear that there is all that much experience with
Lisp in mission-critical applications.

> > Either allow merging everywhere or forbid
> > it everywhere.  I don't see any advantage to this inconsistency.
> 
> Language stability and unreasonable cost of gratuitous change are still
> an advantage that argue for keeping it this way.  At the point that the
> language were open in free-for-all mode, which I hope is no time soon,
> it surely should be changed.  But not until.

Gack!  Why must utter stagnation and open free-for-all be the only two
possibilities?

E.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwlm02nnzw.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
> 
> >  (2) It was, even when it was necessary for EVAL and COMPILE not to do the
> >      merging, still true that file compilation couldn't be forced not to do
> >      this because there was no sharing of object identity (other than the
> >      notion of "similarity as a constant") over file compilation.
> 
> I don't understand this.  Why does it not suffice to use #n= or #.
> notation in a file where object identity is needed?

Both of these are read-time actions and the fact of their use is lost by
the time of compilation.  The file compiler is still free to make

 (eq (car '(#1=(foo bar))) (car '(#1#))) => NIL

It is also free to make:

 #.(let ((foobar '(foo bar)))
     `(car '(,foobar) '(,foobar)))
 => NIL

[Caveat: It's the middle of the night and I have a cold and I'm just
 restless. I didn't  test this, so I might have made an error in this
 code.  But I THINK it mostly illustrates what my head would believe
 on another occasion.]

This is because you should should not and cannot rely on the object
identity of literal constants in compiled files.  The compiled file
loader will reconstruct the literal objects it sees [and those 
literals are what appeared literal AFTER readtime, not before; 
that is, semantics is applied to objects, not to input expressions] 
and to do coalescing and merging.  There is no way for the file
compiler to do otherwise.  It is easy [ok, it's cumbersome, but it's
mechanical] to construct situations that would just be so opaque that
you'd have to admit the file compiler has no choice but to do this.

> > > The non-uniformity of this constraint can cause serious problems, 
>                                           ^^^
> > 
> > It's probably best fixed, but the language has survived for years this way
> > so I dispute this claim if you intend it to mean "will necesssarily cause
> > serious problems".  Just about anything CAN cause serious problems, but
> > decades of experience show this has not been an issue.
> 
> Look up the meaning of the word "can" and contrast it with the word
> "will".  I wrote what I meant.

If I opened Consumer Reports and saw "The choice of steering wheel on this
model car can cause serious problems." I would not infer that you were
using the literal meaning of "can".  Read HP Grice's rules of conversational
implicature to understand why the normal human reading of "can" is not
the reason you cite.  Conveniently, I've cited Grice here before, so the
URL was findable in google from my prior posts.
 http://mh.cla.umn.edu/grice.html

> And it is not at all clear that there is all that much experience with
> Lisp in mission-critical applications.
> 
> > > Either allow merging everywhere or forbid
> > > it everywhere.  I don't see any advantage to this inconsistency.
> > 
> > Language stability and unreasonable cost of gratuitous change are still
> > an advantage that argue for keeping it this way.  At the point that the
> > language were open in free-for-all mode, which I hope is no time soon,
> > it surely should be changed.  But not until.
> 
> Gack!  Why must utter stagnation and open free-for-all be the only two
> possibilities?

_I think_ that's just the ANSI rule.  If you fast-tracked it into
another system in a way that (a) disallowed change during
fast-tracking and in a system that (b) had the property you desire,
then indeed it could be done.  But ANSI, for reasons of fairness
[which is all it's intended to implement in the first place, its
primary purpose being to prevent antitrust actions due to collusion in
advance of an outcome agreed upon by conspiring businesses], simply
doesn't let you make material changes to the language, as far as I
know, without opening the document to arbitrary change.

I think, in part, that it may also be a creeping sense that if you go
to fix something, you need the right to fix it thoroughly, and it's
hard to know who would judge what was and what was not a "related"
fix; certainly the text that would change in a document might not be
limited to a given place just because you were trying to localize a
textual change.

Incidentally, the production of the ANSI standard was held up by almost
a year because during a technical comment period, someone made a 
non-technical comment.  We'd like to have dismissed it as non-technical
but there was no source of authority at ANSI to arbitrate what was 
technical and what was not in borderline cases, since ordinarily borderline
cases are only aptly judged by the committee.  (Note that there was
little ambiguity in this case, but because the expected case did expect
ambiguity, they had no such engine.)  They ended up just throwing it back
to the committee and saying "just treat it as technical, which will take
most of the next year in procedures, because it will take as long for us
to develop an exception.  Why is this related?  Because it emphasises
that what may seem obvious ("we're opening this document to change only X")
may be quite a subjective judgment; ANSI's job is to ensure fairness and
it hasn't the credentials to say whether this was fairly applied.  
Therefore it needs simple rules that it CAN apply.  "Not open for change"
vs "Open for any change" is a simple rule that can be objectively judged,
so it's hardly surprising that this kind of rule is what they sought,
assuming my explanation of the rule is in fact correct.

I am no longer even on the ANSI committee, so my understanding might be
wrong.  Even when I was on the committee, that didn't make me all-fired
right, of course.  But I'll let Steve Haflich speak to any of this if he
disagrees, assuming he's still committee chair.  He's presumably up on the
rules.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2702030934020001@192.168.1.51>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> > > > The non-uniformity of this constraint can cause serious problems, 
> >                                           ^^^
> > > 
> > > It's probably best fixed, but the language has survived for years this way
> > > so I dispute this claim if you intend it to mean "will necesssarily cause
> > > serious problems".  Just about anything CAN cause serious problems, but
> > > decades of experience show this has not been an issue.
> > 
> > Look up the meaning of the word "can" and contrast it with the word
> > "will".  I wrote what I meant.
> 
> If I opened Consumer Reports and saw "The choice of steering wheel on this
> model car can cause serious problems." I would not infer that you were
> using the literal meaning of "can".

But you aren't reading Consumer Reports, you're reading Erann Gat.

Besides, I'm pretty sure that Consumer Reports has in the past written
things like, "The Subara thingamawhatchit has design flaws that CAN
contribute to rollover accidents.  We therefore rate it unacceptable." 
They did not mean that the design flaws will necessarily cause serious
problems, only that they can under certain circumstances.  If you avoid
those circumstances there is no problem, and indeed many people have
driven Subaru thingamawhatchits with no ill effects.

BUT...

> Read HP Grice's rules of conversational
> implicature to understand why the normal human reading of "can" is not
> the reason you cite.  Conveniently, I've cited Grice here before, so the
> URL was findable in google from my prior posts.
>  http://mh.cla.umn.edu/grice.html

Sorry, I have neither the time nor the patience to read some postmodern
deconstructionist hogwash.  I'll stick with Webster to tell me what words
mean.  But just out of curiosity, how would Grice prescribe that one
communicate the concept of "can" (as defined in the dictionary) if it
can't be done using the word itself?

But since I seem to be having such a hard time expressing myself, I'll
also resort to an obscure literary reference.  I'm reading a book called
"A Walk in the Woods" by Bill Bryson, about hiking the Appalachian Trail. 
(It's a wonderful book, I highly recommend it even if you don't care one
whit about hiking.)  Bryson describes how in his pre-trip research he did
a lot of reading about bear attacks.  He goes on for a few pages
describing the various horriffic things that bears have done to people
over the years, then notes that notwithstanding the horror stories, bear
attacks are exceedingly rare and unlikely.  Here's how Bryson assesses the
situation:

"Black bears rarely attack.  But here's the thing.  Sometimes they do. 
... If they want to kill you and eat you, they can, and pretty much
whenever they want.  This doesn't happen often, but -- and here's the
absolutely salient point -- once would be enough."

So: real problems caused by the misfeature rarely happen.  But here's the
thing: sometimes they do.  And in my line of work, when they do it can
cost you a few hundred million dollars.  Once would be enough.

> > Gack!  Why must utter stagnation and open free-for-all be the only two
> > possibilities?
> 
> _I think_ that's just the ANSI rule.

Why must all changes to Lisp necessarily be changes to the ANSI standard? 
Why could the community not adopt something like Scheme's SRFI system?

E.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwel5t611h.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> "Black bears rarely attack.  But here's the thing.  Sometimes they do. 
> ... If they want to kill you and eat you, they can, and pretty much
> whenever they want.  This doesn't happen often, but -- and here's the
> absolutely salient point -- once would be enough."

"People rarely attack.  But here's the thing.  Sometimes they do.
 ... If they want to killy uo and eat you, they can, and pretty much
 whenever they want.  This doesn't happen often, but -- and here's the
 absolutely salient point -- once would be enough."

> So: real problems caused by the misfeature rarely happen.  But here's the
> thing: sometimes they do.  And in my line of work, when they do it can
> cost you a few hundred million dollars.  Once would be enough.

There are tons of equally 'dangerous' misfeatures  in the language
that you have to learn to live with, by writing in good style or by
good testing, and that if they go wrong will go wrong just as often.

Certainly people have cited the ability to do 
  (SETF (SYMBOL-VALUE (INTERN *PICK-YOUR-POISON-STRING*
                              *PICK-YOUR-POISON-PACKAGE*))
        *PICK-YOUR-POISON-VALUE*)
is dangerous, too.  It's a valid CL program.  It uses commonly used
operators.  And it can completely blow away your Lisp if you pick
the wrong values for the special variables out of completely ordinary
strings.  But it doesn't.  Because people try to use operators as
they are intended.

Without a discussion of (a) the likelihood of occurrence and (b) the
necessity of occurrence, you aren't really addressing things.

The issue of identity is a necessary thing to confront.  The proper fix
will be to allow the language definition to be changed so that merging
can occur according to the implementation in all 3 cases.  However, we
would never require it, I believe.  (We discussed this during the X3J13
process, and I believe some implementations didn't want to be forced to
merge all literals, or even to say which they would merge.)  Once the
fix is applied, the language will be "consistent" but the implementation
will STILL have the property that if you test interactively, you might
not get the same behavior in the file compiler as interactively.  So
I can't figure out what you're after.

Lisp objects have identity, but we deliberately do not make identity
the first-class quantity you want it to be.  There are two reasons for
this: first, it would make debugging hard, and, second, it would still
be impossible to reconcile when objects from different Lisp images met
up unless you used addresses that had enough bits to identify the object
in whatever running image it had come from (as CORBA does/did).

> > > Gack!  Why must utter stagnation and open free-for-all be the only two
> > > possibilities?
> > 
> > _I think_ that's just the ANSI rule.
> 
> Why must all changes to Lisp necessarily be changes to the ANSI standard? 
> Why could the community not adopt something like Scheme's SRFI system?

There is a difference between extending the lanuage and gutting it.
To my knowledge, SRFIs do not change the underlying semantics.  They 
merely talk about things that could be implemented atop Scheme; they 
are "Scheme requests for implementation", where the "implementation"
is in a conforming Scheme.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2702031144270001@k-137-79-50-101.jpl.nasa.gov>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> Without a discussion of (a) the likelihood of occurrence and (b) the
> necessity of occurrence, you aren't really addressing things.

Agreed.  So here's a scenario:

Programmer 1 writes:

(defun call-a ()
   (case (baz)
      (1 'a)
      (2 'b)
      (otherwise '(no-result))))

Programmer 2 writes:

(defun call-b ()
  (case (bing)
    (1 'a)
    (2 'b)
    (otherwise '(no-result))))

Programmer 3 writes:

(if (eq (call-a) (call-b))
   (proceed-with-dangerous-operation)
   (go-to-safe-mode))

The idea being that you want to proceed only if the two independently
calculated results match.

This code is correct if eval'd, potentially incorrect if file-compiled.

Smells like a bear to me.

> The proper fix
> will be to allow the language definition to be changed so that merging
> can occur according to the implementation in all 3 cases.

That would be a good start.

> There is a difference between extending the lanuage and gutting it.
> To my knowledge, SRFIs do not change the underlying semantics.  They 
> merely talk about things that could be implemented atop Scheme; they 
> are "Scheme requests for implementation", where the "implementation"
> is in a conforming Scheme.

I don't recall anyone calling for the gutting of the language.  Fixing
this problem in a SRFI-style way is easy: just add an
implementation-dependent flag,
*DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*.  (The default value, of
course, should be T ;-)

E.
From: Tim Bradshaw
Subject: Re: (eq x x) => false
Date: 
Message-ID: <ey38yw1bfs0.fsf@cley.com>
* Erann Gat wrote:

> The idea being that you want to proceed only if the two independently
> calculated results match.

I think you need to find a more plausible example.  I'm not saying
that they don't exist, but, well, I'd be worried if something like
that got through in a system with half-billion-dollar failure costs,
and where this could cause such a failure.  (I'm not saying it might
not be hard to find, but when so much is at stake on a bit of software
I'd expect it to be *very* carefully checked).

And before we get carried away about all this, how hard is it to with
no changes to CL at all?  It isn't that hard to write a readmacro for
#\' which expands into (LOAD-TIME-VALUE ... NIL) and thus will prevent
any constant-folding the compiler might do.  It's quite easy to
arrange life so that the ... is then looked up in a global hashtable
so that, say, (eq '(a) '(a)) is always true, or never true, unless
READ is doing something very funny.

But there are other literals - strings for one (easily dealt with by
the same trick), and then things like arrays and structures (probably
likewise).  Other things?  Things you need MAKE-LOAD-FORM for - well
you should be able to do the same thing there, though if there is no
built-in equality predicate that works I don't see how the compiler
can do much merging there.  Hash tables?  Not sure.

--tim
From: Tim Bradshaw
Subject: Re: (eq x x) => false
Date: 
Message-ID: <ey3of4xdv9c.fsf@cley.com>
* I wrote:
> And before we get carried away about all this, how hard is it to with
> no changes to CL at all?  It isn't that hard to write a readmacro for
> #\' which expands into (LOAD-TIME-VALUE ... NIL) and thus will prevent
> any constant-folding the compiler might do.  It's quite easy to
> arrange life so that the ... is then looked up in a global hashtable
> so that, say, (eq '(a) '(a)) is always true, or never true, unless
> READ is doing something very funny.

However such a readmacro will not work in cases like '(x 'y) ... for
this and other reasons I think that this is actually pretty hard
(although I still don't think the
literals-folded-only-in-the-file-compiler problem is a huge issue).

--tim
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwheapghl9.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
> 
> > Without a discussion of (a) the likelihood of occurrence and (b) the
> > necessity of occurrence, you aren't really addressing things.
> 
> Agreed.  So here's a scenario:
> 
> Programmer 1 writes:
> 
> (defun call-a ()
>    (case (baz)
>       (1 'a)
>       (2 'b)
>       (otherwise '(no-result))))
> 
> Programmer 2 writes:
> 
> (defun call-b ()
>   (case (bing)
>     (1 'a)
>     (2 'b)
>     (otherwise '(no-result))))
> 
> Programmer 3 writes:
> 
> (if (eq (call-a) (call-b))
>    (proceed-with-dangerous-operation)
>    (go-to-safe-mode))
> 
> The idea being that you want to proceed only if the two independently
> calculated results match.

If I were reviewing your code, you wouldn't be calling EQ on lists
without an INTENSE study of the dataflow to avoid problems like this.
In general, EQ should be called on objects that the system is not likely
to perturb the identity of.  The above is simply ill-formed, bad-styled
code, IMO.  I'd want you to be changing '(no-result) to 'no-result unless
you had a strong reason to bump heads with the whole issue of splitting
and merging.  I'd urge you never to compare lists with EQ if you can avoid
it.
 
> This code is correct if eval'd, potentially incorrect if file-compiled.

I'm not sure what 'correct' means in this context.

> Smells like a bear to me.
> 
> > The proper fix
> > will be to allow the language definition to be changed so that merging
> > can occur according to the implementation in all 3 cases.
> 
> That would be a good start.

In that case you could rely on none of the 3 cases to do what you want?
How does that improve things for you?
 
> > There is a difference between extending the lanuage and gutting it.
> > To my knowledge, SRFIs do not change the underlying semantics.  They 
> > merely talk about things that could be implemented atop Scheme; they 
> > are "Scheme requests for implementation", where the "implementation"
> > is in a conforming Scheme.
> 
> I don't recall anyone calling for the gutting of the language.  Fixing
> this problem in a SRFI-style way is easy: just add an
> implementation-dependent flag,
> *DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*.  (The default value, of
> course, should be T ;-)

That is not a "compatible extension".
That is a reimplementation.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2802031237460001@k-137-79-50-101.jpl.nasa.gov>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> > I don't recall anyone calling for the gutting of the language.  Fixing
> > this problem in a SRFI-style way is easy: just add an
> > implementation-dependent flag,
> > *DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*.  (The default value, of
> > course, should be T ;-)
> 
> That is not a "compatible extension".
> That is a reimplementation.

Why?  Didn't you just get through telling me that the main application for
compile-time merging was superceded by the introduction of
load-time-value?

I would think this would be completly trivial to implement.  I would
expect, somewhere in the guts of the compiler, to find a function along
the line of:

(defun find-potential-literal-merges ()
   ...)

or something like that.  One could simply change this to:

(defun find-potential-literal-merges ()
  (if *DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*
    nil
    ...))

Why would it be any harder than that?

E.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfw7kbjx5yy.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
> 
> > > I don't recall anyone calling for the gutting of the language.  Fixing
> > > this problem in a SRFI-style way is easy: just add an
> > > implementation-dependent flag,
> > > *DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*.  (The default value, of
> > > course, should be T ;-)
> > 
> > That is not a "compatible extension".
> > That is a reimplementation.
> 
> Why?  Didn't you just get through telling me that the main application for
> compile-time merging was superceded by the introduction of
> load-time-value?

Because it would cause the semantics of the language would be violated.

It doesn't have anything to do with usefulness.

An extension is not allowed to break existing semantics.

Additionally, global switches like this work terribly if you load one
module that globally requires one value and another module that needs
another, one's setting will break the other.

Big-switch options that change language semantics globally are a disaster.
We did a LOT of this in Maclisp and it made a mess in which certain packages
could only be co-loaded with certain others.

You can make a new language that behaves like this, but it is not Common 
Lisp.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2802032000070001@192.168.1.51>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> ···@jpl.nasa.gov (Erann Gat) writes:
> 
> > In article <···············@shell01.TheWorld.com>, Kent M Pitman
> > <······@world.std.com> wrote:
> > 
> > > > I don't recall anyone calling for the gutting of the language.  Fixing
> > > > this problem in a SRFI-style way is easy: just add an
> > > > implementation-dependent flag,
> > > > *DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*.  (The default value, of
> > > > course, should be T ;-)
> > > 
> > > That is not a "compatible extension".
> > > That is a reimplementation.
> > 
> > Why?  Didn't you just get through telling me that the main application for
> > compile-time merging was superceded by the introduction of
> > load-time-value?
> 
> Because it would cause the semantics of the language would be violated.

How?
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfw8yvzyxz3.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
> 
> > ···@jpl.nasa.gov (Erann Gat) writes:
> > 
> > > In article <···············@shell01.TheWorld.com>, Kent M Pitman
> > > <······@world.std.com> wrote:
> > > 
> > > > > I don't recall anyone calling for the gutting of the language.  Fixing
> > > > > this problem in a SRFI-style way is easy: just add an
> > > > > implementation-dependent flag,
> > > > > *DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*.  (The default value, of
> > > > > course, should be T ;-)
> > > > 
> > > > That is not a "compatible extension".
> > > > That is a reimplementation.
> > > 
> > > Why?  Didn't you just get through telling me that the main application for
> > > compile-time merging was superceded by the introduction of
> > > load-time-value?

No, the main application for NOT compile time merging was superseded by
LOAD-TIME-VALUE, which is capable of protecting something in compiled code
from being merged.  It is no longer necessary to NOT merge constants.

> > Because it would cause the semantics of the language would be violated.
> 
> How?

I take back what I said.  This would be a compatible extension on the
implementation side; my problem is that it would encourage non-conforming
code.  Exactly because it's an implementation-dependent switch that might
not be implemented (or even implementable) in all Lisps, you'd be encouraging
people to rely on identity issues that might not be portably available.

It would be better if implementations aggressively merged constants so that
you weren't lulled into the false sense of security that you are...
From: Tim Bradshaw
Subject: Re: (eq x x) => false
Date: 
Message-ID: <ey3adggh9qq.fsf@cley.com>
* Erann Gat wrote:

> I don't recall anyone calling for the gutting of the language.  Fixing
> this problem in a SRFI-style way is easy: just add an
> implementation-dependent flag,
> *DONT-ALLOW-FILE-COMPILE-CONSTANT-MERGING*.  (The default value, of
> course, should be T ;-)

I think the problem with this is: if you have an implementation which
does not support this, can you make it do so by something you load on
top of it without that thing being vast?

--tim
From: Coby Beck
Subject: Re: (eq x x) => false
Date: 
Message-ID: <b3n718$imi$1@otis.netspace.net.au>
"Erann Gat" <···@jpl.nasa.gov> wrote in message
·························@k-137-79-50-101.jpl.nasa.gov...
> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
>
> > Without a discussion of (a) the likelihood of occurrence and (b) the
> > necessity of occurrence, you aren't really addressing things.
>
> Agreed.  So here's a scenario:
>
> Programmer 1 writes:
>
> (defun call-a ()
>    (case (baz)
>       (1 'a)
>       (2 'b)
>       (otherwise '(no-result))))
>
> Programmer 2 writes:
>
> (defun call-b ()
>   (case (bing)
>     (1 'a)
>     (2 'b)
>     (otherwise '(no-result))))
>
> Programmer 3 writes:
>
> (if (eq (call-a) (call-b))
>    (proceed-with-dangerous-operation)
>    (go-to-safe-mode))
>
> The idea being that you want to proceed only if the two independently
> calculated results match.
>
> This code is correct if eval'd, potentially incorrect if file-compiled.
>
> Smells like a bear to me.

So take your can of bear repellant (people don't like it either ;) and spray
programmer 1 and 2 in the face then tell them to use 'no-result instead of
'(no-result) and spray programmer 3 in the face and tell him he is no longer
allowed to use EQ again.

>
> > The proper fix
> > will be to allow the language definition to be changed so that merging
> > can occur according to the implementation in all 3 cases.
>
> That would be a good start.

This would now be reliably bearish.

No comment on your general point of view, but you'll need a better example
to be convincing, IMO.

--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwwujkbgfl.fsf@shell01.TheWorld.com>
"Coby Beck" <·····@mercury.bc.ca> writes:

> So take your can of bear repellant (people don't like it either ;) and spray
> programmer 1 and 2 in the face then tell them to use 'no-result instead of
> '(no-result) and spray programmer 3 in the face and tell him he is no longer
> allowed to use EQ again.

Actually, this raises the other point I've made over and over again, and
emphasizes why I advocated the extreme position of maybe not including EQUAL
(and certainly not including EQUALP) in the language.  People spend WAY too
little time thinking hard about the significance of equality predicates.
They assume that the predicates provided are "good for something" and then
they try to come up with uses for them that "seem right", rather than
making a list of conditions that the predicate must satisfy and then checking
that the built-in they choose really can test the things they want.  In the
case here, it's not just the use of EQ, which is admittedly suspect, but
the fact that the example seems to presuppose that 
 (a) The choice of EQ was made without examining the nature of all the
     objects that would be getting compared and
 (b) The choice of the return value was to be made without regard to what
     operations (e.g., comparison) might be needed for it.

Since the compiler has license to merge and split conses, it has
license to destroy any identity that EQ could detect.  As such, either
lists are not an appropriate thing to give EQ or else if EQ is going
to be used, you have to handle the lists much more carefully to assure
the compiler doesn't ever see them as literals.

(defvar *some-eq-capable-marker* (list ...))

and then return *some-eq-capable-marker* rather than returning something 
that is ever available as a literal for merging.  

In most cases, I claim (i.e., as a matter of personal style), you're
better off either returning just symbols (as Coby recommends) so that
EQ can work, or returning just relatively simple atoms (symbols and
numbers) that EQL can reliably handle, or else returning a complex
object that has its own creator and its own comparer. e.g.,
(make-state ...) and (state-equal ...), even if these happen to turn
into LIST and EQUAL.
From: Kenny Tilton
Subject: Re: (eq x x) => false
Date: 
Message-ID: <3E5F851E.1040301@nyc.rr.com>
Kent M Pitman wrote:
> "Coby Beck" <·····@mercury.bc.ca> writes:
> 
> 
>>So take your can of bear repellant (people don't like it either ;) and spray
>>programmer 1 and 2 in the face then tell them to use 'no-result instead of
>>'(no-result) and spray programmer 3 in the face and tell him he is no longer
>>allowed to use EQ again.
> 
> 
> Actually, this raises the other point I've made over and over again, and
> emphasizes why I advocated the extreme position of maybe not including EQUAL
> (and certainly not including EQUALP) in the language.

Sounds like EQ needs to go, too, or be renamed not to have the equality 
connotation. How about:

    (HaveSameObjectID? x x)

:)

Don't export it, either.

-- 

  kenny tilton
  clinisys, inc
  http://www.tilton-technology.com/
  ---------------------------------------------------------------
"Cells let us walk, talk, think, make love and realize
  the bath water is cold." -- Lorraine Lee Cudmore
From: Barry Margolin
Subject: Re: (eq x x) => false
Date: 
Message-ID: <%%L7a.5$271.148@paloalto-snr1.gtei.net>
In article <················@nyc.rr.com>,
Kenny Tilton  <·······@nyc.rr.com> wrote:
>Sounds like EQ needs to go, too, or be renamed not to have the equality 
>connotation. How about:
>
>    (HaveSameObjectID? x x)
>
>:)
>
>Don't export it, either.

If the CL designers felt they had the right to break such long-standing
established practice, we might have done this.  But Common Lisp has a bunch
of undesirable features that have been retained because they are so
thoroughly embedded in the collect Lisp consciousness.

It's the same reason why ANSI C still has functions like gets(), even
though it's well known that it's almost never appropriate to use it.

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Nils Goesche
Subject: Re: (eq x x) => false
Date: 
Message-ID: <lyd6lc73gw.fsf@cartan.de>
Barry Margolin <··············@level3.com> writes:

> In article <················@nyc.rr.com>,
> Kenny Tilton  <·······@nyc.rr.com> wrote:
> >Sounds like EQ needs to go, too, or be renamed not to have the equality 
> >connotation. How about:
> >
> >    (HaveSameObjectID? x x)
> >
> >:)
> >
> >Don't export it, either.
> 
> If the CL designers felt they had the right to break such
> long-standing established practice, we might have done this.  But
> Common Lisp has a bunch of undesirable features that have been
> retained because they are so thoroughly embedded in the collect Lisp
> consciousness.
> 
> It's the same reason why ANSI C still has functions like gets(), even
> though it's well known that it's almost never appropriate to use it.

Uhm, aren't we going a bit far here?  `gets� cannot be used correctly
at all, it is an outright evil function.  But I think there are enough
cases where EQ is... just the right thing to use that I would be very
angry if it wasn't included in the next standard.  Note that the
example in question would be just as broken if it had used EQL instead
of EQ.  We don't want to ban EQL, too now, do we?

Regards,
-- 
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x0655CFA0
From: Kenny Tilton
Subject: Re: (eq x x) => false
Date: 
Message-ID: <3E5FE243.3010803@nyc.rr.com>
Nils Goesche wrote:
>  But I think there are enough
> cases where EQ is... just the right thing to use that I would be very
> angry if it wasn't included in the next standard.

Kent, you gonna take this lying down?! :) No, seriously, never having 
used EQ, this whole thread is fascinating. Kent said (I hope) "eschew 
eq". Where do you find it the Right Thing?

>  Note that the
> example in question would be just as broken if it had used EQL instead
> of EQ.

That speaks ill of the example, not well of EQ.

>  We don't want to ban EQL, too now, do we?

Right you are, out with the lot. We'll drop into in-line assembler for 
equality tests, preferably implementation-dependent.

btw, I did say we could keep EQ, we just have to rename it. I hate bad 
names. EQ seems to leak compiler choices into the implemented language.

In my later years, as my brain VO2max ebbs, I have learned many a 
shortcut to avoid thought. Good example: screw this thread, all i need 
to know is that (eq x x) can => false to know EQ has a bad name.

:)


-- 

  kenny tilton
  clinisys, inc
  http://www.tilton-technology.com/
  ---------------------------------------------------------------
"Cells let us walk, talk, think, make love and realize
  the bath water is cold." -- Lorraine Lee Cudmore
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwfzq8vsjz.fsf@shell01.TheWorld.com>
Kenny Tilton <·······@nyc.rr.com> writes:

> Nils Goesche wrote:
> >  But I think there are enough
> > cases where EQ is... just the right thing to use that I would be very
> > angry if it wasn't included in the next standard.
> 
> Kent, you gonna take this lying down?! :) No, seriously, never having
> used EQ, this whole thread is fascinating. Kent said (I hope) "eschew
> eq". Where do you find it the Right Thing?

Well, this entire discussion would be the same if we used EQL instead of EQ.
(eq x y) and (eql x y) have the same semantics [note I did not say "yield
the same values"] for all x and y both being conses.

I do think it's rare for anyone to need EQ instead of EQL.

On the other hand, I don't think doing an EQL test is routinely wrong.

I DO think that people should be quite careful about using EQL on conses
because the pedigree of the cons has to be checked to make sure it hasn't
been through the address-randomization that occurs with coalescing and 
splitting.   But that is not the fault of EQL, which does a very simple
thing--check for object identity on a Lisp virtual machine (whereas EQ
uses the physical machine, which might differ for numbers and characters,
whose addresses are virtualized by EQL).  The problem isn't that EQL is
doing something unreliable, it's that the arguments being given don't have
a reliable home and so checking them for object identity is silly.
From: Nils Goesche
Subject: Re: (eq x x) => false
Date: 
Message-ID: <87u1eovvw5.fsf@darkstar.cartan>
Kenny Tilton <·······@nyc.rr.com> writes:

> Nils Goesche wrote:
> > But I think there are enough cases where EQ is... just the
> > right thing to use that I would be very angry if it wasn't
> > included in the next standard.
> 
> Kent, you gonna take this lying down?! :) No, seriously, never
> having used EQ, this whole thread is fascinating. Kent said (I
> hope) "eschew eq". Where do you find it the Right Thing?

Actually, I don't think that's what he said, or at least not what
he meant.  It is of course safe to use either EQL or EQ on lists
if you /know/ the data-flow and you /know/ that the compiler is
not going to screw you by conflating list constants or whatever.
If you want an example, here is one:

A few days ago I needed a function that takes a list of lists of
strings (called ``specs��) and will enlarge each list by adding
``abbreviations�� of those strings, but won't introduce
ambigousness, like

CL-USER 8 > (make-abbrevs '(("foo" "bar")
                            ("blar" "zom")))
(("ba" "bar" "f" "fo" "foo") ("z" "zo" "zom" "bl" "bla" "blar"))

Here is how I wrote it (speed was not an issue ;-):

(defun make-abbrevs (specs)
  (let ((string-table (make-hash-table :test #'equal))
        (spec-table (make-hash-table :test #'eq)))
    (dolist (spec specs)
      (dolist (opt spec)
        ;; Put all strings into string-table, remembering the
        ;; specs they are in
        (loop for i from (length opt) downto 1
              for abbr = (subseq opt 0 i) do
              (pushnew spec (gethash abbr string-table) :test #'eq)
              (pushnew abbr (gethash spec spec-table) :test #'string=))))
    (maphash (lambda (abbr specs)
               ;; If a string is in more than one spec, to hell
               ;; with it
               (when (cdr specs)
                 (dolist (spec specs)
                   (setf (gethash spec spec-table)
                         (delete abbr (gethash spec spec-table)
                                 :test #'string=)))))
             string-table)
    (mapcar (lambda (spec)
              (union (gethash spec spec-table) spec :test #'string=))
            specs)))

Now I've used EQ on lists.  Who is going to kill me for that?  I
could have simply omitted the :test #'eq keywords, but I /always/
use EQ in such situations (except when characters or numbers are
involved) and I don't think there is anything wrong with that.

> > Note that the example in question would be just as broken if
> > it had used EQL instead of EQ.
> 
> That speaks ill of the example, not well of EQ.

If people take the example as an argument against EQ, they should
take it as an argument against EQL as well -- and I think neither
Barry nor Mr. Identity Kent Pitman have anything against EQL.

> >  We don't want to ban EQL, too now, do we?
> 
> Right you are, out with the lot. We'll drop into in-line assembler for
> equality tests, preferably implementation-dependent.
> 
> btw, I did say we could keep EQ, we just have to rename it. I
> hate bad names. EQ seems to leak compiler choices into the
> implemented language.

I don't think it's such a bad name.  And the reason EQ exists
besides EQL is that it is faster.  If it wasn't, EQ would have
been removed long ago (and EQL would be known as EQ, most
likely).  Have a look (disassemble 'eq) and (disassemble 'eql)...

> In my later years, as my brain VO2max ebbs, I have learned many
> a shortcut to avoid thought. Good example: screw this thread,
> all i need to know is that (eq x x) can => false to know EQ has
> a bad name.

Using EQL instead of EQ all the time is a fine strategy, at least
for beginners.  But I think at some point every Lisp programmer
should learn about the differences, and when using EQ is safe.
It isn't hard, actually -- simply be aware that using it for
numbers or characters is not a good idea.  This is not /that/
much to remember.  Moreover, if you are not aware of the
difference between EQ and EQL, you'll eventually get screwed when
you use either CATCH, GETF, THROW, GET, REMF, GET-PROPERTIES or
REMPROP...

Regards,
-- 
Nils G�sche
Ask not for whom the <CONTROL-G> tolls.

PGP key ID #xD26EF2A0
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2802031526060001@k-137-79-50-102.jpl.nasa.gov>
It should be noted that EQ vs EQL is a red herring W.R.T. the issue that I
raised.  EQ and EQL are equivalent when applied to lists.

E.
From: Barry Margolin
Subject: Re: (eq x x) => false
Date: 
Message-ID: <ZnS7a.39$271.323@paloalto-snr1.gtei.net>
In article <··············@darkstar.cartan>,
Nils Goesche  <···@cartan.de> wrote:
>Kenny Tilton <·······@nyc.rr.com> writes:
>> btw, I did say we could keep EQ, we just have to rename it. I
>> hate bad names. EQ seems to leak compiler choices into the
>> implemented language.
>
>I don't think it's such a bad name.

The problem with the name is that its terseness suggests simplicity, and is
likely to encourage its use by beginners who don't appreciate the subtle
difference between EQ and EQL.  If you're not sure what you're doing, EQL
is a much safer function to use, because its behavior matches the intuitive
expectations.  And a misuse of EQ can be difficult to discover; since its
behavior on numbers and characters is implementation-dependent, it may work
fine on all the systems you test with, but fail when the application is run
on some implementation the developer didn't have access to.  Giving a
potentially dangerous function a short, catchy name makes it an "attractive
nuisance".

Perhaps rather than two different functions, a better interface would have
been a single function with a keyword option.  The default would be the EQL
behavior, and the keyword would enable the EQ behavior.  However, since
these comparison functions are frequently used with :TEST arguments, it's
convenient to have a name for both of them.  But I still think Kenny had
the right idea in suggesting that EQ should have a name that suggests its
complexity.

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Kenny Tilton
Subject: Re: (eq x x) => false
Date: 
Message-ID: <3E6018B8.9050002@nyc.rr.com>
Nils Goesche wrote:
> Kenny Tilton <·······@nyc.rr.com> writes:
>   And the reason EQ exists
> besides EQL is that it is faster.  If it wasn't, EQ would have
> been removed long ago (and EQL would be known as EQ, most
> likely).  Have a look (disassemble 'eq) and (disassemble 'eql)...

OK. How come eql is not as fast as eq when the operands are eq? I see 
some extra heavy lifting in there regardless. Can't eql just test eq 
first? Anyway, here's my benchmark:

(defparameter *xxx* (cons 1 2))

(defun xxx ()
   (let* ((x (yyy))
          )
     (dotimes (z 10000000)
             (unless (eql x (yyy))
               (print 'whoa)))))

(defun yyy () *xxx*)

(time
  (xxx))

230msec with eq, 320msec with eql, 20msec if the test is (unless t...

So I would lose a second every hundred million times.

OK, from now on I will use EQ on lists. :)

btw, all this talk about "when in the same source file...". amazing. 
amazing that the spec gets into such detail (hoorah), amazing that 
splitting  file in two could introduce a bug.

OK, I'll shut up from the peanut gallery. we now return you to your 
otherwise well-informed thread....


-- 

  kenny tilton
  clinisys, inc
  http://www.tilton-technology.com/
  ---------------------------------------------------------------
"Cells let us walk, talk, think, make love and realize
  the bath water is cold." -- Lorraine Lee Cudmore
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfw4r6nyxn1.fsf@shell01.TheWorld.com>
Kenny Tilton <·······@nyc.rr.com> writes:

> Nils Goesche wrote:
> > Kenny Tilton <·······@nyc.rr.com> writes:
> >   And the reason EQ exists
> > besides EQL is that it is faster.  If it wasn't, EQ would have
> > been removed long ago (and EQL would be known as EQ, most
> > likely).  Have a look (disassemble 'eq) and (disassemble 'eql)...
> 
> OK. How come eql is not as fast as eq when the operands are eq? I see
> some extra heavy lifting in there regardless. Can't eql just test eq
> first?

Sure, but if it fails it may have to do additional work to decode the types
and see if the reason for the failure is that some virtualization of numbers
or characters is required.  EQL is as fast as EQ when the compiler knows
for certain the type of at least one of the arguments (and knows it's not
a character or a number).  e.g.,
 (member x '(a b c))
should be as fast as
 (member x '(a b c) :test #'eq)
because the compiler should notice that the list is all symbols and that
:test #'eq can be inferred in place of #'eql.  But
 (defvar *x* '(a b c))
followed later by
 (member x *x*)
might not be equivalently fast because member uses EQL and because while
it might check EQ first, if x is C, then the first two cases will return
NIL from the EQ test and you'll still have to decode the type to make sure
it's not one of the 'bad types'.  In at least some implementations, the
type codes can be arranged so that EQ never has to do type decoding and
is a simple pointer compare.

> btw, all this talk about "when in the same source
> file...". amazing. amazing that the spec gets into such detail
> (hoorah), amazing that splitting  file in two could introduce a bug.

If you think about what it would take to NOT introduce the bug, you'll
understand both why it has to be that way and why the spec is as detailed
as it is.
From: Gabe Garza
Subject: Re: (eq x x) => false
Date: 
Message-ID: <87r89r7t5z.fsf@ix.netcom.com>
Kenny Tilton <·······@nyc.rr.com> writes:

> Nils Goesche wrote:
> >  But I think there are enough
> > cases where EQ is... just the right thing to use that I would be very
> > angry if it wasn't included in the next standard.
> 
> Kent, you gonna take this lying down?! :) No, seriously, never having
> used EQ, this whole thread is fascinating.


Not even EQ hash tables?  I find that I almost never use the function
EQ (except when I'm comparing symbols), but I use EQ hash tables all
the time--I could also have used EQL hash tables, but it would have
been less efficient and it would have been misleading to me if I read
the code again later (because it would make me think that the code
thought numbers and characters should be handled different then other
objects).

Gabe Garza
From: Kenny Tilton
Subject: Re: (eq x x) => false
Date: 
Message-ID: <3E6059EB.3000100@nyc.rr.com>
Gabe Garza wrote:
> Kenny Tilton <·······@nyc.rr.com> writes:
> 
> 
>>Nils Goesche wrote:
>>
>>> But I think there are enough
>>>cases where EQ is... just the right thing to use that I would be very
>>>angry if it wasn't included in the next standard.
>>
>>Kent, you gonna take this lying down?! :) No, seriously, never having
>>used EQ, this whole thread is fascinating.
> 
> 
> 
> Not even EQ hash tables?  

Hell, I've never used hash tables period. I don't chomp gobs of info ala 
Google. In my apps the lists that need searching are few, and max out 
around five. I search bigger trees, but then tree order matters, 
couldn't flatten that into a hash table.



-- 

  kenny tilton
  clinisys, inc
  http://www.tilton-technology.com/
  ---------------------------------------------------------------
"Cells let us walk, talk, think, make love and realize
  the bath water is cold." -- Lorraine Lee Cudmore
From: Barry Margolin
Subject: Re: (eq x x) => false
Date: 
Message-ID: <PJK8a.6$c55.717@paloalto-snr1.gtei.net>
In article <··············@ix.netcom.com>,
Gabe Garza  <·······@ix.netcom.com> wrote:
>Kenny Tilton <·······@nyc.rr.com> writes:
>
>> Nils Goesche wrote:
>> >  But I think there are enough
>> > cases where EQ is... just the right thing to use that I would be very
>> > angry if it wasn't included in the next standard.
>> 
>> Kent, you gonna take this lying down?! :) No, seriously, never having
>> used EQ, this whole thread is fascinating.
>
>
>Not even EQ hash tables?  I find that I almost never use the function
>EQ (except when I'm comparing symbols), but I use EQ hash tables all
>the time--I could also have used EQL hash tables, but it would have
>been less efficient and it would have been misleading to me if I read
>the code again later (because it would make me think that the code
>thought numbers and characters should be handled different then other
>objects).

EQL hash tables should not be significantly less efficient than EQ hash
tables.  Their hash function may need to check for characters and numbers
being used as the key, but not all implementations need to treat them
differently.  On the other hand, characters and numbers probably don't need
to be rehashed after a GC, so you gain back efficiency there.

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Frode Vatvedt Fjeld
Subject: Re: (eq x x) => false
Date: 
Message-ID: <2hllzz6uq6.fsf@vserver.cs.uit.no>
Kenny Tilton <·······@nyc.rr.com> writes:

> No, seriously, never having used EQ, this whole thread is
> fascinating.

I tend to use/view (eq x y) as shorthand notation for something like

  (eql (the (not (or integer character)) x)
       (the (not (or integer character)) y))

-- 
Frode Vatvedt Fjeld
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfw7kbi27on.fsf@shell01.TheWorld.com>
Frode Vatvedt Fjeld <······@cs.uit.no> writes:

> Kenny Tilton <·······@nyc.rr.com> writes:
> 
> > No, seriously, never having used EQ, this whole thread is
> > fascinating.
> 
> I tend to use/view (eq x y) as shorthand notation for something like
> 
>   (eql (the (not (or integer character)) x)
>        (the (not (or integer character)) y))

Interestingly put.  Mostly I do this, too.

There have been occasional exceptions, mostly pre-dating CL though.
http://groups.google.com/groups?selm=sfwvgi8u2ge.fsf%40world.std.com
From: Nils Goesche
Subject: Re: (eq x x) => false
Date: 
Message-ID: <874r6nxh1x.fsf@darkstar.cartan>
Frode Vatvedt Fjeld <······@cs.uit.no> writes:

> Kenny Tilton <·······@nyc.rr.com> writes:
> 
> > No, seriously, never having used EQ, this whole thread is
> > fascinating.
> 
> I tend to use/view (eq x y) as shorthand notation for something like
> 
>   (eql (the (not (or integer character)) x)
>        (the (not (or integer character)) y))

It might be worth pointing out that it is enough if you know that
/one/ of the arguments if neither integer nor character.  If they
aren't EQ then, they can't be EQL, either.

So, something like

CL-USER 3 > (defun foo-d (x)
              (find 'foo x :test #'eq))
FOO-D

CL-USER 4 > (foo-d '(42 #\B foo))
FOO

is fine, too.

Regards,
-- 
Nils G�sche
Ask not for whom the <CONTROL-G> tolls.

PGP key ID #xD26EF2A0
From: Ray Blaak
Subject: Re: (eq x x) => false
Date: 
Message-ID: <usmu6ex23.fsf@telus.net>
Kenny Tilton <·······@nyc.rr.com> writes:
> Sounds like EQ needs to go, too, or be renamed not to have the equality 
> connotation. How about:
> 
>     (HaveSameObjectID? x x)

identical, identical?, is-identical, identicalp

--
Cheers,                                        The Rhythm is around me,
                                               The Rhythm has control.
Ray Blaak                                      The Rhythm is inside me,
·····@telus.net                                The Rhythm has my soul.
From: Bruce Nagel
Subject: Re: (eq x x) => false
Date: 
Message-ID: <fAd8a.2058$se1.1562235@newssvr28.news.prodigy.com>
On Sat, 01 Mar 2003 18:57:39 GMT, Ray Blaak <·····@telus.net> wrote:
 
> identical, identical?, is-identical, identicalp
 
How about "same", "same-p", etc.?  Not only is identical longer,
it's a lot of syllables if you have to repeat it a lot. :-)

Bruce N.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfw4r6msf2h.fsf@shell01.TheWorld.com>
Bruce Nagel <········@nerg.net> writes:

> On Sat, 01 Mar 2003 18:57:39 GMT, Ray Blaak <·····@telus.net> wrote:
>  
> > identical, identical?, is-identical, identicalp
>  
> How about "same", "same-p", etc.?  Not only is identical longer,
> it's a lot of syllables if you have to repeat it a lot. :-)

The CLHS glossary term "same" means "EQL".
The CLHS glossary term "identical" means "EQ".

It would be unfortunate to choose function names that were in conflict.
From: Paolo Amoroso
Subject: Re: (eq x x) => false
Date: 
Message-ID: <LjRfPosCaEYur4XqKCv4TsxjgrMw@4ax.com>
On Thu, 27 Feb 2003 09:34:02 -0800, ···@jpl.nasa.gov (Erann Gat) wrote:

> Why must all changes to Lisp necessarily be changes to the ANSI standard? 
> Why could the community not adopt something like Scheme's SRFI system?

In a way, this has been done more or less formally for years. Think about
Screamer, Iterate, Generators/Gatherers, Simple Streams, etc. They are
layered standards, you don't have to throw away Common Lisp in order to use
any combination of them.


Paolo
-- 
Paolo Amoroso <·······@mclink.it>
From: Pekka P. Pirinen
Subject: Re: (eq x x) => false
Date: 
Message-ID: <uk7fkbb4d.fsf@globalgraphics.com>
Kent M Pitman <······@world.std.com> writes:
> ···@jpl.nasa.gov (Erann Gat) writes:
> > In article <···············@shell01.TheWorld.com>, Kent M Pitman
> > <······@world.std.com> wrote:
> > >  (2) It was, even when it was necessary for EVAL and COMPILE not to do the
> > >      merging, still true that file compilation couldn't be forced not to do
> > >      this because there was no sharing of object identity (other than the
> > >      notion of "similarity as a constant") over file compilation.
> > 
> > I don't understand this.  Why does it not suffice to use #n= or #.
> > notation in a file where object identity is needed?
> 
> Both of these are read-time actions and the fact of their use is lost by
> the time of compilation.

This is true, but not relevant.  The compiler can quite well see that
the values in the source are EQ.

With all due respect to Kent, while there can't sharing of identity
over file compilation, there can be preservation of identity between
two literals in the code.  Two identical objects in the source are
required to map to two identical objects in the loaded file, even
though they can only be similar to their counterparts in the source.

ANS 3.2.4.4 first paragraph: "If two literal objects appearing in the
source code for a single file processed with the file compiler are the
identical, the corresponding objects in the compiled code must also be
the identical."

So your examples are mistaken:

> The file compiler is still free to make
> 
>  (eq (car '(#1=(foo bar))) (car '(#1#))) => NIL
> 
> It is also free to make:
> 
>  #.(let ((foobar '(foo bar)))
>      `(car '(,foobar) '(,foobar)))
>  => NIL
> 
> [Caveat: It's the middle of the night and I have a cold and I'm just
>  restless.

Those things must evaluate to T (assuming CAR in the latter means EQ).
Usenetters are advised not to post while drunk; this advice should
probably be extended to colds.

This requirement does make compiled files a bit complicated, basically
a lookup table of shared structures has to be used at load time.

It makes some code vastly simpler.  In particular, circular data
structures are externalizable; they won't result in infinite loops at
compile or load time.

I've always regarded literal coalescing as a small but useful
optimization.  Even C compilers coalesce strings.  And it's not hard
to implement: Given that the compiler must build a lookup table for
literals, it basically just matter of using an EQUAL hash table,
instead of an EQ table.
-- 
Pekka P. Pirinen
GOOD ADVICE: Something old men give young men when they can no
longer give them a bad example.  - Gideon Wurdz
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwbs0vx65v.fsf@shell01.TheWorld.com>
···············@globalgraphics.com (Pekka P. Pirinen) writes:

> With all due respect to Kent, while there can't sharing of identity
> over file compilation, there can be preservation of identity between
> two literals in the code.  Two identical objects in the source are
> required to map to two identical objects in the loaded file, even
> though they can only be similar to their counterparts in the source.

I don't think this is true if the dataflow passes through separate files.
Consider the following  [again untested--(he holds his breath and hopes)]

File-1.lisp

(setq *foo-1* (list 'foo))
(setq *foo-2* (list 'foo))

File-2.lisp

(defun foo-1a1 () '#.*foo-1*)
(defun foo-1a2 () '#.*foo-1*)

(defun foo-1a () (eq (foo-1a1) (foo-1a2)))

(defun foo-2a1 () '#.*foo-2*)
(defun foo-2a2 () '#.*foo-2*)

(defun foo-2a () (eq (foo-2a1) (foo-2a2)))


File-3.lisp

(defun foo-1b1 () '#.*foo-1*)
(defun foo-1b2 () '#.*foo-1*)

(defun foo-1b () (eq (foo-1b1) (foo-1b2)))

(defun foo-2b1 () '#.*foo-2*)
(defun foo-2b2 () '#.*foo-2*)

(defun foo-2b () (eq (foo-2b1) (foo-2b2)))

File-4.lisp

(defun foo-1ab1 () (eq (foo-1a1) (foo-1b1)))
(defun foo-2ab1 () (eq (foo-2a1) (foo-2b1)))

So identity might tend to be preserved interactively, but in the file
compiler, foo-1ab1 and foo-2ab1 are going to have "surprising" results
(to those in this conversation who insist on being surprised by the
documented semantics) while foo-1a, foo-2a, foo-1b, and foo-2b are going
to lull those same people into a false sense of security.

If instead one simply adopts the attitude that once they've injected these
things into literals, they should stop using EQ/EQL to find them later,
they'd not run into any problem at all.

> ANS 3.2.4.4 first paragraph: "If two literal objects appearing in the
> source code for a single file processed with the file compiler are the
> identical, the corresponding objects in the compiled code must also be
> the identical."

Right.  But the "in the source code for a single file" is quite a strong
restriction, though, and not one you used in the opening of your discussion.
(I'm replying as I go and I would have replied differently if I knew you
were going to acknowledge the file restriction explicitly. ;)
 
> So your examples are mistaken:
> 
> > The file compiler is still free to make
> > 
> >  (eq (car '(#1=(foo bar))) (car '(#1#))) => NIL
> > 
> > It is also free to make:
> > 
> >  #.(let ((foobar '(foo bar)))
> >      `(car '(,foobar) '(,foobar)))
> >  => NIL
> > 
> > [Caveat: It's the middle of the night and I have a cold and I'm just
> >  restless.
> 
> Those things must evaluate to T (assuming CAR in the latter means EQ).
> Usenetters are advised not to post while drunk; this advice should
> probably be extended to colds.

(awwww...)

I'm not convinced that a multi-file situation can't still surprise you,
although I do agree that the simple cases aren't as dangerous as I was
thinking.  Btw, I'd probably have made this same error when more awake;
I'd forgotten there was this requirement because it turns out not to matter
to me since I simply don't rely on identity of literals at all.

> This requirement does make compiled files a bit complicated, basically
> a lookup table of shared structures has to be used at load time.
> 
> It makes some code vastly simpler.  In particular, circular data
> structures are externalizable;

This is quite an important result, though, yes.

> they won't result in infinite loops at
> compile or load time.
> 
> I've always regarded literal coalescing as a small but useful
> optimization.  Even C compilers coalesce strings.  And it's not hard
> to implement: Given that the compiler must build a lookup table for
> literals, it basically just matter of using an EQUAL hash table,
> instead of an EQ table.

Thanks for chiming in with the clarifications, Pekka.   I've always 
admired and have lately missed your clear thinking analyses of things
like this. ;)
From: Barry Margolin
Subject: Re: (eq x x) => false
Date: 
Message-ID: <Kyr7a.5$PE4.156@paloalto-snr1.gtei.net>
In article <····················@k-137-79-50-102.jpl.nasa.gov>,
Erann Gat <···@jpl.nasa.gov> wrote:
>In article <················@paloalto-snr1.gtei.net>, Barry Margolin
><··············@level3.com> wrote:
>
>> In article <····················@k-137-79-50-101.jpl.nasa.gov>,
>> Erann Gat <···@jpl.nasa.gov> wrote:
>> >In article <······················@dls.net>, "Paul F. Dietz"
>> ><·····@dls.net> wrote:
>> >
>> >> The ANSI spec forbids compile and eval (but not the *file*
>> >> compiler) from merging literal objects.
>> >
>> >Youch!  That means I can't rely on my file-compiled code to do the same
>> >thing as my interactively compiled code.  That's not good.
>> 
>> Since the file compiler doesn't have access to the original list
>> representation, but just a printed representation, it would be pretty
>> difficult to require it to maintain all those relationships.
>
>What "original" list representation?  Most code starts out as text, even
>during interactive development.

But during interactive use, you can manipulate the list structure directly:

(setq some-list (list 1 2 3))
(setq some-other-list (list 1 2 3))
(setf (symbol-function 'fun-1)
      (compile nil `(lambda (x) (nthcdr x ',some-list))))
(setf (symbol-function 'fun-2)
      (compile nil `(lambda (y) (nthcdr y ',some-list))))
(setf (symbol-function 'fun-3)
      (compile nil `(lambda (z) (nthcdr z ',some-other-list))))

You're guaranteed that (eq (fun-1 1) (fun-2 1)) => T because they share the
internal list structure of some-list, while (eq (fun-1 1) (fun-3 1)) => NIL
because they don't.

>But this is a red-herring.  The file compiler can be considered to operate
>on the data structures resulting from processing the file through READ.  I
>don't see why, if EVAL should be forbidden to merge equal constants the
>file compiler should not likewise be forbidden from doing so.  The
>non-uniformity of this constraint can cause serious problems, and I don't
>see that it buys you anything.  Either allow merging everywhere or forbid
>it everywhere.  I don't see any advantage to this inconsistency.

When you're creating the lambda expressions interactively, you have easy,
direct control over when structures are shared and when they're distinct,
as in the example above.  So the in-memory compiler is expected to respect
your decisions.

However, READ always allocates new data structures (unless you make use of
things like #. or #n=/#n#), you're generally not going to have any sharing
among all the places that use similar constants.  Since information like
this is lost when going through printed representations, the file compiler
is given license to merge things back when it's generally safe to share
them.

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwvfz65ywn.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <······················@dls.net>, "Paul F. Dietz"
> <·····@dls.net> wrote:
> 
> > The ANSI spec forbids compile and eval (but not the *file*
> > compiler) from merging literal objects.
> 
> Youch!  That means I can't rely on my file-compiled code to do the same
> thing as my interactively compiled code.

No, this is not true.  It means you can't rely on your file-compiled code
to have the same non-promised behaviors as the non-promised behaviors of
your interatively compiled code has.

To the extent that the language has promised you anything, you can rely 
on it.

> That's not good.

A possible (but not the only possible) point of view.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2602031700440001@k-137-79-50-102.jpl.nasa.gov>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> ···@jpl.nasa.gov (Erann Gat) writes:
> 
> > In article <······················@dls.net>, "Paul F. Dietz"
> > <·····@dls.net> wrote:
> > 
> > > The ANSI spec forbids compile and eval (but not the *file*
> > > compiler) from merging literal objects.
> > 
> > Youch!  That means I can't rely on my file-compiled code to do the same
> > thing as my interactively compiled code.
> 
> No, this is not true.  It means you can't rely on your file-compiled code
> to have the same non-promised behaviors as the non-promised behaviors of
> your interatively compiled code has.

Non-promised is not the same thing as promised-not.  If I promise not to
shoot you in the head you can feel more secure than if I simply refrain
from promising that I will shoot you in the head.

The language has made an affirmative promise to refrain from certain
behavior during interactive development that it no longer promises to
refrain from doing during file compilation.  As a result I might be
relying on this promised refraining without even realizing it -- until I
file-compile and my code suddenly stops working.

E.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwptpenp0k.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
> 
> > ···@jpl.nasa.gov (Erann Gat) writes:
> > 
> > > In article <······················@dls.net>, "Paul F. Dietz"
> > > <·····@dls.net> wrote:
> > > 
> > > > The ANSI spec forbids compile and eval (but not the *file*
> > > > compiler) from merging literal objects.
> > > 
> > > Youch!  That means I can't rely on my file-compiled code to do the same
> > > thing as my interactively compiled code.
> > 
> > No, this is not true.  It means you can't rely on your file-compiled code
> > to have the same non-promised behaviors as the non-promised behaviors of
> > your interatively compiled code has.
> 
> Non-promised is not the same thing as promised-not.  If I promise not to
> shoot you in the head you can feel more secure than if I simply refrain
> from promising that I will shoot you in the head.

But if I promise not to shoot you only in certain named circumstances
and not in others, you'd be silly to to breathe a sigh of relief and
say "Thank goodness Kent doesn't plan to shoot me." since that isn't
what I said.
 
> The language has made an affirmative promise to refrain from certain
> behavior during interactive development that it no longer promises to
> refrain from doing during file compilation.  As a result I might be
> relying on this promised refraining without even realizing it -- until I
> file-compile and my code suddenly stops working.

Right, the language has promised not to cause you problems in some but
not all cases.  This is not logically equivalent to a promise not to do
something that is being laxly treated.  When people make conditional
promises on issues that you care about, I'd pay good attention to the
conditions.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2702030912110001@192.168.1.51>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> ···@jpl.nasa.gov (Erann Gat) writes:
> 
> > In article <···············@shell01.TheWorld.com>, Kent M Pitman
> > <······@world.std.com> wrote:
> > 
> > > ···@jpl.nasa.gov (Erann Gat) writes:
> > > 
> > > > In article <······················@dls.net>, "Paul F. Dietz"
> > > > <·····@dls.net> wrote:
> > > > 
> > > > > The ANSI spec forbids compile and eval (but not the *file*
> > > > > compiler) from merging literal objects.
> > > > 
> > > > Youch!  That means I can't rely on my file-compiled code to do the same
> > > > thing as my interactively compiled code.
> > > 
> > > No, this is not true.  It means you can't rely on your file-compiled code
> > > to have the same non-promised behaviors as the non-promised behaviors of
> > > your interatively compiled code has.
> > 
> > Non-promised is not the same thing as promised-not.  If I promise not to
> > shoot you in the head you can feel more secure than if I simply refrain
> > from promising that I will shoot you in the head.
> 
> But if I promise not to shoot you only in certain named circumstances
> and not in others, you'd be silly to to breathe a sigh of relief and
> say "Thank goodness Kent doesn't plan to shoot me." since that isn't
> what I said.

Yes.  Do you not understand that that is precisely the point I am trying
to make?  (There seems to be a basic mismatch of premises here, but I
can't quite figure out what it is.)

> > The language has made an affirmative promise to refrain from certain
> > behavior during interactive development that it no longer promises to
> > refrain from doing during file compilation.  As a result I might be
> > relying on this promised refraining without even realizing it -- until I
> > file-compile and my code suddenly stops working.
> 
> Right, the language has promised not to cause you problems in some but
> not all cases.  This is not logically equivalent to a promise not to do
> something that is being laxly treated.  When people make conditional
> promises on issues that you care about, I'd pay good attention to the
> conditions.

Yes, I understand that.  My point is that this particular conditional
promise has negative value IMO.

E.
From: Tim Bradshaw
Subject: Re: (eq x x) => false
Date: 
Message-ID: <ey3znoi3i16.fsf@cley.com>
* Erann Gat wrote:

> Youch!  That means I can't rely on my file-compiled code to do the same
> thing as my interactively compiled code.  That's not good.

Well, it only means that if you have code which relies on EQness of
textually different literals, or destructively modifies literals.
Assuming you don't do the latter anyway, how much of the former do you
do, and why?

--tim
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2602031528580001@k-137-79-50-102.jpl.nasa.gov>
In article <···············@cley.com>, Tim Bradshaw <···@cley.com> wrote:

> * Erann Gat wrote:
> 
> > Youch!  That means I can't rely on my file-compiled code to do the same
> > thing as my interactively compiled code.  That's not good.
> 
> Well, it only means that if you have code which relies on EQness of
> textually different literals, or destructively modifies literals.

Your conditional is missing a "then" part, but I think I take your
meaning.  The problem is not relying on the EQness of textually different
literals, it's relying on the NON-EQness of textually different literals. 
Non-file compiled code can rely on this property.  File-compiled code
can't.

> Assuming you don't do the latter anyway, how much of the former do you
> do, and why?

Relying on the non-eqness of textually different literals is easy to do
without realizing that you're doing it.  So you write your code
interactively, test it, verify that it works.  Now if you file-compile
that same code it can break.   That's a bad property of a programming
system IMO.  It pretty much totally undermines the advantages of
incremental development as far as I'm concerned.

E.
From: Paul F. Dietz
Subject: Re: (eq x x) => false
Date: 
Message-ID: <lp-cnQC76ZgaxcCjXTWcoQ@dls.net>
Erann Gat wrote:

> Your conditional is missing a "then" part, but I think I take your
> meaning.  The problem is not relying on the EQness of textually different
> literals, it's relying on the NON-EQness of textually different literals. 
> Non-file compiled code can rely on this property.  File-compiled code
> can't.

It can ensure non-EQLness if it uses load-time-value with the appropriate
copying form:

    If two lists (load-time-value form) that are the same under equal
    but are not identical are evaluated or compiled, their values always
    come from distinct evaluations of form. Their values may not be coalesced
    unless read-only-p is t.

	Paul
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwn0ki5y9c.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> > Assuming you don't do the latter anyway, how much of the former do you
> > do, and why?
> 
> Relying on the non-eqness of textually different literals is easy to do
> without realizing that you're doing it.  So you write your code
> interactively, test it, verify that it works.

"test it" is not the same as "verify that it works".

Dear God, I hope NASA uses a higher standard of care than that.

Verifying means checking against the language semantics, not just checking
for felicitous stuff.  Surely some language semantics make it easier to 
check, but there are myriad things you cannot check in the way you suggest,
so you STILL have to do the other kind of checking (inspecting code, 
understanding stuff what test suites will miss and testing those things
in other ways) even if this were fixed.

If it takes this kind of bug to make you realize you need a different kind
of testing, then it is well worth having just for that reason.

I recall a situation in an X3J13 meeting long after we had broken
"compatibility" with CLTL2 (even though formally we had never declared
any allegiance to it) and someone objected to a name change saying "if
you change that, I can't use CLTL2 as a reference any more".  Steele
and I both looked puzzled--the language already doesn't mean what
CLTL2 says, we each explained.  Yes, said the person, but I can
pretend it does... until you change those names.  (I believe these
were the DEFINE-SETF-METHOD, GET-SETF-METHOD, etc. set of functions.)
Steele and I, and probably others present, jumped on this as being as
good a reason as any for changing the names--to make sure that people
realized things had changed ...

> Now if you file-compile
> that same code it can break.   That's a bad property of a programming
> system IMO.  It pretty much totally undermines the advantages of
> incremental development as far as I'm concerned.

I don't agree.  There are numerous other differences that are essential
and cannot be handwaved away.  You simply cannot assume that interactive
compilation is a proper test of file compilation for myriad other reasons.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2602031734090001@k-137-79-50-102.jpl.nasa.gov>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> ···@jpl.nasa.gov (Erann Gat) writes:
> 
> > > Assuming you don't do the latter anyway, how much of the former do you
> > > do, and why?
> > 
> > Relying on the non-eqness of textually different literals is easy to do
> > without realizing that you're doing it.  So you write your code
> > interactively, test it, verify that it works.
> 
> "test it" is not the same as "verify that it works".

That's right.  That's why I listed them separately.

> Dear God, I hope NASA uses a higher standard of care than that.

If you saw what actually goes on here you'd be appalled. 

> Verifying means checking against the language semantics, not just checking
> for felicitous stuff.  Surely some language semantics make it easier to 
> check, but there are myriad things you cannot check in the way you suggest,
> so you STILL have to do the other kind of checking (inspecting code, 
> understanding stuff what test suites will miss and testing those things
> in other ways) even if this were fixed.

The quantity of things you have to check is significant.  If it weren't,
then one may just as well code in C++.

> If it takes this kind of bug to make you realize you need a different kind
> of testing, then it is well worth having just for that reason.

Take this position to an extreme and you get back to the bad old days
where interpreted code used dynamic binding by default.

> I recall a situation in an X3J13 meeting long after we had broken
> "compatibility" with CLTL2 (even though formally we had never declared
> any allegiance to it) and someone objected to a name change saying "if
> you change that, I can't use CLTL2 as a reference any more".  Steele
> and I both looked puzzled--the language already doesn't mean what
> CLTL2 says, we each explained.  Yes, said the person, but I can
> pretend it does... until you change those names.  (I believe these
> were the DEFINE-SETF-METHOD, GET-SETF-METHOD, etc. set of functions.)
> Steele and I, and probably others present, jumped on this as being as
> good a reason as any for changing the names--to make sure that people
> realized things had changed ...

That is not an analogous situation at all.  In fact, this anecdote seems
to me to support my position.  You advocated a name change precisely so
that the real difference would be in people's faces and not subtle.  But
this difference (between eval/compile behavior and file-compile behavior)
is subtle.  I would have far fewer problems with the situation if, e.g.
the file compiler was required to issue a warning if it merged constants.

> > Now if you file-compile
> > that same code it can break.   That's a bad property of a programming
> > system IMO.  It pretty much totally undermines the advantages of
> > incremental development as far as I'm concerned.
> 
> I don't agree.  There are numerous other differences that are essential
> and cannot be handwaved away.  You simply cannot assume that interactive
> compilation is a proper test of file compilation for myriad other reasons.

If they are truly essential then I have no problem with them.  I do have a
problem with a language feature that serves no useful purpose
(notwithstanding that it may at one time have done so) and is potentially
dangerous.

E.
From: Tim Bradshaw
Subject: Re: (eq x x) => false
Date: 
Message-ID: <ey3bs0y2b8k.fsf@cley.com>
* Erann Gat wrote:
> Your conditional is missing a "then" part, but I think I take your
> meaning.  The problem is not relying on the EQness of textually different
> literals, it's relying on the NON-EQness of textually different literals. 
> Non-file compiled code can rely on this property.  File-compiled code
> can't.

Yes, sorry for the crap wording.  I guess that what I mean is that
it's not safe ever to rely on textually distinct (non-symbol) literals
to be either EQ not EQ, sometimes EQ, or anything, as well as not
being safe to modify literals and so on.

Perhaps your programming style is very different than mine, but I
don't find the equality aspects harder to enforce than the
modification aspects - any function which might want to modify an
argument already has to worry if that argument came from some literal
somewhere.  I've been screwed by that kind of problem many more times
than I've been screwed by possible equality of literals.

It would be nice (and perhaps not that implementationally hard) if
implementations had a `LITERALP' predicate - presumably an
implementation could do this by allocating literals in some special
region (possibly only with some compilation options), and then
LITERALP would simply be an address check.

--tim
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2702030941280001@192.168.1.51>
In article <···············@cley.com>, Tim Bradshaw <···@cley.com> wrote:

> * Erann Gat wrote:
> > Your conditional is missing a "then" part, but I think I take your
> > meaning.  The problem is not relying on the EQness of textually different
> > literals, it's relying on the NON-EQness of textually different literals. 
> > Non-file compiled code can rely on this property.  File-compiled code
> > can't.
> 
> Yes, sorry for the crap wording.

Actually, I think it was my crap reading that was at fault.  (Thanks to
Jeremy Yallop for pointing this out.)

>  I guess that what I mean is that
> it's not safe ever to rely on textually distinct (non-symbol) literals
> to be either EQ not EQ, sometimes EQ, or anything, as well as not
> being safe to modify literals and so on.

But that's exactly the problem: it is safe to assume (per the standard),
as long as you're not file-compiling.

> Perhaps your programming style is very different than mine,

Individual programming styles are not the issue here.  The days when one
person can write a significant software system by himself (even in Lisp)
are pretty much over.  What matters is: what invariants can you rely on
obtaining from your development process, and what is left as a burden on
the develoment team?  The issue at hand is that an invariant that ought to
be automatically guaranteed by the development process is instead left as
a burden on the developers.  That's bad.  In fact, it is precisely this
sort of problem that is the reason that C++ sucks so badly: it leaves too
high a burden on the developer.  There is nothing you can do in Lisp that
you can't do in C++, it's just harder.  Likewise, there is nothing that
this particular flaw prevents you from doing, it just makes it harder. 
And if what you care about is reliability it makes it much, much harder.

E.
From: Barry Margolin
Subject: Re: (eq x x) => false
Date: 
Message-ID: <1Er7a.6$PE4.139@paloalto-snr1.gtei.net>
In article <····················@k-137-79-50-102.jpl.nasa.gov>,
Erann Gat <···@jpl.nasa.gov> wrote:
>
>In article <···············@cley.com>, Tim Bradshaw <···@cley.com> wrote:
>
>> * Erann Gat wrote:
>> 
>> > Youch!  That means I can't rely on my file-compiled code to do the same
>> > thing as my interactively compiled code.  That's not good.
>> 
>> Well, it only means that if you have code which relies on EQness of
>> textually different literals, or destructively modifies literals.
>
>Your conditional is missing a "then" part, but I think I take your
>meaning.

There's no clause missing.  The sentence should be parsed as "Well, if you
have code which relies on EQness of textually different literals THEN it
only means that."  The antecedent of "that" is "you can't rely on your
file-compiled code to do the same thing as your interactively compiled
code."

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Tim Bradshaw
Subject: Re: (eq x x) => false
Date: 
Message-ID: <ey31y1tski6.fsf@cley.com>
* Barry Margolin wrote:
> There's no clause missing.  The sentence should be parsed as "Well, if you
> have code which relies on EQness of textually different literals THEN it
> only means that."  The antecedent of "that" is "you can't rely on your
> file-compiled code to do the same thing as your interactively compiled
> code."

No, there isn't, is there?  I think I should have done something to
make it clearer though, maybe a comma after `that', or maybe just
better wording.

--tim
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwznoi5z0d.fsf@shell01.TheWorld.com>
"Paul F. Dietz" <·····@dls.net> writes:

> Rob Warnock wrote:
> > Paul F. Dietz <·····@dls.net> wrote:
> > +---------------
> > | Rob Warnock wrote:
> > | > You think *that's* fun, then try the following, entirely within CMUCL:
> > | > 	> (let ()
> > | > 	    (eq '(a . b) '(a . b)))
> > | > | > 	T
> > | | This behavior of the compiler is a known ANSI bug, and has
> > (IIRC) been
> > | fixed by the cmucl developers in cvs.  Wait for 18e.
> > +---------------
> > Do you mean because ANSI CL *forbids* such merging?
> 
> 
> The ANSI spec forbids compile and eval (but not the *file*
> compiler) from merging literal objects.  It must maintain EQL-ness.
> This form:
> 
>    (funcall (compile nil (let ((x (cons 'a 'b)) (y (cons 'a 'b)))
> 				`(lambda () (eq ',x ',y)))))
> 
> must return NIL.  Section 3.2.4:
> 
>     The functions eval and compile are required to ensure that
>     literal objects referenced within the resulting interpreted
>     or compiled code objects are the same as the corresponding
>     objects in the source code.
> 
> where 'same' is defined in the glossary:
> 
>     2. (of objects if no predicate is implied by context) indistinguishable
>     by eql.
> 
> 	Paul

FWIW, I can explain why this was essential.

It's actually a lingering requirement from before the introduction of
LOAD-TIME-VALUE that we incompletely cleaned up when we added 
LOAD-TIME-VALUE, which offers the right way to assure this behavior.

If you imagine writing something like CLOS, where you need the ability
to have slot accesses turn into efficient accesses to a slot indirect
through an index gotten from a constant that can only be known at load
time, then you realize that you're going to end up with these references
turning into (AREF '#(...) 37) etc. where the thing that is quoted is
a table that must be allowed to dynamically change as more classes
are loaded or as classes are redefined.  It was CRITICAL that there be
a way of getting the equivalent of an access to a constant but where the
constant was only known at load-time.  Once we added LOAD-TIME-VALUE
we could use that, but prior to that we had to use macros that were
expanded and then compiled at runtime in order to get the same effect,
and if the compiler went around substituting a fixed table for the one
that would be the central registry, that was a disaster.

So that's why the language is quite clear on this point (because it was
once very essential) and yet it seems so superfluous now (because at the
last minute an operator was introduced that deals with this way more
cleanly).  The whole issue of the T/NIL second arg to LOAD-TIME-VALUE
allows the user to say which behavior is wanted.
From: Erann Gat
Subject: Re: (eq x x) => false
Date: 
Message-ID: <gat-2602032020230001@192.168.1.51>
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:

> FWIW, I can explain why this was essential.
> 
> It's actually a lingering requirement from before the introduction of
> LOAD-TIME-VALUE that we incompletely cleaned up when we added 
> LOAD-TIME-VALUE, which offers the right way to assure this behavior.
> 
> If you imagine writing something like CLOS, where you need the ability
> to have slot accesses turn into efficient accesses to a slot indirect
> through an index gotten from a constant that can only be known at load
> time, then you realize that you're going to end up with these references
> turning into (AREF '#(...) 37) etc. where the thing that is quoted is
> a table that must be allowed to dynamically change as more classes
> are loaded or as classes are redefined.  It was CRITICAL that there be
> a way of getting the equivalent of an access to a constant but where the
> constant was only known at load-time.  Once we added LOAD-TIME-VALUE
> we could use that, but prior to that we had to use macros that were
> expanded and then compiled at runtime in order to get the same effect,
> and if the compiler went around substituting a fixed table for the one
> that would be the central registry, that was a disaster.
> 
> So that's why the language is quite clear on this point (because it was
> once very essential) and yet it seems so superfluous now (because at the
> last minute an operator was introduced that deals with this way more
> cleanly).  The whole issue of the T/NIL second arg to LOAD-TIME-VALUE
> allows the user to say which behavior is wanted.

This explains why the file compiler is allowed to merge constants, but it
does not seem to explain why eval and compile are forbidden from doing so.

E.
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwheaqnnrd.fsf@shell01.TheWorld.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
> 
> > FWIW, I can explain why this was essential.
> > 
> > It's actually a lingering requirement from before the introduction of
> > LOAD-TIME-VALUE that we incompletely cleaned up when we added 
> > LOAD-TIME-VALUE, which offers the right way to assure this behavior.
> > 
> > If you imagine writing something like CLOS, where you need the ability
> > to have slot accesses turn into efficient accesses to a slot indirect
> > through an index gotten from a constant that can only be known at load
> > time, then you realize that you're going to end up with these references
> > turning into (AREF '#(...) 37) etc. where the thing that is quoted is
> > a table that must be allowed to dynamically change as more classes
> > are loaded or as classes are redefined.  It was CRITICAL that there be
> > a way of getting the equivalent of an access to a constant but where the
> > constant was only known at load-time.  Once we added LOAD-TIME-VALUE
> > we could use that, but prior to that we had to use macros that were
> > expanded and then compiled at runtime in order to get the same effect,
> > and if the compiler went around substituting a fixed table for the one
> > that would be the central registry, that was a disaster.
> > 
> > So that's why the language is quite clear on this point (because it was
> > once very essential) and yet it seems so superfluous now (because at the
> > last minute an operator was introduced that deals with this way more
> > cleanly).  The whole issue of the T/NIL second arg to LOAD-TIME-VALUE
> > allows the user to say which behavior is wanted.
> 
> This explains why the file compiler is allowed to merge constants, but it
> does not seem to explain why eval and compile are forbidden from doing so.

Because absent LOAD-TIME-VALUE there was otherwise no way [see my other
mail explaining why #= and #. occur at the wrong time] to ensure that a 
particular object with any identity at all ended up in compiled code.
And it was critical to the efficiency of things like CLOS that you be able
to do (AREF '#<... array with definite identity not merged nor split...> n)
Having to do (AREF *special-variable-reference* n) was too slow and having
to do (AREF +constant-reference+ n) opened you again to constant folding.
In Maclisp days, you used the undocumented SQUID [self-quoting internal 
datum] facility in the compiler, and in Zetalisp or CLTL1 you used #, 
but it turned out those were ill-specified and often wrongly implemented
[certainly "differently implemented" among implementations, which would
have caused you equal strife in your interactive testing], so we removed
those.  For a while, the language definition had nothing that would 
substitute other than this restriction--the ability to assure that code could
be compiled at load time.
 
From: Jochen Schmidt
Subject: Re: (eq x x) => false
Date: 
Message-ID: <b3fcej$9mg$02$1@news.t-online.com>
Adam Warner wrote:


> So (let ((x "Foo")) (eq x x)) is always true simply because
> implementations are not allowed to copy strings at any time (even though a
> case could probably be made for "tremendous performance improvements" if
> this was allowed).

Strings are mutable datastructures - so you cannot simply allow to make 
copies at any time.

ciao,
Jochen
From: Adam Warner
Subject: Re: (eq x x) => false
Date: 
Message-ID: <pan.2003.02.25.09.31.58.130391@consulting.net.nz>
Hi Jochen Schmidt,

>> So (let ((x "Foo")) (eq x x)) is always true simply because
>> implementations are not allowed to copy strings at any time (even
>> though a case could probably be made for "tremendous performance
>> improvements" if this was allowed).
> 
> Strings are mutable datastructures - so you cannot simply allow to make
> copies at any time.

As I understand it literals are not mutable data structures. "Foo" is a
string literal. (copy-seq "Foo") is a mutable data structure.

Regards,
Adam
From: Jochen Schmidt
Subject: Re: (eq x x) => false
Date: 
Message-ID: <b3ff10$uc$00$1@news.t-online.com>
Adam Warner wrote:

> Hi Jochen Schmidt,
> 
>>> So (let ((x "Foo")) (eq x x)) is always true simply because
>>> implementations are not allowed to copy strings at any time (even
>>> though a case could probably be made for "tremendous performance
>>> improvements" if this was allowed).
>> 
>> Strings are mutable datastructures - so you cannot simply allow to make
>> copies at any time.
> 
> As I understand it literals are not mutable data structures. "Foo" is a
> string literal. (copy-seq "Foo") is a mutable data structure.

You have to distinquish between datastructures you simply cannot mutate or 
for which the consequences of mutations are undefined. To say it in other 
words - ANSI CL left it undefined to specify any mechanisms which would 
prevent you from mutating literals of objects which are mutable when they 
are not literals. On the other side there are simply no means specified to 
mutate a number or a character object - so they are really immutable.

ciao,
Jochen
From: Barry Margolin
Subject: Re: (eq x x) => false
Date: 
Message-ID: <W8O6a.15$tk4.68@paloalto-snr1.gtei.net>
In article <······························@consulting.net.nz>,
Adam Warner <······@consulting.net.nz> wrote:
>So (let ((x "Foo")) (eq x x)) is always true simply because
>implementations are not allowed to copy strings at any time (even though a
>case could probably be made for "tremendous performance improvements" if
>this was allowed).

I find it difficult to imagine an architecture where copying strings would
result in much speed improvement.  Unless the string is real short, such
that the copy can be done in 1 or 2 bus cycles, the overhead of copying is
likely to outweigh any subsequent improvement in accessing the string.

-- 
Barry Margolin, ··············@level3.com
Genuity Managed Services, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Frode Vatvedt Fjeld
Subject: Re: (eq x x) => false
Date: 
Message-ID: <2hptpg83m3.fsf@vserver.cs.uit.no>
Barry Margolin <··············@level3.com> writes:

> I find it difficult to imagine an architecture where copying strings
> would result in much speed improvement.  Unless the string is real
> short, such that the copy can be done in 1 or 2 bus cycles, the
> overhead of copying is likely to outweigh any subsequent improvement
> in accessing the string.

For a 64-bit architecture it would conceivably be feasible to
implement simple-base-strings shorter than about 8 as immediate
values.

For example:

 (loop for s being the external-symbols of :common-lisp
     count (>= 8 (length (symbol-name s))))

 => 403

..which is in the neighborhood of 50% of those symbols.

-- 
Frode Vatvedt Fjeld
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfw1y1wm0bs.fsf@shell01.TheWorld.com>
Frode Vatvedt Fjeld <······@cs.uit.no> writes:

> Barry Margolin <··············@level3.com> writes:
> 
> > I find it difficult to imagine an architecture where copying strings
> > would result in much speed improvement.  Unless the string is real
> > short, such that the copy can be done in 1 or 2 bus cycles, the
> > overhead of copying is likely to outweigh any subsequent improvement
> > in accessing the string.
> 
> For a 64-bit architecture it would conceivably be feasible to
> implement simple-base-strings shorter than about 8 as immediate
> values.
> 
> For example:
> 
>  (loop for s being the external-symbols of :common-lisp
>      count (>= 8 (length (symbol-name s))))
> 
>  => 403
> 
> ..which is in the neighborhood of 50% of those symbols.

But is there benefit to doing this?  The space savings is tiny. The
issue is speed.  Are there machine instructions on such a specialized
representation which would be fast enough to make it worthwhile?

In the case of numbers, there are a lot of primitive operations on
floats and fixnums that are worth getting to.  An implementation can
choose to make these classes, not just types, and then leverage the
fact that users often are willing to type-declare fixnums and floats.
The same is not true of short-named symbols.  No one declares them.
There's not even a way to declare them.  Therefore, they would have to
be runtime discriminated between long symbols and short ones, and
already that would take long enough as to make the so-called speedy
operation questionable--dispatching a declared symbol that had uniform
representation to a shared subroutine that had the speedy operation in
it would probably be faster than decoding the symbol to determine if
it could use the speedy code or would have to jump to the unspeedy
code.
From: Frode Vatvedt Fjeld
Subject: Re: (eq x x) => false
Date: 
Message-ID: <2hlm047xl5.fsf@vserver.cs.uit.no>
Kent M Pitman <······@world.std.com> writes:

> But is there benefit to doing this?

No, I consider that highly unlikely too. I wasn't being entirely
serious.

> There's not even a way to declare them.

Oh no? How about this?

(deftype fixstring ()
  `(or ,@(loop for i from 0 to 8 collect `(simple-base-string ,i))))

:-)

-- 
Frode Vatvedt Fjeld
From: Kent M Pitman
Subject: Re: (eq x x) => false
Date: 
Message-ID: <sfwbs0zrvv4.fsf@shell01.TheWorld.com>
Frode Vatvedt Fjeld <······@cs.uit.no> writes:

> Kent M Pitman <······@world.std.com> writes:
> 
> > But is there benefit to doing this?
> 
> No, I consider that highly unlikely too. I wasn't being entirely
> serious.
> 
> > There's not even a way to declare them.
> 
> Oh no? How about this?
> 
> (deftype fixstring ()
>   `(or ,@(loop for i from 0 to 8 collect `(simple-base-string ,i))))
> 
> :-)

Oh, sure, for strings.  But you'd referred to symbols, and so I was
referring to there being no way to declare symbols (that a compiler
was likely to understand).  Your example talked about the external 
symbols of COMMON-LISP.

Surely you can use SATISFIES, but that's likely to be opaque to compilers
unless they're just doing haphazard idiom-matching.

Incidentally, another aspect of this is that the declaration is
likewise probably useless if it does not force an implementation; that
is, in the case of strings, there is a likelihood that there will be
more than one way to represent strings.  If SIMPLE-BASE-STRING does
not force an implementation, it becomes much less useful as a compiler
declaration.  So if the representation chosen for (SIMPLE-BASE-STRING
3) is very different than that of (SIMPLE-BASE-STRING 13), then you're
making the declaration SIMPLE-BASE-STRING less efficient...  or so it
seems to me.  (I don't do compiler optimizations for a living--I just
make off-the-wall guesses about what it would be like to do that based
on my intuitions.  So once in a while I get this kind of thing way
wrong.  I rely on people like Duane to correct me in cases like
this if I start spewing falsehoods. ;)
From: Alexander Kjeldaas
Subject: Re: (eq x x) => false
Date: 
Message-ID: <b3idmn$26t$1@localhost.localdomain>
Frode Vatvedt Fjeld wrote:

> Barry Margolin <··············@level3.com> writes:
> 
>> I find it difficult to imagine an architecture where copying strings
>> would result in much speed improvement.  Unless the string is real
>> short, such that the copy can be done in 1 or 2 bus cycles, the
>> overhead of copying is likely to outweigh any subsequent improvement
>> in accessing the string.
> 
> For a 64-bit architecture it would conceivably be feasible to
> implement simple-base-strings shorter than about 8 as immediate
> values.
> 
> For example:
> 
>  (loop for s being the external-symbols of :common-lisp
>      count (>= 8 (length (symbol-name s))))
> 
>  => 403
> 
> ..which is in the neighborhood of 50% of those symbols.
> 

Theoretically a compiler might want to bring structs and strings of up to 16
bytes into registers on x86 using sse instructions.  These registers are
128 bit long and you have all 8 of them free to your disposal.

astor
From: Coby Beck
Subject: Re: (eq x x) => false
Date: 
Message-ID: <b3f94g$2tcs$1@otis.netspace.net.au>
"Adam Warner" <······@consulting.net.nz> wrote in message
···································@consulting.net.nz...
> Hi all,
>
> Has any implementation of Lisp every existed where (eq x x) is false? (x
> is a symbol name for a variable containing any contents whatsoever)
>
> The HyperSpec contains this example:
>
> (let ((x 5)) (eq x x))
> =>  true
> OR=>  false

You could add to this
(let ((x #\5)) (eq x x))
=>  true
OR=>  false

>
> I can't understand why anyone would implement a Lisp that doesn't return
> the same generalised reference for an object with the same symbol name.
>
> To compound the strangeness the HyperSpec also contains this example:
>
> (let ((x "Foo")) (eq x x)) =>  true
>
> How the heck can this be guaranteed to be true but the example above
> cannot?

Though you answered your own question, maybe this is a good reason to not
use EQ unless you really mean it (and I can't think of a good reason ATM)

--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
From: Adam Warner
Subject: Re: (eq x x) => false
Date: 
Message-ID: <pan.2003.02.25.08.35.08.332044@consulting.net.nz>
Hi Coby Beck,

> Though you answered your own question, maybe this is a good reason to
> not use EQ unless you really mean it (and I can't think of a good reason
> ATM)

I think a rule of thumb that one should use EQ for nothing else but
checking symbol identity is fine, e.g. (eq x nil), (eq x 'misc).

Regards,
Adam
From: Nils Goesche
Subject: Re: (eq x x) => false
Date: 
Message-ID: <lyr89wa60z.fsf@cartan.de>
"Adam Warner" <······@consulting.net.nz> writes:

> > Though you answered your own question, maybe this is a good reason
> > to not use EQ unless you really mean it (and I can't think of a
> > good reason ATM)
> 
> I think a rule of thumb that one should use EQ for nothing else but
> checking symbol identity is fine, e.g. (eq x nil), (eq x 'misc).

Or for everything except for numbers and characters ;-) Well, we
/need/ a way to test for an object's identity in everyday programming.
And using EQL works just fine for that, but if we keep in mind not to
use it for numbers and characters, EQ works just as well and is
faster.

Regards,
-- 
Nils G�sche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x0655CFA0