hi,
In a certain lisp-library one can find code of the form:
(defconstant +eof+ (cons nil nil))
This "works" in most implementations apparently but did not work when
I tried SBCL yesterday. The file it is defined in is part of a system
and before use it is compiled and then loaded. So I looked at
DEFCONSTANT in the HyperSpec:
"""
If a defconstant form appears as a top level form, the compiler must
recognize that name names a constant variable. An implementation may
choose to evaluate the value-form at compile time, load time, or
both. Therefore, users must ensure that the initial-value can be
evaluated at compile time (regardless of whether or not references to
name appear in the file) and that it always evaluates to the same
value.
"""
So what is the "same value"? Earlier on the page it says:
"""
The consequences are undefined if there are any bindings of the
variable named by name at the time defconstant is executed or if the
value is not eql to the value of initial-value.
"""
So my understanding is that the value should be EQL even if it is
evaluated on compile-time and load-time, which it probably is. EQL
says this about CONS-equality:
"""
(eql (cons 'a 'b) (cons 'a 'b)) => false
"""
So my impression from reading DEFCONSTANT and EQL is that:
(defconstant +eof+ (cons nil nil))
is not 100% kosher to what is specified in the HyperSpec
for portable behaviour. Is my musings on this topic in accordance
with other people's interpretation and the code in question should be
changed to something more portable?
--
------------------------------------------------------------------
Stig Erik Sandoe ····@ii.uib.no http://www.ii.uib.no/~stig/
····@ii.uib.no (Stig E. Sandoe) writes:
> So my impression from reading DEFCONSTANT and EQL is that:
> (defconstant +eof+ (cons nil nil))
> is not 100% kosher to what is specified in the HyperSpec
> for portable behaviour. Is my musings on this topic in accordance
> with other people's interpretation and the code in question should be
> changed to something more portable?
I agree. Do you think
(defconstant +eof+ (load-time-value (cons nil nil) t))
is a portable way to do this?
And would the now missing #, have done the same?
--
Lieven Marchand <···@wyrd.be>
Making laws appears to win votes. Enforcing them doesn't.
See Rule One. Roger Burton West in the monastery.
Lieven Marchand <···@wyrd.be> writes:
> ····@ii.uib.no (Stig E. Sandoe) writes:
>
> > So my impression from reading DEFCONSTANT and EQL is that:
> > (defconstant +eof+ (cons nil nil))
> > is not 100% kosher to what is specified in the HyperSpec
> > for portable behaviour. Is my musings on this topic in accordance
> > with other people's interpretation and the code in question should be
> > changed to something more portable?
>
> I agree. Do you think
>
> (defconstant +eof+ (load-time-value (cons nil nil) t))
>
> is a portable way to do this?
Hmm. Something looks odd about that. I normally only define constants
(rather than mutable variables) if the compiler is going to make assumptions
about the values. There is really no difference between that and
defparameter of the same value, other than the restriction on assigning it,
which is a bit pointless if the compiler hasn't built in any dependency.
> And would the now missing #, have done the same?
No. #, was just basically a slightly-more-general, slightly-less-well-defined
variant of load-time-value.
The problems with #, were two, one real and one imagined:
(a) A lot of people couldn't understand why it was different than #.
That was mostly only an imagined problem, but the world was happier
when there was only one.
(b) For #, we neglected to say if the result self-quoted or required quoting
or was required to occur only in quoted structure. In Maclisp, where it
came from, it was self-quoting. That is, it could be used in data
and would appear as self, as in: '(#,(+ 1 1) 4) which was the same as
'(2 4) by execution time. But it could also in Maclisp be used like:
(list #,(list '+ '3 '4) 7) yielded ((+ 3 4) 7) and not, as would happen
with #.,(7 7). There was a big problem about whether you had to insert
a quotation mark or not to keep it from evaluating there. By making it
a form, as in LOAD-TIME-VALUE, it became clear when the evaluation
would take place and when it would not.
Historical aside for those that like that sort of thing:
In Maclisp, btw, the #, was implemented in a very obscure way. There were
two obarrays--a user one for the program being compiled and a system one for
the compiler. obarrays were very much like packages, but without all the
bookkeeping that let you move conveniently between them. Mostly symbols in
one never touched the other, which is what let you write a program in lisp
that compiled another one without packages and still didn't clobber you.
In the compiler's obarray, which you could get to with some work, you could
find a symbol SQUID (self-quoting internal datum) and if you wrapped it around
an object, as in (list +squid+ thing), or maybe it was (cons +squid+ thing),
you'd mark THING as something to execute
at loadtime and the entire cons would be replaced at runtime by a pointer to
the result of the computation. [Btw, the variable that held the SQUID object
was probably not called +squid+, but I wanted to emphasize that it was the
identity of this object, not the variable that held it, that was relevant. The
user had to make up a holder for it. You couldn't write just (list 'squid x)
because you'd get squid from the wrong obarray if you did. It itself had to
be a variable reference. Just as well--if it had been in literal data, that
literal data would itself not have compiled correctly.] Anyway, this bizarre
treatment of SQUID led to some problems in type analysis,
because it wasn't as well-integrated with the compiler as it could have been.
And there are some remarks in my fortran->lisp translator paper about how this
annoyingly couldn't result in the SQUID-form returning a small integer that
would become the immediate data part of MOVEI instruction; you always got a
MOVE instruction from real, allocated memory, because the CONS that was being
the placeholder took up space until it was replaced. Anyway, a lot of the
confusing semantics of #, dated back to the particular mechanics of how SQUID
was processed in the early days and little thought was given to formalizing it
better when the mechanism changed, which is why I think it went away. Also,
I think, one reason it started off as #, to begin with was that the SQUID token
was very hard to handle correctly once it got mixed into code, so it didn't
lend itself well to cascaded delays in quoting as would have resulted from
macro expansion. Since it was acted upon immediately whether in code or data,
it didn't cascade well. LOAD-TIME-VALUE is better in that regard, and that's
why I think it will survive. Some have advocated putting #, back in as a
shorthand for LOAD-TIME-VALUE, and I mostly wouldn't disagree, now that people
understand that putting it in quoted structure WOULD delay it, something
that didn't used to happen...
In article <···············@world.std.com>,
Kent M Pitman <······@world.std.com> wrote:
>Lieven Marchand <···@wyrd.be> writes:
>
>> ····@ii.uib.no (Stig E. Sandoe) writes:
>>
>> > So my impression from reading DEFCONSTANT and EQL is that:
>> > (defconstant +eof+ (cons nil nil))
>> > is not 100% kosher to what is specified in the HyperSpec
>> > for portable behaviour. Is my musings on this topic in accordance
>> > with other people's interpretation and the code in question should be
>> > changed to something more portable?
>>
>> I agree. Do you think
>>
>> (defconstant +eof+ (load-time-value (cons nil nil) t))
>>
>> is a portable way to do this?
>
>Hmm. Something looks odd about that. I normally only define constants
>(rather than mutable variables) if the compiler is going to make assumptions
>about the values. There is really no difference between that and
>defparameter of the same value, other than the restriction on assigning it,
>which is a bit pointless if the compiler hasn't built in any dependency.
I think one point of using DEFCONSTANT in this case is so the compiler can
generate an error if you ever try to assign or bind +EOF+. If it can also
open-code references to the variable, so much the better.
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
>
> In article <···············@world.std.com>,
> Kent M Pitman <······@world.std.com> wrote:
> >
> >Hmm. Something looks odd about that. I normally only define constants
> >(rather than mutable variables) if the compiler is going to make assumptions
> >about the values. There is really no difference between that and
> >defparameter of the same value, other than the restriction on assigning it,
> >which is a bit pointless if the compiler hasn't built in any dependency.
>
> I think one point of using DEFCONSTANT in this case is so the compiler can
> generate an error if you ever try to assign or bind +EOF+. If it can also
> open-code references to the variable, so much the better.
But if it's not going to open-code the variable (and how could it if it sees
that definition is the load-time-value of something?), then what difference
does it make if the variable is assigned? Everyone is going to access the
variable's value indirect through the name of the variable at runtime, I
would think, and so if the value changed, they'd naturally pick up the new
value and no one would care... or am I missing something?
From: Tim Bradshaw
Subject: Re: DEFCONSTANT and demand for EQL
Date:
Message-ID: <nkjofrvvt0w.fsf@tfeb.org>
Kent M Pitman <······@world.std.com> writes:
>
> But if it's not going to open-code the variable (and how could it if it sees
> that definition is the load-time-value of something?), then what difference
> does it make if the variable is assigned? Everyone is going to access the
> variable's value indirect through the name of the variable at runtime, I
> would think, and so if the value changed, they'd naturally pick up the new
> value and no one would care... or am I missing something?
This is really slightly off-topic but one trick I wondered about with
regard to constants was for a compiler to note that something was a
constant, even though it didn't yet know the value, and essentially
partly-compile the relevant bit of code, leaving the final part of
compilation until load time, when the constant should be known. There are probably some techniques that this would make hard (for instance something like:
(if (eq +x+ 'development-version)
(progn ... huge code ...)
nil)
could probably be optimized away much more aggressively if +X+ was
known completely at compile time, but you should still be able to do
quite a lot, by being willing to `compile' rather late in the day. I'm particularly thinking of something like:
... (car +x+) ...
where the compiler would leave some kind of note that not just +X+ but
the whole expression would be constant, and do the CAR at load time
before splicing i the fixed result.
I always thought this would be a very lisp thing to do - I can hear
static-language / tiny runtime people gnashing their teeth at the
thought of having to have so much intelligence present at load time,
so it's obviously a good thing to do if just to encourage
teeth-gnashing in that sort of person, but perhaps I'm missing
something.
--tim
(The idea is that this would answer the `how could it open-code'
question.)
Tim Bradshaw <···@tfeb.org> writes:
> ... one trick I wondered about with regard to constants was for a
> compiler to note that something was a constant, even though it
> didn't yet know the value, and essentially partly-compile the
> relevant bit of code, leaving the final part of compilation until
> load time, when the constant should be known.
Funny you should mention that. I considered that among the space of things
that could be done when writing the message to which you were replying, and
I concluded that probably no implementations do that, and so I didn't
specifically address it. But I think you're right it's a possible thing to
do. It's the sort of thing I'd expect CMU CL to do.
Tim Bradshaw <···@tfeb.org> wrote:
> [O]ne trick I wondered about with regard to constants was for a compiler
> to note that something was a constant, even though it didn't yet know the
> value, and essentially partly-compile the relevant bit of code, leaving
> the final part of compilation until load time, when the constant should be
> known.
> [...]
>
> I always thought this would be a very lisp thing to do - I can hear
> static-language / tiny runtime people gnashing their teeth at the thought
> of having to have so much intelligence present at load time,
Hmmm, is this true?
Consider how much work gets done by the Solaris[1] runtime linker, for
example. I remember being somewhat shocked when I began fiddling with
LD_DEBUG. Also doesn't a non-PIC shared-object cause some kind of
load-time effect? I can't remember the details.
I also remember years ago being startled when I discovered that
executables on my Atari ST had a section called the fix-up table, and
that the computer could actually alter my program in memory before
running it. Seemed like magic at the time :-)
From not quite so long ago, when FPUs were optional extras (on desktop
PCs), I vaguely recall the Microsoft[2] C compiler & runtime spreading
the cost of a logically load time action over runtime: any
floating-point code got compiled into FPU opcodes. If, when the program
was run, there was no FPU any such instruction would cause the processor
to trap. This would be caught be the runtime which would then patch the
code that caused the trap with a call to the appropriate software maths
function. Self modifying code? Hurrah!
These are just random things that I've stumbled across over the years,
so there are undoubtedly loads of other examples. On the other hand
whether the majority of "static-language / tiny runtime people" actually
realise what's going on when they run their program is another matter.
> so it's obviously a good thing to do if just to encourage teeth-gnashing
> in that sort of person, but perhaps I'm missing something.
<gnash> <gnash>
Footnotes:
[1] This will be the same for other Unixes as well, certainly Digital
Unix, or whatever the damn thing is called now.
[2] Because I used mainly their compiler, not because the technique
is/was unique to them.
--
Duncan Harvey
"Smiling and waving. Before letting himself fall."
-- Anja Garbarek, The Diver
From: Raymond Toy
Subject: Re: Load-time behaviour (was: Re: DEFCONSTANT and demand for EQL)
Date:
Message-ID: <4n1yopg6xz.fsf@rtp.ericsson.se>
>>>>> "Duncan" == Duncan Harvey <····@hubris2.demon.co.uk> writes:
Duncan> From not quite so long ago, when FPUs were optional extras (on desktop
Duncan> PCs), I vaguely recall the Microsoft[2] C compiler & runtime spreading
Duncan> the cost of a logically load time action over runtime: any
Duncan> floating-point code got compiled into FPU opcodes. If, when the program
Duncan> was run, there was no FPU any such instruction would cause the processor
Duncan> to trap. This would be caught be the runtime which would then patch the
Duncan> code that caused the trap with a call to the appropriate software maths
Duncan> function. Self modifying code? Hurrah!
I think it was the other way. All FPU operations were implemented as
a call to some routine(s). The routine would check for an FPU
available, and replace the call instruction with the appropriate FPU
instruction. (The call would have been padded with nops, if
necessary, to make room for any possible FPU instruction).
I thought this was a cool hack back in the days of the 8086 and 8087
where, I think, there weren't traps for illegal opcodes, especially
FPU opcodes.
Ray
>
> Duncan> From not quite so long ago, when FPUs were optional extras (on desktop
> Duncan> PCs), I vaguely recall the Microsoft[2] C compiler & runtime spreading
> Duncan> the cost of a logically load time action over runtime: any
> Duncan> floating-point code got compiled into FPU opcodes. If, when the program
> Duncan> was run, there was no FPU any such instruction would cause the processor
> Duncan> to trap. This would be caught be the runtime which would then patch the
> Duncan> code that caused the trap with a call to the appropriate software maths
> Duncan> function. Self modifying code? Hurrah!
I believe Windows 3.1 did this, though it was somewhat more egregious;
it would modify in-memory the prologs of user-space C functions to
tweak a value loaded into a key register at call-time. It was in
order to support virtualizing the function's address- perhaps
something to do with the "thunking" I believe. Its been too long to
remember the details...
Gregm
Raymond Toy <···@rtp.ericsson.se> wrote:
> I think it was the other way. All FPU operations were implemented as a
> call to some routine(s). The routine would check for an FPU available,
> and replace the call instruction with the appropriate FPU instruction.
> (The call would have been padded with nops, if necessary, to make room for
> any possible FPU instruction).
<slaps head> Yes! This definitly rings a louder bell.
Apologies for the misinformation.
> I thought this was a cool hack back in the days of the 8086 and 8087
Heh. AOL!
--
Duncan Harvey
"Smiling and waving. Before letting himself fall."
-- Anja Garbarek, The Diver
From: Tim Bradshaw
Subject: Re: Load-time behaviour (was: Re: DEFCONSTANT and demand for EQL)
Date:
Message-ID: <nkjr8wq3shm.fsf@tfeb.org>
····@hubris2.demon.co.uk (Duncan Harvey) writes:
> These are just random things that I've stumbled across over the years,
> so there are undoubtedly loads of other examples. On the other hand
> whether the majority of "static-language / tiny runtime people" actually
> realise what's going on when they run their program is another matter.
Well, of course, *we* know that they are doing something much more
slow, complex and badly specified than any Lisp system, but *they*
think they are running on a PDP11, only clocked at a GHz, where that
kind of thing doesn't happen.
--tim
(I used to think that CS was merely prescientific, but I am now
developing a theory that it is in fact a medieval culture, where
things that are not understood must be destroyed as heresy).
From: Bob Bane
Subject: Re: Load-time behaviour (was: Re: DEFCONSTANT and demand for EQL)
Date:
Message-ID: <3B263EC9.5B984E84@removeme.gst.com>
Duncan Harvey wrote:
>
>
> Consider how much work gets done by the Solaris[1] runtime linker, for
> example. I remember being somewhat shocked when I began fiddling with
> LD_DEBUG. Also doesn't a non-PIC shared-object cause some kind of
> load-time effect? I can't remember the details.
>
Linkers can do the strangest things sometimes. I heard a horror story
from a developer at one of the major Lisp vendors. Seems one particular
revision of a particular CPU architecture had a bug where some delayed
branch/load instructions would fail if they were encountered at certain
memory page boundaries. They "fixed" it temporarily by having their
linker detect these cases and insert extra NOOP instructions.
The Lisp environment didn't use the linker, and could relocate code with
its copying GC, so it would get bitten randomly by this bug, which of
course had no external documentation. Ultimately the hardware vendor
admitted the problem to the Lisp vendor and gave all the Lisp customers
a new CPU without the fault, but I can imagine how much it must have
hurt to track that problem down...
--
Remove obvious stuff to e-mail me.
Bob Bane
From: Tim Moore
Subject: Re: Load-time behaviour (was: Re: DEFCONSTANT and demand for EQL)
Date:
Message-ID: <9g68tj$klv$0@216.39.145.192>
On Tue, 12 Jun 2001, Bob Bane wrote:
> Linkers can do the strangest things sometimes. I heard a horror story
> from a developer at one of the major Lisp vendors. Seems one particular
> revision of a particular CPU architecture had a bug where some delayed
> branch/load instructions would fail if they were encountered at certain
> memory page boundaries. They "fixed" it temporarily by having their
> linker detect these cases and insert extra NOOP instructions.
Mips... R4000, I believe.
Tim
From: Stephen J. Bevan
Subject: Re: Load-time behaviour (was: Re: DEFCONSTANT and demand for EQL)
Date:
Message-ID: <m3puc97zgk.fsf@yahoo.com>
Tim Moore <·····@herschel.bricoworks.com> writes:
> On Tue, 12 Jun 2001, Bob Bane wrote:
> > revision of a particular CPU architecture had a bug where some delayed
> > branch/load instructions would fail if they were encountered at certain
> > memory page boundaries. They "fixed" it temporarily by having their
> > linker detect these cases and insert extra NOOP instructions.
>
> Mips... R4000, I believe.
I thought it was a R2000 but it has been a few years. I can confirm
that the MIPS version of MLWorks would check the architecture/revision
and if it was a dodgy one, avoid planting jumps near the end of a page
because of this problem. I assumed that LispWorks did something similar.
From: Tim Moore
Subject: Re: Load-time behaviour (was: Re: DEFCONSTANT and demand for EQL)
Date:
Message-ID: <9g85e0$pdb$0@216.39.145.192>
On Wed, 13 Jun 2001, Stephen J. Bevan wrote:
> Tim Moore <·····@herschel.bricoworks.com> writes:
> > On Tue, 12 Jun 2001, Bob Bane wrote:
> > > revision of a particular CPU architecture had a bug where some delayed
> > > branch/load instructions would fail if they were encountered at certain
> > > memory page boundaries. They "fixed" it temporarily by having their
> > > linker detect these cases and insert extra NOOP instructions.
> >
> > Mips... R4000, I believe.
>
> I thought it was a R2000 but it has been a few years. I can confirm
> that the MIPS version of MLWorks would check the architecture/revision
> and if it was a dodgy one, avoid planting jumps near the end of a page
> because of this problem. I assumed that LispWorks did something similar.
I went and checked the IRIX man page for ld, which is where I'd heard
about this bug (never having encountered it in person). One of ld's many
options is:
-no_jump_at_eop
Works around an R4000 bug present in 2.1 and 2.2 silicon.
This option tries to prevent certain classes of branch
instructions from being the last instruction on a page of
virtual memory. This option is enabled by default for
links
of MIPS3 or lower programs. You can use the hinv(1M)
command to determine what revision of R4000 is present in
the system.
The kernel automatically works around this problem with
early versions of the R4000, but there is a small
performance penalty. Using this option helps avoid the
situation in which a kernel workaround is needed.
Of course, there may be similar bugs in other chips too.
Tim
From: Duane Rettig
Subject: Re: Load-time behaviour (was: Re: DEFCONSTANT and demand for EQL)
Date:
Message-ID: <47kyg2mlc.fsf@beta.franz.com>
Tim Moore <·····@herschel.bricoworks.com> writes:
> On Wed, 13 Jun 2001, Stephen J. Bevan wrote:
>
> > Tim Moore <·····@herschel.bricoworks.com> writes:
> > > On Tue, 12 Jun 2001, Bob Bane wrote:
> > > > revision of a particular CPU architecture had a bug where some delayed
> > > > branch/load instructions would fail if they were encountered at certain
> > > > memory page boundaries. They "fixed" it temporarily by having their
> > > > linker detect these cases and insert extra NOOP instructions.
> > >
> > > Mips... R4000, I believe.
> >
> > I thought it was a R2000 but it has been a few years. I can confirm
> > that the MIPS version of MLWorks would check the architecture/revision
> > and if it was a dodgy one, avoid planting jumps near the end of a page
> > because of this problem. I assumed that LispWorks did something similar.
>
> I went and checked the IRIX man page for ld, which is where I'd heard
> about this bug (never having encountered it in person). One of ld's many
> options is:
>
> -no_jump_at_eop
> Works around an R4000 bug present in 2.1 and 2.2 silicon.
> This option tries to prevent certain classes of branch
> instructions from being the last instruction on a page of
> virtual memory. This option is enabled by default for links
> of MIPS3 or lower programs. You can use the hinv(1M)
> command to determine what revision of R4000 is present in
> the system.
>
> The kernel automatically works around this problem with
> early versions of the R4000, but there is a small
> performance penalty. Using this option helps avoid the
> situation in which a kernel workaround is needed.
Yes, this was the bug. The specific problem was that if a branch
instruction was the last instruction on the page, _and_ if the page
of the next memory location was not paged in (for the delay instruction
which was always executed after the branch) the page-faulting mechanism
did not do the right thing and an illegal-instructions was trapped which
was not restartable.
The fix in a linker was easy, as long as code was not position-independent,
or if the code was at least was fixed on a per-page boundary; the linker
would just insert a nop instruction before the branch if it would have been
the last on the page. One 4-byte nop per 4kbytes; not too bad.
Our problem in lisp was that code vectors are completely relocatable, on
an 8-byte aligned basis. So our code-vector-generation had to include
insertion of nops for any branch which _could_ appear at an odd-multiple-
of-four address. This caused a code bloat of a couple of percent.
Not very much, but bloat nevertheless.
It was actually another developer who had created this fix. But it was
only a little later that I was rebuilding our mips port to incorporate
it into common source and to add the new SVR4 support in Irix, and I
became disgusted with the need for such a hack. I screamed and stirred
up a ruckus, and we approached SGI for a swapout. They were not interested,
but after a year of being a pest and with the combined help of a few strong
customers and the fact that there were likely a very limited number of Lisp
customers who still had the buggy chip (most had moved on to later r4000 and
r4400 chips), we convinced them to offer the swapout.
> Of course, there may be similar bugs in other chips too.
Yes, but this bug was fairly unique in that it could be covered up
by a C-centric (actually, static) language system. What took us a year
to do was for us to convince their sales-types and upper management that
the bug was not in fact "fixed" as they had been told, but only covered
over in a way that did not help us or our customers.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
In article <···············@world.std.com>,
Kent M Pitman <······@world.std.com> wrote:
>Barry Margolin <······@genuity.net> writes:
>
>>
>> In article <···············@world.std.com>,
>> Kent M Pitman <······@world.std.com> wrote:
>> >
>> >Hmm. Something looks odd about that. I normally only define constants
>> >(rather than mutable variables) if the compiler is going to make assumptions
>> >about the values. There is really no difference between that and
>> >defparameter of the same value, other than the restriction on assigning it,
>> >which is a bit pointless if the compiler hasn't built in any dependency.
>>
>> I think one point of using DEFCONSTANT in this case is so the compiler can
>> generate an error if you ever try to assign or bind +EOF+. If it can also
>> open-code references to the variable, so much the better.
>
>But if it's not going to open-code the variable (and how could it if it sees
>that definition is the load-time-value of something?), then what difference
>does it make if the variable is assigned? Everyone is going to access the
>variable's value indirect through the name of the variable at runtime, I
>would think, and so if the value changed, they'd naturally pick up the new
>value and no one would care... or am I missing something?
If someone does
(setq +eof+ 'foo)
then the program could detect a spurious EOF as a result of the input
stream containing the symbol FOO. DEFCONSTANT is intended to prevent this
type of reassignment. I don't think the language requires that this be
detected, but most (if not all) implementations do.
--
Barry Margolin, ······@genuity.net
Genuity, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Barry Margolin <······@genuity.net> writes:
> If someone does
>
> (setq +eof+ 'foo)
>
> then the program could detect a spurious EOF as a result of the input
> stream containing the symbol FOO. DEFCONSTANT is intended to prevent this
> type of reassignment.
Hmmmmm... If they do (defconstant +eof+ 'foo) it also prevents them
from correcting their bug without recompiling. ;-)
Lieven Marchand <···@wyrd.be> writes:
> ····@ii.uib.no (Stig E. Sandoe) writes:
>
> > So my impression from reading DEFCONSTANT and EQL is that:
> > (defconstant +eof+ (cons nil nil))
> > is not 100% kosher to what is specified in the HyperSpec
> > for portable behaviour. Is my musings on this topic in accordance
> > with other people's interpretation and the code in question should be
> > changed to something more portable?
>
> I agree. Do you think
>
> (defconstant +eof+ (load-time-value (cons nil nil) t))
>
> is a portable way to do this?
I am not sure, but it doesn't look pretty. The code in question was
changed in the library after I posted to use DEFVAR which I think is
the most readable in this case.
--
------------------------------------------------------------------
Stig Erik Sandoe ····@ii.uib.no http://www.ii.uib.no/~stig/
Lieven Marchand <···@wyrd.be> writes:
> ····@ii.uib.no (Stig E. Sandoe) writes:
> > [restored a little more context]
> > [quoting from ANS] Therefore, users must ensure that the
> > initial-value can be evaluated at compile time (regardless of
> > whether or not references to name appear in the file) and that it
> > always evaluates to the same value.
> > """
> >
> > So what is the "same value"?
According to the glossary, "same" means EQL, unless a specific
predicate is mentioned.
However, that bit about DEFCONSTANT can't mean literally what it says,
because EQL isn't defined between two objects in different images. It
apparently means "similar (same for repeated loads into the same
image)". The latter restriction comes from the other bit you quoted:
"The consequences are undefined if there are any bindings of the
variable named by name at the time DEFCONSTANT is executed or if the
value is not EQL to the value of initial-value."
> > [...]
> > So my impression from reading DEFCONSTANT and EQL is that:
> > (defconstant +eof+ (cons nil nil))
> > is not 100% kosher to what is specified in the HyperSpec
> > for portable behaviour. [...]
As long as you don't reload that, it's kosher. If you do,
LOAD-TIME-VALUE will help, as long as the file is compiled.
> Do you think
>
> (defconstant +eof+ (load-time-value (cons nil nil) t))
>
> is a portable way to do this?
However, specifying T as the read-only-p argument gives permission to
coalesce the result with other similar constants, which is not what
you'd want for an eof marker. IF you're paranoid enough.
Anyway, Sandoe doesn't say what failed on SBCL. It probably wasn't
reloading, but "splitting", as Kent called it. It's better not to use
DEFCONSTANT for this.
--
Pekka P. Pirinen
"Don't hate the media. Become the media." - Jello Biafra
That doesn't happen to be some of my old code, does it? I think I had
used '(()) as +eof+ though...
My understanding (as brief and shallow as it is) is that a CL
implementation is not obligated to preserve the value of a constant
across compilation units such that the value retains equality in the EQL
sense. In other words, if I were to define this EOF in file A and use it
in file B, after compilation, the implementation is not obligated to
guarantee that the value of EOF in B is still the same as A. I also
believe there are ways around this: look at LOAD-TIME-VALUE, avoid using
defconstant (use defvar instead), and simply avoid using this approach
to handling EOF's. The last may not be feasible since the code is not
yours, but I have found that a local (lexical) value of EOF generally
suffices.
Hope this hasn't muddled the waters too much more...
Sunil
Stig E. Sandoe wrote:
> hi,
>
> In a certain lisp-library one can find code of the form:
> (defconstant +eof+ (cons nil nil))
>
> This "works" in most implementations apparently but did not work when
> I tried SBCL yesterday. The file it is defined in is part of a system
> and before use it is compiled and then loaded. So I looked at
> DEFCONSTANT in the HyperSpec:
>
> """
> If a defconstant form appears as a top level form, the compiler must
> recognize that name names a constant variable. An implementation may
> choose to evaluate the value-form at compile time, load time, or
> both. Therefore, users must ensure that the initial-value can be
> evaluated at compile time (regardless of whether or not references to
> name appear in the file) and that it always evaluates to the same
> value.
> """
>
> So what is the "same value"? Earlier on the page it says:
>
> """
> The consequences are undefined if there are any bindings of the
> variable named by name at the time defconstant is executed or if the
> value is not eql to the value of initial-value.
> """
>
> So my understanding is that the value should be EQL even if it is
> evaluated on compile-time and load-time, which it probably is. EQL
> says this about CONS-equality:
>
> """
> (eql (cons 'a 'b) (cons 'a 'b)) => false
> """
>
> So my impression from reading DEFCONSTANT and EQL is that:
> (defconstant +eof+ (cons nil nil))
> is not 100% kosher to what is specified in the HyperSpec
> for portable behaviour. Is my musings on this topic in accordance
> with other people's interpretation and the code in question should be
> changed to something more portable?
>
>
Sunil Mishra <·······@notmyemail.com> writes:
> That doesn't happen to be some of my old code, does it? I think I had
> used '(()) as +eof+ though...
Sure, and it probably works fine because you don't put the right thing in
your files, not because its identity is rescuing you. The problem
with '(...) as an eof marker is that if you write
(defun foo () '(()))
and then you do #.(foo) in your file, it may return the same value as your
eof marker without your specially conspiring to have it mean that. Of
course, you may regard this as legitimiately rare that you don't car.
I'm sure some people get away with '+eof+ or even just 'eof or NIL as
an eof marker. NIL would work fine in a file full of numbers, for example.
I personally think the safest and least-consing thing to use is the stream
itself, which is known to be a value that is not in the stream, and which
it's unlikely you'll have any functions whose value is that.
> My understanding (as brief and shallow as it is) is that a CL
> implementation is not obligated to preserve the value of a constant
> across compilation units such that the value retains equality in the EQL
> sense. In other words, if I were to define this EOF in file A and use it
> in file B, after compilation, the implementation is not obligated to
> guarantee that the value of EOF in B is still the same as A.
Nothing you say here is wrong per se, but the particular way that EOF works,
the problem scenario you describe is rarely a problem. Usually an EOF value
is going to work as long as it remains distinct from other things in the
file. Consequently, its beecoming "more distinct" by having its identity
be "split" from its other incarnations, rarely leads to a problem. ON THE
OTHER HAND, coalescing, as I described above, is a problem.
> ... look at LOAD-TIME-VALUE, ...
This will fix both splitting and coalescing, so is a fine idea.
Though again I think its simpler just to use the stream.
I sometimes just write
(loop for form = (read stream nil stream)
until (eq form stream)
collect form)
or something like that.
> avoid using defconstant (use defvar instead),
Good advice. I'm glad someone said this because it was making me queasy to
see that (defconstant +eof+ ...) in some prior post but I was too busy to
reply.
> and simply avoid using this approach
> to handling eof's. the last may not be feasible since the code is not
> yours, but I have found that a local (lexical) value of EOF generally
> suffices.
Yes, people often do (let ((eof (list 'eof))) ...) at the cost of one cons.
But since the stream has just been "consed" unique, I might suggest
(let ((eof stream)) ...)
if you just want to write (read stream nil eof) instead of
(read stream nil stream) for visual clarity.
> Hope this hasn't muddled the waters too much more...
Doesn't sound like it should have. Don't think you said anything really
false in there. Hope you don't mind my sticking in a few extra notes to
augment what you'd written, though.
Wouldn't an uninterned symbol do the job as EOF-marker nicely?
(defconstant +eof+ #:eof-marker)
The advantage over using a (cons x y) to achieve uniqueness, is that
once READ the symbol really is a constant (always evaluates the same),
while the CONS form is a function and so uncertainties about its
evaluation-time arise.
Or things like
(loop with eof-marker = #:eof-marker
for form = (read stream nil eof-marker)
until (eq form eof-marker)
collect form)
I find this more readable/explicit than using the stream object as
EOF-marker.
--
Frode Vatvedt Fjeld
Frode Vatvedt Fjeld <······@acm.org> writes:
> Wouldn't an uninterned symbol do the job as EOF-marker nicely?
>
> (defconstant +eof+ #:eof-marker)
>
> The advantage over using a (cons x y) to achieve uniqueness, is that
> once READ the symbol really is a constant (always evaluates the same),
> while the CONS form is a function and so uncertainties about its
> evaluation-time arise.
Well, first, you're not generally going to evaluate it.
Second, gensyms don't evaluate to themselves. They are unbound.
Third, a symbol usually takes up more space than a cons. Usually a cons
is about the lowest overhead reliably-unique object you can make, presuming
you make its car and cdr be objects that exist already. e.g.,
(defvar +eof+ (list '+eof+))
Conses are also much faster to make on the fly than gensyms.
> Or things like
>
> (loop with eof-marker = #:eof-marker
> for form = (read stream nil eof-marker)
> until (eq form eof-marker)
> collect form)
>
> I find this more readable/explicit than using the stream object as
> EOF-marker.
There's nothing really awful about this. But I personally don't like to use
gensyms as literals in code because, while it works here, the I/O behavior is
a bit too odd for my taste. Also, since it's storage-wise cheaper (though by
a near-inconsequential amount), I tend to prefer:
(loop with eof-marker = (load-time-value (list 'eof-marker))
...)
But my preference is just my preference. I don't think you'll get in
trouble with what you're doing.
Kent M Pitman <······@world.std.com> writes:
> Second, gensyms don't evaluate to themselves. They are unbound.
Right, I forgot to quote the symbols at the appropriate places. I do
find the saving of a few tens of bytes a rather peculiar argument, but
like you say I guess it's mostly a matter of personal preference.
--
Frode Vatvedt Fjeld
Frode Vatvedt Fjeld <······@acm.org> writes:
> Kent M Pitman <······@world.std.com> writes:
>
> > Second, gensyms don't evaluate to themselves. They are unbound.
>
> Right, I forgot to quote the symbols at the appropriate places. I do
> find the saving of a few tens of bytes a rather peculiar argument, but
> like you say I guess it's mostly a matter of personal preference.
You have to recall that I learned to program Lisp on a PDP10, whose
maximum address space size was 256K 36-bit words (about 1.25MB). That
space had to hold program + data. A cons took one word (~5 bytes) while
while a symbol took a lot more, several words for header plus a couple
more for the name (say about 5-6 words, or about 25-30 bytes. So the order
of wastage there was not "a few bytes" but ".0001 maximum address space".
On one hand, it is probably right that in modern systems, such tiny amounts
really are so close to nothing that one just shouldn't care. (I have heard
it said that since people generally do not change in their ways, that the way
society changes its ways is through the death of people. I suppose in that
regard society is impeded by those of us from that era who stubbornly hang
on for now, perhaps worse if we share our neuroses.)
On the other hand, it has been said that an ounce of prevention is worth a
pound of cure. I never recommend to anyone that they go around optimizing
out such small uses in their programs. I'm sure it's demonstrably not worth
the time to do that. But training yourself to take note of the cost of doing
various different ways and then just trying to quietly apply the lower cost
methods where it's convenient to do so, without being neurotic about it, as
my generation was taught to be, may get you enough benefit over time to make
it worth your while.
I also do funny little other things like try to name my
variables in ways that are not gratuitously different on the assumption that
(defun f (x) x) and (defun g (y) y) can be "optimized" by using (defun g (x) x)
and not having an extra variable "y" around. In truth, I never worry about
X and Y because I figure *someone somewhere* will always have snuck in an
X and a Y, so it's pointless to do that particular th ing. But I try to
avoid things like
(defun frob (first-thing-to-frob second-thing-to-frob)
...)
when
(defun frob (thing1 thing2) ...)
on the assumption that it's quite likely I'll find
(defun frotz (thing1 thing2) ...)
somewhere else and get some symbol sharing (and incrementally less image
bloat), while I'm not likely to find any further use for a symbol named
first-thing-to-frob. Again, just a few bytes here and a few bytes there.
But if it's just second nature to do it the cheap way, it doesn't take any
time (you had to write *some* variable, it might as well be a
likely-to-be-shared one), and the practice can add up.
At least until people stop complaining that some images don't fit on their
floppy disks, there is some reason to believe that people care about space
use.
Kent M Pitman wrote:
> I personally think the safest and least-consing thing to use is the stream
> itself, which is known to be a value that is not in the stream, and which
> it's unlikely you'll have any functions whose value is that.
The primary reason I had defined a constant +eof+ was that I wanted to
check for the EOF value at a location other than where the value had
been read. I have found since then that it is far easier to rearrange
your code to check for EOF right away rather than attempt to keep a
globally consistent value. The idea of using the stream itself as the
EOF value is one that I have heard before, but constantly keep
forgetting :-I
Sunil
>