So the thread on "An Idiot's Guide to Special Variables" managed to
confuse me enough that I spent some time banging my head against the
CLHS myself supplemented with some experiments at my local REPL. I
think I basically get how special variables work but I'm wondering if
someone could say whether this mental model is basically right:
When a dynamic binding is "established", such as by a 'LET' containing
a '(DECLARE (SPECIAL ...))', that binding is the binding in the
current dynamic environment until it is "disestablished" at which
point whatever dynamic binding (if any) existed before the
establishment just mentioned resumes it's position as the binding in
the current dynamic environment. In other words, for each symbol that
may be dynamically bound, there is (conceptually at least) a stack of
bindings and the dynamic environment only exposes the top of that
stack. There is--I take it--no way to access the dynamic bindings
below the top of this notional stack. The value that the accessor
SYMBOL-VALUE is accessing is either the value of the most-recently
established dynamic binding or the value of the constant variable
named by the symbol.
On a related note: is there any way to undo a global declaration?
During my experimenting I found the following results initially
surprising:
CL-USER(1): (defun show-x (label) (format t "~A: ~A~%" label *x*))
SHOW-X
CL-USER(2): (let ((*x* 10))
(declare (special *x*))
(show-x "outer")
(let ((*x* 20)) (show-x "inner")))
outer: 10
inner: 10 ;; makes sense -- the second let establishes a lexical binding
NIL
CL-USER(3): (defvar *x*)
*X*
CL-USER(4): (let ((*x* 10))
(declare (special *x*))
(show-x "outer")
(let ((*x* 20)) (show-x "inner")))
outer: 10
inner: 20 ;; also makes sense -- defvar made *x* globally special
NIL
CL-USER(5): (makunbound '*x*)
*X*
CL-USER(6): (let ((*x* 10))
(declare (special *x*))
(show-x "outer")
(let ((*x* 20)) (show-x "inner")))
outer: 10
inner: 20 ;; whoops! -- i was expecting to be back to the first case
NIL
CL-USER(7):
I was surprised that makunbound didn't completely undo the effect of
the defvar. But I think I get why it didn't--defvar had two effects,
one being the equivalent of (proclaim '(special *x*)) and the other
being (setf (symbol-value '*x*) nil), and only the setf part was
undone by makunbound. Or forget undoing the declarations; is there a
standard way to see what global declarations are in effect?
-Peter
--
Peter Seibel
·····@javamonkey.com
In article <··············@localhost.localdomain>, Peter Seibel
<·····@javamonkey.com> wrote:
> I'm wondering if
> someone could say whether this mental model is basically right:
Yep.
> On a related note: is there any way to undo a global declaration?
Nope. The best you can do is unintern the symbol, and then re-read and
re-evaluate all the forms that reference the symbol.
E.
Peter Seibel <·····@javamonkey.com> writes:
> I was surprised that makunbound didn't completely undo the effect of
> the defvar. But I think I get why it didn't--defvar had two effects,
> one being the equivalent of (proclaim '(special *x*)) and the other
> being (setf (symbol-value '*x*) nil), and only the setf part was
> undone by makunbound. Or forget undoing the declarations; is there a
> standard way to see what global declarations are in effect?
Well, the point is that the semantic treatment of a variable ("what
happens if I bind it?") is orthogonal to the dynamic state of the
variable ("is it bound, and to what?").
I read through the HyperSpec on this just now and agree it is poorly
explained, but here's the point:
Lexical bindings always have a value so are not really interesting to
MAKUNBOUND. There's no way even to name a lexical binding. Lexical
bindings always have a value and never get unbound errors.
Lexical Variable -----------> Lexical Variable
Binding Value
(a hidden internal place) (an object)
Special bindings always refer to a place called the value cell which is
always accessible via SYMBOL-VALUE without reference to the lexical
environment. That value cell is either unbound (has no associated
value object) or bound (has an associated value object).
Symbol --------> Special Variable [ ] unbound
Value Cell [x] bound ------> Special Variable
(a place named by Value
a certain symbol) (an object)
MAKUNBOUND merely unchecks the 'bound' box, it does not change the nature
of the variable itself.
There used to be, in older Lisp dialects (Maclisp, for example) an
UNSPECIAL declaration that could undo a SPECIAL declaration. However,
this was a source of program error because programmers would become
confused about which files had been compiled with and without the
special declaration. Indeed it's handy interactively to undo these
things if you've made an error, but this is considered an "environment
issue" (to be dealt with by a vendor's programmer interface), not a
"language issue" (which would be dealt with at the language level).
Side note only partly related to a bunch of the above: I like to think
that when you bind a lexical variable, you get a new binding--a
different storage location into which a value goes. By contrast, I
like to think that when you bind a special variable, you do not get a
new place, you get a new value--the place is unchanged. (There's some
opportunity for dispute about this because there are implementation
strategies that may in fact get you a new place, but if they do this
they are obliged to somehow connect that new place to SYMBOL-VALUE so
that it seamlessly indirects through the new place, so this is a
difference without real significance like when the GC relocates an
object but everyone knows about the new object so we say it's still
the same object...)
In article <···············@shell01.TheWorld.com>, Kent M Pitman
<······@world.std.com> wrote:
> By contrast, I
> like to think that when you bind a special variable, you do not get a
> new place, you get a new value--the place is unchanged.
This model works only in a single-threaded world. In a multi-threaded
world it fails to account for the behavior of:
(defvar *x* 1)
(defun thread1 () *x*)
(defun thread2 () (setf *x* 2))
(defun thread3 () (let ( (*x* 3) ) (setf *x* 4)))
If all three threadN functions are run concurrently then thread1 can
return 1 or 2, but not 3 or 4.
I don't know of any way to salvage the one-place-per-special-variable
model in a multithreaded world.
E.
From: Tim Bradshaw
Subject: Re: dynamic bindings and global declarations
Date:
Message-ID: <ey3k7j28w7x.fsf@cley.com>
* Erann Gat wrote:
> I don't know of any way to salvage the one-place-per-special-variable
> model in a multithreaded world.
I think the classic hack is to have something which happens on
context/thread switches and swaps over the current values of specials
bound in a thread. Then you can expose this via something like
DYNAMIC-WIND so you can implement LETF and so on. And some people
think this is a good idea, I suspect, regardless of its catastrophic
failure to work at all in the presence of a more-than-one-cpu machine.
(So I agree: I don't know any way to salvage it...)
--tim
Tim Bradshaw <···@cley.com> writes:
> * Erann Gat wrote:
>
> > I don't know of any way to salvage the one-place-per-special-variable
> > model in a multithreaded world.
>
> I think the classic hack is to have something which happens on
> context/thread switches and swaps over the current values of specials
> bound in a thread. Then you can expose this via something like
> DYNAMIC-WIND so you can implement LETF and so on. And some people
> think this is a good idea, I suspect, regardless of its catastrophic
> failure to work at all in the presence of a more-than-one-cpu machine.
>
> (So I agree: I don't know any way to salvage it...)
Oh, that's easy: have a concept of global places, and thread-local
places. If only thread places can be "bound" with LETF, you got no
problem. Sure it's a little sick, but ...
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
From: Joe Marshall
Subject: Re: dynamic bindings and global declarations
Date:
Message-ID: <lm3h4ugd.fsf@ccs.neu.edu>
···@jpl.nasa.gov (Erann Gat) writes:
> I don't know of any way to salvage the one-place-per-special-variable
> model in a multithreaded world.
Well, you *could* have one place per special variable and associate a
mutex with each one. It's an idiotic idea, but it *does* have the
property of one place per special variable.
From: Duane Rettig
Subject: Re: dynamic bindings and global declarations
Date:
Message-ID: <4adjq7tia.fsf@beta.franz.com>
[Sorry this is late; I was gone for a while. I decided to answer this
article for both posters, because it was convenient.]
···@jpl.nasa.gov (Erann Gat) writes:
> In article <···············@shell01.TheWorld.com>, Kent M Pitman
> <······@world.std.com> wrote:
> > By contrast, I
> > like to think that when you bind a special variable, you do not get a
> > new place, you get a new value--the place is unchanged.
This depends on the implementation, and what definition you give to
"place". If in deep binding the "place" you define is "the topmost
entry on the stack for this special's binding", then that
conceptualization works, even though the actual memory location or
relationship to the name symbol might change dramatically.
> This model works only in a single-threaded world.
This also depends on what defininitions you give for "place".
> In a multi-threaded
> world it fails to account for the behavior of:
[ example moved below ]
Allow me to [re]introduce a new term: lambda-bind. It is really a
fairly old term, but it is not used in the CL spec. We have always
used it internally in Allegro CL source annotation, and, if I recall
correctly, in Franz Lisp before that. Its use might help to
delineate the distinctions and clear up some of the confusions, even
with respet to multithreading.
Definition:
In at least Haskel and ML, a lambda-bound variable is one which appears
as a lambda argument. At Franz, we give a slightly different definition;
On one hand, lambda-binding only applies to special symbols as names, but
on the other hand it also applies to any such names that appear in the
initialization of LET forms as well as those which appear in lambda-lists.
Thus in the following situations:
1. (defvar *a*)
2. (defun foo (*a* ...) ... )
3. (let ((*a* ...)) ...)
4. (setq *a* ...)
According to the CL spec, #2, #3, and #4 will establish a binding.
But (ex-CL) only each of #2 and #3 will establish a lambda-binding. Note
that *a* may already be lambda-bound before #1 or #4, and each of those
forms will not change the symbol's lambda-bound state. And note further
that I am only discussing symbols which are globally-special, or those
which are declared special within the let or lambda-list forms.
Application:
So we think of a symbol as being in one of three states:
a. unbound: no bindings, either lambda or otherwise, are established
b. bound, but not lambda-bound: The easiest way to think of this is
that a symbol which had been unbound is simply setq'd.
c. lambda-bound: This state always involved saving the current binding
(or lack thereof) on a stack, and establishing a new binding in "the
same place" (i.e. in such a way that eval will find this one instead
of the saved one).
I am not sure if deep binding lends itself to this distinction, but I know
that shallow-binding (which most CL implementations use) does lend itself
to the distinction; it is in fact the reason why so many view the "location"
of a special variable's value to be constant, sort of.
However, when we move to multiprocessing, we are faced with the question
of how much state to save and restore when switching processes. Long ago
(and I think that this is true of most MP-capable lisps, but someone will
probably correct me if I'm wrong) it was decided that those symbols which
were lambda-bound would be saved and restored, but those symbols that
had no lambda-binding (whether bound or not) would be left alone. This
explains the behavior below:
> (defvar *x* 1)
>
> (defun thread1 () *x*)
>
> (defun thread2 () (setf *x* 2))
>
> (defun thread3 () (let ( (*x* 3) ) (setf *x* 4)))
>
>
> If all three threadN functions are run concurrently then thread1 can
> return 1 or 2, but not 3 or 4.
The example above only works if *x* had not been previously bound in any
way (perhaps a defparameter should have been used for the example).
Allegro CL provides ways to get at various bindings of symbols. For that
location that represents the bound-but-not-lambda-bound value of a symbol,
there is sys:global-symbol-value:
http://www.franz.com/support/documentation/6.2/doc/pages/operators/system/global-symbol-value.htm
and for the current binding of any symbol in any thread, mp:symeval-in-process:
http://www.franz.com/support/documentation/6.2/doc/pages/operators/mp/symeval-in-process.htm
There are also internal acessors which allow stacked (hidden) bindings to
be accessed, but we do not export them.
Incidentally, in Allegro CL 6.2, we switched from straight shallow-binding
(which requires unwinding and rewinding bindstacks for process switches)
to what we call "wide-binding", which provides a place per thread for each
lambda-bound symbol's current binding. Roger Corman has described his
method, which is to create a vector per thread for a common set of
symbols; we access this conceptual two-dimentsional matrix the opposite way,
using a vector of value cells in the symbol rather than a vector of
symbol-values for each thread. Either technique has tradeoffs, but each
has the advantage over the old style of not requiring binding-manipulation
on context switches, which would preclude any simultaneous multiple-processor
symbol access. Wide-binding also makes trivial the implementation of the
accessors described above.
The Allegro CL description of wide-binding is given in
http://www.franz.com/support/documentation/6.2/doc/multiprocessing.htm#wide-binding-1
Note that we only reference "lambda bind" only once in the whole
mutiprocessing.htm document, it is presumably used sparingly to not confuse
the issue by introducing a non-CL term. Hopefully I have not confused
things here by also [re]introducing it.
> I don't know of any way to salvage the one-place-per-special-variable
> model in a multithreaded world.
It's really one-place-per-non-lambda-bound-special-variable, and
one-place-per-thread-per-lambda-bound-special-variable.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Kent M Pitman <······@world.std.com> writes:
> Side note only partly related to a bunch of the above: I like to
> think that when you bind a lexical variable, you get a new
> binding--a different storage location into which a value goes. By
> contrast, I like to think that when you bind a special variable, you
> do not get a new place, you get a new value--the place is unchanged.
> (There's some opportunity for dispute about this because there are
> implementation strategies that may in fact get you a new place, but
> if they do this they are obliged to somehow connect that new place
> to SYMBOL-VALUE so that it seamlessly indirects through the new
> place, so this is a difference without real significance like when
> the GC relocates an object but everyone knows about the new object
> so we say it's still the same object...)
Okay, there's one thing here that I don't quite get yet: if every time
you establish a dynamic binding named by a given symbol you are
putting the value of the new binding in the same place, i.e. the
"value cell" of that symbol, then where, conceptually, do the old
values go? That is, when *x* is special and show-x is:
(defun show-x () (format t "~A~%" *x*))
and I evaluate:
(let ((*x* 10))
(show-x)
(let ((*x* 20)) (show-x))
(show-x))
I get:
10
20
10
NIL
That suggests that during the dynamic extent of the inner let, when
the value cell of *x* held 20, the 10 didn't just get clobbered--it
came back after the dynamic extent of the inner let was
disestablished.
I guess what I'm trying to understand is what words I'm understanding
differently than you're intending them when you say:
"[W]hen you bind a lexical variable, you get a new binding--a
different storage location into which a value goes. By contrast, I
like to think that when you bind a special variable, you do not get
a new place, you get a new value--the place is unchanged"
I interpret that (in a probably too literal way) as something like:
"When you bind a lexical variable, the value is stored in a freshly
allocated spot in memory. But when you bind a special variable the
value of the new binding is put into the same memory location as the
value of the old binding."
But I know that's not a correct interpretation of what you're saying
because that implies that the old value would just be clobbered which
is not the behavior I see.
One way I think I can reconcile what you're saying with the behavior I
see is if the value cell "place" (that stays the same) is something
like the car of a list and that creating a dynamic binding pushes the
value of the binding on the list and disestablishing the binding pops
it off. Under that interpretation the place, in the sense of the
generalized reference (car <something>) stays the same.
In other words, if we were implementing it in Lisp our system might
contain some definitions like this:
(defstruct my-symbol special-values)
(defmacro value-cell (sym)
`(car (my-symbol-special-values ,sym)))
(defun establish-dynamic-binding (sym value)
(push value (my-symbol-special-values sym)))
(defun disestablish-dynamic-binding (sym)
(pop (my-symbol-special-values sym)))
Is *that* the kind of "place" you mean?
-Peter
--
Peter Seibel
·····@javamonkey.com
Peter Seibel <·····@javamonkey.com> writes:
> I interpret that (in a probably too literal way) as something like:
>
> "When you bind a lexical variable, the value is stored in a freshly
> allocated spot in memory. But when you bind a special variable the
> value of the new binding is put into the same memory location as the
> value of the old binding."
>
> But I know that's not a correct interpretation of what you're saying
> because that implies that the old value would just be clobbered which
> is not the behavior I see.
I'm pretty sure that's what Kent meant.
> One way I think I can reconcile what you're saying with the behavior I
> see is if the value cell "place" (that stays the same) is something
> like the car of a list and that creating a dynamic binding pushes the
> value of the binding on the list and disestablishing the binding pops
> it off. Under that interpretation the place, in the sense of the
> generalized reference (car <something>) stays the same.
That's one way it could be implemented. Another way is to deal with
this:
(let ((*x* (1+ *x*)))
(declare (special *x*))
(foo))
as being equivalent to this:
(let ((saved *x*))
(unwind-protect (progn
(setf *x* (1+ *x*))
(foo))
(setf *x* saved)))
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
···@famine.OCF.Berkeley.EDU (Thomas F. Burdick) writes:
...
> > One way I think I can reconcile what you're saying with the behavior I
> > see is if the value cell "place" (that stays the same) is something
> > like the car of a list and that creating a dynamic binding pushes the
> > value of the binding on the list and disestablishing the binding pops
> > it off. Under that interpretation the place, in the sense of the
> > generalized reference (car <something>) stays the same.
>
> That's one way it could be implemented. Another way is to deal with
> this:
>
> (let ((*x* (1+ *x*)))
> (declare (special *x*))
> (foo))
>
> as being equivalent to this:
>
> (let ((saved *x*))
> (unwind-protect (progn
> (setf *x* (1+ *x*))
> (foo))
> (setf *x* saved)))
Right. (Not worrying about process switches. But since CL doesn't have
them by default, that's an adequate model. An implementation that adds
multiprocessing must do something more complicated.)
[Cue Roger Corman, who recently did this in a very interesting way.]
From: Steven M. Haflich
Subject: Re: dynamic bindings and global declarations
Date:
Message-ID: <3DE0F9A7.8040900@alum.mit.edu>
Peter Seibel wrote:
...
> CL-USER(3): (defvar *x*)
> *X*
...
> I was surprised that makunbound didn't completely undo the effect of
> the defvar. But I think I get why it didn't--defvar had two effects,
> one being the equivalent of (proclaim '(special *x*)) and the other
> being (setf (symbol-value '*x*) nil), ...
I'm afraid you persist in trying to confusing yourself. Where do
you get this business about setting any value to nil? defvar
_never_ sets a variable value to a default value, nil or otherwise.
If the value subform to defvar is omitted, defvar neither sets any
value nor changes whether the variable is bound or unbound. The
ANS is quite explicit about this.
"Steven M. Haflich" <·················@alum.mit.edu> writes:
> Peter Seibel wrote:
> ...
> > CL-USER(3): (defvar *x*)
> > *X*
> ...
> > I was surprised that makunbound didn't completely undo the effect of
> > the defvar. But I think I get why it didn't--defvar had two effects,
> > one being the equivalent of (proclaim '(special *x*)) and the other
> > being (setf (symbol-value '*x*) nil), ...
>
> I'm afraid you persist in trying to confusing yourself. Where do
> you get this business about setting any value to nil? defvar
> _never_ sets a variable value to a default value, nil or otherwise.
> If the value subform to defvar is omitted, defvar neither sets any
> value nor changes whether the variable is bound or unbound. The
> ANS is quite explicit about this.
Whoops. Quite right. My mistake.
-Peter
--
Peter Seibel
·····@javamonkey.com