Hi All,
I have a question about the way CMUCL handles lexically scoped
variables. Consider the following code:
(defun foo (x)
#'(lambda (y)
(+ x y)))
From my understanding, the binding of x in the lambda from should be to
the argument supplied to foo. A call to foo should return a function
that simply adds its argument to the argument supplied to foo. For
example, the call:
(funcall (foo 4) 5)
should return 9. This happens in the normal case.
However, if *_before_* defining the function foo, I define a special
variable x, like:
(setf x '(1 2))
And then define foo as above, on running (funcall (foo 4) 5), I get the
error:
Argument X is not a NUMBER: (1 2).
Restarts:
0: [ABORT] Return to Top-Level.
Debug (type H for help)
(KERNEL:TWO-ARG-+ (1 2) 5)
I don't get the same error if I declare the special variable x after
defining foo.
What am I missing about lisp's scoping rules?
BTW, this problem seems to be specific to CMUCL. It works fine in CLisp
and SBCL.
Thanks for any help.
Regards,
Sachin.
> However, if *_before_* defining the function foo, I define a special
> variable x, like:
>
> (setf x '(1 2))
You are not defining any special variable here, but invoking undefined
behavior.
"Kaz Kylheku" <········@gmail.com> writes:
> > However, if *_before_* defining the function foo, I define a special
> > variable x, like:
> >
> > (setf x '(1 2))
>
> You are not defining any special variable here, but invoking undefined
> behavior.
But the same thing would happen with defvar.
The OP was trying to do lexical closures over dynamically bound
variables.
--
Fred Gilham ······@csl.sri.com
linus doesnt have what they call a GUI which means that you cant use a
mouse and click on things you have to type in words in a different
language probably finnish I dont know but it wont work unless you
speak at least 10 different languages and one of those languages has
to be a slavic language and you have a phd in 10 different languages
and also can do math real real good - From geraldholmes.freeyellow.com
Fred Gilham <······@snapdragon.csl.sri.com> writes:
> "Kaz Kylheku" <········@gmail.com> writes:
>
>> > However, if *_before_* defining the function foo, I define a special
>> > variable x, like:
>> >
>> > (setf x '(1 2))
>>
>> You are not defining any special variable here, but invoking undefined
>> behavior.
>
> But the same thing would happen with defvar.
No, all implementations would behave in the same way with defvar,
because defvar's effects are specified. The OP was confused because
of cmucl's different treatment of the above, undefined form. (He may
_also_ have been confused about what dynamic scope means, of course.)
Christophe
Sachin Kamboj <······@cis.udel.edu> writes:
> Hi All,
>
> I have a question about the way CMUCL handles lexically scoped
> variables. Consider the following code:
> [...]
> (setf x '(1 2))
Question 2 on the CMUCL FAQ and its answer are pertinent.
Christophe
Thanks for your reply. However, my question still remains.
Declaring x as special is fine and is what I would expect if setf were
used without a previous special declaration using defvar or defparameter.
However, my question is about the defun form. Shouldn't defun create a
new 'lexical scope' for the parameter x in:
(defun foo (x) ...)
That is, any reference to x in the function foo should refer to the
parameter x and not the special variable x.
Also shouldn't the x in foo have *indefinite* extent, That is the
reference to x should remain even after the function has exited as the
possibility of reference still exists if, for example, the function foo
returns a lambda form.
This seems to be consistent with what Steele says in Chapter 3 of Common
Lisp, The language on Scope and Extent. (Linked Below)
http://www.supelec.fr/docs/cltl/clm/node43.html
The book even uses the compose function as an example of a similar case:
(defun compose (f g)
#'(lambda (x)
(funcall f (funcall g x))))
Are you saying that if I declare f and g as special before defining the
compose function I can no longer use the compose function? Because that
seems to be the CMUCL behavior.
I might be misunderstanding something about the way scope and extent
works and the CMUCL behavior might be correct. Hence the question.
Regards,
Sachin.
Christophe Rhodes wrote:
> Sachin Kamboj <······@cis.udel.edu> writes:
>
>
>>Hi All,
>>
>>I have a question about the way CMUCL handles lexically scoped
>>variables. Consider the following code:
>>[...]
>>(setf x '(1 2))
>
>
> Question 2 on the CMUCL FAQ and its answer are pertinent.
>
> Christophe
Sachin Kamboj <······@cis.udel.edu> writes:
> Thanks for your reply. However, my question still remains.
>
> Declaring x as special is fine and is what I would expect if setf were
> used without a previous special declaration using defvar or
> defparameter.
>
> However, my question is about the defun form. Shouldn't defun create a
> new 'lexical scope' for the parameter x in:
>
> (defun foo (x) ...)
>
> That is, any reference to x in the function foo should refer to the
> parameter x and not the special variable x.
However when something has been declared globally special via DEFVAR,
DEFPARAMETER, or whatever other mechanism, it is pervasively special
and you can't make it not be special. That's why it's so important to
use the *x* naming convention for all globally special variables.
> Also shouldn't the x in foo have *indefinite* extent, That is the
> reference to x should remain even after the function has exited as the
> possibility of reference still exists if, for example, the function
> foo returns a lambda form.
>
> This seems to be consistent with what Steele says in Chapter 3 of
> Common Lisp, The language on Scope and Extent. (Linked Below)
>
> http://www.supelec.fr/docs/cltl/clm/node43.html
>
> The book even uses the compose function as an example of a similar case:
>
> (defun compose (f g)
> #'(lambda (x)
> (funcall f (funcall g x))))
>
> Are you saying that if I declare f and g as special before defining
> the compose function I can no longer use the compose function? Because
> that seems to be the CMUCL behavior.
Yes. And it's not just CMUCL--that's the specified behavior of
globally special variables in Common Lisp.
-Peter
--
Peter Seibel * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
In article <··············@gigamonkeys.com>,
Peter Seibel <·····@gigamonkeys.com> wrote:
> Yes. And it's not just CMUCL--that's the specified behavior of
> globally special variables in Common Lisp.
But what I think is unique about CMUCL is that using a free variable at
top-level proclaims it special pervasively. Most other implementations
just do it temporarily. I.e. in CMUCL
(setq x 3)
is equivalent to
(defvar x)
(setq x 3)
but in most other implementations it's equivalent to
(locally (declare (special x))
(setq x 3))
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
Sachin Kamboj <······@cis.udel.edu> writes:
> I have a question about the way CMUCL handles lexically scoped
> variables. Consider the following code:
>
> (defun foo (x)
> #'(lambda (y)
> (+ x y)))
>
> However, if *_before_* defining the function foo, I define a special
> variable x, like:
>
> (setf x '(1 2))
>
> And then define foo as above, on running (funcall (foo 4) 5), I get the
> error:
It is a bug in CMUCL. The bug is that the text of the
warning message is wrong.
Notice that you can mix special and lexical variables and
closures. They play nicely together, and although the code
can be hard to understand it does the right thing. For
example:
CL-USER> (let ((x 1/100))
(declare (special x))
(flet ((add-to(x)
(print (symbol-value 'x))
(lambda(y)(+ x y))))
(funcall (add-to 5) 7)))
1/100
=> 12
However, special declarations work at two different levels
of granularity, binding by binding, or symbol by symbol.
Binding by binding
When you use (declare (special x)) you make that binding
and its references dynamic, but not pervasively so. The
inner binding created by (flet ((add-to (x) ..)))
shadows the outer one, and reverts to the default lexical
scoping rule. So the closure works and we may resort to
(symbol-value 'x) to get at the now shadowed dynamic
binding.
Symbol by symbol
When you use (proclaim '(special x)), whether explicitly
or with (declaim (special x)) or (defvar x) or
(defparameter x) it is the symbol that become special,
not just one binding. This violates referential
transparency. Inner closures get broken by specialness
leaking in unasked. Hence the use of the *asterisk*
convention to create a separate namespace for symbols
that have been proclaimed special.
The specification doesn't say what is to be done with
(setf x 'something) at the top level, except in the case
that it is preceded by (defvar x) or (defparameter x). Well
it is clear what happens in those cases and you almost
certainly wanted to say (defvar *x*)(setf *x* 'something)
instead.
So CMUCL gets to chose what to do with a bare, toplevel
(setf x). It choses to proclaim it to be special, just as if
you had preceded it by (defvar x). That's OK.
Then it gives a warning message
Warning: Declaring X special.
It could have said "Proclaiming X special" but it
doesn't. There is only one possible interpretation of choice
of "Declaring" instead of "Proclaiming" in the text of the
warning message: the text of the warning message is telling
its reader that X has been declared special in the binding
by binding sense, and will play nicely with closures.
In fact, when it issues this warning message, CMUCL is
proclaiming that the symbol is special, which will break
closures. So the text of the warning message is wrong.
Imagining myself as Dr. Evil, plotting to harm Common Lisp
by the malicious use of incorrect warning messages, I find
myself at a loss to imagine how I could make this warning
message worse. Indeed I feel outclassed. Any crude attempt
to mislead users with incorrect warning messages will fail
as users learn to distrust warnings. The way in which
"Declaring X special" offers a false reassurance in an
especially difficult area of the language blocks any attempt
by a beginner to work out for himself that it is the warning
message itself that is in error. No other wording could vex
a beginner more greviously nor frustrate him more
completely. It is the perfection of evil.
Alan Crowe
Edinburgh
Scotland
Alan Crowe <····@cawtech.freeserve.co.uk> wrote:
+---------------
| It is a bug in CMUCL. The bug is that the text of the
| warning message is wrong.
...
| The specification doesn't say what is to be done with
| (setf x 'something) at the top level [absent a decl.]...
|
| So CMUCL gets to chose what to do with a bare, toplevel
| (setf x). It choses to proclaim it to be special, just as if
| you had preceded it by (defvar x). That's OK.
|
| Then it gives a warning message
| Warning: Declaring X special.
| It could have said "Proclaiming X special" but it doesn't.
+---------------
This is made more amusing(?) by the fact that it really *is* merely
PROCLAIM'ing! From the CMUCL code for EVAL in "src/code/eval.lisp":
...
(let ((symbol (first name)))
(case (info variable kind symbol)
(:special)
(:global
(case *top-level-auto-declare*
(:warn
(warn "Declaring ~S special." symbol))
((t))
((nil)
(return (eval:internal-eval original-exp))))
#| Here ==> |# (proclaim `(special ,symbol)))
(t
(return (eval:internal-eval original-exp)))))
...
+---------------
| In fact, when it issues this warning message, CMUCL is
| proclaiming that the symbol is special, which will break
| closures. So the text of the warning message is wrong.
+---------------
Easily fixed... ;-} ;-}
cmu> (compile-file "eval-patch.lisp")
...[chatter]
#p"/usr/u/rpw3/eval-patch.x86f"
NIL
NIL
cmu> (load *)
; Loading #p"/usr/u/rpw3/eval-patch.x86f".
...[drop into debugger because of locked package, select CONTINUE]...
T
cmu> (setf foo 37)
Warning: Proclaiming FOO special.
37
cmu>
Is that better? ;-} ;-}
-Rob
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Thanks to everyone who responded.
I guess I wasn't aware of the exact behavior of special variables and
the difference between declare and proclaim. This really helped clarify
things.
Thanks,
Sachin.
····@rpw3.org (Rob Warnock) writes:
> Alan Crowe <····@cawtech.freeserve.co.uk> wrote:
> +---------------
> | It is a bug in CMUCL. The bug is that the text of the
> | warning message is wrong.
> +---------------
>
> cmu> (compile-file "eval-patch.lisp")
> ...[chatter]
> #p"/usr/u/rpw3/eval-patch.x86f"
> NIL
> NIL
> cmu> (load *)
>
> ; Loading #p"/usr/u/rpw3/eval-patch.x86f".
> ...[drop into debugger because of locked package, select CONTINUE]...
> T
> cmu> (setf foo 37)
> Warning: Proclaiming FOO special.
>
> 37
> cmu>
>
> Is that better? ;-} ;-}
Yes. It is a huge improvement, but are you just toying with
me? Are you one of the maintainers for CMUCL? Is that it
really fixed, or will 19c be back to saying "Warning:
Declaring FOO special."?
Alan Crowe
Edinburgh
Scotland
Alan Crowe <····@cawtech.freeserve.co.uk> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > ; Loading #p"/usr/u/rpw3/eval-patch.x86f".
| > ...[drop into debugger because of locked package, select CONTINUE]...
| > T
| > cmu> (setf foo 37)
| > Warning: Proclaiming FOO special.
| >
| > 37
| > cmu>
| >
| > Is that better? ;-} ;-}
|
| Yes. It is a huge improvement, but are you just toying with me?
+---------------
Well, maybe. ;-} It depends on how you look at it. The patch code
was real, and the messages were real.
+---------------
| Are you one of the maintainers for CMUCL? Is that it really fixed,
| or will 19c be back to saying "Warning: Declaring FOO special."?
+---------------
No. No, and probably.
But the *real* point was that you can fix it yourself, just as I did,
*without* needing to build all of CMUCL. That's one of the really nice
things about Lisp, IMHO. You don't have to sit on your thumbs and wait
for the "official" maintainers to fix something.[1]
-Rob
[1] Well, usually. At least for small stuff like the above, or, for
another example, the CMUCL & SBCL problem with place-mutating forms
applied to symbol macros that somebody posted a fix for recently.
Doubtless from time to time there appear pervasive and subtle
problems that can't be fixed with a simple overlay patch, but
they seem to be much more rare.
-----
Rob Warnock <····@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
In article <······················@speakeasy.net>,
····@rpw3.org (Rob Warnock) wrote:
> But the *real* point was that you can fix it yourself, just as I did,
> *without* needing to build all of CMUCL. That's one of the really nice
> things about Lisp, IMHO. You don't have to sit on your thumbs and wait
> for the "official" maintainers to fix something.[1]
But there's little point in fixing this warning message yourself. If
you know enough to fix it, you don't really need the fix -- you know
that when it says "declaring" it means "proclaiming". It needs to be
fixed in the official version for the benefit of new users, like the OP.
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
>>>>> "Barry" == Barry Margolin <······@alum.mit.edu> writes:
Barry> In article <······················@speakeasy.net>,
Barry> ····@rpw3.org (Rob Warnock) wrote:
>> But the *real* point was that you can fix it yourself, just as I did,
>> *without* needing to build all of CMUCL. That's one of the really nice
>> things about Lisp, IMHO. You don't have to sit on your thumbs and wait
>> for the "official" maintainers to fix something.[1]
Barry> But there's little point in fixing this warning message yourself. If
Barry> you know enough to fix it, you don't really need the fix -- you know
Barry> that when it says "declaring" it means "proclaiming". It needs to be
Barry> fixed in the official version for the benefit of new users, like the OP.
If you really want "Declaring" changed to "Proclaiming", that's easy
enough to fix.
I submit, though, that new users will still be confused.
Ray
Raymond Toy <···········@ericsson.com> writes:
> If you really want "Declaring" changed to "Proclaiming", that's easy
> enough to fix.
>
> I submit, though, that new users will still be confused.
I find it a little tricky to understand what is meant by
this cavil.
It is very easy, reading the documentation, to pick up the
usuage that "declaring special" denotes declaring special at
the binding-by-binding level of granularity, while "proclaiming
special" denotes declaring special at the symbol-by-symbol
level of granularity. For example, on page 224 of CLtL2
Steele writes
A |special| declaration does /not/ affect bindings
pervasively. Inner bindings of a variable implicitly
shadow a |special| declaration and must be explicitly
redeclared to be special. (However, a |special|
proclamation /does/ prevasively affect bindings; this
exception is made for reasons of convenience and
compatibility with MacLisp.)
Against this background it is natural to read "Declaring X
special" contrastively. The reader considers that the writer
of the text of the warning message chose between
Declaring X special
and
Proclaiming X special
in order to convey the distinction that Steele is making.
Changing the wording from "Declaring" to "Proclaiming" will
succeed completely in protecting new users from this
particular confusion.
Obviously new users will continue to be confused by scope
and extent in CL because it is a difficult area with many
sources of confusion. However I think it is reasonable to
model the difficulties caused by multiple sources of
confusion as a non-linear process. Perhaps the difficulties
caused by multiple sources of confusion go as the square of
the number of sources of confusion. Thus resolving one
source of confusion in a difficult area confers a much
greater benefit on new users than resolving a source of
confusion that stands on its own. From this perspective
changing from Declaring to Proclaiming is progress in
untangling a knotty area of language. The fact that users
will continue to be confused only heightens the importance
of the change by pointing out that it is focussed on a
problem area in which small changes can be expected to
confer large benefits.
I urge attention to the double significance of the fact that
the change is easy to make. First the ratio of the benefit
to new users to the effort required to make the change is
especially favourable. Second new users are themselves
programmers and aware that changing the text of warning
messages is easily done. Consequently they are likely infer
that since they are using version 19 all infelicities in the
wording of the warning messages must have been smoothed out
long ago. It is perfectly possible that some-one who has
wasted a couple of hours because they over-interpreted
"Declaring X special" will then have the thought: "Oh, they
must mean `Proclaiming X special.'." and then reject that
hypothesis and waste another couple of hours because they
assume that such a simple matter of wording would have been
fixed ages ago, version 3 or version 4.
Changing the wording of the warning message from Declaring
to Proclaiming would be a big improvement to CMUCL, and
no-one should be lead into under estimating the improvement
to the user experience by the fact that the change is
technically very easy.
Alan Crowe
Edinburgh
Scotland
Alan Crowe wrote:
> Changing the wording of the warning message from Declaring
> to Proclaiming would be a big improvement to CMUCL, and
> no-one should be lead into under estimating the improvement
> to the user experience by the fact that the change is
> technically very easy.
I've been thinking about this issue, in a broader sense. Here's some ideas.
1. Suppress the message if the symbol is *enclosed-in-splats* but make
the message all caps and add the blink tag (and maybe an URL) if it
isn't. Downsides: The messages will be more annoying for legacy code,
and could exhibit slight performance problems with pathological code.
2. Don't allow the behaviour at all, by default. Let the user push a
flag onto *features* or something if he desires the old behaviour.
Rationale: Current behaviour is undefined by the spec, and not DEFVARing
specials beforehand is now just sloppy practice, as well as not being
portable. As Rob Warnock has pointed out, connoisseurs who know what
they're doing can easily change the implementation's behaviour exactly
to their liking anyway. Downsides: Breaks legacy code, which means a
spike in support list traffic upon release. Performance problems for
pathological code if *features* has to be checked every time.
If people in general think Alan's sense of the problem is overblown and
few newbies get bitten by this, I'm fine with the status quo. But I
definitely don't agree with Fred Gilham's attitude that this is a rite
of passage; once our intrepid explorer learns about 'undefined
behaviour' there's nowhere in the spec he can see them all enumerated,
nor will his implementation's docs typically contain a single list of
implementation-specific undefined-but-defined behaviours, thus all he
can learn is that he's been bitten by one issue and will likely be
bitten in future by others -- and that's no lesson at all. If the noob
experience can be enhanced without detracting from everyone else's
experience, I think it should be. Cryptic diagnostics are a curse.
····@rpw3.org (Rob Warnock) writes:
> +---------------
> | Are you one of the maintainers for CMUCL? Is that it really fixed,
> | or will 19c be back to saying "Warning: Declaring FOO special."?
> +---------------
>
> No. No, and probably.
>
> But the *real* point was that you can fix it yourself, just as I did,
> *without* needing to build all of CMUCL.
That is a true in general, but it doesn't apply to this
particular case. My concern is that I don't want others to
have the same hard struggle that I had to understand special
variables. I want that warning message fixed for the sake of
other beginners. Fixing it on my machine doesn't help
anyone. I've already been through special variable boot
camp, and it is not an experience that I'm likely to
forget. To help others, it has to be fixed in the official
source tree.
Alan Crowe
Ediburgh
Scotland
Alan Crowe <····@cawtech.freeserve.co.uk> writes:
> ····@rpw3.org (Rob Warnock) writes:
> > +---------------
> > | Are you one of the maintainers for CMUCL? Is that it really fixed,
> > | or will 19c be back to saying "Warning: Declaring FOO special."?
> > +---------------
> >
> > No. No, and probably.
> >
> > But the *real* point was that you can fix it yourself, just as I did,
> > *without* needing to build all of CMUCL.
>
> That is a true in general, but it doesn't apply to this
> particular case. My concern is that I don't want others to
> have the same hard struggle that I had to understand special
> variables. I want that warning message fixed for the sake of
> other beginners. Fixing it on my machine doesn't help
> anyone. I've already been through special variable boot
> camp, and it is not an experience that I'm likely to
> forget. To help others, it has to be fixed in the official
> source tree.
I bet it would have a much greater chance of finding it into the next
CMUCL release if you posted your patch to the cmucl-devel or -help
list, instead of in the middle of a thread called "Scoping Question"
on comp.lang.lisp.
--
/|_ .-----------------------.
,' .\ / | Free Mumia Abu-Jamal! |
,--' _,' | Abolish the racist |
/ / | death penalty! |
( -. | `-----------------------'
| ) |
(`-. '--.)
`. )----'
The CMUCL behavior is customizable using the *top-level-auto-declare*
variable.
* (setf *top-level-auto-declare* nil)
NIL
* (setf foo 3)
;
; Warning: This variable is undefined:
; FOO
;
3
* foo
3
*
*TOP-LEVEL-AUTO-DECLARE* is an external symbol in the EXTENSIONS
package.
It is a special variable; its value is :WARN.
WARN is an external symbol in the KEYWORD package.
It is a constant; its value is :WARN.
Special documentation:
This variable controls whether assignments to unknown variables at top-level
(or in any other call to EVAL of SETQ) will implicitly declare the variable
SPECIAL. These values are meaningful:
:WARN -- Print a warning, but declare the variable special (the default.)
T -- Quietly declare the variable special.
NIL -- Never declare the variable, giving warnings on each use.
So set it to "NIL" in your init.lisp.
As far as noobs are concerned, sometimes you have to take a few
lumps....setq or setf at the top level is undefined behavior. I think
many lisp noobs learn the meaning of undefined behavior by stumbling
over this issue, and it certainly brings the issue of what a special
variable is to the forefront.
--
Fred Gilham ······@csl.sri.com
I think it's pretty obvious that the worship of that false idol known
as the State has, in the 20th Century, had some very bad effects. The
historians I'm familiar with have settled on the number of dead as 177
million, although I've seen estimates of up to 200 million.
-Bob Wallace