From: Vladimir V. Zolotych
Subject: compiled/interpreted fuctions
Date: 
Message-ID: <3AD01F52.4BBAF0DC@eurocom.od.ua>
Hello

I'm using CMUCL 18c.

I've encountered some behavior that I can't understand. Would you mind
help me a bit ? The problem is. I have the following function.

(defun make-linear (data)
  "Returns linear regression Y = A0 + A1 * X for DATA 
or NIL if it is impossible to build one."
  (multiple-value-bind (a0 a1 valid-p)
      (get-l-h-coefficients data)
    (when valid-p
      #'(lambda (x)
	  (+ a0 (* a1 x))))))

While MAKE-LINEAR is interpreted function all seems work.
1] (make-linear foo-data)

#<Interpreted Function "LAMBDA (#:G1297)" {4820E899}>
1] (funcall * 0)

3954.0625
1] 

If I load MAKE-LINEAR from compiled file (FASL)
I see.
1] (load "home:lisp/src/regression.x86f")
; Loading #p"/home/vlz/lisp/src/regression.x86f".
T
1] (make-linear foo-data)

#<Function "DEFUN MAKE-LINEAR" {48008711}>

;;; What does the line above mean, it seems to me not obvious.

1] (describe *)

#<Function "DEFUN MAKE-LINEAR" {48008711}> is function.
Arguments:
  (x)
Its defined argument types are:
  (T)
Its result type is:
  NUMBER
On Saturday, 4/7/01 05:40:22 pm [-2] it was compiled from:
/home/vlz/lisp/src/./regression.lisp
  Created: Saturday, 4/7/01 05:40:01 pm [-2]

1] (funcall ** 0)

Error in KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER:  the variable A0 is unbound.

Restarts:
  0: [ABORT] Return to debug level 1.
  1:         Return to Top-Level.

Debug  (type H for help)

("DEFUN MAKE-LINEAR" 0)
Source: (+ A0 (* A1 X))
0]] 

Please show me my misunderstanding point.

-- 
Vladimir Zolotych                         ······@eurocom.od.ua

From: Kent M Pitman
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <sfwwv8vik60.fsf@world.std.com>
"Vladimir V. Zolotych" <······@eurocom.od.ua> writes:

> (defun make-linear (data)
>   "Returns linear regression Y = A0 + A1 * X for DATA 
> or NIL if it is impossible to build one."
>   (multiple-value-bind (a0 a1 valid-p)
>       (get-l-h-coefficients data)
>     (when valid-p
>       #'(lambda (x)
> 	  (+ a0 (* a1 x))))))
> 
> While MAKE-LINEAR is interpreted function all seems work.
>
> If I load MAKE-LINEAR from compiled file (FASL)

Of course, we don't know the source was the same as what you've shown
here.  But supposing it is, I'm going to guess that you compiled it in
an environment where you had been debugging something and had
mentioned a0 as a variable to the read-eval-print loop, which in CMU
CL will auto-proclaim the variable special for you.  If you had then
subsequently compiled this file in the same environment, it would be
erroneously not creating closures over a0.  

In all cases where a compiled file behaves weirdly, I recommend a clean
recompilation in an environment you haven't been debugging in.  If the
problem persists, it's probably something else.

Note that I'm not a CMU CL user.  I've just seen a lot of reports about
this questionable "feature" of CMU CL.  (I really don't know why someone
doesn't "fix" CMU CL not to do this.)
From: Pierre R. Mai
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <87n19rpfo9.fsf@orion.bln.pmsf.de>
Kent M Pitman <······@world.std.com> writes:

> Note that I'm not a CMU CL user.  I've just seen a lot of reports about
> this questionable "feature" of CMU CL.  (I really don't know why someone
> doesn't "fix" CMU CL not to do this.)

Well, the behaviour is under complete user control, so anyone can
change the it as they like:

* (describe 'ext:*top-level-auto-declare*)

*TOP-LEVEL-AUTO-DECLARE* is an external symbol in the EXTENSIONS package.
It is a special variable; its value is :WARN.
   WARN is an external symbol in the KEYWORD package.
   It is a constant; its value is :WARN.
Special documentation:
  This variable controls whether assignments to unknown variables at top-level
   (or in any other call to EVAL of SETQ) will implicitly declare the variable
   SPECIAL.  These values are meaningful:
     :WARN  -- Print a warning, but declare the variable special (the default.)
      T     -- Quietly declare the variable special.
      NIL   -- Never declare the variable, giving warnings on each use.
* (setq ext:*top-level-auto-declare* nil)

With the default value it will warn the user that it is going to
declare the variable special, so it's not as if CMU CL is silently
doing something "special".

But I'll see if there is support for changing the default to nil for 
*top-level-auto-declare*.

Regs, Pierre.

-- 
Pierre R. Mai <····@acm.org>                    http://www.pmsf.de/pmai/
 The most likely way for the world to be destroyed, most experts agree,
 is by accident. That's where we come in; we're computer professionals.
 We cause accidents.                           -- Nathaniel Borenstein
From: Raymond Toy
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <4nwv8u42df.fsf@rtp.ericsson.se>
>>>>> "Pierre" == Pierre R Mai <····@acm.org> writes:

    Pierre> * (setq ext:*top-level-auto-declare* nil)

    Pierre> With the default value it will warn the user that it is going to
    Pierre> declare the variable special, so it's not as if CMU CL is silently
    Pierre> doing something "special".

When I do this, I sometimes get messages about "undefined variable",
depending on how I use that variable.  This is also annoying.

Ray
From: Kent M Pitman
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <sfwlmpa87lx.fsf@world.std.com>
Raymond Toy <···@rtp.ericsson.se> writes:

> >>>>> "Pierre" == Pierre R Mai <····@acm.org> writes:
> 
>     Pierre> * (setq ext:*top-level-auto-declare* nil)
> 
>     Pierre> With the default value it will warn the user that it is
>     Pierre> going to declare the variable special, so it's not as if
>     Pierre> CMU CL is silently doing something "special".
> 
> When I do this, I sometimes get messages about "undefined variable",
> depending on how I use that variable.  This is also annoying.

Might it be because you haven't defined the variable?

Seems to me that either you have defined it and it's an implementation
bug that you should be tracking down and reporting so that not only you
but everyone can benefit from a fix, OR it's a bug in your usage of 
DECLARE, DECLAIM, PROCLAIM, DEFVAR, and/or DEFPARAMETER.  In the latter
case, Common Lisp requires that you do these operations explicitly, so 
muffling the message isn't going to make it all better.

Perhaps you occasionally need 
 (LOCALLY (DECLARE (SPECIAL ...))
   ...)
around top-level SETQ's when you have your files not loading in the right
order to see the declaration before the SETQ of the declared variable.

I have to say that whenever I hear someone trying to avoid an annoying
message, I get very suspicious that they are treating the symptom (the
message) and not the cause (the thing the message is telling them).
From: Pierre R. Mai
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <87n19qgksv.fsf@orion.bln.pmsf.de>
Raymond Toy <···@rtp.ericsson.se> writes:

> >>>>> "Pierre" == Pierre R Mai <····@acm.org> writes:
> 
>     Pierre> * (setq ext:*top-level-auto-declare* nil)
> 
>     Pierre> With the default value it will warn the user that it is going to
>     Pierre> declare the variable special, so it's not as if CMU CL is silently
>     Pierre> doing something "special".
> 
> When I do this, I sometimes get messages about "undefined variable",
> depending on how I use that variable.  This is also annoying.

Can you elaborate on this?  If CMU CL warns about undefined variable
for conforming code, then this is a serious bug IMHO, and we should
fix it ASAP.

After a quick glance at the CMU CL code base though, it seems to me
that there is no way you could get an undefined variable warning with
*top-level-auto-declare* being nil, where you wouldn't get at least
one "Declaring ~S special." warning with *top-level-auto-declare*
being :warn (the default).  Furthermore you can only get undefined
variable warnings when there is no special declaration in effect,
which would either indicate a very serious bug in CMU CL, or
non-conforming code.

Of all the code samples I've tried out, the only ones where I come
across "undefined variable" warnings are those that try to use
"global lexicals", but that is non-conforming in any case, since CL
doesn't have those.

Or maybe I'm misunderstanding you, and you would just like to have a
fourth option for *top-level-auto-declare*, that is to nil what t is
to :warn?  Though I don't see how that would be terribly useful unless
we define and document the exact semantics of "global lexicals", but
in that case no warnings would be issued anyway, so this option is
again a bit useless...

Regs, Pierre.

PS: Maybe we should continue this on the implementors mailing list?

-- 
Pierre R. Mai <····@acm.org>                    http://www.pmsf.de/pmai/
 The most likely way for the world to be destroyed, most experts agree,
 is by accident. That's where we come in; we're computer professionals.
 We cause accidents.                           -- Nathaniel Borenstein
From: Raymond Toy
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <4nwv8ut3dl.fsf@rtp.ericsson.se>
>>>>> "Pierre" == Pierre R Mai <····@acm.org> writes:

    Pierre> Raymond Toy <···@rtp.ericsson.se> writes:
    >> When I do this, I sometimes get messages about "undefined variable",
    >> depending on how I use that variable.  This is also annoying.

    Pierre> Of all the code samples I've tried out, the only ones where I come
    Pierre> across "undefined variable" warnings are those that try to use
    Pierre> "global lexicals", but that is non-conforming in any case, since CL
    Pierre> doesn't have those.

This is the situation I'm seeing.  I setq a var, and it's nicely not
declared special.  But then whenever I use it, I get a warning about it
being undefined, but then the right value is used.

As a user, it's rather odd that this "undefined variable" actually has
a value.  But declaring it special is also yucky.

I just live with the warnings or do a defvar for the variables I want,
with the understanding of what defvar does (and I put *'s around the
var now too, just to be sure).

Ray

P.S.  I hope this clears up Kent's comments too.
From: Kent M Pitman
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <sfw4rvyt1ql.fsf@world.std.com>
Raymond Toy <···@rtp.ericsson.se> writes:

> >>>>> "Pierre" == Pierre R Mai <····@acm.org> writes:
> 
>     Pierre> Raymond Toy <···@rtp.ericsson.se> writes:
>     >> When I do this, I sometimes get messages about "undefined variable",
>     >> depending on how I use that variable.  This is also annoying.
> 
>     Pierre> Of all the code samples I've tried out, the only ones where I come
>     Pierre> across "undefined variable" warnings are those that try to use
>     Pierre> "global lexicals", but that is non-conforming in any case, since CL
>     Pierre> doesn't have those.
> 
> This is the situation I'm seeing.  I setq a var, and it's nicely not
> declared special.  But then whenever I use it, I get a warning about it
> being undefined, but then the right value is used.
> 
> As a user, it's rather odd that this "undefined variable" actually has
> a value.  But declaring it special is also yucky.
>
> I just live with the warnings or do a defvar for the variables I want,
> with the understanding of what defvar does (and I put *'s around the
> var now too, just to be sure).
> 
> Ray
> 
> P.S.  I hope this clears up Kent's comments too.

Sort of.  But let me be clear.  As a user, you should not be making up
semantics you want and then calling deviations from your wishes
"yucky".  You should be relying on either the language definition or
the vendor documentation to guide your use.  No matter how strange it
seems to you from a design point of view, the language definition does
say that the only global variables are special variables.  There is no
other meaning to reading or assigning a free variable, however much
you (and I, incidentally) might wish there was.  The unwanted messages
you're getting are reminding you that the implementation's behavior is
dictated by reality and not wishes.  What you are doing is not
portable and works only by accident.  When you write
correct/conforming/portable code, you won't get the messages.
From: Raymond Toy
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <4nk84tubzy.fsf@rtp.ericsson.se>
>>>>> "Kent" == Kent M Pitman <······@world.std.com> writes:

    Kent> Raymond Toy <···@rtp.ericsson.se> writes:
    >> P.S.  I hope this clears up Kent's comments too.

    Kent> Sort of.  But let me be clear.  As a user, you should not be making up
    Kent> semantics you want and then calling deviations from your wishes
    Kent> "yucky".  You should be relying on either the language definition or
    Kent> the vendor documentation to guide your use.  No matter how strange it
    Kent> seems to you from a design point of view, the language definition does
    Kent> say that the only global variables are special variables.  There is no
    Kent> other meaning to reading or assigning a free variable, however much
    Kent> you (and I, incidentally) might wish there was.  The unwanted messages
    Kent> you're getting are reminding you that the implementation's behavior is
    Kent> dictated by reality and not wishes.  What you are doing is not
    Kent> portable and works only by accident.  When you write
    Kent> correct/conforming/portable code, you won't get the messages.

Sorry, I didn't mean to imply these things.  I just wish the behavior
of CMUCL were more consistent.  I have no problem with CMUCL
automatically making variables special, since that seems to be allowed
by the standard.

I just find it annoying that when you set the magic variable to NIL,
CMUCL nicely doesn't make variables special, but then complains that
the variable is undefined but gives it a value anyway.  It should be
either be undefined and have no value or stop complaining about it
while giving it a value.  Of course, doing this would probably make
others complain even more loudly.
 
This also explains why I haven't seen this warning in a long time,
even though I had set the magic variable to NIL.  I *always* use
defvar when I'm in the top-level repl.

Ray
From: Mike McDonald
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <xFHA6.844$Hp.39619@typhoon.aracnet.com>
In article <··············@rtp.ericsson.se>,
	Raymond Toy <···@rtp.ericsson.se> writes:

> I just find it annoying that when you set the magic variable to NIL,
> CMUCL nicely doesn't make variables special, but then complains that
> the variable is undefined but gives it a value anyway.  It should be
> either be undefined and have no value or stop complaining about it
> while giving it a value.  Of course, doing this would probably make
> others complain even more loudly.

  Just for clarity, here's an example:

USER(1): (setq fuzzy t)

Warning: This variable is undefined:
  FUZZY

T
USER(2): 

> This also explains why I haven't seen this warning in a long time,
> even though I had set the magic variable to NIL.  I *always* use
> defvar when I'm in the top-level repl.
> 
> Ray
> 

  I never use defvar at the interactive read-eval-print loop. defvar won't
change the value if it's already set. My main use of assignment of the r-v-p
loop is for setting temp vars to keep track of old results, not for setting up
globals that functions will implicitly use. But then I don't use foo, bar, and
baz in any of my code so I don't have any wierd interactions.

  Mike McDonald
  ·······@mikemac.com
From: Raymond Toy
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <4ng0fg4nr8.fsf@rtp.ericsson.se>
>>>>> "Mike" == Mike McDonald <·······@mikemac.com> writes:

    Mike> In article <··············@rtp.ericsson.se>,
    Mike> 	Raymond Toy <···@rtp.ericsson.se> writes:

    Mike>   I never use defvar at the interactive read-eval-print loop. defvar won't
    Mike> change the value if it's already set. My main use of assignment of the r-v-p
    Mike> loop is for setting temp vars to keep track of old results, not for setting up
    Mike> globals that functions will implicitly use. But then I don't use foo, bar, and
    Mike> baz in any of my code so I don't have any wierd interactions.


Same here. I never have so many I can't keep track of them, so I know
if it's been set or not.  They are definitely NOT meant as globals for
functions to use, other than some hack functions I typed into the
listener.

I've been bitten by CMUCL's special treatment before and spent lots of
time wondering why things suddenly changed.  I admit being lazy.  Some
of my functions had args named x, and I set x at top-level, and so on.
Why should I write and remember 50-char names for some junk variable
to hold some possibly interesting result? :-)

Ray
From: Tim Moore
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <9asuap$har$0@216.39.145.192>
On Mon, 9 Apr 2001, Kent M Pitman wrote:

> Raymond Toy <···@rtp.ericsson.se> writes:
> 
> > >>>>> "Pierre" == Pierre R Mai <····@acm.org> writes:
> > This is the situation I'm seeing.  I setq a var, and it's nicely not
> > declared special.  But then whenever I use it, I get a warning about it
> > being undefined, but then the right value is used.
> > 
> > As a user, it's rather odd that this "undefined variable" actually has
> > a value.  But declaring it special is also yucky.
...
> Sort of.  But let me be clear.  As a user, you should not be making up
> semantics you want and then calling deviations from your wishes
> "yucky".  You should be relying on either the language definition or
> the vendor documentation to guide your use.  No matter how strange it

First off, why not?  If the semantics are "yucky" they are "yucky,"
whether or not they are a user's wish or mandated by the standard.
Second, Raymond and Pierre are as about as close as we get to vendors of
CMUCL; I think their comments should be taken in that light, not that of
J. Random Luser whining about the language.

 > seems to you from a design point of view, the language definition
does
> say that the only global variables are special variables.  There is no
> other meaning to reading or assigning a free variable, however much
> you (and I, incidentally) might wish there was.  The unwanted messages
> you're getting are reminding you that the implementation's behavior is
> dictated by reality and not wishes.  What you are doing is not
> portable and works only by accident.  When you write
> correct/conforming/portable code, you won't get the messages.

Tim
From: Raymond Toy
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <4ng0fhubqf.fsf@rtp.ericsson.se>
>>>>> "Tim" == Tim Moore <·····@herschel.bricoworks.com> writes:

    Tim> First off, why not?  If the semantics are "yucky" they are "yucky,"
    Tim> whether or not they are a user's wish or mandated by the standard.
    Tim> Second, Raymond and Pierre are as about as close as we get to vendors of
    Tim> CMUCL; I think their comments should be taken in that light, not that of
    Tim> J. Random Luser whining about the language.

Thanks for the kind words, but I'm not a "vendor" of CMUCL.  I use it,
make changes to it, and help support it, but vendor is too much.  I'm
just a luser who knows how to modify some parts of CMUCL to my needs.

Ray
From: Vladimir V. Zolotych
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <3AD08DE8.FAD8C900@eurocom.od.ua>
Kent M Pitman wrote:
> 
> ...But supposing it is, I'm going to guess that you compiled it in
> an environment where you had been debugging something and had
> mentioned a0 as a variable to the read-eval-print loop,

Thanks.
(SETQ A0 ...) was the cause.
I've added 
(setf ext:*top-level-auto-declare* nil)
to my .cmucl-init.lisp file.

-- 
Vladimir Zolotych                         ······@eurocom.od.ua
From: Pierre R. Mai
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <87u23zplri.fsf@orion.bln.pmsf.de>
"Vladimir V. Zolotych" <······@eurocom.od.ua> writes:

> (defun make-linear (data)
>   "Returns linear regression Y = A0 + A1 * X for DATA 
> or NIL if it is impossible to build one."
>   (multiple-value-bind (a0 a1 valid-p)
>       (get-l-h-coefficients data)
>     (when valid-p
>       #'(lambda (x)
> 	  (+ a0 (* a1 x))))))
> 
> While MAKE-LINEAR is interpreted function all seems work.
> 1] (make-linear foo-data)
> 
> #<Interpreted Function "LAMBDA (#:G1297)" {4820E899}>
> 1] (funcall * 0)
> 
> 3954.0625
> 1] 
> 
> If I load MAKE-LINEAR from compiled file (FASL)
> I see.
> 1] (load "home:lisp/src/regression.x86f")
> ; Loading #p"/home/vlz/lisp/src/regression.x86f".
> T
> 1] (make-linear foo-data)
> 
> #<Function "DEFUN MAKE-LINEAR" {48008711}>
> 
> ;;; What does the line above mean, it seems to me not obvious.

It says that make-linear returned a compiled function object, for an
anonymous lambda that was created in MAKE-LINEAR.  In that way you can
differentiate between different anonymous functions more easily.

The interesting point to note here, though is that the returned
compiled function object is _not_ a closure, which is what you
probably expected, i.e. I get:

* (load *)

; Loading #p"/tmp/demo.x86f".
T
* (make-linear 0)

#<Closure Over Function "DEFUN MAKE-LINEAR" {4810E441}>
* (funcall * 0)

1

Since you don't get a closure, that means that the function didn't
close over a0 and a1, and hence you get this error:

> 1] (funcall ** 0)
> 
> Error in KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER:  the variable A0 is unbound.

The usual reason why your code doesn't create a closure for you, but
does create a closure for me, is that in the core where you compiled
make-linear, a0 and a1 are somehow declared special[1].  If this happened
accidentally, you might want to recompile make-linear (or the file
that contained its definition) in a new, clean core, and try again.

BTW: Why are you doing all this stuff while in the debugger?

Regs, Pierre.

Footnotes: 
[1]  Either because of a defvar or defparameter form, a manual special
     proclamation, or in CMU CL through a simple setq form at
     top-level for an unbound variable, if ext:*top-level-auto-declare*
     is true (the default).  I.e. a simple (setq a0 1 a1 2) at
     top-level could have resulted in your problems.

-- 
Pierre R. Mai <····@acm.org>                    http://www.pmsf.de/pmai/
 The most likely way for the world to be destroyed, most experts agree,
 is by accident. That's where we come in; we're computer professionals.
 We cause accidents.                           -- Nathaniel Borenstein
From: Vladimir V. Zolotych
Subject: Re: compiled/interpreted fuctions
Date: 
Message-ID: <3AD08C0F.91A59D9B@eurocom.od.ua>
"Pierre R. Mai" wrote:

> 
> > #<Function "DEFUN MAKE-LINEAR" {48008711}>
> >
> > ;;; What does the line above mean, it seems to me not obvious.
> 
> It says that make-linear returned a compiled function object, for an
> anonymous lambda that was created in MAKE-LINEAR.  In that way you can
> differentiate between different anonymous functions more easily.

Yes, its more convenient. I didn't guess such meaning.

> The usual reason why your code doesn't create a closure for you, but
> does create a closure for me, is that in the core where you compiled
> make-linear, a0 and a1 are somehow declared special[1].  If this happened
> accidentally, you might want to recompile make-linear (or the file
> that contained its definition) in a new, clean core, and try again.

You're right. I had recompiled these files. Now all works.

> BTW: Why are you doing all this stuff while in the debugger?

Nothing special here. Just had been finding a bug in debugger
and forget to quit it. SETQ for A0 I had done much earlier
in the environment which I had been using for compilation.

Thanks.

-- 
Vladimir Zolotych                         ······@eurocom.od.ua