From: Howard R. Stearns
Subject: Idiom For Context Sensitive Dynamic Context:
Date:
Message-ID: <36E6AA39.F8390CDC@elwood.com>
What is the Common Lisp idiom for executing a body of code in a
dynamic context in which some parameter will be set, but only if it
hasn't been already set? This comes up often enough in systems
programming, and is hard enough to get right, that I'd like to use
and/or develop a standard idiom for it. How this is done also effects
other things which we'd like to standardize, such as DEFSYSTEM and
perhaps other interfaces to system code (streams, CORBA, ...)
I'm going to give some example situations and some sucessively more
complicated implementations. I'm also going to show how this effects
WITH-COMPILATION-UNIT/DEFSYSTEM, and show an analogy to the condition
system.
Please note that I'm just asking questions here and not proposing a
particular solution or any standardization. In fact, I'm hoping that
someone will just say, "Oh, I just use xxx as follows..." I know that
before asking "how do I do this specific thing", one should generally
simplify and spell out the specific example of what one is trying to
do. In this case, I can't, because in this system programming
situation, I want to write code that cooperates with whatever the user
writes to solve the problems described here. That's why I think I
need to recognize (and implement) a standard idiom.
-----
There are situations in systems programming where the action taken by
some internal routine is coupled to a dynamic context. Some examples
include:
- Whether and when a compiler should issue warnings about undefined
or apparently misused entities.
- Details of how users should recieve notifications, such as which
stream or formatting style to use.
- Whether some distributed operation should compute things locally or
use some networked resource, and which resource and communication
path should be used.
In all these cases, it is insufficient to expect the direct caller of
the utility to have all the answers, so special variables tend to be
used.
SIMPLE SPECIAL VARIABLES:
At first glance, one might imagine that special variables would be
enough:
(let ((*warning-environment* (make-warning-environment)))
...
(let ((*standard-output* (make-some-stream))
(*user-notification-prefix* ";;;; "))
...
(let ((*default-lisp-engine* :localhost))
...
(free-engine *default-lisp-engine*))
(force-output *standard-output*))
(dump-warnings *warning-environment*))
In each case, a dynamic context is created in which the parameter is
bound appropriately. At the end of this time, a cleanup operation may
need to be performed.
A MACRO:
One could imagine abstracting this idiom into a macro that provided
for the right values to be returned, and used unwind-protect to make
sure that the right cleanups always happened:
(defmacro WITH-CONTEXT ((parameter &optional initform cleanup) &body
body)
`(let ((,parameter ,initform))
(unwind-protect (progn ,@body)
,cleanup)))
(with-context (*warning-environment* (make-warning-environment)
(dump-warnings *warning-environment*))
...
(with-context (*standard-output* *foo* (force-output
*standard-output*))
...
(with-context (*notification-prefix* ";;;;")
(with-context (*default-lisp-engine* :localhost
(free-host *default-lisp-engine*))
...))))
CONDITIONAL BINDING:
At this point, WITH-CONTEXT still doesn't handle the common problem of
not initializing unless necessarry. For example, to really defer
warnings, one person might write:
(with-context (*warning-environment* (make-warning-environment)
(dump-warnings *warning-environment*))
...)
but the whole thing might be executed in a dynamic context in which
someone else has wrapped an outer WITH-CONTEXT *warning-environment*.
Now we want something like this:
(defmacro WITH-CONTEXT ((parameter &optional initform cleanup) &body
body)
(with-unique-names (new-context?)
`(let* ((,new-context? (or (not (boundp ',parameter))
(null ,parameter)))
(,parameter (if ,new-context?
,initform
,parameter)))
(unwind-protect (progn ,@body)
(when ,new-context? ,cleanup)))))
The idea of binding a special variable to its old value might be
wrong. For example, some of the internal tools which make use of this
context information might need to ASSIGN a new value to the parameter.
They would expect this new value to get propogated back to the outer
context. Let's pretend, for now, that such assignment is simply
prohibited.
We might also want to give the user some control over whether a new
context should be introduced, just like :override in
WITH-COMPILATION-UNIT:
(defmacro WITH-CONTEXT ((parameter override &optional initform cleanup)
&body body)
(with-unique-names (new-context?)
`(let* ((,new-context? (or ,override ; This is the only new code
(not (boundp ',parameter))
(null ,parameter)))
(,parameter (if ,new-context?
,initform
,parameter)))
(unwind-protect (progn ,@body)
(when ,new-context? ,cleanup)))))
COMMUNICATING BETWEEN CONTEXTS:
Here's where I loose it. I'm not convinced that this is enough. I
have an intuition that instead of indicating a simple boolean for
override, we instead want to be able to specify a "situation" in which
we want to override or otherwise define the parameter.
Instead of just looking at the value of a global variable, the
interior routine which is coupled to the parameter really needs to
access a hierarchical situation. For example, instead of COMPILE doing
something like:
(note-warning *warning-environment*)
it instead does something like:
(note-warning (context-value *warning-environment*
'clos-gf-call))
where clos-gf-call is a subsituation of clos.
In the ANSI writeup for WITH-COMPILATION-UNIT, Kim Barrett wrote:
Consider a stream system built on an object system which will
compose and compile functions on the fly on an as needed basis. It
would be very strange for the functions so generated while doing
file io for the user's compile-file to have any relationship with
said compile-file.
And so Kim didn't want the warnings associated with the internal CLOS
compilation to appear with the warnings for the user's compile-file.
I suspect that it isn't enough to just say: "Give me warnings about my
code, but not warnings about system code that's done in service of the
things I ask for." The problem is that it's not always possible for
the system to distinguish between the two without guidance form the
user.
For example, what happens if one has a defconstant with a value form
that causes compilation "internally"? Well, depending on several
factors, this compilation might need to happen at load time as well,
so the user might need to see those warnings in association with all
of his other "user code" warnings.
Another example comes from a WITH-COMPILATION-UNIT extension, where a
user wants to specify that some block compilation activities are to be
accumulated in one way, while other block compilation activities are
accumulated in another way.
Thus I think we need to be able to label a situation in which the
binding will be relevent. Perhaps T denotes all contexts. It also
seems that, like conditions, one want to be able to access a parameter
with respect to a fairly broad situation (perhaps even T) and get back
any applicable value. In other words, a context is really defined by
TWO labels: a parameter and a situation. It is this complex context
that we wish to bind over some dynamic extent.
Now we're entering into a maze of twisty little passages, all
different:
Is this complex context best represented by two (or more) variables,
or by a single combined context object? (In other words, are
parameter and situation really indepent axes in the context space?)
Do users need to control situational inheritance/applicability? Is
multiple inheritance needed (i.e., can a situation be encompased
by two otherwise unrelated situations)?
How is this efficiently implemented?
I suspect that there I could find a lesson in the design of the
condition system. There one specifies a dynamic context which sets up
a handler specified in outer code for a condition signalled from
interior code. The condition is represented as an object. It seems
like what I've been describing is the inverse: A dynamic context is
set up in which a context is specified or "signaled" by outer code and
is read or "handled" by inner code. Perhaps there is even a way to
turn the existing condition system inside out to accomplish this
without a new set of constructs.
I'm not sure how nasty all this has to be. It might be the case that
not only WITH-COMPILATION-UNIT, but also EVAL-WHEN could be
implemented in terms of WITH-CONTEXT. Maybe its "the ultimate
<whatever special binding and unwind-protect are examples of>" (I
vaguely remember Barbara Liskov trying and apparently failing to teach
me something about strength and coupling...)
I don't want to get carried away. All I really know is that to do
"the right thing" for DEFSYSTEM, one needs to have something like
WITH-COMPILATION-UNIT that can be controlled by the user, and I'm not
at all sure that the design of WITH-COMPILATION-UNIT isn't broken.
Comments?
"Howard R. Stearns" <······@elwood.com> writes:
> What is the Common Lisp idiom for executing a body of code in a
> dynamic context in which some parameter will be set, but only if it
> hasn't been already set?
Is this a homework problem?
:-)
> This comes up often enough in systems
> programming, and is hard enough to get right, that I'd like to use
> and/or develop a standard idiom for it. How this is done also effects
> other things which we'd like to standardize, such as DEFSYSTEM and
> perhaps other interfaces to system code (streams, CORBA, ...)
My first reaction to your first paragraph was "use DEFVAR".
However, it becomes clear later that what you really want is
"environments". One version of this was proposed in CLtL2 (<174>
SYNTACTIC-ENVIRONMEN-ACCESS:SMALL, as described on pp 206-214 of
CLtL2), but was then withdrawn due to the facts that nobody had
implemented it yet, and there were semantic difficulties with the
system.
I am doing some research in the environments area, and it is my
intention (if I am successful) to release an environments implementation
(with source, of course) to the public, and further to propose such
additions to the ANSI spec. Anyone with ideas or problems is welcome
to respond on this ng or by mail; the more I know about what does or
doesn't work, the more likely I would be to succeed.
> I'm going to give some example situations and some sucessively more
> complicated implementations. I'm also going to show how this effects
> WITH-COMPILATION-UNIT/DEFSYSTEM, and show an analogy to the condition
> system.
Rather than quote and answer your whole article, I will put forth
what I am doing and maybe you can then analyze how what you want to
do relates to what I am proposing to propose. I must first say that
what I am doing is driven by two needs we have in our product:
source-level debugging and user-declared inline function support.
First, I am using the CLtL2 documentation as a guide for a first
implementation for lexical environments. I am not sure this will
stick, because of the lack of support for dynamic environments and
lack of extensibility. But it is a start, for now.
For definitions, I use the ANSI spec for some, and add a term of my own:
The spec distinguishes between "lexical" and "dynamic" environments,
but only talks about environment "objects" for the lexical environments.
The only dynamic environment that is discussed is the "global"
environment. However, anyone who deals with file compilation (or any
other manifestation of with-compilation-unit) knows that during such
time, things may be added to the "global" environment that will then
go away after compilation. The extent to which implementations clean
up this after-compilation global environment varies between
implementations. I have given this intermediate state of the global
environment a name: a "shadow" environment. If we could represent a
shadow environment as an object which is created for each outside or
overridden with-compilation-unit form, then cleanup would become more
specifiable, and there would be the potential for more precise
specification of the scope of various operations by identifying the
environments to with they pertain.
Another difference between lexical and dynamic environments is the
complexity of lexical-environment access and creation: Every time
an augment-environment returns a new environment, this info must be
kept/recorded somewhere in heirarchical fashion so that various
stages of compilation can analyze the entire lexical system within
the compilation unit. Such information could even be stored away to
become the information needed for such things like block-compilation,
code-rewriting, and source-level-debugging. Every CL implementation
has some method of storing just enough info to complete the compilation
process. But I am convinced that lexical environments, properly
designed, will regularize this info and make amazing tools available
portably that are not available now.
[Steve Haflich has worked with me to get a straw implementation that
allows for lexical environments to be implemented with the above
behaviors; I am not close to being ready to release it, but am
always implementing the code in such a way that the source can be
released when the concept is solid.]
Continuing with the difference discussion, any additions to a dynamic
environment tend not to want to be logged; instead, the environment as
a whole (whether it is a "shadow" object or simply the current state
of the lisp as the global environment).
I have not yet considered the best implementation for shadow
environments. How they are implemented will probably depend
on what CL functionality should access them. This seems to me to be
an easier problem than that of lexical environments, but any help
I can get would be welcome.
I have only worked for a couple of months on the nitty-gritty of
this environments problem, so those of you who know much more than
I about the details, please be gentle with me. I intend to learn
as much as possible, and at the very least report on what I have
found, if an implementation does not itself come of it.
Howard, let me know if this concept strikes a chord (or not) with what
you are trying to do. To me, environments seem like the natural way
to do what you are trying to do.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: Howard R. Stearns
Subject: Re: Idiom For Context Sensitive Dynamic Context:
Date:
Message-ID: <36E81484.ECF1E5DC@elwood.com>
Duane, I'm going to address your environment strategy in another
thread.
For this issue, though, I think you're a step (or more) ahead of me.
Are you saying that one ought to be able to exercise dynamic control
over lexical countours?
For example, in general, contours allow one to say "this variable
shadows that one -- use this value for any references to this name."
You want to have the inner code say "get met the 'relevent' value for
this name, and then have the environment mechanism supply the right
value.
Meanwhile, some outer code might want to say: "Extend the 'current'
environment with these new countours and have this become the new
'current' environment?
I guess I'm a little overwhelmed about how these concepts get
interleaved. It also seems like a bigger hammer than I had hoped to
need. I'm also not sure how making all this available in all compiled
code would effect performance.
"Howard R. Stearns" <······@elwood.com> writes:
> Duane, I'm going to address your environment strategy in another
> thread.
OK, I'll look for it.
> For this issue, though, I think you're a step (or more) ahead of me.
>
> Are you saying that one ought to be able to exercise dynamic control
> over lexical countours?
No, lexical contours should be controlled by lexical environments
(CL gives us this control already). However, dynamic control
could be controlled by capturing global and dynamic environments.
Currently, there is only one global and dynamic environment at a
time, and it essentially constitutes the state of the lisp (barring
multiprocessing considerations).
> For example, in general, contours allow one to say "this variable
> shadows that one -- use this value for any references to this name."
> You want to have the inner code say "get met the 'relevent' value for
> this name, and then have the environment mechanism supply the right
> value.
Yes, but in a dynamic sense. Compile-file already does this; as
compilation progresses (it encounters a defmacro, for example) the
definition is added temporarily to the global environment (i.e.,
a compile-time-too evaluation _will_ see the definition). But when
the compile-file is done, the macro definition might disappear. It is
this shadowing of global environments that I'd like to see formalized
in some way.
> Meanwhile, some outer code might want to say: "Extend the 'current'
> environment with these new countours and have this become the new
> 'current' environment?
Yes, of course there would be issues to deal with, and dependencies
on enclosing environments by outer ones would have to be defined.
But please bear in mind that my major goal with these is for
compilation. Your references to with-compilation-unit may have
been only as an example, but if you want truly dynamic shadowing
of environments, then perhaps this is not the right way to go.
However, if you are making the references to with-compilation-unit
because the problem you are trying to solve is either related to
compiling, or bears a similarity to compiling, then maybe this idea
would be apropos.
> I guess I'm a little overwhelmed about how these concepts get
> interleaved. It also seems like a bigger hammer than I had hoped to
> need. I'm also not sure how making all this available in all compiled
> code would effect performance.
Yep. After having lived on a ranch for many years, I came to
appreciate the difference between jobs that call for a shovel and
jobs that call for a front-loader on a tractor. However, the
tractor's controls take so little energy to manipulate, I would
sometimes go ahead and do the shovel-job with the tractor anyway;
the overkill only really hurts when the size of the tractor
causes me to start breaking things.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 ·····@Franz.COM (internet)
From: Kent M Pitman
Subject: Re: Idiom For Context Sensitive Dynamic Context:
Date:
Message-ID: <sfw3e3bz64r.fsf@world.std.com>
Duane Rettig <·····@franz.com> writes:
> "Howard R. Stearns" <······@elwood.com> writes:
>
> > Duane, I'm going to address your environment strategy in another
> > thread.
>
> OK, I'll look for it.
>
> > For this issue, though, I think you're a step (or more) ahead of me.
> >
> > Are you saying that one ought to be able to exercise dynamic control
> > over lexical countours?
>
> No, lexical contours should be controlled by lexical environments
> (CL gives us this control already). However, dynamic control
> could be controlled by capturing global and dynamic environments.
> Currently, there is only one global and dynamic environment at a
> time, and it essentially constitutes the state of the lisp (barring
> multiprocessing considerations).
This area of the language was hotly debated because it has lots of
effect on low-level implementation details and efficiency. My
sense is that the definition we have now is not there because everyone
thought it was best, but rather because different camps wanted different
things and we picked a compromise that would tolerate the various models.
I don't think more ground can be gotten in the area you cite without
giving up some implementation compatibility.
The same problem was what made it so hard to make macro environments
as first class objects work in the way people wanted them to, e.g.,
the stuff that was in CLTL2 and later removed for ANSI CL. The problem
is that it's easy to write the words "the global environment is
represented as an object" but it imposes an enormous coding, quality,
support, and customer impact burden on the implementations that don't
already do it, some of which may not even want to do it. (The camps
are at least three: those who don't want or need it, those who want it
but think the cost is too high, and those who want it and are willing
to tolerate the cost.)
The MOP is a third example. It essentially purports to describe the
internals of CLOS, but it wasn't written before CLOS, so some people
did their CLOS a different way. Agreeing to conform means for some a
trivial dusting of some protocols that already mostly conform, but
agreeing to conform for other vendors means a huge cost since the MOP
doesn't describe how their system is implemented and they have to either
fake it or reimplement it. (Genera is such an example.)
> > For example, in general, contours allow one to say "this variable
> > shadows that one -- use this value for any references to this name."
> > You want to have the inner code say "get met the 'relevent' value for
> > this name, and then have the environment mechanism supply the right
> > value.
>
> Yes, but in a dynamic sense. Compile-file already does this; as
> compilation progresses (it encounters a defmacro, for example) the
> definition is added temporarily to the global environment (i.e.,
> a compile-time-too evaluation _will_ see the definition). But when
> the compile-file is done, the macro definition might disappear. It is
> this shadowing of global environments that I'd like to see formalized
> in some way.
Genera does pretty well at this by having its definitional operators
actually manipulate a definition environment. But it's remarkably
painful. There are a great many details that make it nearly impossible
to win with, because the package universe intrudes. You essentially have
to make all new packages, but since they must inherit from existing symbols
you end up with pointers out to the other universe, and some programs
can find their way out of whatever isolated environment you create.
I personally think it was a mistake to say there's a separate
compilation environment, not because it was a bad wish, but because
in practice no one does what the manual might be read to say happens.
And that can be confusing to users.
In article <·················@elwood.com>,
Howard R. Stearns <······@elwood.com> wrote:
>What is the Common Lisp idiom for executing a body of code in a
>dynamic context in which some parameter will be set, but only if it
>hasn't been already set?
PROGV can be used to compute dynamic bindings at runtime. Is that what
you're looking for?
--
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Howard R. Stearns
Subject: Re: Idiom For Context Sensitive Dynamic Context:
Date:
Message-ID: <36E81271.2CD907F@elwood.com>
Barry Margolin wrote:
>
> In article <·················@elwood.com>,
> Howard R. Stearns <······@elwood.com> wrote:
> >What is the Common Lisp idiom for executing a body of code in a
> >dynamic context in which some parameter will be set, but only if it
> >hasn't been already set?
>
> PROGV can be used to compute dynamic bindings at runtime. Is that what
> you're looking for?
Well, I think that's PART of the story...
I think we have the following needs:
1. A systems programmer is going to define some internal stuff who's
action is going to be coupled with some sort of dynamic context. I
think that in most cases the identification of the context (i.e. a
variable name) will be known at compile time. (But even if it isn't
access to the value of special variable known only at run time is just
as easy (symbol-value).) The catch is that I suspect that this code
will not simply want to specify a single parameter name to define the
environment, but instead, some designator for an entity that can be
satisfied in multiple ways, just like an instance or condition can be
TYPEP of more than one class.
2. A user is going to define a dynamic context over which the context
will be bound. Again, I don't think the issue is knowing the parameter
to bind, because the context is really going to be some fairly arbitrary
user-defined thing.
3. More system code (and this is what I'm trying to write), is going
to have to make it possible for a user to specify what context should be
bound, and what the value will be. For this, I think PROGV is the
obvious way to let the user specify both. The catch is that in order to
make sure that the separately written code in [1] and [2] cooperate with
each other, I feel as though I need to spell out more about what
protocol will be used to communicate between the two. In short, I'm not
sure if I'm comfortable just saying: "And here's where we'll use PROGV
to bind whatever variables the user wants to whatever values they want."
Nonetheless, you make a good point, for which I thank you: "If you don't
know all the answers, at least go with what you think you do know and
move on. Progv MAY be all you need to know for now."
Howard R. Stearns wrote:
> What is the Common Lisp idiom for executing a body of code in a
> dynamic context in which some parameter will be set, but only if it
> hasn't been already set?
The Common Lisp idiom is probably using special variables,
as in
(defvar *warn*)
(let ((*warn* (if (boundp '*warn*) *warn* local-default-value)))
..)
If I understand correctly, the problem is that one wants to
change the global value of *warn* inside the local context,
which doesn't work with the above, it will revert to its old
value on return. I recall this problem comes up with having
an initialization file being loaded that wants to change default value
of globals for loading a file, e.g *redefinition-warnings*.
You can't do it in Common Lisp.
ACL has something called tpl:setq-default that apparently? changes
the global value, ignoring the local binding, which seems like
a straightforward solution.
I recall doing
checks if the call to load/compile was passed explicit values
or not. If no values were passed, it doesn't bind the variables.
E.g.
(defun load (&key (warnings *warnings* warnings-p))
(if warnings-p ;; if current value wasn't used, don't bind
(real-load ..)
(let ((*warnings* warnings))
(real-load ...)
)))
Then you can load an init file with (load :warnings nil)
and it can setq the warning variable which will change the
global value, or even (load :warnings *warnings*) which
actually uses the default value, but still allows it to be changed.
To basically, the caller decides whether the value can be changed
by the callee.
Hope that helps.
-Kelly Murray ···@niclos.com
BTW, a not-Common-Lisp idiom is to have a structure that contains
the switches which is always passed as an argument.
The caller can then decide to allocate another one and fill in
the defaults from the current one, or just pass the current one.
> programming, and is hard enough to get right, that I'd like to use
> and/or develop a standard idiom for it. How this is done also effects
> other things which we'd like to standardize, such as DEFSYSTEM and
> perhaps other interfaces to system code (streams, CORBA, ...)
>
> I'm going to give some example situations and some sucessively more
> complicated implementations. I'm also going to show how this effects
> WITH-COMPILATION-UNIT/DEFSYSTEM, and show an analogy to the condition
> system.
>
> Please note that I'm just asking questions here and not proposing a
> particular solution or any standardization. In fact, I'm hoping that
> someone will just say, "Oh, I just use xxx as follows..." I know that
> before asking "how do I do this specific thing", one should generally
> simplify and spell out the specific example of what one is trying to
> do. In this case, I can't, because in this system programming
> situation, I want to write code that cooperates with whatever the user
> writes to solve the problems described here. That's why I think I
> need to recognize (and implement) a standard idiom.
>
> -----
> There are situations in systems programming where the action taken by
> some internal routine is coupled to a dynamic context. Some examples
> include:
>
> - Whether and when a compiler should issue warnings about undefined
> or apparently misused entities.
> - Details of how users should recieve notifications, such as which
> stream or formatting style to use.
> - Whether some distributed operation should compute things locally or
> use some networked resource, and which resource and communication
> path should be used.
>
> In all these cases, it is insufficient to expect the direct caller of
> the utility to have all the answers, so special variables tend to be
> used.
>
> SIMPLE SPECIAL VARIABLES:
>
> At first glance, one might imagine that special variables would be
> enough:
>
> (let ((*warning-environment* (make-warning-environment)))
> ...
> (let ((*standard-output* (make-some-stream))
> (*user-notification-prefix* ";;;; "))
> ...
> (let ((*default-lisp-engine* :localhost))
> ...
> (free-engine *default-lisp-engine*))
> (force-output *standard-output*))
> (dump-warnings *warning-environment*))
>
> In each case, a dynamic context is created in which the parameter is
> bound appropriately. At the end of this time, a cleanup operation may
> need to be performed.
>
> A MACRO:
>
> One could imagine abstracting this idiom into a macro that provided
> for the right values to be returned, and used unwind-protect to make
> sure that the right cleanups always happened:
>
> (defmacro WITH-CONTEXT ((parameter &optional initform cleanup) &body
> body)
> `(let ((,parameter ,initform))
> (unwind-protect (progn ,@body)
> ,cleanup)))
>
> (with-context (*warning-environment* (make-warning-environment)
> (dump-warnings *warning-environment*))
> ...
> (with-context (*standard-output* *foo* (force-output
> *standard-output*))
> ...
> (with-context (*notification-prefix* ";;;;")
> (with-context (*default-lisp-engine* :localhost
> (free-host *default-lisp-engine*))
> ...))))
>
> CONDITIONAL BINDING:
>
> At this point, WITH-CONTEXT still doesn't handle the common problem of
> not initializing unless necessarry. For example, to really defer
> warnings, one person might write:
>
> (with-context (*warning-environment* (make-warning-environment)
> (dump-warnings *warning-environment*))
> ...)
>
> but the whole thing might be executed in a dynamic context in which
> someone else has wrapped an outer WITH-CONTEXT *warning-environment*.
>
> Now we want something like this:
>
> (defmacro WITH-CONTEXT ((parameter &optional initform cleanup) &body
> body)
> (with-unique-names (new-context?)
> `(let* ((,new-context? (or (not (boundp ',parameter))
> (null ,parameter)))
> (,parameter (if ,new-context?
> ,initform
> ,parameter)))
> (unwind-protect (progn ,@body)
> (when ,new-context? ,cleanup)))))
>
> The idea of binding a special variable to its old value might be
> wrong. For example, some of the internal tools which make use of this
> context information might need to ASSIGN a new value to the parameter.
> They would expect this new value to get propogated back to the outer
> context. Let's pretend, for now, that such assignment is simply
> prohibited.
>
> We might also want to give the user some control over whether a new
> context should be introduced, just like :override in
> WITH-COMPILATION-UNIT:
>
> (defmacro WITH-CONTEXT ((parameter override &optional initform cleanup)
> &body body)
> (with-unique-names (new-context?)
> `(let* ((,new-context? (or ,override ; This is the only new code
> (not (boundp ',parameter))
> (null ,parameter)))
> (,parameter (if ,new-context?
> ,initform
> ,parameter)))
> (unwind-protect (progn ,@body)
> (when ,new-context? ,cleanup)))))
>
> COMMUNICATING BETWEEN CONTEXTS:
>
> Here's where I loose it. I'm not convinced that this is enough. I
> have an intuition that instead of indicating a simple boolean for
> override, we instead want to be able to specify a "situation" in which
> we want to override or otherwise define the parameter.
>
> Instead of just looking at the value of a global variable, the
> interior routine which is coupled to the parameter really needs to
> access a hierarchical situation. For example, instead of COMPILE doing
> something like:
>
> (note-warning *warning-environment*)
>
> it instead does something like:
>
> (note-warning (context-value *warning-environment*
> 'clos-gf-call))
>
> where clos-gf-call is a subsituation of clos.
>
> In the ANSI writeup for WITH-COMPILATION-UNIT, Kim Barrett wrote:
>
> Consider a stream system built on an object system which will
> compose and compile functions on the fly on an as needed basis. It
> would be very strange for the functions so generated while doing
> file io for the user's compile-file to have any relationship with
> said compile-file.
>
> And so Kim didn't want the warnings associated with the internal CLOS
> compilation to appear with the warnings for the user's compile-file.
>
> I suspect that it isn't enough to just say: "Give me warnings about my
> code, but not warnings about system code that's done in service of the
> things I ask for." The problem is that it's not always possible for
> the system to distinguish between the two without guidance form the
> user.
>
> For example, what happens if one has a defconstant with a value form
> that causes compilation "internally"? Well, depending on several
> factors, this compilation might need to happen at load time as well,
> so the user might need to see those warnings in association with all
> of his other "user code" warnings.
>
> Another example comes from a WITH-COMPILATION-UNIT extension, where a
> user wants to specify that some block compilation activities are to be
> accumulated in one way, while other block compilation activities are
> accumulated in another way.
>
> Thus I think we need to be able to label a situation in which the
> binding will be relevent. Perhaps T denotes all contexts. It also
> seems that, like conditions, one want to be able to access a parameter
> with respect to a fairly broad situation (perhaps even T) and get back
> any applicable value. In other words, a context is really defined by
> TWO labels: a parameter and a situation. It is this complex context
> that we wish to bind over some dynamic extent.
>
> Now we're entering into a maze of twisty little passages, all
> different:
>
> Is this complex context best represented by two (or more) variables,
> or by a single combined context object? (In other words, are
> parameter and situation really indepent axes in the context space?)
>
> Do users need to control situational inheritance/applicability? Is
> multiple inheritance needed (i.e., can a situation be encompased
> by two otherwise unrelated situations)?
>
> How is this efficiently implemented?
>
> I suspect that there I could find a lesson in the design of the
> condition system. There one specifies a dynamic context which sets up
> a handler specified in outer code for a condition signalled from
> interior code. The condition is represented as an object. It seems
> like what I've been describing is the inverse: A dynamic context is
> set up in which a context is specified or "signaled" by outer code and
> is read or "handled" by inner code. Perhaps there is even a way to
> turn the existing condition system inside out to accomplish this
> without a new set of constructs.
>
> I'm not sure how nasty all this has to be. It might be the case that
> not only WITH-COMPILATION-UNIT, but also EVAL-WHEN could be
> implemented in terms of WITH-CONTEXT. Maybe its "the ultimate
> <whatever special binding and unwind-protect are examples of>" (I
> vaguely remember Barbara Liskov trying and apparently failing to teach
> me something about strength and coupling...)
>
> I don't want to get carried away. All I really know is that to do
> "the right thing" for DEFSYSTEM, one needs to have something like
> WITH-COMPILATION-UNIT that can be controlled by the user, and I'm not
> at all sure that the design of WITH-COMPILATION-UNIT isn't broken.
> Comments?
From: Howard R. Stearns
Subject: Re: Idiom For Context Sensitive Dynamic Context:
Date:
Message-ID: <36E82664.675E66E7@elwood.com>
Indeed, having inner code change the binding, and letting that propogate
to outer contexts is one of the problems, and I'd like a portable
solution to that. However, this isn't my biggest concern.
I'm more worried that the "definition space" of contexts isn't just a
single axis -- i.e. the parameter name. Instead, I'm getting carried
away worrying that one wants to specify complex sets of situations in
which some flag is meant to apply.
I have received correspondence that suggests that at least one other
person ran into this:
A tracing system for a large application had code littered with
thousands of print statements. A utility was provided to enable the
user to "turn on" the printing associated with each tag. The soon found
that they wanted to turn on groups of statements and that these groups
overlapped. They found they need to switch on different parts of a
lattice of sets and subsets.
I'm saying the same thing, but further complicated by the fact that
(different parts of) this lattice might want to change in time (i.e.
over some dynamic extent).
When you combine both these issues, and add the occasional need to
introduce a new context conditionally (i.e. only when it hasn't already
been defined in the current context -- the :override stuff) and the
occasional need to have cleanup code (that is always and only executed
whenever a truly new context was introduced), I think a fullly defined
utility is in order.
Note that these issue must work together. For example, if there is
cleanup code that gets executed because of new context is closed, the
cleanup code will need to know exactly which context is being closed
(i.e., all the data and type information about the context).
Kelly Murray wrote:
>
> Howard R. Stearns wrote:
> > What is the Common Lisp idiom for executing a body of code in a
> > dynamic context in which some parameter will be set, but only if it
> > hasn't been already set?
>
> The Common Lisp idiom is probably using special variables,
> as in
> (defvar *warn*)
> (let ((*warn* (if (boundp '*warn*) *warn* local-default-value)))
> ..)
> If I understand correctly, the problem is that one wants to
> change the global value of *warn* inside the local context,
> which doesn't work with the above, it will revert to its old
> value on return. I recall this problem comes up with having
> an initialization file being loaded that wants to change default value
> of globals for loading a file, e.g *redefinition-warnings*.
>
> You can't do it in Common Lisp.
> ACL has something called tpl:setq-default that apparently? changes
> the global value, ignoring the local binding, which seems like
> a straightforward solution.
> I recall doing
> checks if the call to load/compile was passed explicit values
> or not. If no values were passed, it doesn't bind the variables.
> E.g.
> (defun load (&key (warnings *warnings* warnings-p))
> (if warnings-p ;; if current value wasn't used, don't bind
> (real-load ..)
> (let ((*warnings* warnings))
> (real-load ...)
> )))
>
> Then you can load an init file with (load :warnings nil)
> and it can setq the warning variable which will change the
> global value, or even (load :warnings *warnings*) which
> actually uses the default value, but still allows it to be changed.
> To basically, the caller decides whether the value can be changed
> by the callee.
>
> Hope that helps.
>
> -Kelly Murray ···@niclos.com
>
> BTW, a not-Common-Lisp idiom is to have a structure that contains
> the switches which is always passed as an argument.
> The caller can then decide to allocate another one and fill in
> the defaults from the current one, or just pass the current one.
>
> > programming, and is hard enough to get right, that I'd like to use
> > and/or develop a standard idiom for it. How this is done also effects
> > other things which we'd like to standardize, such as DEFSYSTEM and
> > perhaps other interfaces to system code (streams, CORBA, ...)
> >
> > I'm going to give some example situations and some sucessively more
> > complicated implementations. I'm also going to show how this effects
> > WITH-COMPILATION-UNIT/DEFSYSTEM, and show an analogy to the condition
> > system.
> >
> > Please note that I'm just asking questions here and not proposing a
> > particular solution or any standardization. In fact, I'm hoping that
> > someone will just say, "Oh, I just use xxx as follows..." I know that
> > before asking "how do I do this specific thing", one should generally
> > simplify and spell out the specific example of what one is trying to
> > do. In this case, I can't, because in this system programming
> > situation, I want to write code that cooperates with whatever the user
> > writes to solve the problems described here. That's why I think I
> > need to recognize (and implement) a standard idiom.
> >
> > -----
> > There are situations in systems programming where the action taken by
> > some internal routine is coupled to a dynamic context. Some examples
> > include:
> >
> > - Whether and when a compiler should issue warnings about undefined
> > or apparently misused entities.
> > - Details of how users should recieve notifications, such as which
> > stream or formatting style to use.
> > - Whether some distributed operation should compute things locally or
> > use some networked resource, and which resource and communication
> > path should be used.
> >
> > In all these cases, it is insufficient to expect the direct caller of
> > the utility to have all the answers, so special variables tend to be
> > used.
> >
> > SIMPLE SPECIAL VARIABLES:
> >
> > At first glance, one might imagine that special variables would be
> > enough:
> >
> > (let ((*warning-environment* (make-warning-environment)))
> > ...
> > (let ((*standard-output* (make-some-stream))
> > (*user-notification-prefix* ";;;; "))
> > ...
> > (let ((*default-lisp-engine* :localhost))
> > ...
> > (free-engine *default-lisp-engine*))
> > (force-output *standard-output*))
> > (dump-warnings *warning-environment*))
> >
> > In each case, a dynamic context is created in which the parameter is
> > bound appropriately. At the end of this time, a cleanup operation may
> > need to be performed.
> >
> > A MACRO:
> >
> > One could imagine abstracting this idiom into a macro that provided
> > for the right values to be returned, and used unwind-protect to make
> > sure that the right cleanups always happened:
> >
> > (defmacro WITH-CONTEXT ((parameter &optional initform cleanup) &body
> > body)
> > `(let ((,parameter ,initform))
> > (unwind-protect (progn ,@body)
> > ,cleanup)))
> >
> > (with-context (*warning-environment* (make-warning-environment)
> > (dump-warnings *warning-environment*))
> > ...
> > (with-context (*standard-output* *foo* (force-output
> > *standard-output*))
> > ...
> > (with-context (*notification-prefix* ";;;;")
> > (with-context (*default-lisp-engine* :localhost
> > (free-host *default-lisp-engine*))
> > ...))))
> >
> > CONDITIONAL BINDING:
> >
> > At this point, WITH-CONTEXT still doesn't handle the common problem of
> > not initializing unless necessarry. For example, to really defer
> > warnings, one person might write:
> >
> > (with-context (*warning-environment* (make-warning-environment)
> > (dump-warnings *warning-environment*))
> > ...)
> >
> > but the whole thing might be executed in a dynamic context in which
> > someone else has wrapped an outer WITH-CONTEXT *warning-environment*.
> >
> > Now we want something like this:
> >
> > (defmacro WITH-CONTEXT ((parameter &optional initform cleanup) &body
> > body)
> > (with-unique-names (new-context?)
> > `(let* ((,new-context? (or (not (boundp ',parameter))
> > (null ,parameter)))
> > (,parameter (if ,new-context?
> > ,initform
> > ,parameter)))
> > (unwind-protect (progn ,@body)
> > (when ,new-context? ,cleanup)))))
> >
> > The idea of binding a special variable to its old value might be
> > wrong. For example, some of the internal tools which make use of this
> > context information might need to ASSIGN a new value to the parameter.
> > They would expect this new value to get propogated back to the outer
> > context. Let's pretend, for now, that such assignment is simply
> > prohibited.
> >
> > We might also want to give the user some control over whether a new
> > context should be introduced, just like :override in
> > WITH-COMPILATION-UNIT:
> >
> > (defmacro WITH-CONTEXT ((parameter override &optional initform cleanup)
> > &body body)
> > (with-unique-names (new-context?)
> > `(let* ((,new-context? (or ,override ; This is the only new code
> > (not (boundp ',parameter))
> > (null ,parameter)))
> > (,parameter (if ,new-context?
> > ,initform
> > ,parameter)))
> > (unwind-protect (progn ,@body)
> > (when ,new-context? ,cleanup)))))
> >
> > COMMUNICATING BETWEEN CONTEXTS:
> >
> > Here's where I loose it. I'm not convinced that this is enough. I
> > have an intuition that instead of indicating a simple boolean for
> > override, we instead want to be able to specify a "situation" in which
> > we want to override or otherwise define the parameter.
> >
> > Instead of just looking at the value of a global variable, the
> > interior routine which is coupled to the parameter really needs to
> > access a hierarchical situation. For example, instead of COMPILE doing
> > something like:
> >
> > (note-warning *warning-environment*)
> >
> > it instead does something like:
> >
> > (note-warning (context-value *warning-environment*
> > 'clos-gf-call))
> >
> > where clos-gf-call is a subsituation of clos.
> >
> > In the ANSI writeup for WITH-COMPILATION-UNIT, Kim Barrett wrote:
> >
> > Consider a stream system built on an object system which will
> > compose and compile functions on the fly on an as needed basis. It
> > would be very strange for the functions so generated while doing
> > file io for the user's compile-file to have any relationship with
> > said compile-file.
> >
> > And so Kim didn't want the warnings associated with the internal CLOS
> > compilation to appear with the warnings for the user's compile-file.
> >
> > I suspect that it isn't enough to just say: "Give me warnings about my
> > code, but not warnings about system code that's done in service of the
> > things I ask for." The problem is that it's not always possible for
> > the system to distinguish between the two without guidance form the
> > user.
> >
> > For example, what happens if one has a defconstant with a value form
> > that causes compilation "internally"? Well, depending on several
> > factors, this compilation might need to happen at load time as well,
> > so the user might need to see those warnings in association with all
> > of his other "user code" warnings.
> >
> > Another example comes from a WITH-COMPILATION-UNIT extension, where a
> > user wants to specify that some block compilation activities are to be
> > accumulated in one way, while other block compilation activities are
> > accumulated in another way.
> >
> > Thus I think we need to be able to label a situation in which the
> > binding will be relevent. Perhaps T denotes all contexts. It also
> > seems that, like conditions, one want to be able to access a parameter
> > with respect to a fairly broad situation (perhaps even T) and get back
> > any applicable value. In other words, a context is really defined by
> > TWO labels: a parameter and a situation. It is this complex context
> > that we wish to bind over some dynamic extent.
> >
> > Now we're entering into a maze of twisty little passages, all
> > different:
> >
> > Is this complex context best represented by two (or more) variables,
> > or by a single combined context object? (In other words, are
> > parameter and situation really indepent axes in the context space?)
> >
> > Do users need to control situational inheritance/applicability? Is
> > multiple inheritance needed (i.e., can a situation be encompased
> > by two otherwise unrelated situations)?
> >
> > How is this efficiently implemented?
> >
> > I suspect that there I could find a lesson in the design of the
> > condition system. There one specifies a dynamic context which sets up
> > a handler specified in outer code for a condition signalled from
> > interior code. The condition is represented as an object. It seems
> > like what I've been describing is the inverse: A dynamic context is
> > set up in which a context is specified or "signaled" by outer code and
> > is read or "handled" by inner code. Perhaps there is even a way to
> > turn the existing condition system inside out to accomplish this
> > without a new set of constructs.
> >
> > I'm not sure how nasty all this has to be. It might be the case that
> > not only WITH-COMPILATION-UNIT, but also EVAL-WHEN could be
> > implemented in terms of WITH-CONTEXT. Maybe its "the ultimate
> > <whatever special binding and unwind-protect are examples of>" (I
> > vaguely remember Barbara Liskov trying and apparently failing to teach
> > me something about strength and coupling...)
> >
> > I don't want to get carried away. All I really know is that to do
> > "the right thing" for DEFSYSTEM, one needs to have something like
> > WITH-COMPILATION-UNIT that can be controlled by the user, and I'm not
> > at all sure that the design of WITH-COMPILATION-UNIT isn't broken.
> > Comments?