Hi all,
Let me preface this question by saying that I'm not trying to solve any actual
problem, just trying to understand better how Lisp works.
I'm trying to figure out if it's possible, when defining a function, to capture
in it a reference to another existing function, at the time my function's
definition is evaluated. For example:
(defun my-list (&rest args) (apply #'list args))
This function will behave like list, as long as nobody redefines list in the
meantime. (Whoops, can't redefine a system function. Please pretend I picked a
better example.)
But as soon as someone redefines list, this will refer to the new version,
rather than the old one. My understanding is that this is because global
bindings are dynamic in nature, which means they don't go in closures -- a
reference to a function binding always means "the binding in effect at time of
use" (assuming no flet or anything like that, which just confuses me for the
moment.)
So suppose I wanted to write a my-list which referred to the function list
which existed at the evaluation time of my-list's definition, and not at the
time of the call to my-list. Is there a way I could do this?
One way to accomplish very nearly what I want is to use #. to refer to the
value of list which exists at _read time_ when the definition of my-list is
_read_. This strikes me as a terrible hack:
(defun my-list (&rest args) (apply #.#'list args))
Is there another way? (For that matter, does this way always do what I think it
does?)
Thanks!
gwillen
From: Kent M Pitman
Subject: Re: Capturing a function in a closure
Date:
Message-ID: <uhcahbpnt.fsf@nhplace.com>
Glenn Willen <·······@nerdnet.org> writes:
> Hi all,
>
> Let me preface this question by saying that I'm not trying to solve
> any actual problem, just trying to understand better how Lisp works.
>
> I'm trying to figure out if it's possible, when defining a function,
> to capture in it a reference to another existing function, at the
> time my function's definition is evaluated. For example:
>
> (defun my-list (&rest args) (apply #'list args))
>
> This function will behave like list, as long as nobody redefines
> list in the meantime. (Whoops, can't redefine a system
> function. Please pretend I picked a better example.)
This function will always behave like list (or whatever name you pick).
If the definition of list is updated, then that's what list behaves like,
and your function will continue to behave the same.
You should understand that a closure is a closure over a location that
something is stored, not over a name. Redefining something does not mean
getting a new location to store it, so the closure still points to it.
Consider:
(defun make-a-cell (initial-value)
(let ((value initial-value))
(lambda (operation)
(cond ((eq operation 'get) (lambda () value))
((eq operation 'set) (lambda (v) (setq value v)))))))
(funcall foo 'get)
#<closure>
(funcall (funcall foo 'get)) => 3
(funcall (funcall foo 'set) 4) => 4
(funcall (funcall foo 'get)) => 4
By the logic you advance, the notion that redefining a value should get
you the old value, the SET should have no effect. I doubt you really mean
that.
> But as soon as someone redefines list, this will refer to the new version,
> rather than the old one. My understanding is that this is because global
> bindings are dynamic in nature,
Global variables are dynamic in nature, but this is not an effect of that.
First, because functions are not variables, and don't behave the same.
Functions are lexical. But even if it were a variable you were using,
if there _were_ global lexicals (and you can find any of a number of
implementations of global lexicals by searching this newsgroup), they
would behave the same.
> which means they don't go in closures -- a
> reference to a function binding always means "the binding in effect
> at time of use" (assuming no flet or anything like that, which just
> confuses me for the moment.)
The issue is entirely lexical. To know what LIST in the car of a form
refers to, or what #'LIST refers to, you search outward lexically (after
having expanded all macros) and find out if the thing is lexically fbound.
LIST can't be, but other functions can be. If it is fbound, then the value
of that lexical binding is used. Otherwise, the global functional
binding is used is used.
> So suppose I wanted to write a my-list which referred to the function list
> which existed at the evaluation time of my-list's definition, and not at the
> time of the call to my-list. Is there a way I could do this?
As explained, this is not an issue. This is an issue for special variables,
but you did not exercise that situation. The situation is exercised only
if there is a dynamic variable binding intervening between the point of call
and the global value, since [by my terminology; full disclosure: others
disagree with my personal terminology and will explain this differently]
all special bindings use the same cell, so when you special-bind something,
you're affecting the value seen by all closures since they continue to access
the same cell with a different value.
> One way to accomplish very nearly what I want is to use #. to refer to the
> value of list which exists at _read time_ when the definition of my-list is
> _read_. This strikes me as a terrible hack:
>
> (defun my-list (&rest args) (apply #.#'list args))
It's also pointless. It accomplishes nothing and might confuse some
compilers. Read in the hyperspec about externalizable objects (3.2.4).
http://www.lispworks.com/documentation/HyperSpec/Body/03_bd.htm
Functions are not
> Is there another way? (For that matter, does this way always do what I
> think it does?)
Rainer made the obvious LOAD-TIME-VALUE suggestion, which I didn't think was
really the question you were asking, though maybe it was. The other way to
do that if you didn't have that operation is:
(let ((original-list #'list))
(defun my-list (&rest args) (apply original-list args)))
But the reason I don't recommend this, unless you're literally intending
to implement function versioning, is that ordinarily the "meaning" of
updating a function is to say "I have here a bug fix" not to say "I have
a new and different function". If you only update a function and not its
callers in Lisp, the informal intention is supposed to be that all the callers
will see the update. If you start to second-guess that by guarding yourself
against such a feature, you are refusing to accept bug fixes. This is a
case of an unstoppable force meets and immovable object. Something has to
give. But the usual situation in Lisp is not to permanently wire the value,
which is why it's not often done.
Incidentally, as a point of style, if a DEFUN _is_ included in a closure,
it's usually better to indent it at column 0 so as not to confuse text
editors that look for definitions to begin there. Hence you might write:
(let ((original-list #'list))
(defun my-list (&rest args) (apply original-list args))
);tel
But that's yet another argument for using LOAD-TIME-VALUE in the cases that
it works, and if you really need it. Which I'm guessing you mostly don't.
On 22 Jul 2008 17:00:06 -0400, Kent M Pitman <······@nhplace.com> said:
| ...
| But the reason I don't recommend this, unless you're literally intending
| to implement function versioning, is that ordinarily the "meaning" of
| updating a function is to say "I have here a bug fix" not to say "I have
| a new and different function". If you only update a function and not its
| callers in Lisp, the informal intention is supposed to be that all the callers
| will see the update. If you start to second-guess that by guarding yourself
| against such a feature, you are refusing to accept bug fixes. This is a
| case of an unstoppable force meets and immovable object. Something has to
| give. But the usual situation in Lisp is not to permanently wire the value,
| which is why it's not often done.
So to further complicate the explanation, note that the compiler
must heed a NOTINLINE declaration, unlike an INLINE declaration.
---Vassil.
--
Peius melius est. ---Ricardus Gabriel.
Glenn Willen wrote:
> One way to accomplish very nearly what I want is to use #. to refer to the
> value of list which exists at _read time_ when the definition of my-list is
> _read_. This strikes me as a terrible hack:
>
> (defun my-list (&rest args) (apply #.#'list args))
>
> Is there another way? (For that matter, does this way always do what I think it
> does?)
Yes, this is a terrible hack. It's ugliness will manifest itself when
you attempt to file-compile code with such references:
(defun foo (x) x)
(defun bar (x) (apply #.#'foo (list x)))
If you file-compile such code, the compiler will complain because the
function foo is not defined yet at readtime.
What you want is load-time-value:
(defun foo (x) x)
(defun bar (x) (apply (load-time-value #'foo) (list x)))
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
In article <··············@mid.individual.net>, Pascal Costanza wrote:
> Yes, this is a terrible hack. It's ugliness will manifest itself when
> you attempt to file-compile code with such references:
>
> (defun foo (x) x)
>
> (defun bar (x) (apply #.#'foo (list x)))
>
> If you file-compile such code, the compiler will complain because the
> function foo is not defined yet at readtime.
Right, this is why I emphasized read-time. What I really want to capture is
function-definition-evaluation-time. Unfortunately I've done very little
working with or thinking about compilation versus interpretation, which means
there are more 'times' than I'm really comfortable with.
>
> What you want is load-time-value:
>
> (defun foo (x) x)
>
> (defun bar (x) (apply (load-time-value #'foo) (list x)))
Interesting! That's a form I've never heard of before. The CLHS suggests it is
indeed precisely what I was looking for. Is "load time" always the same as the
time at which the function definition is evaluated? (I guess in the compiled
universe, there's no such time, just the time at which it's compiled and the
time at which the image is loaded, this being the latter. In the interpreted
universe I guess "load-time" is treated as precisely what I want, though.)
Thanks a lot!
gwillen
In article <······················@gwillen-mac.local>,
Glenn Willen <·······@nerdnet.org> wrote:
> In article <··············@mid.individual.net>, Pascal Costanza wrote:
> > Yes, this is a terrible hack. It's ugliness will manifest itself when
> > you attempt to file-compile code with such references:
> >
> > (defun foo (x) x)
> >
> > (defun bar (x) (apply #.#'foo (list x)))
> >
> > If you file-compile such code, the compiler will complain because the
> > function foo is not defined yet at readtime.
>
> Right, this is why I emphasized read-time. What I really want to capture is
> function-definition-evaluation-time. Unfortunately I've done very little
> working with or thinking about compilation versus interpretation, which means
> there are more 'times' than I'm really comfortable with.
> >
> > What you want is load-time-value:
> >
> > (defun foo (x) x)
> >
> > (defun bar (x) (apply (load-time-value #'foo) (list x)))
>
> Interesting! That's a form I've never heard of before.
On the Lisp Machine one could display a random hint with a command.
I propose a random CLHS page of the day. The Hyperspec has
a file with links for each Common Lisp symbol. One could for example
email a clhs html file for a random CL symbol each day. After
a few years you can then start over... ;-) Planet Lisp
could also display the CLHS symbol of the day. Etc...
> The CLHS suggests it is
> indeed precisely what I was looking for. Is "load time" always the same as the
> time at which the function definition is evaluated? (I guess in the compiled
> universe, there's no such time, just the time at which it's compiled and the
> time at which the image is loaded, this being the latter. In the interpreted
> universe I guess "load-time" is treated as precisely what I want, though.)
Also make sure you read the entry for FUNCTION. (function foo) is
the expanded version of #'foo).
http://www.lispworks.com/documentation/HyperSpec/Body/s_fn.htm
>
> Thanks a lot!
> gwillen
--
http://lispm.dyndns.org/
On Tue, 22 Jul 2008 21:35:38 +0200 (CEST), Glenn Willen <·······@nerdnet.org> said:
| Right, this is why I emphasized read-time. What I really want to capture is
| function-definition-evaluation-time. Unfortunately I've done very little
| working with or thinking about compilation versus interpretation, which means
| there are more 'times' than I'm really comfortable with.
It seems you ain't seen everything yet... There is ((-1) design
time, (0) compose time,) (1) read time, (2) compile time, (3) load
time, and (4) run time.
(0) Figuring out what code the programmer can have executed at
compose time or earlier is left as an exercise.
(1) The programmer can have code executed at read time by means of
character macros.
(2) The programmer can have code executed at compile time by means
of macros. Other possibilities are using EVAL-WHEN for the
:COMPILE-TOPLEVEL situation and defining MAKE-LOAD-FORM methods.
(3) The programmer can have code executed at load time by using
EVAl-WHEN for the :LOAD-TOPLEVEL situation; and by having a
MAKE-LOAD-FORM return such code.
(The above are very simplistic hints at what EVAL-WHEN does; note
also that sometimes the :EXECUTE situation needs to be specified
explicitly.)
---Vassil.
P.S. There is also common-lisp:time, of course...
--
Peius melius est. ---Ricardus Gabriel.
Vassil Nikolov <···············@pobox.com> writes:
> On Tue, 22 Jul 2008 21:35:38 +0200 (CEST), Glenn Willen <·······@nerdnet.org> said:
>
> | Right, this is why I emphasized read-time. What I really want to capture is
> | function-definition-evaluation-time. Unfortunately I've done very little
> | working with or thinking about compilation versus interpretation, which means
> | there are more 'times' than I'm really comfortable with.
>
> It seems you ain't seen everything yet... There is ((-1) design
> time, (0) compose time,) (1) read time, (2) compile time, (3) load
> time, and (4) run time.
And don't forget that this is recursive. When you are in any time,
you can use (load (compile-file (ed "x.lisp"))) #|assuming an
implementation whose ED returns the path of the edited file ;-) |# and
therefore recurse over -1,0,1,2,3,4 again.
> P.S. There is also common-lisp:time, of course...
--
__Pascal Bourguignon__ http://www.informatimago.com/
Pour moi, la grande question n'a jamais �t�: �Qui suis-je? O� vais-je?�
comme l'a formul� si adroitement notre ami Pascal, mais plut�t:
�Comment vais-je m'en tirer?� -- Jean Yanne
In article <······················@gwillen-mac.local>,
Glenn Willen <·······@nerdnet.org> wrote:
> Hi all,
>
> Let me preface this question by saying that I'm not trying to solve any
> actual
> problem, just trying to understand better how Lisp works.
>
> I'm trying to figure out if it's possible, when defining a function, to
> capture
> in it a reference to another existing function, at the time my function's
> definition is evaluated. For example:
>
> (defun my-list (&rest args) (apply #'list args))
>
> This function will behave like list, as long as nobody redefines list in the
> meantime. (Whoops, can't redefine a system function. Please pretend I picked
> a
> better example.)
>
> But as soon as someone redefines list, this will refer to the new version,
> rather than the old one. My understanding is that this is because global
> bindings are dynamic in nature, which means they don't go in closures -- a
> reference to a function binding always means "the binding in effect at time
> of
> use" (assuming no flet or anything like that, which just confuses me for the
> moment.)
(let ((orig-list #'list))
(defun my-list (&rest args) (apply orig-list args)))
--
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Barry Margolin <······@alum.mit.edu> writes:
> In article <······················@gwillen-mac.local>,
> Glenn Willen <·······@nerdnet.org> wrote:
>
>> Hi all,
>>
>> Let me preface this question by saying that I'm not trying to solve any
>> actual
>> problem, just trying to understand better how Lisp works.
>>
>> I'm trying to figure out if it's possible, when defining a function, to
>> capture
>> in it a reference to another existing function, at the time my function's
>> definition is evaluated. For example:
>>
>> (defun my-list (&rest args) (apply #'list args))
>>
>> This function will behave like list, as long as nobody redefines list in the
>> meantime. (Whoops, can't redefine a system function. Please pretend I picked
>> a
>> better example.)
>>
>> But as soon as someone redefines list, this will refer to the new version,
>> rather than the old one. My understanding is that this is because global
>> bindings are dynamic in nature, which means they don't go in closures -- a
>> reference to a function binding always means "the binding in effect at time
>> of
>> use" (assuming no flet or anything like that, which just confuses me for the
>> moment.)
>
> (let ((orig-list #'list))
> (defun my-list (&rest args) (apply orig-list args)))
The problem with closures is that they don't go well thru .fasl files...
That's why it's better to use LOAD-TIME-VALUE.
--
__Pascal Bourguignon__
On Wed, 23 Jul 2008 11:38:17 +0200, ···@informatimago.com (Pascal J. Bourguignon) said:
| Barry Margolin <······@alum.mit.edu> writes:
|| ...
|| (let ((orig-list #'list))
|| (defun my-list (&rest args) (apply orig-list args)))
| The problem with closures is that they don't go well thru .fasl files...
| That's why it's better to use LOAD-TIME-VALUE.
But note that
(defun your-list (&rest args) (apply (load-time-value #'their-list) args))
would match
(let ((their-list #'their-list))
(defun my-list (&rest args) (apply their-list args)))
only if we made sure that YOUR-LIST is compiled no later than
definition time (explicitly or implicitly) [*]. If we _really_ have
to worry about closures in compiled files (and I don't know if we
do), then it seems we would have to resort to something like
(progn
(defvar #1=#:*their-list* #'their-list)
(defun our-list (&rest args) (apply #1# args)))
(where the Uninterned Symbol Dance is only to imitate lexical
scope), unless, of course, there is an elelgant solution that
escapes me. Note that using DEFCONSTANT instead of DEFVAR here
would not be clean because by the very context of the problem the
values may well be different between compile time and run time.
_________
[*] To illustrate the effect of a lack of compilation without
introducing implementation-specific controls, consider
(defun your-list (&rest args) (apply (eval '(load-time-value #'their-list)) args))
as a simulation of the uncompiled ("interpreted") version.
(Except for the sake of the example, the use of EVAL here is not
justified, of course.)
---Vassil.
--
Peius melius est. ---Ricardus Gabriel.
On 2008-07-24, Vassil Nikolov <···············@pobox.com> wrote:
>
> On Wed, 23 Jul 2008 11:38:17 +0200, ···@informatimago.com (Pascal J. Bourguignon) said:
>
>| Barry Margolin <······@alum.mit.edu> writes:
>|| ...
>|| (let ((orig-list #'list))
>|| (defun my-list (&rest args) (apply orig-list args)))
>
>| The problem with closures is that they don't go well thru .fasl files...
>| That's why it's better to use LOAD-TIME-VALUE.
>
> But note that
>
> (defun your-list (&rest args) (apply (load-time-value #'their-list) args))
>
> would match
>
> (let ((their-list #'their-list))
> (defun my-list (&rest args) (apply their-list args)))
Here, THEIR-LIST /is/ a de facto load-time value, since the closure is
set up in a top-level form, which is evaluated at load time.
LOAD-TIME-VALUE allows you to do this without having to create
next-to-top-level lexical environment. You just use LOAD-TIME-VALUE wherever
you need it and you're done.
You don't always have the luxury of being able to set up this lexical
environment. For instance you have no way to do this in a macro.
What if this is the case:
(defun my-list (&rest args) (some-macro args))
SOME-MACRO cannot generate the (let ...) around the DEFUN.
From: Kent M Pitman
Subject: Re: Capturing a function in a closure
Date:
Message-ID: <ubq0m3lbf.fsf@nhplace.com>
Kaz Kylheku <········@gmail.com> writes:
> You don't always have the luxury of being able to set up this lexical
> environment. For instance you have no way to do this in a macro.
>
> What if this is the case:
>
> (defun my-list (&rest args) (some-macro args))
>
> SOME-MACRO cannot generate the (let ...) around the DEFUN.
Right. Well, this is the thing for which there is LOAD-TIME-VALUE.
For anyone not remembering the history... and there are days where I'm
not sure I do either...
I vaguely recall that in Maclisp there was a magic list [a random
place in my memory says it was called the "pratt stack" (probably
named after Vaughan Pratt, of CGOL fame, since JonL White maintained
the compiler and liked to name language features after people that
requested them)] onto which you could push definitions when you were
expanding a macro in compiled code (in interpreted code, I think you
just called EVAL to effect the definition immediately--Maclisp had
tons of places where the interpreter and compiler behaved in utterly
different ways. Things pushed onto this list would end up compiled in
along with whatever was being prepared for evaluation, so macros could
rely on auxiliary functions getting compiled to support them. It
might even have gotten the order right (making sure the auxiliary
function was compiled before the macro that needed it), though I don't
recall for sure ... that might be why it was called a stack.
In CL for a while, one used to just arrange to construct and compile
objects at load time, with quoted constant values that could still be
modified. The language definition is still a little blurry around that
question, since at some point we intended LOAD-TIME-VALUE to be used for
that but I'm not sure we 100% closed the door to modifiable constants
for evaluation... but I don't recommend anyone use them since there's no
need.
LOAD-TIME-VALUE, incidentally, had its origin in an earlier weird compiler
only constant [a magic otherwise-uninterned symbol, whose name was SQUID,
for self-quoting internal datum] that you could find on a secret obarray
(kind of like a package, but not as high tech) and could cons in with an
expression causing the compiler to emit code for runtime ... I think that
was how Maclisp's #, was implemented in the compiler. (In the interpreter
it acted like #. as I recall.)
The CL way of doing this is much cleaner.
From: Kent M Pitman
Subject: Re: Capturing a function in a closure
Date:
Message-ID: <uiquu3lvb.fsf@nhplace.com>
···@informatimago.com (Pascal J. Bourguignon) writes:
> Barry Margolin <······@alum.mit.edu> writes:
> ...
> > (let ((orig-list #'list))
> > (defun my-list (&rest args) (apply orig-list args)))
>
> The problem with closures is that they don't go well thru .fasl files...
> That's why it's better to use LOAD-TIME-VALUE.
There's no compile-time closure happening here. The LET is evaluated
at execution time (i.e., at load time). No closure ends up in a FASL
file with the above code. The DEFUN does create a closure, but not
until load time. (The definition of MY-LIST is presumably not
inlineable even if there is an INLINE definition prior to the above
code not shown in the example.)
On 2008-07-22, Glenn Willen <·······@nerdnet.org> wrote:
> One way to accomplish very nearly what I want is to use #. to refer to the
> value of list which exists at _read time_ when the definition of my-list is
> _read_. This strikes me as a terrible hack:
>
> (defun my-list (&rest args) (apply #.#'list args))
It's an undefined hack. You're in effect turning a function object into
a literal datum embedded in your program's source code.
For this to work under file compilation, the compiler has to be able to
externalize the function object.
Common Lisp implementations are not required to be able to externalize
functions.
>
> Is there another way? (For that matter, does this way always do what I think it
> does?)
Here is something that may be of interest to you:
(funcall (load-time-value #'func) arg)
Forms wrapped in LOAD-TIME-VALUE are evaluated once at load time, and the
values are then cached.
If you reload the surrounding module, then this will resolve to the latest
binding of FUNC.