I am pleased to announce that Lexicons are finally ready for their
close-up. Get them here:
http://www.flownet.com/ron/lisp
You'll want to start by reading lexicons.pdf. To run the code you'll
need lexicons.lisp, dictionary.lisp and utilities.lisp. There's also
some demo code in lexdemo.lisp.
This code was developed in Clozure Common Lisp, and has been briefly
tested under SBCL and CLIsp. It should under no circumstances be
considered ready for production. This is beta-test at best, probably
more like alpha. Consider this a request for peer review.
Comments, bug reports and (constructive) criticisms are much appreciated.
Enjoy!
rg
On Feb 25, 5:28 pm, Ron Garret <·········@flownet.com> wrote:
> I am pleased to announce that Lexicons are finally ready for their
> close-up. Get them here:
>
> http://www.flownet.com/ron/lisp
You know, that whole `(,foo ...) thing is wrong. The unquote ,FOO
means ``stick the value of the /variable/ FOO here''. Since you
haven't made Lexicons a one-namespace system (Lisp-1), it's obvious
you have a hack going on. The programmer wants the FOO function there,
not the FOO variable.
The very fact that (LDEFUN FOO ...) does anything at all to the
variable binding stinks to high heaven! If you LDEFVAR FOO to some
value, and follow that with a LDEFUN or LDEFMACRO in the same scope,
either of these will clobber that value!
Come on, wouldn't the following Just Work, without duplicating the
function cell into the value cell?
`(,(lfunction foo) ...)
You just don't have a nice shorthand read notation for (lfunction foo)
that's all, so it's verbose. What this says is ``insert a lexical
reference to the function foo here''. A lexical reference is
understood to be a special object which (by the permission of a
special rule) can be spliced into the CAR position of the source code
of a list expression, where its action is that it resolves to the
intended function. A lexical function reference inserted in the
source code in a position where it is evaluated is undefined behavior,
so `(funcall ,(lfunction foo) ...) is erroneous. A lexical function
reference passed into funcall can be defined as working so `(funcall ',
(lfunction foo) ...) is equivalent to `(,(lfunction foo) ...).
(Obviously, since wherever you see (lfunction foo) you can pretend
there is an uninterned symbol).
But, there is a big problem. Both solutions suffer from the obvious
problem that they ``bake in'' in the actual Lisp function at macro-
expansion time. If the function is redefined, the code generated by
the macro continues to call the old FOO. The macro-generated code has
captured the actual function, bypassing the symbol FOO. Why? Because
it is calling the secret DEFUN through its uninterned name. Each
LDEFUN manufactures a new secret symbol (the value of the LNAME
variable in LDEFUN), and writes a new DEFUN for it. Your macro
expansions are tied to this secret DEFUN, because the macro's
backquote pulls it out as the value of the cell, and so this whole
approach is shot to shit, really.
And of course, what about hygienic references to variables? How does a
macro expansion refer to the variable FOO such that there is no
capture, and yet newly assigned values to the variable will be seen?
Something like `(print foo) won't work, because it's not a lexical
reference. On the other hand `(print ,foo) causes FOO to be completely
resolved at macro-expansion time down to a value which is inserted
into the source code. So the macro-expanded code will not track
changes of the variable FOO. Moreover, the reduction to a value only
works for objects that evaluate to themselves (okay that can be fixed
if we insert ',FOO instead of ,FOO which is hacky). The bigger problem
is that, those have to be externalizeable for the purposes of
compilation.
I think how this mess can be rescued is if you define some additional
special operators that work with symbols, and produce some useful
snippets of code that does the right thing in macros. You may have to
make use of the splice operator in backquotes:
`(,@(oper foo) ...)
This OPER, whatever it is, will generate code specifically designed
for this situation. The reason it is a splice is because (OPER FOO)
may have to produce a list: a list which most likely starts with
FUNCALL, if you catch my drift. The goal is to generate the code
(FUNCALL (CELL-VALUE '#:CELL366) ...). This code is the correct way to
lexically call the function FOO, which is permanently associated with
#:CELL366 even under multiple LDEFUNS.
You cuold have a similar operator for variables, so that
`(print ,(oper2 foo))
will generate the code
(PRINT (CELL-VALUE `#:CELL367))
which is the correct way to lexically refer to FOO in a transparent
way that can be planted anywhere, which follows changes in the value
of the cell, and which is SETF-able.
What are good names for OPER and OPER2? I'd call them HCALL and HVAR
(hygienic call to a function, hygienic reference to var). A third one,
HFUN, could be a hygienic way to obtain the lexical function binding.
HCALL just conses a FUNCALL onto whatever HFUN returns, so that
`(FUNCALL ,(HFUN FOO) ...) is long-winded for `(,@(HCALL FOO) ...).
In article
<····································@64g2000hsw.googlegroups.com>,
Kaz Kylheku <········@gmail.com> wrote:
> Come on, wouldn't the following Just Work, without duplicating the
> function cell into the value cell?
>
> `(,(lfunction foo) ...)
Sure, but that's more typing.
> You just don't have a nice shorthand read notation for (lfunction foo)
> that's all, so it's verbose.
Yep.
> But, there is a big problem. Both solutions suffer from the obvious
> problem that they ``bake in'' in the actual Lisp function at macro-
> expansion time.
Yeah, that's a bug. It's supposed to pick up the binding, not the
value. Damn.
> I think how this mess can be rescued is if you define some additional
> special operators that work with symbols, and produce some useful
> snippets of code that does the right thing in macros. You may have to
> make use of the splice operator in backquotes:
>
> `(,@(oper foo) ...)
>
> This OPER, whatever it is, will generate code specifically designed
> for this situation. The reason it is a splice is because (OPER FOO)
> may have to produce a list: a list which most likely starts with
> FUNCALL, if you catch my drift. The goal is to generate the code
> (FUNCALL (CELL-VALUE '#:CELL366) ...). This code is the correct way to
> lexically call the function FOO, which is permanently associated with
> #:CELL366 even under multiple LDEFUNS.
>
> You cuold have a similar operator for variables, so that
>
> `(print ,(oper2 foo))
>
> will generate the code
>
> (PRINT (CELL-VALUE `#:CELL367))
>
> which is the correct way to lexically refer to FOO in a transparent
> way that can be planted anywhere, which follows changes in the value
> of the cell, and which is SETF-able.
>
> What are good names for OPER and OPER2? I'd call them HCALL and HVAR
> (hygienic call to a function, hygienic reference to var). A third one,
> HFUN, could be a hygienic way to obtain the lexical function binding.
> HCALL just conses a FUNCALL onto whatever HFUN returns, so that
> `(FUNCALL ,(HFUN FOO) ...) is long-winded for `(,@(HCALL FOO) ...).
Let me sleep on this. Thanks for the feedback.
rg
On Feb 26, 12:16 am, Ron Garret <·········@flownet.com> wrote:
> In article
> <····································@64g2000hsw.googlegroups.com>,
> Kaz Kylheku <········@gmail.com> wrote:
> > What are good names for OPER and OPER2? I'd call them HCALL and HVAR
> > (hygienic call to a function, hygienic reference to var). A third one,
> > HFUN, could be a hygienic way to obtain the lexical function binding.
> > HCALL just conses a FUNCALL onto whatever HFUN returns, so that
> > `(FUNCALL ,(HFUN FOO) ...) is long-winded for `(,@(HCALL FOO) ...).
>
> Let me sleep on this. Thanks for the feedback.
Here is another idea. Since references to all these global lexicals
are done through symbol and regular macros, the semantics of
evaluation of variable and function references is under your control.
Normally FOO macroexpands to (CELL-VALUE ...) which is called to fetch
the value (or turns into SET-CELL-VALUE in a SETF assignment context).
But this macroexpansion is inconvenient when you are writing macros.
In macros you don't so much want to access global lexicals as you want
to insert references to them into the generated code. These references
have to be already compiled to unevaluated (CELL-VALUE ...)
expressions.
So, you could provide a binding construct which rebinds names to their
corresponding CELL-VALUE expressions:
(with-lexical-vars (a b c)
;; a, b and d hold (CELL-VALUE ...) exps denoting variables
...)
(with-lexical-fun-refs (f g)
;; f and g holds (CELL-VALUE ...) exp denoting functions
If your macro is wrapped with these things, you can write `(,f
(function ,g) ,h) which does the right thing: the lexical function F
will be called, with the current value of the lexical variable A, and
the function value of lexical G, all lexicals being resolved in the
original lexicon of the macro, no matter where it is expanded.
The LNAME problem should be easy to fix; instead of calling MAKE-
SYMBOL in LDEFUN every time, LDEFUN should check that a cell exists
for the given function NAME. If so, the only job it has to do is to
redo the Lisp DEFUN. There is no need to LEXIFY, no need to generate a
new LNAME, and no need to store it. The LNAME for the DEFUN is simply
pulled out from the function cell, where the first LDEFUN stored it.
The way you have it now, LDEFUN is oblivious to previous calls. The
only code which is aware of the existing bindings is *LBIND, and
you've told it to shut up via :IF-BOUND IGNORE.
Another thing I noticed is that CELL-VALUE is redundant. (CELL-VALUE
X) reduces to (SYMBOL-VALUE X). X is always an uninterned symbol, so
unintended capture is not an issue. Thus (SYMBOL-VALUE 'Y) is
equivalent to Y and can be replaced by Y. Instead of generating (CELL-
VALUE 'SYM) all over the place, just generate SYM. You can then do
away with SET-CELL-VALUE also.
In article
<····································@e41g2000hsc.googlegroups.com>,
Kaz Kylheku <········@gmail.com> wrote:
> On Feb 26, 12:16�am, Ron Garret <·········@flownet.com> wrote:
> > In article
> > <····································@64g2000hsw.googlegroups.com>,
> > �Kaz Kylheku <········@gmail.com> wrote:
> > > What are good names for OPER and OPER2? I'd call them HCALL and HVAR
> > > (hygienic call to a function, hygienic reference to var). A third one,
> > > HFUN, could be a hygienic way to obtain the lexical function binding.
> > > HCALL just conses a FUNCALL onto whatever HFUN returns, so that
> > > `(FUNCALL ,(HFUN FOO) ...) is long-winded for `(,@(HCALL FOO) ...).
> >
> > Let me sleep on this. �Thanks for the feedback.
>
> Here is another idea. Since references to all these global lexicals
> are done through symbol and regular macros, the semantics of
> evaluation of variable and function references is under your control.
> Normally FOO macroexpands to (CELL-VALUE ...) which is called to fetch
> the value (or turns into SET-CELL-VALUE in a SETF assignment context).
> But this macroexpansion is inconvenient when you are writing macros.
> In macros you don't so much want to access global lexicals as you want
> to insert references to them into the generated code. These references
> have to be already compiled to unevaluated (CELL-VALUE ...)
> expressions.
>
> So, you could provide a binding construct which rebinds names to their
> corresponding CELL-VALUE expressions:
>
> (with-lexical-vars (a b c)
> ;; a, b and d hold (CELL-VALUE ...) exps denoting variables
> ...)
>
> (with-lexical-fun-refs (f g)
> ;; f and g holds (CELL-VALUE ...) exp denoting functions
>
> If your macro is wrapped with these things, you can write `(,f
> (function ,g) ,h) which does the right thing: the lexical function F
> will be called, with the current value of the lexical variable A, and
> the function value of lexical G, all lexicals being resolved in the
> original lexicon of the macro, no matter where it is expanded.
>
> The LNAME problem should be easy to fix; instead of calling MAKE-
> SYMBOL in LDEFUN every time, LDEFUN should check that a cell exists
> for the given function NAME. If so, the only job it has to do is to
> redo the Lisp DEFUN. There is no need to LEXIFY, no need to generate a
> new LNAME, and no need to store it. The LNAME for the DEFUN is simply
> pulled out from the function cell, where the first LDEFUN stored it.
>
> The way you have it now, LDEFUN is oblivious to previous calls. The
> only code which is aware of the existing bindings is *LBIND, and
> you've told it to shut up via :IF-BOUND IGNORE.
>
> Another thing I noticed is that CELL-VALUE is redundant. (CELL-VALUE
> X) reduces to (SYMBOL-VALUE X). X is always an uninterned symbol, so
> unintended capture is not an issue. Thus (SYMBOL-VALUE 'Y) is
> equivalent to Y and can be replaced by Y. Instead of generating (CELL-
> VALUE 'SYM) all over the place, just generate SYM. You can then do
> away with SET-CELL-VALUE also.
That's true, but only because cells happen to be implemented as symbols.
They don't have to be, and if you look at the code you'll see two
alternate implementations I was playing with. CELL-VALUE is there to
make it easy to swap out implementations.
But I'm actually tending towards a radically different solution which
entails completely restructuring how lexicons are implemented. I really
want very much to keep the current API because one of the goals is to be
easily accessible to newcomers, and I worry that going from (,helper-fn
...) to (,@(href helper-fn) ...) or something like that will cross a
threshold of complexity that will make people balk.
The new idea is to radically embrace symbols under the hood. Every
lexicon will have a package associated with it. The symbol-macro for a
symbol will expand into the corresponding symbol in the package
corresponding to the current lexicon. The value and function namespaces
will be stored in the value and function cells of the symbol (and
probably classes too). Additional namespaces, if any, will be stored
somewhere else. This way, the current strategy of having a symbol
macroexpand to another symbol will do the Right Thing because the second
symbol will *be* the binding. The implementation will be a little
inelegant, but I think it will work, and it will have the added bonus of
making fans of packages a little more comfortable. (Lexicons will in
some sense *be* packages whose name resolution happens at compile time
rather than read time.)
All this would all be a lot simpler if CL just handled ((...) ...) in a
sane way. :-(
rg
On Feb 26, 10:54 am, Ron Garret <·········@flownet.com> wrote:
> In article
> <····································@e41g2000hsc.googlegroups.com>,
> Kaz Kylheku <········@gmail.com> wrote:
>
>
>
>
>
> > On Feb 26, 12:16 am, Ron Garret <·········@flownet.com> wrote:
> > > In article
> > > <····································@64g2000hsw.googlegroups.com>,
> > > Kaz Kylheku <········@gmail.com> wrote:
> > > > What are good names for OPER and OPER2? I'd call them HCALL and HVAR
> > > > (hygienic call to a function, hygienic reference to var). A third one,
> > > > HFUN, could be a hygienic way to obtain the lexical function binding.
> > > > HCALL just conses a FUNCALL onto whatever HFUN returns, so that
> > > > `(FUNCALL ,(HFUN FOO) ...) is long-winded for `(,@(HCALL FOO) ...).
>
> > > Let me sleep on this. Thanks for the feedback.
>
> > Here is another idea. Since references to all these global lexicals
> > are done through symbol and regular macros, the semantics of
> > evaluation of variable and function references is under your control.
> > Normally FOO macroexpands to (CELL-VALUE ...) which is called to fetch
> > the value (or turns into SET-CELL-VALUE in a SETF assignment context).
> > But this macroexpansion is inconvenient when you are writing macros.
> > In macros you don't so much want to access global lexicals as you want
> > to insert references to them into the generated code. These references
> > have to be already compiled to unevaluated (CELL-VALUE ...)
> > expressions.
>
> > So, you could provide a binding construct which rebinds names to their
> > corresponding CELL-VALUE expressions:
>
> > (with-lexical-vars (a b c)
> > ;; a, b and d hold (CELL-VALUE ...) exps denoting variables
> > ...)
>
> > (with-lexical-fun-refs (f g)
> > ;; f and g holds (CELL-VALUE ...) exp denoting functions
>
> > If your macro is wrapped with these things, you can write `(,f
> > (function ,g) ,h) which does the right thing: the lexical function F
> > will be called, with the current value of the lexical variable A, and
> > the function value of lexical G, all lexicals being resolved in the
> > original lexicon of the macro, no matter where it is expanded.
>
> > The LNAME problem should be easy to fix; instead of calling MAKE-
> > SYMBOL in LDEFUN every time, LDEFUN should check that a cell exists
> > for the given function NAME. If so, the only job it has to do is to
> > redo the Lisp DEFUN. There is no need to LEXIFY, no need to generate a
> > new LNAME, and no need to store it. The LNAME for the DEFUN is simply
> > pulled out from the function cell, where the first LDEFUN stored it.
>
> > The way you have it now, LDEFUN is oblivious to previous calls. The
> > only code which is aware of the existing bindings is *LBIND, and
> > you've told it to shut up via :IF-BOUND IGNORE.
>
> > Another thing I noticed is that CELL-VALUE is redundant. (CELL-VALUE
> > X) reduces to (SYMBOL-VALUE X). X is always an uninterned symbol, so
> > unintended capture is not an issue. Thus (SYMBOL-VALUE 'Y) is
> > equivalent to Y and can be replaced by Y. Instead of generating (CELL-
> > VALUE 'SYM) all over the place, just generate SYM. You can then do
> > away with SET-CELL-VALUE also.
>
> That's true, but only because cells happen to be implemented as symbols.
> They don't have to be, and if you look at the code you'll see two
> alternate implementations I was playing with. CELL-VALUE is there to
> make it easy to swap out implementations.
You can swap out implementations anyway! In theory, though, CELL-VALUE
should make it possible to swap out an implementation at run-time,
rather than compile time. Is that something anyone should care about?
Also, it's only possible if the CELL-VALUE interface is consistently
being used at run time, and never reduced at compile time. Currently,
this is not true, because your (LDEFMACRO NAME) writes a Lisp
(DEFMACRO NAME ...) which evaluates (CELL-VALUE ...) at macro-
expansion time:
(defmacro ,name (&rest args &environment env1)
`(,(cell-value (*ref (env-lexicon env1) ',name
'function)) ,@args))
The (CELL-VALUE ...) expression is reduced to a single uninterned
symbol, and so the macaroexpansion bypasses that interface. You can't
swap to a different representation of cells at run-time; these macros
must all be recompiled.
> But I'm actually tending towards a radically different solution which
> entails completely restructuring how lexicons are implemented. I really
> want very much to keep the current API because one of the goals is to be
> easily accessible to newcomers, and I worry that going from (,helper-fn
> ...) to (,@(href helper-fn) ...) or something like that will cross a
> threshold of complexity that will make people balk.
>
> The new idea is to radically embrace symbols under the hood. Every
> lexicon will have a package associated with it.
Whoa hold it. One thing you could do is simply give up the idea that
you can have a function and binding for the same name in the same
context come from different lexicons.
What if lexicons just mapped from concrete symbols to renamed
uninterned symbols, without regard for function versus value binding.
So for instance in some given context, FOO expands to #:CELL123
through the lexicon. The value of FOO is the value of #:CELL123, and
the function is the function of #:CELL123. There goes the ability to
separately import function and value bindings, right out the window.
Is that a huge deal?
Now all you need is for a macro-defining environment to implicitly
bind all of the global lexicals to their renamed symbols.
That is to say, you could have a macro called LEX-RENAME which scans
the entire lexical environment visible from that point, and sets up
symbol macrolets which map the available symbols to their translated
counterparts. LEX-RENAME would be implicit around macros (it would
surround the DEFMACRO so that the macro's arguments could shadow the
renaming macrolets).
So in a macro you would just naively write things like `(,f ,a ,b).
The global lexicals F, A and B have /implicitly/ become local
variables whose values are #:CELL235 #:CELL123 and #:CELL025. The
programmer didn't have to write anything to make this work. And so the
backquote pumps out the code (#:CELL235 #:CELL123 #:CELL025), correct
code which does the Right Thing in any context: it calls the correct
global function, and passes it the values of the correct global
variables. (Only problem is, that could be one mighty darn big SYMBOL-
MACROLET. To optimize its size, you'd have to walk the macro's code to
ferret out which translations are actually required).
Of course `(,f ,f) means pass the variable f to function f. Both of
these f's translate through the lexicon to the same symbol, so you
don't have to have any additional syntax to tell Lexicons which
translation you want. The underlying Lisp takes care of the fact that
the f (or rather the symbol that f renames to) has two different
meanings based on position within the form.
If someone wants to refer to the value of a global lexical within a
macro body, they can get it through the (LVALUE SYM) interface, which
is impervious to the symbol macrolets, since it is a special form.
(LVALUE SYM) simply translates to (SYMBOL-VALUE '#:CELLnnn), or rather
just #:CELLnnn.
> The symbol-macro for a
> symbol will expand into the corresponding symbol in the package
> corresponding to the current lexicon. The value and function namespaces
> will be stored in the value and function cells of the symbol (and
> probably classes too).
Ah, you're thinking of the same thing. Well, as long as you're willing
to give up separate translation cells for separate namespaces.
In article
<····································@s37g2000prg.googlegroups.com>,
Kaz Kylheku <········@gmail.com> wrote:
> >�The symbol-macro for a
> > symbol will expand into the corresponding symbol in the package
> > corresponding to the current lexicon. �The value and function namespaces
> > will be stored in the value and function cells of the symbol (and
> > probably classes too).
>
> Ah, you're thinking of the same thing. Well, as long as you're willing
> to give up separate translation cells for separate namespaces.
I am reluctant to give that up, because that means giving up on the
ability to do convert-lexicon-to-lisp1-semantics, which is a hack that
I'm rather fond of. But I can't think of any way to make macros do the
Right Thing and retain this, at least not without hacking the
implementation to support ((...) ...) semantics.
rg
In article
<····································@s37g2000prg.googlegroups.com>,
Kaz Kylheku <········@gmail.com> wrote:
> What if lexicons just mapped from concrete symbols to renamed
> uninterned symbols, without regard for function versus value binding.
> So for instance in some given context, FOO expands to #:CELL123
> through the lexicon. The value of FOO is the value of #:CELL123, and
> the function is the function of #:CELL123.
Turns out this doesn't work. It's because in the context of `(,f ...),
F doesn't just get macroexpanded, it gets *evaluated*. So you end up
not with (#:CELL123 ...) but rather a list whose car is the *value* of
#:CELL123, which is exactly the same problem as the current design.
I am beginning to think that the real underlying problem here is that
backquote/comma really doesn't do the Right Thing, that what is really
needed is an extension to backquote so that you can write, e.g.:
`(^foo ,foo foo)
which would yield:
(list (macroexpand 'foo) foo 'foo)
Or something like that.
rg
On Feb 27, 11:44 am, Ron Garret <·········@flownet.com> wrote:
> In article
> <····································@s37g2000prg.googlegroups.com>,
> Kaz Kylheku <········@gmail.com> wrote:
>
> > What if lexicons just mapped from concrete symbols to renamed
> > uninterned symbols, without regard for function versus value binding.
> > So for instance in some given context, FOO expands to #:CELL123
> > through the lexicon. The value of FOO is the value of #:CELL123, and
> > the function is the function of #:CELL123.
>
> Turns out this doesn't work. It's because in the context of `(,f ...),
> F doesn't just get macroexpanded, it gets *evaluated*.
Yes, exactly. It can't just work naively like that.
To make this work, you have to create an environment in which F, and
other global lexicals, are rebound to their translations as literal
values.
Normally, the global lexical F is a symbol-macro which expands to the
expression (SYMBOL-VALUE '#:CELL123) (or basically just #:CELL123).
Either way, since this is the output of a macro, it evaluates to the
contents of #:CELL123.
What I'm talking about is this:
(let ((f '#:CELL123) ...)
`(,f ...))
-> (#:CELL123 ...)
Or, if you'd like:
(symbol-macrolet ((f '#:CELL123) ...)
`(,f ...))
-> (#:CELL123 ...)
This LET wouldn't be written by the programmer, of course. There are a
number of ways to do it. One is a binding construct which names all of
the symbols to be renamed to their lexicon translations:
;; least attractive, but could be a useful tool anyway
(lex-rename (f g a b c)
`(,f (,g ,a) ,b ,c))
-> (#:CELL123 (#:CELL125 #:CELL231) #:CELL322 #:CELL309)
Another option is a construct which takes no parameters, and just does
all of the visible global lexicals:
(lex-rename-all
`(,f (,g ,a) ,b ,c))
-> ;; same as before
This is not ``Pascal-with-evil'' because it doesn't introduce any new
sybmols into the scope; it just flips the evaluation semantics of
existing symbols.
Another idea is to simply make lex-rename-all implicit around
LDEFMACROS:
;; you still need lex-rename-all when breaking macro into helpers
(ldefmacro mac (arg)
`(,foo ,global ,arg))
;; expands to (#:CELL123 #:CELL495 <arg-source-code>)
(ldefun fun (arg)
`(,foo ,global ,arg))
;; returns (<value-of-foo> <value-of-global> <value-of-arg>)
Of course rename is scoped around the entire macro function, so arg of
course shadows any global lexical translation of the same name..
I could definitely live with something like that. Inside a macro
function, it's going to be more common to want global lexicals to
denote their lexicon translation rather than their value. Should you
want the value at macro time you can get it explicitly:
(ldefmacro mac (arg)
`(,foo ,(lvalue global) ,arg))
;; expands to (#:CELL123 <baked-value-of-global> <arg-source-code>)
Because LVALUE is a special operator, it is impervious to the local
rebinding of GLOBAL. It gets the GLOBAL symbol unexpanded and
unevaluated and just continues to work as normal.
With this approach, it might be also good to have a function-writing
macro specifically for writing macro helpers.
;; implicit lex-rebind-all around ldefmacrofun's.
(ldefmacrofun macfun (arg)
`(,foo ,global ,arg))
(ldefmacro mac (arg)
(macfun arg))
LDEFMACROFUN could also put in the EVAL-WHEN to ensure that MACFUN is
available at macroexpansion time.
In article
<····································@s19g2000prg.googlegroups.com>,
Kaz Kylheku <········@gmail.com> wrote:
> On Feb 27, 11:44�am, Ron Garret <·········@flownet.com> wrote:
> > In article
> > <····································@s37g2000prg.googlegroups.com>,
> > �Kaz Kylheku <········@gmail.com> wrote:
> >
> > > What if lexicons just mapped from concrete symbols to renamed
> > > uninterned symbols, without regard for function versus value binding.
> > > So for instance in some given context, FOO expands to #:CELL123
> > > through the lexicon. The value of FOO is the value of #:CELL123, and
> > > the function is the function of #:CELL123.
> >
> > Turns out this doesn't work. �It's because in the context of `(,f ...),
> > F doesn't just get macroexpanded, it gets *evaluated*.
>
> Yes, exactly. It can't just work naively like that.
>
> To make this work, you have to create an environment in which F, and
> other global lexicals, are rebound to their translations as literal
> values.
>
> Normally, the global lexical F is a symbol-macro which expands to the
> expression (SYMBOL-VALUE '#:CELL123) (or basically just #:CELL123).
> Either way, since this is the output of a macro, it evaluates to the
> contents of #:CELL123.
Only if F is bound when it is defined. If it isn't, then it becomes a
deferred binding, and F expands into (cell-value (funcall #<a closure>)).
> What I'm talking about is this:
>
> (let ((f '#:CELL123) ...)
> `(,f ...))
>
> -> (#:CELL123 ...)
>
> Or, if you'd like:
>
> (symbol-macrolet ((f '#:CELL123) ...)
> `(,f ...))
>
> -> (#:CELL123 ...)
>
> This LET wouldn't be written by the programmer, of course. There are a
> number of ways to do it. One is a binding construct which names all of
> the symbols to be renamed to their lexicon translations:
>
> ;; least attractive, but could be a useful tool anyway
> (lex-rename (f g a b c)
> `(,f (,g ,a) ,b ,c))
>
> -> (#:CELL123 (#:CELL125 #:CELL231) #:CELL322 #:CELL309)
>
> Another option is a construct which takes no parameters, and just does
> all of the visible global lexicals:
>
> (lex-rename-all
> `(,f (,g ,a) ,b ,c))
>
> -> ;; same as before
>
> This is not ``Pascal-with-evil'' because it doesn't introduce any new
> sybmols into the scope; it just flips the evaluation semantics of
> existing symbols.
>
> Another idea is to simply make lex-rename-all implicit around
> LDEFMACROS:
>
> ;; you still need lex-rename-all when breaking macro into helpers
> (ldefmacro mac (arg)
> `(,foo ,global ,arg))
> ;; expands to (#:CELL123 #:CELL495 <arg-source-code>)
>
> (ldefun fun (arg)
> `(,foo ,global ,arg))
> ;; returns (<value-of-foo> <value-of-global> <value-of-arg>)
>
> Of course rename is scoped around the entire macro function, so arg of
> course shadows any global lexical translation of the same name..
Interesting idea. I've taken a different approach. Instead of:
`(,foo ...)
you now write:
`(^foo ...)
where the carat is intended to evoke "top level". This is currently
implemented with a reader macro and a little code walker, which is ugly
but it seems to work (at least it passes the one test case that the
previous implementation failed on). And it keeps the API nice and
simple for beginners.
(I note in passing that there is a simpler way of implementing the same
API, namely, to define a synonym for every lexified symbol that adds a
prefix carat to the name, and then simply adopting the convention of
never binding a symbol whose name starts with a carat. This is not real
hygiene, but it's serviceable. Also, CL already has a typographical
convention for special variables (*...*) so a similar convention for
global lexicals would seem to be well within the spirit of current
practice.)
New code has been uploaded to the usual place
(http://www.flownet.com/ron/lisp). There's a new lexicons.lisp and a
new lexdemo.lisp. BTW, I also took out the code that set the value cell
for ldefun and ldefmacro, so it now works more like regular CL. This is
still bleeding-edge not-quite-ready-for-prime-time code, but it does
seem to mostly work.
BTW, thank you for taking the time to really dig into this. Your
feedback has been very helpful.
rg
On Mon, 25 Feb 2008 17:28:31 -0800, Ron Garret wrote:
> I am pleased to announce that Lexicons are finally ready for their
> close-up. Get them here:
> ...
> rg
My suggestion is that you add to your posting a paragraph saying what
lexicons are or do. Not everyone has all the context in their head.
Something like this.
Lexicons are first-class global lexical environments, what are sometimes
called “modules” or “namespaces” in other languages. They are designed as
an adjunct (or replacement) for Common Lisp packages. Lexicons were
designed to address two shortcomings of the Common Lisp package system:
1. The need to manually manage list of exported symbols, and
2. The fact that common mistakes have read-time side-effects that can be
difficult to recover from, especially for beginners.
Tim
In article <···············@corp.supernews.com>,
tim <····@internet.com> wrote:
> On Mon, 25 Feb 2008 17:28:31 -0800, Ron Garret wrote:
>
> > I am pleased to announce that Lexicons are finally ready for their
> > close-up. Get them here:
> > ...
> > rg
>
> My suggestion is that you add to your posting a paragraph saying what
> lexicons are or do. Not everyone has all the context in their head.
>
> Something like this.
>
> Lexicons are first-class global lexical environments, what are sometimes
> called “modules” or “namespaces” in other languages. They are designed as
> an adjunct (or replacement) for Common Lisp packages. Lexicons were
> designed to address two shortcomings of the Common Lisp package system:
>
> 1. The need to manually manage list of exported symbols, and
>
> 2. The fact that common mistakes have read-time side-effects that can be
> difficult to recover from, especially for beginners.
Yeah... what he said.
;-)
rg