Hi Group!
This topic originates from the following discussion:
http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/00e2aa56a0f84b20/feaa03484c1c653a?lnk=raot#feaa03484c1c653a
But in the course of that discussion I suggested an obviously
incorrect solution to that problem, so I dare to restart discussion
from the scratch.
Problem is the following:
> (defpackage :p1 (:exports :sym :sym1))
> (defpackage :p2 (:exports :sym :sym2))
> (defpackage :p3 (:use :cl :p1 :p2)) ; error: sym is ambigious.
Sometimes there are many symbols in p1 and p2, we intend use some of
them, but never going to use :sym. Even if we are going to use that,
we agree to qualify it with it's package name. But we want to refer
all non-clashing symbols by their direct names, w/o package prefix.
We'd wanted to just use both p1 and p2. Lisp still allows us to do
that, but we need to write something like:
> (defpackage :p3 (:use :cl)
(:import-from :p1 . list-of-all-external-symbols-from-p1-but-sym)
(:import-from :p2 . list-of-all-external-symbols-from-p2-but-
sym))
Moreover, if we are acquainted to :p3 namespace and then want to
create some package :p4, which would have p3's look-and-feel, we need
to copy and alter package definition of p3 like that:
> (defpackage :p3 (:use :cl)
(:import-from :p1 . list-of-all-external-symbols-from-p1-but-sym)
(:import-from :p2 . list-of-all-external-symbols-from-p2-but-
sym))
Then we might occasionally add new symbols to :p1 and/or :p2. For
those to be exported to p3 and p4, we might need review and modify
definitions of both p3 and p4.
All this seem to be a bit of mess, especially when the number of
packages is large and then some of the packages are external
libraries. It is especially bad when trying to use external libraries
which often export such actively used symbols as for, while, collect,
etc.
Solution suggested is the following. Add a new macro
(in-packages package1 &rest more-packages)
which behaves approximately as follows:
0. (in-packages package1) behaves exactly as (in-package package1)
1. Construct is dynamic, like in-package. If we are in scope of in-
packages, we are not in scope of in-package and vice versa.
2. When in-packages construct is in effect, reader acts as following:
2.1. When symbol name "SYM1" is unique between all packages
designated, the symbol can be read by its unqualified name "SYM1"
2.2. Otherwise, when symbol name "SYM" is interned in package1 and it
is in shadowing-symbols list of package1, it can be read from that
package by its unqualified name "SYM"
2.3. Otherwise, when symbol name "SYM" occurs in more than one package
designated, and these symbols are not EQ, the symbol can be read only
by its qualified name, e.g. "P2:SYM". Attempt to read an unqualified
name is an error. In package1, "occurs" means it is either internal or
external. In other packages, "occurs" means it is external.
2.4. New symbols which are not found in any package and do not cause a
error, are interned by default to package1.
3. Add a support in development tools (SLIME etc).
4. Add a read-time construct #with-packages(packages &rest body) which
reads body in the scope of packages defined. Packages need to be
defined at compile time for this to work correctly. And, maybe, #with-
added-packages which words as temporarily adding more packages to
current list of (in-packages) arguments.
How would this alter cl's usage? We would seldom employ "use-package"
and defpackage's "uses" clause. Instead of
(defpackage foo (:use :cl :bar) (:exports ...))
(in-package foo)
we would write
(defpackage foo (:use) (:exports ...))
(in-packages :foo :cl :bar)
This way, we won't get a symbol clash at the time use-package is
issued. Instead, we get the clash only when clashing symbol is about
to be interned.
This behavior of reporting conflict only when it really occurs
(instead of the time it gets potentially possible) is observed in
other popular languages:
e.g. C++ "using namespace"
SQL "select sym from p1 inner join p2"
C++ is a messy language and in fact it shouldn't be taken as an
example, but SQL is actually a Good Thing, it proven itself reliable,
scalable (in dimension of application complexity) and convinient to
use. In fact, every join between tables in SQL creates something like
a new namespace where many symbols would clash. But SQL solves this
problem by requiring to disambiguate only the symbols that cause
conflicts and only at the time the symbol is referred (not at the time
inner join operation creates merged namespace). This gives a good
safety of a code while keeping the code reasonably short.
Requirements:
Lisp implementation core modification seem to be required. Likely this
is a reader, intern and/or find-symbol functions and their core
counterparts.
Prior art:
conduit packages:
http://www.tfeb.org/lisp/hax.html#CONDUITS
Seem to allow just the same behaviour as proposed (and many more) with
the use of "extend" keyword, but is a bit more complex and requires
more modifications to package system. When addressing a symbol clashes
between external libraries,
package definition of that libraries seem to need modification.
With respect to reducing a pain caused by symbol clashes, my solution
seem to be more elegant and effective.
lexicons:
http://www.flownet.com/ron/lisp/Lexicons.pdf
http://www.flownet.com/ron/lisp/lexicons.lisp
Addresses the same problem in part, but solution is completely
different.
1. Deals with bindings, not with symbols itself
2. Redefines defun, so it would conflict with any lisp extension
which redefines defun too.
3. Would not help resolving symbol clashes in legacy code.
4. I found no way to refer to other lexicons/namespaces from within
some lexicon/namespace.
budden wrote:
> Hi Group!
>
> This topic originates from the following discussion:
>
> http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/00e2aa56a0f84b20/feaa03484c1c653a?lnk=raot#feaa03484c1c653a
>
> But in the course of that discussion I suggested an obviously
> incorrect solution to that problem, so I dare to restart discussion
> from the scratch.
>
> Problem is the following:
This reads like typical introductions to promoting solutions for
non-issues. ("Untyped programs may give runtime type errors.",
"Traditional macro systems may break hygiene.", etc.)
How often does this problem really arise?
I don't have that problem.
>> (defpackage :p1 (:exports :sym :sym1))
>> (defpackage :p2 (:exports :sym :sym2))
>> (defpackage :p3 (:use :cl :p1 :p2)) ; error: sym is ambigious.
>
> Sometimes there are many symbols in p1 and p2, we intend use some of
> them, but never going to use :sym. Even if we are going to use that,
> we agree to qualify it with it's package name. But we want to refer
> all non-clashing symbols by their direct names, w/o package prefix.
> We'd wanted to just use both p1 and p2. Lisp still allows us to do
> that, but we need to write something like:
>
>> (defpackage :p3 (:use :cl)
> (:import-from :p1 . list-of-all-external-symbols-from-p1-but-sym)
> (:import-from :p2 . list-of-all-external-symbols-from-p2-but-
> sym))
(defpackage :p3 (:use :cl :p1 :p2)
(:shadow :sym))
?!?
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
> How often does this problem really arise?
I faced the problem when tried to add iterate to symbolicweb. Real-
world task, not a library, not a DSL experiment. I like iterate as it
is more lispy then loop. It is _much_ better than loop. First of all,
loop is not extensible (do you remember we are coding in an extensible
language!). But it is convinient that loop uses symbols as keywords,
so it does not matter which "for" to use with loop. It is not the case
with iterate - it is truly lispy sublanguage and it obeys all lisp
rules. Lisp says DSL should rely upon its own symbols and don't mess
everything around.
So, (to simplify things) iterate requires "for" to be from the same
package as the iteration variable. But... some other library exports
"for" too. So: you have no internal "for" in the current package. And
you need write something like
(iter
(for that-lib:i in '(1 2 3))
(collect that-lib:i))
that is super ugly!
There is a synonym mechanism in iterate. But there is other problem:
that-lib exports "collect" too, and collect is a macro. Synonyms
mechanism has lower priority than a macro mechanism. So... I define
synonyms for "for" and "collect" and write (moderately ugly)
(iter
(ifor i in '(1 2 3))
(icollect i))
Ok. We solved it, hurray!
But... All this code is not a final package. It is a library too.
How do I'll use this in my final code?
I'll use
(defpackage :mylib-user (:use :hunchentoot :cl-who :mylib :that-
lib :iterate)
shadow all clashing symbols that we shadowed in my-lib...
)
redefine all synonyms...
final defpackage looks very funny... In fact I need not only to copy
defpackage symbolicweb, I need to do it again at again as symbolicweb
itself changes. Cool...
Yes, there IS an alternative. Don't use anything, prefix everything.
(iter:iter
(ifor i in mylist)
(cl-who:with-html-output-to-string (:esc (that-lib:that-fun i))
(ps:ps-inline (alert "ouch"))))
etc. Well. Where did we started? We simply wanted to generate html+js
in a convinient way. What we get in fact?
A. Difficult to start. Defpackage is hard to write
B. Difficult to use. We should remember from which package which
symbol comes from. Should final use care that our project consists of
several packages? It is a special sublanguage for generating html+js
and rendering it from http server, so we need all frequently used
symbols in one package. We want to use completion for entering
symbols, not an apropos for finding them and then entering. Even
worse, If I'll enter unexported symbol w/o prefix by an omission, it
gets interned to my package, but remains unbound. So I'll be very
surprised at compile time. And then I will need to investigate with
apropos which symbol is wrong, then unintern it. All this is very hard
for serios DSLs with hundreds of symbols.
C. When we want to improve something we have to learn this.
D. When library changes, we get unexpected symbol clashes.
Ok, friendly lispers explained me already that I am dumb and that lisp
is perfect... Hmm... After using lisp for nine years, I started to
think about learning PHP...
All this disappointment stems from experience with real system called
Symbolicweb. I took a small part in its development (testing, etc).
Someday I decided that it would be fine to have examples running not
in a :symbolicweb package. In fact, final user will not work
in :symbolicweb package. He would use it. Moreover, in context of web
development, Symbolicweb can itself be considered as a library. So, if
we give an example we would give a defpackage example too. I get much
trouble with it.
For me it looks like that lisp is not scalable enough in the sence of
using many different libraries in one project. Simplest and very
evident solution of importing just all non-clashing symbols is
rejected by any and every right lisper.
Of course, it seems that iterate is not designed will... It tried to
be lispy so hard, so it fell into all lispy traps... It would better
use symbols as keywords, or just allow keywords to be used, like this:
(iter
(:for i :from 1 :to 10)
(:collect i))
From: Kaz Kylheku
Subject: Re: Symbol clashes: how to avoid them. Part 2
Date:
Message-ID: <20081031115242.5@gmail.com>
On 2008-10-31, budden <········@mtu-net.ru> wrote:
> Hi Group!
>
> This topic originates from the following discussion:
You have overlooked one serious problem with USE-PACKAGE and its ilk.
Suppose that you are using two packages A and B, which happen to be
conflict-free.
The A package exports a function called FOO which you use by
its unqualified symbol.
Now a new version of both A and B comes out. It so happens that A:FOO is
removed from A (or no longer exported), and at the same time,
a new symbol B:FOO appears in B, also naming a function.
Your package use of A and B is still conflict-free. But now, your code
is calling the wrong FOO. No diagnostic is produced.
So you see, carelessly inheriting from multiple namespaces is dangerous and
stupid.
Handpicking symbols is the way to go, if you want to get it right, and in a way
that is future-proof.
(:shadowing-import-from :a :foo)
will blow up if FOO disappears from A. And it won't pick up any new symbols
that have appeared in a new revision of A.