This is part of Sacla project.
http://homepage1.nifty.com/bmonkey/lisp/sacla/index-en.html
You can find the latest sacla loop at.
http://homepage1.nifty.com/bmonkey/lisp/sacla/lisp/loop.lisp
Some design goals of this loop implementation are
* understanding the loop facility by implementing it.
* being as portable as possible.
* producing a good expansion code for a relatively common compiler.
* adopting optimizations which boost the execution speed of idioms which
are commonly used.
* code clarity (I guess I should refine the code more).
Some features which comes to mind are
* when the correct behavior is arguable, followes the behavior of CMUCL's
or GNU CLISP's implementation.
* passes all loop tests in Paul Dietz's ANSI-TEST suite.
* tested to run on CMUCL and GNU CLISP. Probably runs on almost all the
conforming implementation.
* destructuring on the result of multiple-value-list form is transformed
to a multiple-value-setq form, reducing the amout of consing.
e.g. (macroexpand '(loop for (a b c) = (multiple-value-list (foo))))
=> (BLOCK NIL
(LET ((A NIL) (B NIL) (C NIL))
(MACROLET ((LOOP-FINISH NIL '(GO #:LOOP-EPILOGUE-7002)))
(TAGBODY
#:LOOP-BODY-7001
(MULTIPLE-VALUE-SETQ (A B C) (FOO))
(GO #:LOOP-BODY-7001)
#:LOOP-EPILOGUE-7002))))
* when compared to cmucl's loop, attains almost the same execution speed
in cmucl. Some optimizations are omitted, in such cases the speed is
slightly slow. pays more attention to canstant argument cases. The code
size is about half long. (but cmucl's MIT code contains lots of
historical code.)
Any comments are welcome.
Yuji.
Yuji Minejima <········@nifty.ne.jp> writes:
> Some design goals of this loop implementation are
...
> Some features which comes to mind are
...
> Any comments are welcome.
I'd love to see the following design goal:
- A flexible and clearly documented extension mechanism,
enabling programmers to make LOOP loop in new ways
efficiently and without great pain.
As I understand it, the MIT implementation manages
at least "flexible". I don't know how it does in other
respects :-).
--
Gareth McCaughan
.sig under construc
Gareth McCaughan wrote:
> I'd love to see the following design goal:
>
> - A flexible and clearly documented extension mechanism,
> enabling programmers to make LOOP loop in new ways
> efficiently and without great pain.
>
> As I understand it, the MIT implementation manages
> at least "flexible". I don't know how it does in other
> respects :-).
I'd like to see:
- A syntax for iterating over arbitrary sequences.
Paul
Paul Dietz <············@motorola.com> writes:
> Gareth McCaughan wrote:
>
> > I'd love to see the following design goal:
> >
> > - A flexible and clearly documented extension mechanism,
> > enabling programmers to make LOOP loop in new ways
> > efficiently and without great pain.
> >
> > As I understand it, the MIT implementation manages
> > at least "flexible". I don't know how it does in other
> > respects :-).
>
> I'd like to see:
>
> - A syntax for iterating over arbitrary sequences.
I remark that this is straightforward to provide
if my proposed goal is met. On the other hand, it's
probably a lot easier than mine :-).
--
Gareth McCaughan
.sig under construc
On Tue, 28 Sep 2004 23:17:54 +0000, Gareth McCaughan wrote:
> I remark that this is straightforward to provide
> if my proposed goal is met. On the other hand, it's
> probably a lot easier than mine :-).
I agree that Paul's feature is a lot easier to implement than your
proposal.
From what I've heard and just skimming over its manual, iterate macro
has some good features (an extension mecanism, a collect clause can appear
deep inside the code, etc..) which is lacking in Loop. But since Loop is
already in the standard, I guess we're gonna have to live with it for some
time long. So I think adding a nice extension mechanism is a relevant
thing to do.
Any ideas are welcome.
Yuji.
From: Alexander Schmolck
Subject: Re: [ANN] Sacla loop: a new loop implementation
Date:
Message-ID: <yfspt44q7py.fsf@black4.ex.ac.uk>
Yuji Minejima <········@nifty.ne.jp> writes:
> From what I've heard and just skimming over its manual, iterate macro
> has some good features (an extension mecanism, a collect clause can appear
> deep inside the code, etc..) which is lacking in Loop. But since Loop is
> already in the standard, I guess we're gonna have to live with it for some
> time long.
Uhm why exactly? Apart from not coming "preinstalled", is there any area in
which ITERATE is *not* superior to loop (OK and type declarations I guess)?
I've only recently started using iterate a little bit and I'm surely no
authority on LOOP either (although I've written quite a few LOOPs) but from
this limited perspective, it seems to me that:
- you can write working, maintainable code with ITERATE (IMO not true for
LOOP: there are countless rather innocuous looking LOOPs that in fact yield
undefined behavior; while a LOOP wizard might spot them many lispers aren't
LOOP wizards -- if nothing else because its merrits are controversial, to
put it midly)
- the scope of conditional and progn constructs etc. is immediately clear in
ITERATE -- not only because the body indents correctly in emacs (and is
generally much easier to edit), but also because there are far fewer unlispy
and baroque syntax rules to remember (is WHEN the same as IF? Do I need and
END here? Where does this ELSE belong to? was there an implicit PROGN here?
etc. etc.).
- rather than LOOPs IMO horrible parallel iteration (AND), iterate has a much
more mnemonic and useable PREVIOUS construct.
- it' easy to learn (I spent much less time reading the iterate documentation
than that of loop and yet so far every time I got frustrated with a LOOP that
didn't work for some reason, rewritting it in iterate immediately solved the
problem; so someone who knows LOOP should have little difficulty in
switching)
- as far as I can tell it can do everything you can do with LOOP, plus plenty
more -- in particular it has an extension mechanism.
- finally there is a single implementation (I've often encountered completely
different behavior of the same LOOP under different implementations; I am
not sure whether that was because it's so easy to enter undefined territory
with LOOP or whether it was due to implementation bugs; presumably mostly
the former)
> So I think adding a nice extension mechanism is a relevant thing to do.
I think ditching loop and working on improving iterate might at least be worth
considering[1]. If I interpret Dan Weinreb's posting on ll1 some time ago
correctly then even some of the people involved in the design of LOOP seem to
think it's broken (and possibly beyond repair).
Here's a relevant quote:
"Rather, the problem with LOOP was that it turned out to be hard to
predict what it would do, when you started using a lot of different facets
of LOOP all together. This is a serious problem since the whole idea of
LOOP was to let you use many facets together; if you're not doing that,
LOOP is overkill. Moon struggled hard to make LOOP more predictable and
understandable. Even at the time I did not fully comprehend all the
issues; now, over 20 years later, I've almost forgotten them. But if Dave
Moon could not figure out how to solve the problem, I am quite confident
that it's a very hard problem to solve."
Maybe iterate suffers ultimately suffers from similar problems, but I haven't
encountered them so far.
'as
Footnotes:
[1] For example by tighter integration into specific implementions: Like Tim
Bradshaw I also think there ought to be some generic iteration syntax
that will work for any "iterable" but tries to be as efficient if
possible if the type can be deduced; this is I think not possible without
using implementation specific extensions.
On Thu, 30 Sep 2004 13:09:13 +0100, Alexander Schmolck wrote:
> Uhm why exactly? Apart from not coming "preinstalled", is there any area in
> which ITERATE is *not* superior to loop (OK and type declarations I guess)?
Mainly because LOOP is already in the standard (although I don't think the
word superior doesn't apply in this case). Although I haven't thoroughly
read the ITERATE manual, and never used it in actual code, as far as the
design is concerned, I prefer ITERATE to LOOP. Mainly because of its use
of parentheses to group a clause (hence a clause is just a macro form) and
definining clauses as macros, and various good results that are gained
from these (easy indentation, extension mechanism using a macro which
directly relfects the corresponding clause, clauses can appear deep inside
the nested code, etc).
> - you can write working, maintainable code with ITERATE (IMO not true for
> LOOP: there are countless rather innocuous looking LOOPs that in fact
> yield undefined behavior; while a LOOP wizard might spot them many
> lispers aren't LOOP wizards -- if nothing else because its merrits are
> controversial, to put it midly)
I admit there are many pitfalls in LOOP that I noticed implementing it. I
might write them down somewhere.
> - the scope of conditional and progn constructs etc. is immediately
> clear in
> ITERATE -- not only because the body indents correctly in emacs (and
> is generally much easier to edit), but also because there are far
> fewer unlispy and baroque syntax rules to remember (is WHEN the same
> as IF? Do I need and END here? Where does this ELSE belong to? was
> there an implicit PROGN here? etc. etc.).
I think if LOOP used parentheses around a clause, these problems wouldn't
occured.
> - it' easy to learn (I spent much less time reading the iterate documentation
> than that of loop and yet so far every time I got frustrated with a LOOP that
> didn't work for some reason, rewritting it in iterate immediately solved the
> problem; so someone who knows LOOP should have little difficulty in
> switching)
I agree that switching from LOOP to ITERATE seems rather easy.
> - finally there is a single implementation (I've often encountered completely
> different behavior of the same LOOP under different implementations; I am
> not sure whether that was because it's so easy to enter undefined territory
> with LOOP or whether it was due to implementation bugs; presumably mostly
> the former)
If ITERATE gains popularity and eventually replaces LOOP, I expect
different ITERATE implementations appear. I agree that LOOP has so many
unspecified fields.
>> So I think adding a nice extension mechanism is a relevant thing to do.
>
> I think ditching loop and working on improving iterate might at least be worth
> considering[1]. If I interpret Dan Weinreb's posting on ll1 some time ago
> correctly then even some of the people involved in the design of LOOP seem to
> think it's broken (and possibly beyond repair).
I don't deny the usefullness of developing ITERATE. But at the same time I
don't think LOOP would be abolished any time soom.
> Here's a relevant quote:
>
> "Rather, the problem with LOOP was that it turned out to be hard to
> predict what it would do, when you started using a lot of different facets
> of LOOP all together. This is a serious problem since the whole idea
> of LOOP was to let you use many facets together; if you're not doing
> that, LOOP is overkill. Moon struggled hard to make LOOP more
> predictable and understandable. Even at the time I did not fully
> comprehend all the issues; now, over 20 years later, I've almost
> forgotten them. But if Dave Moon could not figure out how to solve the
> problem, I am quite confident that it's a very hard problem to solve."
I think LOOP is just underspecified. I expect the same problem could
occure in ITERATE if sufficient details aren't nailed down. For example, I
think the following code might have a defferent side effect in different
LOOP implementations.
(loop for a from 0 to -1 initially (incf x))
Is the correct behavior for this case is specified in ITERATE?
If we specify more details, I think LOOP would become more predictable and
easier to understand.
One idea is to define the last shape of the expansion of clauses more
specifically, like specifying how the binding forms look like which
correspond to for-as-clauses (`with' caluses are explained in terms of do
do* in the spec, I think we should do this kind of thing), and the
relationship of code location between initial stepping code and initially
clauses and so on.
Regards,
Yuji.
Paul Dietz wrote:
>
> I'd like to see:
>
> - A syntax for iterating over arbitrary sequences.
>
I'd like to see a syntax for iterating over arbitrary
`iterable-things', which could be made convincingly efficient in the
good cases. Python has the nice syntax, although I'm unsure about the
efficiency, and Java now has stuff like
for (MyClass o: MyBlobOfMyClassInstances) {
... do things with o ...
}
although I'm not sure this can be made efficient either. Outside of
LOOP, I have stuff which lets you do things like
(for (x my-thing)
... do things with X ...)
And this works by asking MY-THING for an iterator, then calling that
until it's exhausted which is something like the Python approach, and
I think like Java's. But it is definitely not efficient - it always
conses an iterator. And it's not part of LOOP.
Of course you can do this in LOOP:
(loop with iter = (iter my-ob)
with item and alivep
do (setf (values item alivep) (next iter))
while alivep
do ... things with item ...)
But it's syntactically horrible. It's probably easy enough to extend
something like the FOR clause:
(loop for item iterating-over my-ob
do ... things with item ...)
But how to arrange life so that if my-ob is some known-good-type then
no iterator object needs to be made? Java can, I guess, often win by
knowing the type statically, but this is harder for Lisp...
--tim
"Tim Bradshaw" <··········@tfeb.org> wrote in message
·····························@k26g2000oda.googlegroups.com...
> Paul Dietz wrote:
> >
> > I'd like to see:
> >
> > - A syntax for iterating over arbitrary sequences.
> >
>
> I'd like to see a syntax for iterating over arbitrary
> `iterable-things', which could be made convincingly efficient in the
> good cases. Python has the nice syntax, although I'm unsure about the
> efficiency, and Java now has stuff like
>
> for (MyClass o: MyBlobOfMyClassInstances) {
> ... do things with o ...
> }
Does this work if MyBlobOfMyClassInstances happens to be an array? I think
this only works with Collections.
> although I'm not sure this can be made efficient either. Outside of
> LOOP, I have stuff which lets you do things like
>
> (for (x my-thing)
> ... do things with X ...)
>
> And this works by asking MY-THING for an iterator, then calling that
> until it's exhausted which is something like the Python approach, and
> I think like Java's. But it is definitely not efficient - it always
> conses an iterator. And it's not part of LOOP.
*snip*
> But how to arrange life so that if my-ob is some known-good-type then
> no iterator object needs to be made? Java can, I guess, often win by
> knowing the type statically, but this is harder for Lisp...
Yes, but Java doesn't I don't think.
Can't you compile up the body of the statement into a closure (or
FLET/LABELS) and then have a select few "iteration functions" that then call
that closure to do the work?
Then on loop entry, it does a TYPEP on the sequence, and then calls the
appropriate LIST-ITERATOR, ARRAY-ITERATOR, or GENERIC-ITERATOR wrapper
function passing the closure to it.
Actually, this doesn't help. The loop body is now a function call, whereas
before the "get-next" of the iterator is a function call, but you save
consing the actual iterator.
I think that's why LOOP is the way it is, you simply have to "tell" it the
type of sequence so it can craft the right code.
Regards,
Will Hartung
(·····@msoft.com)
Will Hartung wrote:
> Does this work if MyBlobOfMyClassInstances happens to be an array? I
think
> this only works with Collections.
Yes, it works with arrays:
int[] a = new int[10000];
for (int i: a) { ...}
is fine.
--tim
"Tim Bradshaw" <··········@tfeb.org> writes:
> But it's syntactically horrible. It's probably easy enough to extend
> something like the FOR clause:
>
> (loop for item iterating-over my-ob
> do ... things with item ...)
There is already a syntax for this in the standard LOOP.
For example
(loop for item being the hash-keys of my-hash-table
do ... )
> But how to arrange life so that if my-ob is some known-good-type then
> no iterator object needs to be made? Java can, I guess, often win by
> knowing the type statically, but this is harder for Lisp...
This test would be contained in the prologue code of a user defined
LOOP path.
Carl Shapiro wrote:
>
> There is already a syntax for this in the standard LOOP.
>
> For example
>
> (loop for item being the hash-keys of my-hash-table
> do ... )
>
only if `arbitrary iterable things' means `hashtables'. I think there
might be some other kinds of iterable things... I'd be happy with
something like:
(loop for item being the contents of my-thing
do ...)
--tim
"Tim Bradshaw" <··········@tfeb.org> writes:
> But it's syntactically horrible. It's probably easy enough to extend
> something like the FOR clause:
>
> (loop for item iterating-over my-ob
> do ... things with item ...)
>
> But how to arrange life so that if my-ob is some known-good-type
> then no iterator object needs to be made? Java can, I guess, often
> win by knowing the type statically, but this is harder for Lisp...
So doesn't it seem like The Lisp Way to use type information if its
available and fall back on a more general solution if necessary. Then
type declarations can optimize things but you don't otherwise need to
change the code. So why can't we have:
(loop for item iterating-over (the list my-ob) do ...)
be just another way of writing:
(loop for item in my-ob do ...)
The next step, of course, would be to give LOOP away to do the right
thing with this:
(defun foo (my-ob)
(declare (list my-ob))
...
(loop for item iterating-over my-ob do ...))
This, obviously, relies on making type information known to the
compiler available to LOOP. Since LOOP is part of the implementation
that is obviously doable. Better yet would be for the long-awaited
"first-class environments" to facilitate this kind of macrology.
(Duane, will your environments carry this kind of type information?)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> "Tim Bradshaw" <··········@tfeb.org> writes:
>
> > But it's syntactically horrible. It's probably easy enough to extend
> > something like the FOR clause:
> >
> > (loop for item iterating-over my-ob
> > do ... things with item ...)
> >
> > But how to arrange life so that if my-ob is some known-good-type
> > then no iterator object needs to be made? Java can, I guess, often
> > win by knowing the type statically, but this is harder for Lisp...
>
> So doesn't it seem like The Lisp Way to use type information if its
> available and fall back on a more general solution if necessary. Then
> type declarations can optimize things but you don't otherwise need to
> change the code.
I have an iteration scheme for a domain area which does just this. It
knows about the basic types of set objects (sequences and otherwise)
and does type inference to pick the actual implementation at compile
time or if it can't figure this out punts to a generic solution.
So why can't we have:
>
> (loop for item iterating-over (the list my-ob) do ...)
>
> be just another way of writing:
>
> (loop for item in my-ob do ...)
>
> The next step, of course, would be to give LOOP away to do the right
> thing with this:
>
> (defun foo (my-ob)
> (declare (list my-ob))
> ...
> (loop for item iterating-over my-ob do ...))
I don't think you even need the second form. You can simply have the
(loop for item in my-ob do ...)
do the right thing in either case.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87k6ud9g48.fsf@evaluator.pt>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Paul Dietz wrote:
>>
>> I'd like to see:
>>
>> - A syntax for iterating over arbitrary sequences.
>>
>
> I'd like to see a syntax for iterating over arbitrary
> `iterable-things', which could be made convincingly efficient in the
> good cases. Python has the nice syntax, although I'm unsure about the
> efficiency, and Java now has stuff like
>
> for (MyClass o: MyBlobOfMyClassInstances) {
> ... do things with o ...
> }
>
> although I'm not sure this can be made efficient either. Outside of
> LOOP, I have stuff which lets you do things like
>
> (for (x my-thing)
> ... do things with X ...)
>
> And this works by asking MY-THING for an iterator, then calling that
> until it's exhausted which is something like the Python approach, and
> I think like Java's. But it is definitely not efficient - it always
> conses an iterator. And it's not part of LOOP.
>
> Of course you can do this in LOOP:
>
> (loop with iter = (iter my-ob)
> with item and alivep
> do (setf (values item alivep) (next iter))
> while alivep
> do ... things with item ...)
>
> But it's syntactically horrible. It's probably easy enough to extend
> something like the FOR clause:
>
> (loop for item iterating-over my-ob
> do ... things with item ...)
>
> But how to arrange life so that if my-ob is some known-good-type then
> no iterator object needs to be made? Java can, I guess, often win by
> knowing the type statically, but this is harder for Lisp...
I'm afraid it's not only harder: it's impossible. You don't have type
information available at macro expansion time to allow different
expansions according to the type of the (iterated) object. This means
that either you cons an iterator or you cons a closure.
Just for comparison, Linj (which can be seen as a kind of statically
typed Common Lisp) allows you to do what you want by providing
'macros' with type information. Here is one example that does more or
less what you want:
(def-linj-macro statement (for-each (?var ?form/expression) . ?body/statement-list)
(let ((form-type (get-type ?form)))
(cond ((cons-type-p form-type)
`(dolist (,?var ,?form) . ,?body)) ;;this is fast
((array-type-reference-p form-type)
`(dovector (,?var ,?form) . ,?body)) ;;this is fast too
((super-type-p (iterator-type) form-type)
`(let ((iter ,?form))
(while (has-next iter)
(let ((,?var (next iter))) . ,?body))))
((super-type-p (enumeration-type) form-type)
`(let ((iter ,?form))
(while (has-more-elements iter)
(let ((,?var (next-element iter))) . ,?body))))
(t
(error "Unknown type for iteration ~A" form-type)))))
Note that the macro expansion depends on the type of the ?form
argument. With this macro you can write:
(defun main ()
(let ((x '(1 2 3 4 5)) ;;a list
(y #(5 4 3 2 1)) ;;a vector
(z (new 'string-tokenizer "1 2 3 4 5"))) ;;an enumeration
(for-each (e1 x)
(print e1))
(for-each (e1 y)
(print e1))
(for-each (e1 z)
(print e1))))
and the Linj compiler will translate this to the following 'assembly'
code (note that although the code isn't explicitely typed, the
compiler can derive the necessary type information):
import java.util.StringTokenizer;
import linj.Bignum;
import linj.Cons;
public class A extends Object {
public static void main(String[] outsideArgs) {
Cons x =
Cons.list(Bignum.valueOf(1), Bignum.valueOf(2), Bignum.valueOf(3), Bignum.valueOf(4), Bignum.valueOf(5));
int[] y = new int[] { 5, 4, 3, 2, 1 };
StringTokenizer z = new StringTokenizer("1 2 3 4 5");
for (Cons list = x; ! list.endp(); list = list.rest()) {
Object e1 = list.first();
System.out.print("" + '\n' + e1 + " ");
}
{
int limit = y.length;
for (int i = 0; i < limit; ++i) {
int e1 = y[i];
System.out.print("" + '\n' + e1 + " ");
}
}
StringTokenizer iter = z;
while (iter.hasMoreElements()) {
Object e1 = iter.nextElement();
System.out.print("" + '\n' + e1 + " ");
}
}
}
Ant�nio Leit�o.
From: Christophe Rhodes
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <sq3c11c7j1.fsf@cam.ac.uk>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> "Tim Bradshaw" <··········@tfeb.org> writes:
>
>> But how to arrange life so that if my-ob is some known-good-type then
>> no iterator object needs to be made? Java can, I guess, often win by
>> knowing the type statically, but this is harder for Lisp...
>
> I'm afraid it's not only harder: it's impossible. You don't have type
> information available at macro expansion time to allow different
> expansions according to the type of the (iterated) object. This means
> that either you cons an iterator or you cons a closure.
This is approximately false approximately three times.
Once, because in fact type information is available at macroexpansion
time, through the &environment parameter to the macroexpander. If a
type declaration is available in the environment, there will be
(implementation-specific) functions to extract it.
Twice, because even if there weren't, it would be poor indeed if a
compiler macro expander did not have declared type information
available to it, and, again, implementation-specific functions to
access such type information. So expanding into a form which has a
compiler macro available to transform, given this type information, to
more efficient implementations would likewise avoid consing iterators
or closures.
Thirdly, because even if neither of these applies one can fall back on
the lower-level substrate provided by your implementation: in SBCL the
DEFTRANSFORM macros. This has the advantage that it applies after
macroexpansion, which means that type propagation has had a chance to
run, so we are not even dependent on declared types any more, but on
derived types.
So if you were saying that in completely portable Common Lisp, it is
impossible, then you're trivially right, because an implementation is
free to discard any and all type information declared and derived. On
the non-trivial point, I don't believe you are correct.
Christophe
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87r7ok96zx.fsf@evaluator.pt>
Christophe Rhodes <·····@cam.ac.uk> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> "Tim Bradshaw" <··········@tfeb.org> writes:
>>
>>> But how to arrange life so that if my-ob is some known-good-type then
>>> no iterator object needs to be made? Java can, I guess, often win by
>>> knowing the type statically, but this is harder for Lisp...
>>
>> I'm afraid it's not only harder: it's impossible. You don't have type
>> information available at macro expansion time to allow different
>> expansions according to the type of the (iterated) object. This means
>> that either you cons an iterator or you cons a closure.
>
> This is approximately false approximately three times.
I'm afraid you missed my point.
> Once, because in fact type information is available at macroexpansion
> time, through the &environment parameter to the macroexpander. If a
> type declaration is available in the environment, there will be
> (implementation-specific) functions to extract it.
Even considering non-standard functions to obtain the type
declarations, I would like to know how will you get them when the
programmer does not provide them. If you looked carefully at my
example you'll notice that there were none.
> Twice, because even if there weren't, it would be poor indeed if a
> compiler macro expander did not have declared type information
> available to it, and, again, implementation-specific functions to
> access such type information. So expanding into a form which has a
> compiler macro available to transform, given this type information, to
> more efficient implementations would likewise avoid consing iterators
> or closures.
Again, where are the declared type information when the programmer did
not provide it?
> Thirdly, because even if neither of these applies one can fall back on
> the lower-level substrate provided by your implementation: in SBCL the
> DEFTRANSFORM macros. This has the advantage that it applies after
> macroexpansion, which means that type propagation has had a chance to
> run, so we are not even dependent on declared types any more, but on
> derived types.
That's much more powerfull but it still doesn't solve the problem.
Let's consider my previous for-each macro and let's also consider the
following code fragment:
(defun foo (arg)
(for-each (e1 arg)
(print e1)))
Can you tell me what is the secret machinery that you have in whatever
Common Lisp compiler you want that is able to expand the for-each
macro without knowing the type of arg? I hope you are not requesting
that the compiler must know that type in order to expand the macro.
Because if you are, then you are not talking about Common Lisp but
instead about a statically typed Common Lisp.
> So if you were saying that in completely portable Common Lisp, it is
> impossible, then you're trivially right, because an implementation is
> free to discard any and all type information declared and derived. On
> the non-trivial point, I don't believe you are correct.
I still believe that if you have macros whose expansion depends on the
actual type of the argument expressions, then you need a statically
typed language. But I would be glad if you could show me otherwise.
Thanks for your comments,
Ant�nio Leit�o.
From: Christophe Rhodes
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <sqhdpgbzdf.fsf@cam.ac.uk>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Christophe Rhodes <·····@cam.ac.uk> writes:
>
>> Thirdly, because even if neither of these applies one can fall back on
>> the lower-level substrate provided by your implementation: in SBCL the
>> DEFTRANSFORM macros. This has the advantage that it applies after
>> macroexpansion, which means that type propagation has had a chance to
>> run, so we are not even dependent on declared types any more, but on
>> derived types.
>
> That's much more powerfull but it still doesn't solve the problem.
> Let's consider my previous for-each macro and let's also consider the
> following code fragment:
>
> (defun foo (arg)
> (for-each (e1 arg)
> (print e1)))
>
> Can you tell me what is the secret machinery that you have in whatever
> Common Lisp compiler you want that is able to expand the for-each
> macro without knowing the type of arg? I hope you are not requesting
> that the compiler must know that type in order to expand the macro.
> Because if you are, then you are not talking about Common Lisp but
> instead about a statically typed Common Lisp.
Obviously when no type information is either provided or deriveable,
one would have to fall back to runtime type-investigation mechanisms.
However, let's expand your code fragment a little.
(declaim (inline foo)) ; not strictly necessary, but it's easier.
(defun foo (arg)
(for-each (e1 arg)
(print e1)))
(defun bar ()
(let ((x (mapcar #'random '(1 2 3 4 5 6)))
(y (make-array 10 :initial-element 4)))
(> (max (foo x) (foo y)))))
In this case, while full calls to FOO will of course have to go
through a runtime type check, the calls to FOO from within BAR need
not, and indeed will not in any implementation with even a rudimentary
knowledge of types. And lest you quibble over the fact that I've
asked for FOO to be inline, note that this is absolutely not necessary
if the functions FOO and BAR as defined above reside in the same file
and are compiled by COMPILE-FILE: in that case, again, the code can
conformingly be rewritten by the compiler as
(defun foo (arg)
(for-each (e1 arg)
(print e1)))
(flet ((%foo (arg) (for-each (e1 arg) (print e1))))
(defun bar ()
(let ((x (mapcar #'random '(1 2 3 4 5 6)))
(y (make-array 10 :initial-element 4)))
(> (max (%foo x) (%foo y))))))
and have the type information propagated again. This isn't
theoretical, either: see CMUCL's documentation on block compilation.
None of this is secret machinery.
Christophe
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87lles92b7.fsf@evaluator.pt>
Christophe Rhodes <·····@cam.ac.uk> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> Christophe Rhodes <·····@cam.ac.uk> writes:
>>
>>> Thirdly, because even if neither of these applies one can fall back on
>>> the lower-level substrate provided by your implementation: in SBCL the
>>> DEFTRANSFORM macros. This has the advantage that it applies after
>>> macroexpansion, which means that type propagation has had a chance to
>>> run, so we are not even dependent on declared types any more, but on
>>> derived types.
>>
>> That's much more powerfull but it still doesn't solve the problem.
>> Let's consider my previous for-each macro and let's also consider the
>> following code fragment:
>>
>> (defun foo (arg)
>> (for-each (e1 arg)
>> (print e1)))
>>
>> Can you tell me what is the secret machinery that you have in whatever
>> Common Lisp compiler you want that is able to expand the for-each
>> macro without knowing the type of arg? I hope you are not requesting
>> that the compiler must know that type in order to expand the macro.
>> Because if you are, then you are not talking about Common Lisp but
>> instead about a statically typed Common Lisp.
>
> Obviously when no type information is either provided or deriveable,
> one would have to fall back to runtime type-investigation mechanisms.
Given the fact that macros don't have access to the runtime
information I wonder how do you plan to make a macro that computes its
expansion using that information. Obviouly, if you are thinking about
including typep forms in the macro expansion then you are solving a
different problem.
>
> However, let's expand your code fragment a little.
>
> (declaim (inline foo)) ; not strictly necessary, but it's easier.
> (defun foo (arg)
> (for-each (e1 arg)
> (print e1)))
>
> (defun bar ()
> (let ((x (mapcar #'random '(1 2 3 4 5 6)))
> (y (make-array 10 :initial-element 4)))
> (> (max (foo x) (foo y)))))
>
> In this case, while full calls to FOO will of course have to go
> through a runtime type check, the calls to FOO from within BAR need
> not, and indeed will not in any implementation with even a rudimentary
> knowledge of types. And lest you quibble over the fact that I've
> asked for FOO to be inline, note that this is absolutely not necessary
> if the functions FOO and BAR as defined above reside in the same file
> and are compiled by COMPILE-FILE: in that case, again, the code can
> conformingly be rewritten by the compiler as
>
> (defun foo (arg)
> (for-each (e1 arg)
> (print e1)))
> (flet ((%foo (arg) (for-each (e1 arg) (print e1))))
> (defun bar ()
> (let ((x (mapcar #'random '(1 2 3 4 5 6)))
> (y (make-array 10 :initial-element 4)))
> (> (max (%foo x) (%foo y))))))
>
> and have the type information propagated again. This isn't
> theoretical, either: see CMUCL's documentation on block compilation.
I know about block compilation. However, as you say, it doesn't solve
the problem with foo. You still need to show me how is it possible for
the for-each macro to compute an expansion according to type
information that a non-statically typed Common Lisp can't have.
Let's think about another "trivial" example:
(defun bar ()
(for-each (e1 (foobar))
(print e1)))
How do you expand the macro if you don't have the definition of foobar
available? And what happens if (as is possible in Common Lisp),
foobar returns objects of many different types?
I obviously don't know enough about CMUCL and/or SBCL internals as you
do but I'm also surprised that, in your first example, you can inline
a function before expanding all its macros. And if you have to expand
the macros, I wonder how can the for-each macro call access the type
information _before_ you inline it. In you second example, I don't
understand how is it possible to compile %foo (which you need to
compile in order for type information to be propagated) if the macro
needs information that isn't available yet. I was under the
impression that most optimizations could only be done on an
intermediate form where macros are already expanded but I might be
wrong about this. Is it possible for CMUCL or SBCL to optimize forms
that have unexpanded macros?
Anyway, it seems to me that you are adding requirements to the problem
in order to allow precise static inference. As you well know, you
can't extend that indefinitely: sooner or later you will reach a point
where you just can't find any more static type information. The
for-each macro might depend on that type information that you just
can't get (in Common Lisp), no matter how clever the type inferencer
is.
So, I think my claim is not only "trivially" true.
> None of this is secret machinery.
But none of this works in the general case.
Ant�nio Leit�o.
From: Christophe Rhodes
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <sqvfdwadve.fsf@cam.ac.uk>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Christophe Rhodes <·····@cam.ac.uk> writes:
>
>> Obviously when no type information is either provided or deriveable,
>> one would have to fall back to runtime type-investigation mechanisms.
>
> Given the fact that macros don't have access to the runtime
> information I wonder how do you plan to make a macro that computes its
> expansion using that information. Obviouly, if you are thinking about
> including typep forms in the macro expansion then you are solving a
> different problem.
The macro expands into generic code, which can be specialized. Let's
be concrete, and say (ignoring hygiene issues)
(defmacro for-each ((var obj) &body body)
`(do ((state (initial-state ,obj) (next-state ,obj state)))
((end ,obj state))
(let ((,var (next-var ,obj state)))
,@body)))
;; these can be generic functions if you're worried about the lack
;; of extensibility.
(defun initial-state (obj)
(typecase obj
(list obj)
(vector 0)))
(defun next-state (obj state)
(typecase obj
(list (cdr state))
(vector (1+ state))))
(defun next-var (obj state)
(typecase obj
(list (car state))
(vector (aref obj state))))
(defun end (obj state)
(typecase obj
(list (null state))
(vector (= state (length obj)))))
Does this meet your criterion for a macroexpansion which does not
depend on type information? I hope so. I hope also that it meets
your specification, enough at least for illustrative purposes.
If so, then the presence of either inline definitions for
INITIAL-STATE, NEXT-STATE, NEXT-VAR, END, or transforms for the same,
will allow the compiler to optimize away all runtime type queries.
Are we arguing at cross-purposes here, or what?
>> None of this is secret machinery.
>
> But none of this works in the general case.
I never argued about the general case! Tim Bradshaw wanted a
non-consing iterator when the object being iterated over was some
known-good type. You asserted that this was impossible. I'm
asserting that it's possible when there's enough information around at
compile-time to know that the object is of a good type (that's my
working understanding of "known-good").
Christophe
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87d6048wx4.fsf@evaluator.pt>
Christophe Rhodes <·····@cam.ac.uk> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> Christophe Rhodes <·····@cam.ac.uk> writes:
>>
>>> Obviously when no type information is either provided or deriveable,
>>> one would have to fall back to runtime type-investigation mechanisms.
>>
>> Given the fact that macros don't have access to the runtime
>> information I wonder how do you plan to make a macro that computes its
>> expansion using that information. Obviouly, if you are thinking about
>> including typep forms in the macro expansion then you are solving a
>> different problem.
>
> The macro expands into generic code, which can be specialized. Let's
> be concrete, and say (ignoring hygiene issues)
>
> (defmacro for-each ((var obj) &body body)
> `(do ((state (initial-state ,obj) (next-state ,obj state)))
> ((end ,obj state))
> (let ((,var (next-var ,obj state)))
> ,@body)))
>
> ;; these can be generic functions if you're worried about the lack
> ;; of extensibility.
> (defun initial-state (obj)
> (typecase obj
> (list obj)
> (vector 0)))
> (defun next-state (obj state)
> (typecase obj
> (list (cdr state))
> (vector (1+ state))))
> (defun next-var (obj state)
> (typecase obj
> (list (car state))
> (vector (aref obj state))))
> (defun end (obj state)
> (typecase obj
> (list (null state))
> (vector (= state (length obj)))))
>
> Does this meet your criterion for a macroexpansion which does not
> depend on type information? I hope so. I hope also that it meets
> your specification, enough at least for illustrative purposes.
Except that you are again using the iterator idea that Tim Bradshaw
was trying to avoid and that is used in Java--the only difference is
that, for the specific datatypes you used in your example, you know
that you don't need to generate any garbage. But let's consider
iterating over something different such as a two-dimensional structure
that needs one loop inside another. As soon as you split your
iteration protocol into the functions 'next-state' and 'next-variable'
how can you use your macro without consing an iterator that keeps the
state of the iteration? I was under the impression that that's
precisely what Tim Bradshaw was trying to avoid.
The real problem, as I understand it, is to build a macro that expands
into efficient iteration code for some given set of datatypes (not
necessarily lists or vectors) without using neither iterators nor
closures (as they both will cons). Maybe Tim Bradshaw can jump in and
clearly states what he wants.
> Are we arguing at cross-purposes here, or what?
Maybe. But I'm not really sure you are solving the same problem that
Tim Bradshaw presented.
>>> None of this is secret machinery.
>>
>> But none of this works in the general case.
>
> I never argued about the general case! Tim Bradshaw wanted a
> non-consing iterator when the object being iterated over was some
> known-good type. You asserted that this was impossible. I'm
> asserting that it's possible when there's enough information around
> at compile-time to know that the object is of a good type (that's my
> working understanding of "known-good").
Ok. Maybe I didn't understand what he meant by known-good. Let me be
completely clear:
What I'm trying to convey here is that you can write a particular
pattern of code that iterates a well-known object in some place
without any obvious consing. However, when you try to generalize
several of those patterns into a macro you either have to use
iterators to keep the state of the iteration or you have to use a
generic function that accepts a closure and that is specialized for
all the types you want to iterate.
My first claim is that to avoid both of these (iterator and closure),
you need to have type information available at compile time to allow
your macro to expand into the appropriate iteration pattern.
My second claim is that it is impossible to do this in Common Lisp in
the general case.
I must confess that your first reply (when you first said I was wrong)
seemed to me to be in tune with these thoughs:
> > I'm afraid it's not only harder: it's impossible. You don't have type
> > information available at macro expansion time to allow different
> > expansions according to the type of the (iterated) object. This means
> > that either you cons an iterator or you cons a closure.
>
> This is approximately false approximately three times.
Either I didn't understand what you were saying or you were saying
that my second claim was wrong.
Am I correct about this interpretation?
Ant�nio Leit�o.
Christophe Rhodes wrote:
>
> I never argued about the general case!
Exactly: arguing about the general case is a complete waste of time,
because it's just obvious that you have to do all the work then.
> Tim Bradshaw wanted a
> non-consing iterator when the object being iterated over was some
> known-good type. You asserted that this was impossible. I'm
> asserting that it's possible when there's enough information around
at
> compile-time to know that the object is of a good type (that's my
> working understanding of "known-good").
>
Exactly! Any my hope is that for a fairly serious type-inferencing
compiler (like CMUCL/SBCL) the compiler can do enough work that a fair
amount is known about types, and a macro can (non-portably, so far)
make enquiries about this information and use this to optimise many
cases.
I am fairly sure that CMUCL (and I assume SBCL) can do clever stuff
with `raising' (this is not the right term I'm sure) runtime type
checks so that it knows a good deal more about types at compile time
than you'd think (well, not more then you:Cristophe would think, but
more than you:random-programmer would think...). So for things like
this:
(defun foo (x i)
(setf (aref x i) nil)
...code...)
then the compiler can emit suitable checks for X and I, and then let
...code... know that, in fact they are now of good types, because it
will never run if they are not.
--tim
Tim Bradshaw wrote:
> I am fairly sure that CMUCL (and I assume SBCL) can do clever stuff
> with `raising' (this is not the right term I'm sure) runtime type
> checks so that it knows a good deal more about types at compile time
> than you'd think (well, not more then you:Cristophe would think, but
> more than you:random-programmer would think...). So for things like
> this:
>
> (defun foo (x i)
> (setf (aref x i) nil)
> ...code...)
>
> then the compiler can emit suitable checks for X and I, and then let
> ...code... know that, in fact they are now of good types, because it
> will never run if they are not.
Are you sure about this? What if, say, you had a function like this:
(defun foo (x i)
(if (listp x)
(nth i x)
(aref x i)))
Does the compiler figure out that X isn't a vector or a list, but a
sequence? I'm sure you can see that this type of thing can get very subtle
very quickly, and I'm inclined to think that the compiler, if it does this
type of inference at all, can do it in only some of the simpler cases.
Course, I don't really know much about compilers...
Chris Capel
Chris Capel wrote:
>
> Are you sure about this? What if, say, you had a function like this:
>
Reasonably. Try compiling something like this with CMUCL:
(defun foo (x)
(cons (car x)
(aref x 0)))
You'll get something like:
; Warning: The binding of ARRAY is a LIST, not a (VALUES VECTOR).
;
; Note: Deleting unreachable code.
or, in other words, the compiler has worked out that X can't be a list
*and* a vector.
> (defun foo (x i)
> (if (listp x)
> (nth i x)
> (aref x i)))
>
> Does the compiler figure out that X isn't a vector or a list, but a
> sequence?
I don't know what it can do in this case. I'd expect it might well be
able to work out that in one branch it's a list and the other it's
some kind of vector. I'm sure a CMUCL/SBCL maintainer can make a
better comment than I.
> I'm sure you can see that this type of thing can get very subtle
> very quickly, and I'm inclined to think that the compiler, if it does
this
> type of inference at all, can do it in only some of the simpler
cases.
Yes, it can get subtle, and of course there are limits to what can be
done. I'm just glad that CMUCL's authors didn't do the stupid CS
`just give up because there are cases we know can't be done' thing.
--tim
From: Christophe Rhodes
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <sqr7ojqvhn.fsf@cam.ac.uk>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Chris Capel wrote:
>> (defun foo (x i)
>> (if (listp x)
>> (nth i x)
>> (aref x i)))
>>
>> Does the compiler figure out that X isn't a vector or a list, but a
>> sequence?
>
> I don't know what it can do in this case. I'd expect it might well be
> able to work out that in one branch it's a list and the other it's
> some kind of vector. I'm sure a CMUCL/SBCL maintainer can make a
> better comment than I.
It doesn't rejoin the branches of the conditional except in certain
cases (when the branches rejoin into some iteration-like variable, to
allow type inference over e.g. index variables with varying stride).
On the other hand, the compiler can certainly propagate type
information within the branches: if subsequent operations on x happen
in the branches of the above conditional, the following operations can
use the fact that certain functions are defined to signal errors in
certain type mismatch conditions.
I should say that I'm a long way from being an expert on this area of
the "Python" compiler: ask Alexey Dejneka or Douglas Crosher for the
inside dope.
Christophe
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87brfnquqs.fsf@evaluator.pt>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Chris Capel wrote:
>>
>> Are you sure about this? What if, say, you had a function like this:
>>
>
> Reasonably. Try compiling something like this with CMUCL:
>
> (defun foo (x)
> (cons (car x)
> (aref x 0)))
>
> You'll get something like:
>
> ; Warning: The binding of ARRAY is a LIST, not a (VALUES VECTOR).
> ;
> ; Note: Deleting unreachable code.
>
> or, in other words, the compiler has worked out that X can't be a list
> *and* a vector.
>
>> (defun foo (x i)
>> (if (listp x)
>> (nth i x)
>> (aref x i)))
>>
>> Does the compiler figure out that X isn't a vector or a list, but a
>> sequence?
>
> I don't know what it can do in this case. I'd expect it might well be
> able to work out that in one branch it's a list and the other it's
> some kind of vector. I'm sure a CMUCL/SBCL maintainer can make a
> better comment than I.
>
>> I'm sure you can see that this type of thing can get very subtle
>> very quickly, and I'm inclined to think that the compiler, if it does
> this
>> type of inference at all, can do it in only some of the simpler
> cases.
>
> Yes, it can get subtle, and of course there are limits to what can be
> done. I'm just glad that CMUCL's authors didn't do the stupid CS
> `just give up because there are cases we know can't be done' thing.
You are mixing apples and oranges. You started by asking for clever
macros that explore type information but now you are talking about
clever type inference.
If you look carefully at my previous posts you will see that I never
claimed that type inference could not be applied. It can, _after_
macro expansion. What I said was that you can't write a macro that
explores type information to influence the expansion simply because
not all macro calls will have the necessary type information
available.
Ant�nio Leit�o.
Antonio Menezes Leitao wrote:
>
> Even considering non-standard functions to obtain the type
> declarations, I would like to know how will you get them when the
> programmer does not provide them. If you looked carefully at my
> example you'll notice that there were none.
>
Because the compiler has done a whole bunch of type inference and
worked some stuff out for you. For instance:
(defun foo (x &optional (i 0) (v nil))
(setf (aref foo i) v)
;; now we know that FOO is a one-dimensional array, I is a FIXNUM,
;; but nothing about V
(loop for x being the contents of foo
;; so the above should be optimisable to something pretty quick
do ...))
--tim
Christophe Rhodes <·····@cam.ac.uk> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
> > "Tim Bradshaw" <··········@tfeb.org> writes:
> >
> >> But how to arrange life so that if my-ob is some known-good-type then
> >> no iterator object needs to be made? Java can, I guess, often win by
> >> knowing the type statically, but this is harder for Lisp...
> >
> > I'm afraid it's not only harder: it's impossible. You don't have type
> > information available at macro expansion time to allow different
> > expansions according to the type of the (iterated) object. This means
> > that either you cons an iterator or you cons a closure.
>
> This is approximately false approximately three times.
>
> Once, because in fact type information is available at macroexpansion
> time, through the &environment parameter to the macroexpander. If a
> type declaration is available in the environment, there will be
> (implementation-specific) functions to extract it.
>
> Twice, because even if there weren't, it would be poor indeed if a
> compiler macro expander did not have declared type information
> available to it, and, again, implementation-specific functions to
> access such type information. So expanding into a form which has a
> compiler macro available to transform, given this type information, to
> more efficient implementations would likewise avoid consing iterators
> or closures.
>
> Thirdly, because even if neither of these applies one can fall back on
> the lower-level substrate provided by your implementation: in SBCL the
> DEFTRANSFORM macros. This has the advantage that it applies after
> macroexpansion, which means that type propagation has had a chance to
> run, so we are not even dependent on declared types any more, but on
> derived types.
I agree - this is exactly what I've done in a version of this stuff
for a domain area (actually, I suppose there is no reason why it
couldn't be made to work more generally). Well, the Twice and Thirdly
anyway - Once probably needs compiler support.
> So if you were saying that in completely portable Common Lisp, it is
> impossible, then you're trivially right, because an implementation is
> free to discard any and all type information declared and derived. On
> the non-trivial point, I don't believe you are correct.
If you go with Twice and Thirdly, why would this not be portable? I
don't recall stepping over this line in the case I've done.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
Christophe Rhodes <·····@cam.ac.uk> writes:
> jayessay <······@foo.com> writes:
>
> >> So if you were saying that in completely portable Common Lisp, it is
> >> impossible, then you're trivially right, because an implementation is
> >> free to discard any and all type information declared and derived. On
> >> the non-trivial point, I don't believe you are correct.
> >
> > If you go with Twice and Thirdly, why would this not be portable? I
> > don't recall stepping over this line in the case I've done.
>
> Are you labelling the in the same order that I'm labelling them? In
> any case, there's no portable way to access any type declaration
> information from the environment, so anything which wants to extract
> it is doomed to non-portability.
The way I read it, it looked like this only made sense in the context
of One. Two and Three could be done without accessing type
information from the "implementations compilation environment", but
rather from one built up for use by the expander. Yes, this requires
code walkers and blah, blah, blah. Having the information available
from the implementation (via environments, say) would make all this
simpler and easier.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
Ant�nio Leit�o wrote:
> > But how to arrange life so that if my-ob is some known-good-type then
> > no iterator object needs to be made? Java can, I guess, often win by
> > knowing the type statically, but this is harder for Lisp...
>
> I'm afraid it's not only harder: it's impossible. You don't have type
> information available at macro expansion time to allow different
> expansions according to the type of the (iterated) object.
Impossible is a little strong. The compiler is allowed to
discard type information, but READ isn't, so macros can spot
the THE special operator.
(defmacro doseq ((variable-name thing-form) &body code)
(cond ((= (mismatch '(the list) thing-form) 2)
`(dolist (,variable-name ,thing-form) ,@code))
((= (mismatch '(the vector) thing-form) 2)
`(loop for ,variable-name across ,thing-form do ,@code))
(t (let ((peek-holder-name (make-symbol "PEEK-HOLDER")))
`(let ((,peek-holder-name ,thing-form))
(typecase ,peek-holder-name
(list (dolist (,variable-name ,peek-holder-name) ,@code))
(vector (loop for ,variable-name across ,peek-holder-name
do ,@code))))))))
(macroexpand-1 '(doseq (x (the list '(a b c)))(print x)))
=> (DOLIST (X (THE LIST '(A B C))) (PRINT X))
(doseq (x (the vector '(1 2 3))) (print x))
=> Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER:
(1 2 3) is not of type ARRAY
* (defparameter l (list 'a 'b 'c))
* (defparameter v (vector 1 2 3))
* (doseq (x (the vector v)) (princ x))
123
NIL
* (doseq (x (the list l)) (princ x))
ABC
NIL
* (doseq (x (case (random 2)(0 l)(1 v))) (princ x))
123
NIL
* (doseq (x (case (random 2)(0 l)(1 v))) (princ x))
ABC
NIL
(macroexpand '(dothing (x (case (random 2)(0 l)(1 v))) (princ x)))
=>
(LET ((#:PEEK-HOLDER (CASE (RANDOM 2) (0 L) (1 V))))
(TYPECASE #:PEEK-HOLDER
(LIST (DOLIST (X #:PEEK-HOLDER) (PRINC X)))
(VECTOR (LOOP FOR X ACROSS #:PEEK-HOLDER DO (PRINC X)))))
One could imagine a much fancier version, do-object, that
worked with a macro, define-object-iterator, to let you say
how to iterate over objects that were not going to have
their classes redefined.
Alan Crowe
Edinburgh
Scotland
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87hdpg90ip.fsf@evaluator.pt>
Alan Crowe <····@cawtech.freeserve.co.uk> writes:
> Ant�nio Leit�o wrote:
>
>> > But how to arrange life so that if my-ob is some known-good-type then
>> > no iterator object needs to be made? Java can, I guess, often win by
>> > knowing the type statically, but this is harder for Lisp...
>>
>> I'm afraid it's not only harder: it's impossible. You don't have type
>> information available at macro expansion time to allow different
>> expansions according to the type of the (iterated) object.
>
> Impossible is a little strong. The compiler is allowed to
> discard type information, but READ isn't, so macros can spot
> the THE special operator.
>
> (defmacro doseq ((variable-name thing-form) &body code)
> (cond ((= (mismatch '(the list) thing-form) 2)
> `(dolist (,variable-name ,thing-form) ,@code))
> ((= (mismatch '(the vector) thing-form) 2)
> `(loop for ,variable-name across ,thing-form do ,@code))
> (t (let ((peek-holder-name (make-symbol "PEEK-HOLDER")))
> `(let ((,peek-holder-name ,thing-form))
> (typecase ,peek-holder-name
> (list (dolist (,variable-name ,peek-holder-name) ,@code))
> (vector (loop for ,variable-name across ,peek-holder-name
> do ,@code))))))))
>
> (macroexpand-1 '(doseq (x (the list '(a b c)))(print x)))
> => (DOLIST (X (THE LIST '(A B C))) (PRINT X))
>
> (doseq (x (the vector '(1 2 3))) (print x))
> => Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER:
> (1 2 3) is not of type ARRAY
Your trick (for the first two cond clauses) is exactly the same as
using two different-macros, one named 'deseq-the-list' and the other
named 'doseq-the-vector'. You are just asking the user of your macro
to choose the correct expansion. But if you change your example just
a little bit it stops working:
(let ((l (the list '(a b c))))
(doseq (x l)
(print x)))
Maybe I didn't understand correctly the problem correctly by I thinks
that's not what Tim Bradshaw was asking.
Your third clause expands into a form with branches for all possible
datatypes, using run-time type information to choose one branch.
Again, I think that's not what Tim Bradshaw was asking. But maybe he
is happy enough with that solution.
Anyway, it still looks to me that "impossible" is the correct word to
use in this case: you can't create a macro that allows creates
different expansions according to the type of the (iterated) object.
> * (defparameter l (list 'a 'b 'c))
> * (defparameter v (vector 1 2 3))
> * (doseq (x (the vector v)) (princ x))
> 123
> NIL
> * (doseq (x (the list l)) (princ x))
> ABC
> NIL
> * (doseq (x (case (random 2)(0 l)(1 v))) (princ x))
> 123
> NIL
> * (doseq (x (case (random 2)(0 l)(1 v))) (princ x))
> ABC
> NIL
>
> (macroexpand '(dothing (x (case (random 2)(0 l)(1 v))) (princ x)))
-------
I presume you meant doseq.
Ant�nio Leit�o.
Antonio Menezes Leitao wrote:
> "Tim Bradshaw" <··········@tfeb.org> writes:
> >
> > But how to arrange life so that if my-ob is some known-good-type
then
> > no iterator object needs to be made? Java can, I guess, often win
by
> > knowing the type statically, but this is harder for Lisp...
>
> I'm afraid it's not only harder: it's impossible. You don't have
type
> information available at macro expansion time to allow different
> expansions according to the type of the (iterated) object. This
means
> that either you cons an iterator or you cons a closure.
>
Oh come on: try and think like a human being, not a computer
scientist!
*Sometimes* in a dynamically typed language you will not have enough
type information. Then you need to fall back on the general case, and
be slow (just as you do in more `static' languages like Java in fact).
But sometimes, you do have type information available at compile time,
and you *can* compile decent code.
Currently, unfortunately, CL doesn't provide a standard way for macros
to enquire about what is known about types. Probably all
implementations do provide this information, though, and some
(LispWorks and I think Allegro, probably others) expose this
interface.
Even without such an interface, you can do OK with only a small amount
of overhead such as:
(loop for x being the contents of my-type my-object
do ...)
say.
--tim
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87mzz7n4td.fsf@evaluator.pt>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Antonio Menezes Leitao wrote:
>> "Tim Bradshaw" <··········@tfeb.org> writes:
>> >
>> > But how to arrange life so that if my-ob is some known-good-type
> then
>> > no iterator object needs to be made? Java can, I guess, often win
> by
>> > knowing the type statically, but this is harder for Lisp...
>>
>> I'm afraid it's not only harder: it's impossible. You don't have
> type
>> information available at macro expansion time to allow different
>> expansions according to the type of the (iterated) object. This
> means
>> that either you cons an iterator or you cons a closure.
>>
>
> Oh come on: try and think like a human being, not a computer
> scientist!
It seems to me that I didn't understand what you were asking. I got
the impression that you wanted a macro which expands into different
things according to the type of some argument. This, I claimed, it's
impossible in general due to lack of type information at compile time.
> *Sometimes* in a dynamically typed language you will not have enough
> type information. Then you need to fall back on the general case, and
> be slow (just as you do in more `static' languages like Java in fact).
>
> But sometimes, you do have type information available at compile time,
> and you *can* compile decent code.
In Common Lisp, at least, that seems like a rare situation. Maybe you
have some statistics that I don't but I would say that even for simple
things such as optimizing sequence functions or even arithmetic
functions, I rarely see Common Lisp code that contains the necessary
type declarations. But if you accept creating an iterator in this
(IMHO, very frequent) case, only optimizing critical code that
contains the necessary type declaration on a Common Lisp
implementation that provides that information at macro expansion time
(or that can do sufficient type propagation to remove unnecessary
tests and useless code), then that's an solution. Obviously, you
still have to provide the iterator so that the general case can work.
> Currently, unfortunately, CL doesn't provide a standard way for
> macros to enquire about what is known about types. Probably all
> implementations do provide this information, though, and some
> (LispWorks and I think Allegro, probably others) expose this
> interface.
>
> Even without such an interface, you can do OK with only a small
> amount of overhead such as:
>
> (loop for x being the contents of my-type my-object
> do ...)
>
IMHO, the extended loop macro (your extensions included) provides a
statically typed fragment of Common Lisp. If you look carefully at
the loop keywords (in, on, across, being, etc) you'll see that you are
providing the necessary type information for the macro to generate the
appropriate expansion. Extending the (extended) loop macro with more
keywords, is just more of the same. But that's perfectly fine with
me. As I told before, I got this strange idea that you were looking
for something more clever than directly providing the type
information. (Maybe it was the reference to the statically typed
nature of Java...)
Sorry for the noise.
Ant�nio Leit�o.
Antonio Menezes Leitao wrote:
>
> It seems to me that I didn't understand what you were asking. I got
> the impression that you wanted a macro which expands into different
> things according to the type of some argument. This, I claimed, it's
> impossible in general due to lack of type information at compile time
Yes, that is what I want. yes, it is impossible in general, however,
if you'd used an implementation which does type inferencing you'd know
it was possible in many interesting cases.
> IMHO, the extended loop macro (your extensions included) provides a
> statically typed fragment of Common Lisp. If you look carefully at
> the loop keywords (in, on, across, being, etc) you'll see that you
are
> providing the necessary type information for the macro to generate
the
> appropriate expansion.
That is one of the things I wish to avoid. We need to move forward
from 60s compiler technology.
--tim
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87fz4zqv9t.fsf@evaluator.pt>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Antonio Menezes Leitao wrote:
>>
>> It seems to me that I didn't understand what you were asking. I got
>> the impression that you wanted a macro which expands into different
>> things according to the type of some argument. This, I claimed, it's
>> impossible in general due to lack of type information at compile time
>
> Yes, that is what I want. yes, it is impossible in general, however,
> if you'd used an implementation which does type inferencing you'd know
> it was possible in many interesting cases.
Ok. Now I see what you are talking about. Let me rephrase my claim:
If what you want is a macro that expands into different things
according to the type of some argument then you will not be able to do
it in Common Lisp (even extended with environment inspection). This
is a result of the fact that the compiler must be able to compile a
macro call even when its arguments are unknown function calls. In
this case (and assuming that the programmer didn't include type
declarations), there is no type information available that might be
used to guide the expansion process. [Obviously, your macro might
expand according to the _lack_ of type information.]
On the other hand, if what you want is a macro whose expansion doesn't
depend on the type of some argument but that is further optimizable by
a SSC that uses type inference, then that's trivially true. In this
case, you don't need any extra support besides what Common Lisp
already offers.
Just to illustrate this, let's suppose that you want a macro for-each
that can iterate over foos, bars and foobars. One obvious solution
is:
(defmacro for-each ((var object) &body body)
(let ((obj (gensym)))
`(let ((,obj ,object))
(typecase ,obj
(foo (do-foo (,var ,obj) ,@body))
(bar (do-bar (,var ,obj) ,@body))
(foobar (do-foobar (,var ,obj) ,@body))))))
Note that the expansion phase doesn't need any type information.
Now, let's suppose you have a function myf that has declared type
information:
(defun myf (arg)
(declare (foo arg))
(for-each (e arg)
(print 2)))
The macroexpansion will produce the following:
(defun myf (arg)
(declare (foo arg))
(let ((obj001 arg))
(typecase obj001
(foo (do-foo (e obj001) (print e)))
(bar (do-bar (e obj001) (print e)))
(foobar (do-foobar (e obj001) (print e))))))
And now, the type inference will be able to eliminate all typecase
branches except the first one, producing:
(defun myf (arg)
(declare (foo arg))
(let ((obj001 arg))
(do-foo (e obj001) (print e))))
All of this can be done in standard Common Lisp. I really don't see
any problem here. Why do you need a macro that explores type
information (at compile time)?
>> IMHO, the extended loop macro (your extensions included) provides a
>> statically typed fragment of Common Lisp. If you look carefully at
>> the loop keywords (in, on, across, being, etc) you'll see that you
> are
>> providing the necessary type information for the macro to generate
> the
>> appropriate expansion.
>
> That is one of the things I wish to avoid. We need to move forward
> from 60s compiler technology.
I're puzzling me. You said before that you would be happy with:
(loop for x being the contents of my-type my-object
do ...)
Now you are saying that you wish to avoid the type-specific loop
keywords. I hope you agree that being-the-contents-of-my-type is just
a different syntax for a type-specific loop keywords.
Maybe it's better to forget this entire discussion.
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> "Tim Bradshaw" <··········@tfeb.org> writes:
>
> > But how to arrange life so that if my-ob is some known-good-type then
> > no iterator object needs to be made? Java can, I guess, often win by
> > knowing the type statically, but this is harder for Lisp...
>
> I'm afraid it's not only harder: it's impossible. You don't have type
> information available at macro expansion time to allow different
> expansions according to the type of the (iterated) object. This means
> that either you cons an iterator or you cons a closure.
Why don't you think the type information could be made available by
the implementation? I don't see any reason why this can't be done in
many (most?) typical cases. If you can't determine it, then you punt
off to runtime.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87is9vn3zs.fsf@evaluator.pt>
jayessay <······@foo.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> "Tim Bradshaw" <··········@tfeb.org> writes:
>>
>> > But how to arrange life so that if my-ob is some known-good-type then
>> > no iterator object needs to be made? Java can, I guess, often win by
>> > knowing the type statically, but this is harder for Lisp...
>>
>> I'm afraid it's not only harder: it's impossible. You don't have type
>> information available at macro expansion time to allow different
>> expansions according to the type of the (iterated) object. This means
>> that either you cons an iterator or you cons a closure.
>
> Why don't you think the type information could be made available by
> the implementation? I don't see any reason why this can't be done in
> many (most?) typical cases. If you can't determine it, then you punt
> off to runtime.
>
Let's forget the original request made by Tim Bradshaw and let us make
a new request: I have a convoluted data structure that I need to
iterate but I don't want to use neither an iterator (a la Java) nor a
closure. Never.
Can you make a macro that is applicable to, say, lists, vectors and to
my convoluted data-structure without using typep at runtime? I don't
mind if you use some non-standard form of environment inspection.
That's the problem I was thinking about (I understand now that that's
not the problem Tim Bradshaw was trying to solve).
Ant�nio Leit�o.
Antonio Menezes Leitao wrote:
>
> Can you make a macro that is applicable to, say, lists, vectors and
to
> my convoluted data-structure without using typep at runtime? I don't
> mind if you use some non-standard form of environment inspection.
>
No, obviously not in general.
--tim
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> jayessay <······@foo.com> writes:
>
> > Antonio Menezes Leitao <··············@evaluator.pt> writes:
> >
> >> "Tim Bradshaw" <··········@tfeb.org> writes:
> >>
> >> > But how to arrange life so that if my-ob is some known-good-type then
> >> > no iterator object needs to be made? Java can, I guess, often win by
> >> > knowing the type statically, but this is harder for Lisp...
> >>
> >> I'm afraid it's not only harder: it's impossible. You don't have type
> >> information available at macro expansion time to allow different
> >> expansions according to the type of the (iterated) object. This means
> >> that either you cons an iterator or you cons a closure.
> >
> > Why don't you think the type information could be made available by
> > the implementation? I don't see any reason why this can't be done in
> > many (most?) typical cases. If you can't determine it, then you punt
> > off to runtime.
> >
>
> Let's forget the original request made by Tim Bradshaw and let us make
> a new request: I have a convoluted data structure that I need to
> iterate but I don't want to use neither an iterator (a la Java) nor a
> closure. Never.
^^^^^ ----- !!
>
> Can you make a macro that is applicable to, say, lists, vectors and to
> my convoluted data-structure without using typep at runtime?
No. But then I'm not sure anyone really cares about this sort of
requirement.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <871xgi7191.fsf@evaluator.pt>
jayessay <······@foo.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> Can you make a macro that is applicable to, say, lists, vectors and to
>> my convoluted data-structure without using typep at runtime?
>
> No. But then I'm not sure anyone really cares about this sort of
> requirement.
That's the same answer that John McCarthy got from the IBM group that
developed the FORTRAN compiler when he asked them to make the
necessary changes to allow for recursive definitions.
I think it's a good principle to never reject some idea based on our
lack of understanding of its usefulness.
Just to give you some food for though, consider the fact that this
requirement allows you to generate highly efficient code to iterate
complex data structures without using neither closures nor iterators
(and thus, minimizing GC impact) and without requiring type
information at runtime.
Ant�nio Leit�o.
From: Matthew Danish
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87fz4x3nlx.fsf@mapcar.org>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> jayessay <······@foo.com> writes:
> > Antonio Menezes Leitao <··············@evaluator.pt> writes:
> >> Can you make a macro that is applicable to, say, lists, vectors and to
> >> my convoluted data-structure without using typep at runtime?
> > No. But then I'm not sure anyone really cares about this sort of
> > requirement.
> That's the same answer that John McCarthy got from the IBM group that
> developed the FORTRAN compiler when he asked them to make the
> necessary changes to allow for recursive definitions.
This is not about politics. To do what you ask for requires a static
type system. When jay says ``not sure anyone cares about [it]'' he
means that not everyone cares to have a static type system.
Certainly, some Lisps offer some static type analysis and inference;
and even have an interface to access this information. But in Lisp
you won't be guaranteed any kind of static type information; so it is
only natural that you will have to fall back to typep at run-time for
those cases. I don't see what is wrong in principle here. If you
don't like this then you need to reconsider using Lisp at all, since
you quite clearly would not want a dynamic type system.
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87sm8xzf48.fsf@evaluator.pt>
Matthew Danish <··········@cmu.edu> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>> jayessay <······@foo.com> writes:
>> > Antonio Menezes Leitao <··············@evaluator.pt> writes:
>> >> Can you make a macro that is applicable to, say, lists, vectors and to
>> >> my convoluted data-structure without using typep at runtime?
>> > No. But then I'm not sure anyone really cares about this sort of
>> > requirement.
>> That's the same answer that John McCarthy got from the IBM group that
>> developed the FORTRAN compiler when he asked them to make the
>> necessary changes to allow for recursive definitions.
>
> This is not about politics. To do what you ask for requires a static
> type system.
You are preaching to the choir. Maybe you didn't read the entire
thread.
> When jay says ``not sure anyone cares about [it]'' he means that not
> everyone cares to have a static type system.
> Certainly, some Lisps offer some static type analysis and inference;
> and even have an interface to access this information. But in Lisp
> you won't be guaranteed any kind of static type information;
IMHO, 'Lisp' is a family of languages. Your last statement suggests
that you apply to the entire family a property that is true just for
some members.
> so it is only natural that you will have to fall back to typep at
> run-time for those cases. I don't see what is wrong in principle
> here.
In some cases, I would like to be able to write macros that could ask
for specific type information (inferred or asserted by the programmer)
and that could expand according to that type information or refuse to
expand when that information isn't available. In this case, the
programmer would be forced to provide the information that the type
inferencer couldn't derive.
Would all macros be like that? Of course not. Most probably only a
very small number would explore that possibility. But it's good to
have the option.
> If you don't like this then you need to reconsider using Lisp at
> all, since you quite clearly would not want a dynamic type system.
Thanks for the advice.
Ant�nio Leit�o.
Matthew Danish <··········@cmu.edu> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
> > jayessay <······@foo.com> writes:
> > > Antonio Menezes Leitao <··············@evaluator.pt> writes:
> > >> Can you make a macro that is applicable to, say, lists, vectors and to
> > >> my convoluted data-structure without using typep at runtime?
> > > No. But then I'm not sure anyone really cares about this sort of
> > > requirement.
> > That's the same answer that John McCarthy got from the IBM group that
> > developed the FORTRAN compiler when he asked them to make the
> > necessary changes to allow for recursive definitions.
>
> This is not about politics. To do what you ask for requires a static
> type system. When jay says ``not sure anyone cares about [it]'' he
> means that not everyone cares to have a static type system.
> Certainly, some Lisps offer some static type analysis and inference;
> and even have an interface to access this information. But in Lisp
> you won't be guaranteed any kind of static type information; so it is
> only natural that you will have to fall back to typep at run-time for
> those cases. I don't see what is wrong in principle here. If you
> don't like this then you need to reconsider using Lisp at all, since
> you quite clearly would not want a dynamic type system.
I think that about sums it up. Except for one point: you can't make
the sort of guarantee Leitao seems to want even in static type systems
unless you make even further restrictions. Think about iterating over
objects read in at runtime.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Matthew Danish
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87brfk3nau.fsf@mapcar.org>
jayessay <······@foo.com> writes:
> I think that about sums it up. Except for one point: you can't make
> the sort of guarantee Leitao seems to want even in static type systems
> unless you make even further restrictions. Think about iterating over
> objects read in at runtime.
In a statically typed language, ``objects you read in at run-time''
would still have a statically-determinable type. The API of the
function returning the objects dictates some type. That type may be a
polymorphic disjoint type, but it is still a static type. Perhaps you
are thinking of Java's crappy system where everything must be
downcasted from Object at run-time? A real static type system, like
one based on Hindley-Milner (think ML), would not have this problem.
Of course, it has other problems, relating to extensibility, but that
is a separate issue.
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Matthew Danish <··········@cmu.edu> writes:
> jayessay <······@foo.com> writes:
> > I think that about sums it up. Except for one point: you can't make
> > the sort of guarantee Leitao seems to want even in static type systems
> > unless you make even further restrictions. Think about iterating over
> > objects read in at runtime.
>
> In a statically typed language, ``objects you read in at run-time''
> would still have a statically-determinable type. The API of the
> function returning the objects dictates some type. That type may be a
> polymorphic disjoint type, but it is still a static type. Perhaps you
> are thinking of Java's crappy system where everything must be
> downcasted from Object at run-time? A real static type system, like
> one based on Hindley-Milner (think ML), would not have this problem.
> Of course, it has other problems, relating to extensibility, but that
> is a separate issue.
Sure, but it will still need to dispatch (or jump via a jump table
indexed by type or whatever) to the proper iterator code for any given
instance. The programmer may not see any of this, but he doesn't see
any of it in the macro case either, nevertheless it is still there.
OTOH, maybe I'm really missing something??
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
"Tim Bradshaw" <··········@tfeb.org> writes:
> I'd like to see a syntax for iterating over arbitrary
> `iterable-things', which could be made convincingly efficient in the
Can Jonathan Amsterdam's ITERATE do that?
Paolo
--
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
Recommended Common Lisp libraries/tools (Google for info on each):
- ASDF/ASDF-INSTALL: system building/installation
- CL-PPCRE: regular expressions
- UFFI: Foreign Function Interface
From: Rahul Jain
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87sm8lhm8q.fsf@nyct.net>
"Tim Bradshaw" <··········@tfeb.org> writes:
> (loop for item iterating-over my-ob
> do ... things with item ...)
>
> But how to arrange life so that if my-ob is some known-good-type then
> no iterator object needs to be made? Java can, I guess, often win by
> knowing the type statically, but this is harder for Lisp...
Java never optimizes this, and I'm not sure if it can. Objects (i.e.,
Iterators) can't be inlined, after all. There's no way to intercede in
the compiler, either.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
Rahul Jain wrote:
> "Tim Bradshaw" <··········@tfeb.org> writes:
>
>
>>(loop for item iterating-over my-ob
>>do ... things with item ...)
>>
>>But how to arrange life so that if my-ob is some known-good-type then
>>no iterator object needs to be made? Java can, I guess, often win by
>>knowing the type statically, but this is harder for Lisp...
>
>
> Java never optimizes this, and I'm not sure if it can.
It supplies the java.util.RandomAccess interface to flag whether or not
it should optimise the iterator away, though as to whether any extant
implementations do I don't know.
> Objects (i.e., Iterators) can't be inlined, after all.
If, in C++ parlance, an Iterator of known type (hence all virtual
methods may be inlined) is allocated on the stack rather than the heap,
what do you suppose is not 'inlined' in the iteration calls, vs a stack
allocated index and increment operations? IIRC Jikes attempted such
optimisations if an object reference did not escape a method.
> There's no way to intercede in the compiler, either.
Unless you change the source of an open source JVM, or build your own
using BCEL (depending whether you mean bytecode or machine code compiler).
Pete
Rahul Jain wrote:
> "Tim Bradshaw" <··········@tfeb.org> writes:
>
> > (loop for item iterating-over my-ob
> > do ... things with item ...)
> >
> > But how to arrange life so that if my-ob is some known-good-type
then
> > no iterator object needs to be made? Java can, I guess, often win
by
> > knowing the type statically, but this is harder for Lisp...
>
> Java never optimizes this, and I'm not sure if it can. Objects (i.e.,
> Iterators) can't be inlined, after all. There's no way to intercede
in
> the compiler, either.
>
[Google groups is fouled up so this may be a near duplicate, if it
eventually decides to post the previous one or, I suppose, this one.]
Yes, Java *does* optimise iteration in important cases. Consider the
following:
/*
* Just playing with iteration in Java 1.5
*/
package org.tfeb.play;
public class TsIter {
// how many times to loop
private static final int iters = 1000000;
public static void main(String[] args) {
int[] a = new int[10000];
long start = System.currentTimeMillis();
for (int i = 0; i < iters; i++) {
int sum = 0;
for (int j : a) {
sum += j;
}
}
System.out.format("New %dms%n", System.currentTimeMillis() -
start);
start = System.currentTimeMillis();
for (int i = 0; i < iters; i++) {
int sum = 0;
for (int j = 0; j < a.length; j++) {
sum += a[j];
}
}
System.out.format("Old %dms%n", System.currentTimeMillis() -
start);
}
}
Trying this on the Sun JVM results in pretty similar timings for the
two loops: I think it's fairly clearly generating the nearly the same
code for both kinds of iteration.
Now someone is going to say `but it's easy for arrays'. Yes: it *is*
easy. And for CL many other types are probably fairly hard because
there's no sealing (so you could always get a subtype which behaved
differently, except for things like arrays and so on which can't be
subclassed by user code).
But that's OK: if I can have a common iteration mechanism which will
generate reasonably good code for arrays (and lists) that are known to
be such, and will work adequately for user-defined iterable-things,
then that is a lot better than having to rewrite loops if I change the
type of thing I'm iterating over.
I'm depressed by how hard it has been to get this across to people. I
don't want to solve a halting problem or have a static type system:
where did I say that I did?
--tim
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87y8iasuu7.fsf@evaluator.pt>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Now someone is going to say `but it's easy for arrays'. Yes: it *is*
> easy. And for CL many other types are probably fairly hard because
> there's no sealing (so you could always get a subtype which behaved
> differently, except for things like arrays and so on which can't be
> subclassed by user code).
>
> But that's OK: if I can have a common iteration mechanism which will
> generate reasonably good code for arrays (and lists) that are known to
> be such, and will work adequately for user-defined iterable-things,
> then that is a lot better than having to rewrite loops if I change the
> type of thing I'm iterating over.
I don't see this as a problem at all. Here is one solution:
(defmacro for-each ((var object) &body body)
(let ((obj (gensym)))
`(let ((,obj ,object))
(typecase ,obj
(list (dolist (,var ,obj) ,@body))
(array ,(let ((i (gensym)))
`(dotimes (,i (length ,obj))
(let ((,var (aref ,object ,i))) ,@body))))
(t (iterate ,obj #'(lambda (,var) ,@body)))))))
I presented a very similar solution a long time ago. In order to
generate efficient code for lists and arrays all you need to do is to
include a type declaration and use a good compiler:
(defun test-list (obj)
(declare (list obj))
(for-each (e obj)
(print e)))
If you compile the previous functions you will see that the compiler
will delete the code for the two other branches of the typecase.
And if you don't include the type declaration you will only have to
pay a small price for a few typechecks.
> I'm depressed by how hard it has been to get this across to people. I
> don't want to solve a halting problem or have a static type system:
> where did I say that I did?
In your first post you said:
"Tim Bradshaw" <··········@tfeb.org> writes:
> I'd like to see a syntax for iterating over arbitrary
> `iterable-things', which could be made convincingly efficient in the
> good cases. Python has the nice syntax, although I'm unsure about the
> efficiency, and Java now has stuff like
>
> for (MyClass o: MyBlobOfMyClassInstances) {
> ... do things with o ...
> }
>
> although I'm not sure this can be made efficient either. Outside of
> LOOP, I have stuff which lets you do things like
>
> (for (x my-thing)
> ... do things with X ...)
>
> And this works by asking MY-THING for an iterator, then calling that
> until it's exhausted which is something like the Python approach, and
> I think like Java's. But it is definitely not efficient - it always
> conses an iterator. And it's not part of LOOP.
In the previous three sentences, you mentioned three times the word
"efficient". I understood that you were concerned about efficiency and
that you didn't want to create an iterator. I'm sorry if I didn't
understand your problem.
> [...]
>
> But how to arrange life so that if my-ob is some known-good-type then
> no iterator object needs to be made? Java can, I guess, often win by
> knowing the type statically, but this is harder for Lisp...
If, as you said, you don't mind creating an iterator (or a closure)
for the not-so-good-types, then I don't see any problem that CL can't
easily solve.
Ant�nio Leit�o.
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87r7o1zuf6.fsf@qrnik.zagroda>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> (defmacro for-each ((var object) &body body)
> (let ((obj (gensym)))
> `(let ((,obj ,object))
> (typecase ,obj
> (list (dolist (,var ,obj) ,@body))
> (array ,(let ((i (gensym)))
> `(dotimes (,i (length ,obj))
> (let ((,var (aref ,object ,i))) ,@body))))
> (t (iterate ,obj #'(lambda (,var) ,@body)))))))
It has a potential problem: it duplicates body. With nested loops it
leads to exponential code explosion.
--
__("< Marcin Kowalczyk
\__/ ······@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
> > (defmacro for-each ((var object) &body body)
> > (let ((obj (gensym)))
> > `(let ((,obj ,object))
> > (typecase ,obj
> > (list (dolist (,var ,obj) ,@body))
> > (array ,(let ((i (gensym)))
> > `(dotimes (,i (length ,obj))
> > (let ((,var (aref ,object ,i))) ,@body))))
> > (t (iterate ,obj #'(lambda (,var) ,@body)))))))
>
> It has a potential problem: it duplicates body.
Not in clisp,
(macroexpand-1 '(FOR-EACH (i '(1 2 3)) (print i) (print (* 2 i))))
-->
(LET ((#1=#:G3155 #2='(1 2 3)))
(TYPECASE #1# (LIST (DOLIST (I #1#) . #3=((PRINT I) (PRINT (* 2 I)))))
(ARRAY (DOTIMES (#4=#:G3156 (LENGTH #1#)) (LET ((I (AREF #2# #4#))) . #3#)))
(T (ITERATE #1# #'(LAMBDA (I) . #3#))))), T
Not in sbcl:
* (macroexpand-1 '(FOR-EACH (i '(1 2 3)) (print i) (print (* 2 i))))
(LET ((#1=#:G3823 #2='(1 2 3)))
(TYPECASE #1#
(LIST (DOLIST (I #1#) . #3=((PRINT I) (PRINT (* 2 I)))))
(ARRAY
(DOTIMES (#4=#:G3824 (LENGTH #1#))
(LET ((I (AREF #2# #4#)))
. #3#)))
(T (ITERATE #1# #'(LAMBDA (I) . #3#)))))
T
Not in cmucl:
* (macroexpand-1 '(FOR-EACH (i '(1 2 3)) (print i) (print (* 2 i))))
(LET ((#1=#:G951 #2='(1 2 3)))
(TYPECASE #1#
(LIST (DOLIST (I #1#) . #3=((PRINT I) (PRINT (* 2 I)))))
(ARRAY
(DOTIMES (#4=#:G952 (LENGTH #1#))
(LET ((I (AREF #2# #4#)))
. #3#)))
(T (ITERATE #1# #'(LAMBDA (I) . #3#)))))
T
Yes in gcl:
>(macroexpand-1 '(FOR-EACH (i '(1 2 3)) (print i) (print (* 2 i))))
(LET ((#0=#:G1392 #1='(1 2 3)))
(TYPECASE #0#
(LIST (DOLIST (I #0#) (PRINT I) (PRINT (* 2 I))))
(ARRAY (DOTIMES (#3=#:G1393 (LENGTH #0#))
(LET ((I (AREF #1# #3#))) (PRINT I) (PRINT (* 2 I)))))
(T (ITERATE #0# #'(LAMBDA (I) (PRINT I) (PRINT (* 2 I)))))))
T
Not in ecl:
> (macroexpand-1 '(FOR-EACH (i '(1 2 3)) (print i) (print (* 2 i))))
(LET ((#1=#:G7 #2='(1 2 3)))
(TYPECASE #1# (LIST (DOLIST (I #1#) . #3#))
(ARRAY (DOTIMES (#4=#:G8 (LENGTH #1#))
(LET ((I (AREF #2# #4#))) . #3#)))
(T (ITERATE #1# #'(LAMBDA (I) . #3#)))))
T
> With nested loops it
> leads to exponential code explosion.
In all cases, the compiler should be able to easily reduce the common
subexpressions.
Or you can write:
(defmacro for-each ((var object) &body body)
(let ((obj (gensym)))
`(let ((,obj ,object))
(typecase ,obj
(list (dolist (,var ,obj) #1=(progn ,@body)))
(array ,(let ((i (gensym)))
`(dotimes (,i (length ,obj))
(let ((,var (aref ,object ,i))) #1#))))
(t (iterate ,obj (lambda (,var) #1#)))))))
--
__Pascal Bourguignon__ http://www.informatimago.com/
Voting Democrat or Republican is like choosing a cabin in the Titanic.
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <871xg1vh78.fsf@qrnik.zagroda>
Pascal Bourguignon <····@mouse-potato.com> writes:
>> It has a potential problem: it duplicates body.
>
> Not in clisp,
>
> (macroexpand-1 '(FOR-EACH (i '(1 2 3)) (print i) (print (* 2 i))))
> -->
> (LET ((#1=#:G3155 #2='(1 2 3)))
> (TYPECASE #1# (LIST (DOLIST (I #1#) . #3=((PRINT I) (PRINT (* 2 I)))))
> (ARRAY (DOTIMES (#4=#:G3156 (LENGTH #1#)) (LET ((I (AREF #2# #4#))) . #3#)))
> (T (ITERATE #1# #'(LAMBDA (I) . #3#))))), T
Do you say that the compiler will process the contents of #3# only
once and generate code from it in one copy? While it is threoretically
possible (after it determines how their environments differ and
manages to accommodate the differences by putting all versions of I
in the same location relative to the body), I don't believe it will
happen in practice.
--
__("< Marcin Kowalczyk
\__/ ······@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> writes:
> Pascal Bourguignon <····@mouse-potato.com> writes:
> >> It has a potential problem: it duplicates body.
> > Not in clisp,
> > (macroexpand-1 '(FOR-EACH (i '(1 2 3)) (print i) (print (* 2 i))))
> > -->
> > (LET ((#1=#:G3155 #2='(1 2 3)))
> > (TYPECASE #1# (LIST (DOLIST (I #1#) . #3=((PRINT I) (PRINT (* 2 I)))))
> > (ARRAY (DOTIMES (#4=#:G3156 (LENGTH #1#)) (LET ((I (AREF #2# #4#))) . #3#)))
> > (T (ITERATE #1# #'(LAMBDA (I) . #3#))))), T
> Do you say that the compiler will process the contents of #3# only
> once and generate code from it in one copy?
I would guess that most compilers don't take shared source code
structure into account when generating executable code.
An obvious attempt to avoid duplication is to create a local function
with the body:
(defmacro for-each ((var object) &body body)
(let ((obj (gensym)) (fn (gensym)))
`(let ((,obj ,object))
(flet ((,fn () ,@body))
(typecase ,obj
(list (dolist (,var ,obj) (,fn)))
(array ,(let ((i (gensym)))
`(dotimes (,i (length ,obj))
(let ((,var (aref ,object ,i))) (,fn)))))
(t (iterate ,obj #'(lambda (,var) (,fn)))))))))
--
Lars Brinkhoff, Services for Unix, Linux, GCC, HTTP
Brinkhoff Consulting http://www.brinkhoff.se/
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87r7o1v3kz.fsf@evaluator.pt>
Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> (defmacro for-each ((var object) &body body)
>> (let ((obj (gensym)))
>> `(let ((,obj ,object))
>> (typecase ,obj
>> (list (dolist (,var ,obj) ,@body))
>> (array ,(let ((i (gensym)))
>> `(dotimes (,i (length ,obj))
>> (let ((,var (aref ,object ,i))) ,@body))))
>> (t (iterate ,obj #'(lambda (,var) ,@body)))))))
>
> It has a potential problem: it duplicates body. With nested loops it
> leads to exponential code explosion.
Not if you include type declarations. In this case, the compiler will
remove the other branches of the typecase (and the typecase itself).
But you are right in the general case. However, I would say that it
is a controlled situation: I don't expect to see many nested loops,
(even assuming macro-generated loops) simply because we are talking
about iterating data-structures and these have limited depth
(otherwise they would also require recursion).
Ant�nio Leit�o.
Antonio Menezes Leitao wrote:
>
> Not if you include type declarations. In this case, the compiler
will
> remove the other branches of the typecase (and the typecase itself).
This is the kind of thing I'm thinking of. It's mildly
system-dependent - CMUCL probably can just elide the code, but other
systems may need some kind of (implementation-dependent)
environment-enquiry stuff in the macro to allow it to do the same
thing. And (as I said) I'm happy with mild implementation-dependence
in the macro.
--tim
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87fz4fi0xq.fsf@evaluator.pt>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Antonio Menezes Leitao wrote:
>>
>> Not if you include type declarations. In this case, the compiler
> will
>> remove the other branches of the typecase (and the typecase itself).
>
> This is the kind of thing I'm thinking of. It's mildly
> system-dependent - CMUCL probably can just elide the code, but other
> systems may need some kind of (implementation-dependent)
> environment-enquiry stuff in the macro to allow it to do the same
> thing. And (as I said) I'm happy with mild implementation-dependence
> in the macro.
My solution isn't system-dependent at all. You might say that
different compilers will optimize the macro expansion differently but,
other than that, it's fully portable code.
OTOH, you mention (once again) that you hope to make "the same thing"
at macro expansion time. For the last time: you can't make a macro
that employs whatever environment-enquiry stuff you can imagine to
decide, at macro expansion time, the form of the expansion based on
the type of the macro arguments. Note that I said "the type of the
macro arguments" and not "the type declarations associated with the
macro arguments".
I'm depressed by how hard it has been to get this across to people. :-)
In short: macro expansion + type inference is not the same as
type declarations + macro expansion.
Why? Because (at least in Common Lisp) type inference can only be
applied _after_ macro expansion and, moreover, type declarations are
just a really tiny part of type inference. The net result is that, at
macro expansion time, you only have type declarations and these might
not be enough for your macro to decide its expansion.
Obviously, you can go very far as long as you demand that all the type
declarations needed by the macro are provided, but this is more or
less the same (just more annoying) as having different macros for
different iterations.
Let me give you an example:
(defun iterate-them (x y z)
(let ((l (list x y z)))
(for-each (e l)
(print e))))
In this case, any compiler mildly intelligent will optimize the code
by removing the unneeded branches. But your macro will not have any
type declarations available to decide the expansion. The most it can
do is to expand according to the lack of type declarations. And if
you think the (list x y z) initialization is enough for the
environment-enquiry stuff to work, change it to:
(defun iterate-them (x y z)
(let ((l (f x y z)))
(for-each (e l)
(print e))))
(defun f (x y z)
(list x y z))
Even in this form, CMUCL is clever enough to optimize the macro
expansion. Other compilers should be able to do the same. Your
environment-enquiry stuff will have a hard time. And you can always
make it harder.
Even if you provide type declarations they may come in different forms
that aren't available in the environment. For example, consider the
following macro:
(defmacro the-list (form)
`(the list ,form))
A normal user would think that he is being nice to the for-each macro
when he writes:
(defun test-list (obj)
(for-each (e (the-list obj))
(print e)))
But, as you know, the type declaration isn't visible when the
for-each macro decides the expansion, unless, obviously, the macro
also expands its arguments internally to see if some of them provide
more type information. Obviously, this will get arbitrarily
complicated and, in the end, you will be implementing a statically
typed Lisp. OTOH, if you trust the type inference then it will work
as expected. You don't even have to look at the macro arguments.
So, if you are thinking about replacing one compiler (weak) type
inference by some "environment-enquiry stuff" you better replace the
entire compiler.
Ant�nio Leit�o.
Ps: I would love to see the "environment-enquiry stuff" that you have
been thinking that can solve this problem. Honestly, I'm interested.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> at macro expansion time. For the last time: you can't make a macro
> that employs whatever environment-enquiry stuff you can imagine to
> decide, at macro expansion time, the form of the expansion based on
> the type of the macro arguments. Note that I said "the type of the
> macro arguments" and not "the type declarations associated with the
> macro arguments".
>
> I'm depressed by how hard it has been to get this across to people. :-)
Maybe it's because it isn't really true?
> In short: macro expansion + type inference is not the same as
> type declarations + macro expansion.
>
> Why? Because (at least in Common Lisp) type inference can only be
> applied _after_ macro expansion and, moreover, type declarations are
> just a really tiny part of type inference. The net result is that, at
> macro expansion time, you only have type declarations and these might
> not be enough for your macro to decide its expansion.
Maybe you mean something very particular by "_in_ Common Lisp"
(emphasis mine). If you are willing to write the code walkers, and
the (user extensible) maps for functions to types, and the compile
time stacks for variable types, etc. you can indeed do type inference
at macro expansion time. I've done it. Old "Murphy Law": If it
happens it must be possible.
Maybe you are talking about having all the nice type information
handed to you on a platter via the environment stuff? Well, in that
case AFAIK, you are correct, but that is not the same as the blanket
assertion you make.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <878ya6is2w.fsf@evaluator.pt>
jayessay <······@foo.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> at macro expansion time. For the last time: you can't make a macro
>> that employs whatever environment-enquiry stuff you can imagine to
>> decide, at macro expansion time, the form of the expansion based on
>> the type of the macro arguments. Note that I said "the type of the
>> macro arguments" and not "the type declarations associated with the
>> macro arguments".
>>
>> I'm depressed by how hard it has been to get this across to people. :-)
>
> Maybe it's because it isn't really true?
>
>> In short: macro expansion + type inference is not the same as
>> type declarations + macro expansion.
>>
>> Why? Because (at least in Common Lisp) type inference can only be
>> applied _after_ macro expansion and, moreover, type declarations are
>> just a really tiny part of type inference. The net result is that, at
>> macro expansion time, you only have type declarations and these might
>> not be enough for your macro to decide its expansion.
>
> Maybe you mean something very particular by "_in_ Common Lisp"
> (emphasis mine). If you are willing to write the code walkers, and
> the (user extensible) maps for functions to types, and the compile
> time stacks for variable types, etc. you can indeed do type inference
> at macro expansion time. I've done it. Old "Murphy Law": If it
> happens it must be possible.
If it is possible, it might happens. Can you, please, show me your
implementation of the for-each macro that uses type information at
macro expansion time to compute the expansion?
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> jayessay <······@foo.com> writes:
>
> > Maybe you mean something very particular by "_in_ Common Lisp"
> > (emphasis mine). If you are willing to write the code walkers, and
> > the (user extensible) maps for functions to types, and the compile
> > time stacks for variable types, etc. you can indeed do type inference
> > at macro expansion time. I've done it. Old "Murphy Law": If it
> > happens it must be possible.
>
> If it is possible, it might happens. Can you, please, show me your
> implementation of the for-each macro that uses type information at
> macro expansion time to compute the expansion?
I can't legally do that at this point. Beyond that show stopper, it's
several thousand "lines", so I suppose it would have to be put up on a
server. However, I can show a few simple examples:
($
;; The '$' operator is a macro which forces code walking, type
;; inference and requested opimizations (such as loop hoisting) for
;; dynamic queries that build their function implementation at runtime,
;; but is also used to check expansions at the top level.
;;
(for-each x :in (vector 1 2 3)
(print x)))
Expanding this gives:
(LET ((#:GSET3342753 (VECTOR 1 2 3)))
(LET ((#:GSIZE3342755 (LENGTH #:GSET3342753)))
(DO ((#:GI3342754 0 (1+ #:GI3342754)))
((= #:GI3342754 #:GSIZE3342755))
(LET ((X (SVREF #:GSET3342753 #:GI3342754))) (PRINT X)))))
Same thing only with list:
($
(for-each x :in (list 1 2 3)
(print x)))
Expanding gives:
(LET ((#:GSET3342756 (LIST 1 2 3)))
(DOLIST (X #:GSET3342756) (PRINT X)))
Another slightly more interesting one:
($
(let ((model (the-model "alice:general-conversation")))
(for-each c :in model
(print c))))
(LET ((#:GSET3342757 (THE-MODEL "alice:general-conversation")))
(LET ((#:GTHE-SET3342760
(IF (NOT (CHASH-TABLE-P #:GSET3342757))
(NODES #:GSET3342757)
#:GSET3342757)))
(WITH-CHASH-TABLE-ITERATOR
(#:NEXT3342758 #:KEY3342759 X #:GTHE-SET3342760)
(WHILE (#:NEXT3342758) (PRINT X)))))
One that shows a couple of scopes and punting off to runtime
($
(let ((sx (vector 1 2 3))
(sy (list :one :two :three))
(m (the-model "std-models:models"))
(su *jsa*)) ; Punt to runtime on this one
(for-each ((x :in sx)
(y :in sy)
(c :in m)
(u :in su)
(z :in "123"))
(let ((m u)) ; Nested scope redefines M
(for-each i :in m ; Don't know anything about this M
(print i)))
(format t "~%~S -> ~S -> ~S -> ~S" x y z u)
(print c))))
(LET ((SX (VECTOR 1 2 3))
(SY (LIST :ONE :TWO :THREE))
(M (THE-MODEL "std-models:models"))
(SU *JSA*))
(LET* ((#:|SET-3342886| SX)
(#:|SET-3342889| SY)
(#:|SET-3342890| (NODES M))
(#:|SET-3342893| SU)
(#:|NEXT-RTFN-3342894|
(ARIADNE.THESEUS::MAKE-RUNTIME-NEXT #:|SET-3342893|))
(#:|SET-3342896| "123")
(X NIL)
(Y NIL)
(U NIL)
(Z NIL)
(#:|NOTDONE-3342899| T))
(LET* ((#:|SIZE-3342887| (LENGTH #:|SET-3342886|))
(#:|INDEX-3342888| 0))
(WITH-CHASH-TABLE-ITERATOR
(#:|NEXT-FN-3342891| #:|K-3342892| C #:|SET-3342890|)
(LET* ((#:|SIZE-3342897| (LENGTH #:|SET-3342896|))
(#:|INDEX-3342898| 0))
(LOOP (IF (< #:|INDEX-3342888| #:|SIZE-3342887|)
(SETF X
(SVREF
#:|SET-3342886|
#:|INDEX-3342888|)
#:|INDEX-3342888|
(1+ #:|INDEX-3342888|))
(SETF #:|NOTDONE-3342899| NIL))
(SETF Y (CAR #:|SET-3342889|)
#:|NOTDONE-3342899|
(AND #:|NOTDONE-3342899| #:|SET-3342889|)
#:|SET-3342889| (CDR #:|SET-3342889|))
(SETF #:|NOTDONE-3342899|
(AND #:|NOTDONE-3342899|
(#:|NEXT-FN-3342891|)))
(MULTIPLE-VALUE-BIND (#:G3342900 #:G3342901)
(FUNCALL #:|NEXT-RTFN-3342894|)
(SETF U #:G3342901
#:|NOTDONE-3342899|
(AND
#:|NOTDONE-3342899|
(OR #:G3342900 #:G3342901))))
(IF (< #:|INDEX-3342898| #:|SIZE-3342897|)
(SETF Z
(CHAR
#:|SET-3342896|
#:|INDEX-3342898|)
#:|INDEX-3342898|
(1+ #:|INDEX-3342898|))
(SETF #:|NOTDONE-3342899| NIL))
(UNLESS #:|NOTDONE-3342899| (RETURN))
(LET ((M U))
(LET ((#:GSET3342902 M))
(BLOCK NIL
(ARIADNE.THESEUS::FOR-ALL-RUNTIME-FN
#:GSET3342902 #'(LAMBDA (I) (PRINT I))
NIL))))
(FORMAT T "~%~S -> ~S -> ~S -> ~S" X Y Z U)
(PRINT C)))))))
We also take advantage of type declarations if given (typically this
would be used for the parameters of functions across definition
boundaries):
($
(let ((a-set *jsa*)) ; Would normally punt to runtime on this
(declare (type hash-table a-set)) ; But here we give the type to use
(for-all x :in a-set
(print x))))
(LET ((A-SET *JSA*))
(DECLARE (TYPE HASH-TABLE A-SET))
(LET ((#:GSET3342914 A-SET))
(WITH-HASH-TABLE-ITERATOR (#:NEXT3342915 #:GSET3342914)
(LOOP (MULTIPLE-VALUE-BIND (#:MORE3342916 #:KEY3342917 X)
(#:NEXT3342915)
(DECLARE (IGNORE #:KEY3342917))
(UNLESS #:MORE3342916 (RETURN))
(PRINT X))))))
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
Hi
What you show is interesting, but how does it compare with the (non
standard) CLTL2 "environment" interface? It seems to me that with that
interface you wouldn't need the '$' form at all. I may be mistaken, but
AFAIU, you could just use that to achieve what you achieve now.
Cheers
--
marco
jayessay wrote:
> I can't legally do that at this point. Beyond that show stopper, it's
> several thousand "lines", so I suppose it would have to be put up on a
> server. However, I can show a few simple examples:
>
>
> ($
> ;; The '$' operator is a macro which forces code walking, type
> ;; inference and requested opimizations (such as loop hoisting) for
> ;; dynamic queries that build their function implementation at runtime,
> ;; but is also used to check expansions at the top level.
> ;;
> (for-each x :in (vector 1 2 3)
> (print x)))
>
> Expanding this gives:
>
> (LET ((#:GSET3342753 (VECTOR 1 2 3)))
> (LET ((#:GSIZE3342755 (LENGTH #:GSET3342753)))
> (DO ((#:GI3342754 0 (1+ #:GI3342754)))
> ((= #:GI3342754 #:GSIZE3342755))
> (LET ((X (SVREF #:GSET3342753 #:GI3342754))) (PRINT X)))))
>
>
> Same thing only with list:
>
> ($
> (for-each x :in (list 1 2 3)
> (print x)))
>
>
> Expanding gives:
>
> (LET ((#:GSET3342756 (LIST 1 2 3)))
> (DOLIST (X #:GSET3342756) (PRINT X)))
>
>
>
> Another slightly more interesting one:
>
> ($
> (let ((model (the-model "alice:general-conversation")))
> (for-each c :in model
> (print c))))
>
> (LET ((#:GSET3342757 (THE-MODEL "alice:general-conversation")))
> (LET ((#:GTHE-SET3342760
> (IF (NOT (CHASH-TABLE-P #:GSET3342757))
> (NODES #:GSET3342757)
> #:GSET3342757)))
> (WITH-CHASH-TABLE-ITERATOR
> (#:NEXT3342758 #:KEY3342759 X #:GTHE-SET3342760)
> (WHILE (#:NEXT3342758) (PRINT X)))))
>
....
>
> --
> 'j' - a n t h o n y at romeo/charley/november com
Marco Antoniotti <·······@cs.nyu.edu> writes:
> Hi
>
> What you show is interesting, but how does it compare with the (non
> standard) CLTL2 "environment" interface? It seems to me that with
> that interface you wouldn't need the '$' form at all. I may be
> mistaken, but AFAIU, you could just use that to achieve what you
> achieve now.
AFAIU what Duane and company are doing at Franz (which is the only
example I know with enough detail to comment), the "environment" would
significantly reduce what is involved to get this to work. That would
be from the angle of gathering and propagating the type information.
It wouldn't do anything for the rest of what is happening: choosing
the proper expansions or picking the runtime punts or refactoring, or
any of the optimization stuff, etc.
It may be worth mentioning that the same general strategy is used for
certain optimization techniques. For example, users can declare
within a scope (or globally), that a function is side effect free (or
can be so considered for the scope) and can thus be refactored or
hoisted out of loops.
The '$' operator really doesn't do anything except provide a user
visible hook to the top level code walker. As I mentioned before,
this was all set up within a certain domain scope - it is fairly
general but not as general as CL. Within that, there is a general
definer mechanism for defining various kinds of procedures for the
context: agents, "bin operations", events and their handlers, various
"classes" of functions, et. al. That mechanism makes the machinery
here completely transparent within this context.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87zn2jhcm2.fsf@evaluator.pt>
jayessay <······@foo.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> jayessay <······@foo.com> writes:
>>
>> > Maybe you mean something very particular by "_in_ Common Lisp"
>> > (emphasis mine). If you are willing to write the code walkers, and
>> > the (user extensible) maps for functions to types, and the compile
>> > time stacks for variable types, etc. you can indeed do type inference
>> > at macro expansion time. I've done it. Old "Murphy Law": If it
>> > happens it must be possible.
>>
>> If it is possible, it might happens. Can you, please, show me your
>> implementation of the for-each macro that uses type information at
>> macro expansion time to compute the expansion?
>
> I can't legally do that at this point. Beyond that show stopper, it's
> several thousand "lines", so I suppose it would have to be put up on a
> server. However, I can show a few simple examples:
>
> [some very interesting examples ...]
I presume that the $ macro is a solution to avoid redefining the
standard function definition forms.
From what I could see in your examples, either you include local type
information (via initialization expressions or explicit declarations)
or you punt to runtime. I think you can achieve the exact same effect
using normal macros that expand into a typecase form and then just
wait for the normal code optimization to remove the unneded branches.
Unless I'missing something, the final effect would look exactly the
same.
However you mentioned one thing in your previous message that I was
expecting to see here and that is the thing that makes me thing that
macros that depend on type information can't go very far without lots
of type declarations: assuming that f, g, and h are defined as:
(defun f (x y z)
(vector x y z))
(defun g (x y z)
(list x y z))
(defun h (x)
(the-model x))
what can your macro do in the following case?
($
(let ((sx (f 1 2 3))
(sy (g :one :two :three))
(m (h "std-models:models"))
(su *jsa*)) ; Punt to runtime on this one
(for-each ((x :in sx)
(y :in sy)
(c :in m)
(u :in su)
(z :in "123"))
(let ((m u)) ; Nested scope redefines M
(for-each i :in m ; Don't know anything about this M
(print i)))
(format t "~%~S -> ~S -> ~S -> ~S" x y z u)
(print c))))
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> > [some very interesting examples ...]
>
> I presume that the $ macro is a solution to avoid redefining the
> standard function definition forms.
No, it's for what the comment stated: use in dynamic function creation
(for example for generating specific code for a query) or for testing
at the top level. As I have said this stuff was built within the
context of a certain domain scope (which is fairly general but not as
general as for CL). There is a definer mechanism which supports
defining various kinds of procedures (agents, events and event
handlers, various classes of functions, et. al.) so, within that
context, all this machinery becomes transparent, as the definer knows
how, where, and when to invoke the code walkers for type inference &
propagation, optimization, etc.
> From what I could see in your examples, either you include local type
> information (via initialization expressions or explicit declarations)
> or you punt to runtime.
This is incorrect. As I stated in an earlier message there are
extensible maps that are used to capture and convey this information.
> I think you can achieve the exact same effect using normal macros
> that expand into a typecase form and then just wait for the normal
> code optimization to remove the unneded branches. Unless I'missing
> something, the final effect would look exactly the same.
These _are_ "normal" macros. In theory, what you are saying _might_
work on some implementations in some cases.
> However you mentioned one thing in your previous message that I was
> expecting to see here and that is the thing that makes me thing that
> macros that depend on type information can't go very far without lots
> of type declarations: assuming that f, g, and h are defined as:
>
> (defun f (x y z)
> (vector x y z))
>
> (defun g (x y z)
> (list x y z))
>
> (defun h (x)
> (the-model x))
>
> what can your macro do in the following case?
There is nothing special about this. As I mentioned before
(admittedly, only in passing), if a user cares about this, and
declares the type of these functions (thereby updating the maps), this
will work just like the original example. If they don't, then these
functions will be flagged as having "dynamic" type and the code will
punt to runtime.
If you used the definer mechanism, these sort of simple cases would
work even without declarations (since the type would be inferred and
the maps implicitly updated). But, while using the definer mechanism
for these sort of functions would work, that is really an abuse of its
intent.
However, that very thing is what makes this is a good example of where the
"environment" stuff would really offer some nice help. For various
reasonable examples (certainly the dead simple ones you give here) an
implementation could easily infer the result type of such functions
and tag them as such in the environment.
(defun f (x y z)
(vector x y z))
(add-form-type f vector)
($
(let ((sx (f 1 2 3)))
(for-all x :in sx
(print x))))
(LET ((SX (F 1 2 3)))
(LET ((#:GSET3342996 SX))
(LET ((#:GSIZE3342998 (LENGTH #:GSET3342996)))
(DO ((#:GI3342997 0 (1+ #:GI3342997)))
((= #:GI3342997 #:GSIZE3342998))
(LET ((X (SVREF #:GSET3342996 #:GI3342997)))
(PRINT X))))))
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> A normal user would think that he is being nice to the for-each
> macro when he writes:
>
> (defun test-list (obj)
> (for-each (e (the-list obj))
> (print e)))
>
> But, as you know, the type declaration isn't visible when the
> for-each macro decides the expansion, unless, obviously, the macro
> also expands its arguments internally to see if some of them provide
> more type information.
I have no idea what Tim had in mind but as I've been following this
thread I've been imagining a system where whatever type information
the compiler knows about is available during macro expansion. Imagine
for instance that there was a function TYPE-OF-EXPRESSION that took
the &environment argument to a macro expander and an expression and
returned some representation of whatever the compiler knew about the
type of that expression in the lexical environment represented by the
&environment argument. Then we could write for-each:
(defmacro for-each ((var thing) &body body &environment env)
(let ((thing-type (type-of-expression thing env)))
(cond
((subtypep type-info 'list)
(generate-list-iterator var thing body))
((subtypep type-info 'vector
(generate-list-iterator var thing body)))
(t (generate-generic-iterator var thing body)))))
This macro should work for your example because the compiler can
presumably (trivially) determine that the expression '(the-list obj)
has type 'list. Likewise it would generate a list-specific iteration
for this because when the compiler processes the declaration it would
affect the &environment argument that would be passed to for-each.
(defun test-list (obj)
(declare (type list obj))
(for-each (e obj)
(print e)))
And I'd even imagine that it would work for this case:
(defun foo (stuff)
(let ((obj (mapcar #'bar stuff)))
(foreach (e obj)
(print e))))
because the type inferencer, knowing that MAPCAR returns a list, knows
that OBJ is a list so when the FOR-EACH macroexpander calls
TYPE-OF-EXPRESSION with 'obj it can find out it's a list.
Obviously it's would be up to macro authors to decide what to do if
there wasn't enough type information available--for instance if we had
no generic iteration strategy we might have to signal an error during
macro expansion if TYPE-OF-EXPRESSION returned anything other than
'list or 'vector.
-Peter
P.S. Obviously this is just a sketch--using SUBTYPEP implies that
TYPE-OF-EXPRESSION actually returns a type and since (as has recently
been discussed in another thread) there isn't a single answer or even
a single "most specific" answer to the question "what type is this
thing/expression/whatever" we might have to return some more complex
object that can then be used with some other predicate to answer
SUBTYPEP-style questions.
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87y8i5ybkq.fsf@evaluator.pt>
Peter Seibel <·····@javamonkey.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> A normal user would think that he is being nice to the for-each
>> macro when he writes:
>>
>> (defun test-list (obj)
>> (for-each (e (the-list obj))
>> (print e)))
>>
>> But, as you know, the type declaration isn't visible when the
>> for-each macro decides the expansion, unless, obviously, the macro
>> also expands its arguments internally to see if some of them provide
>> more type information.
>
> I have no idea what Tim had in mind but as I've been following this
> thread I've been imagining a system where whatever type information
> the compiler knows about is available during macro expansion.
The problem, in Common Lisp, is that there isn't much type information
available during macro expansion. Common Lisp is a dynamically typed
language but macro expansion is a compile-time operation. The amount
of type information that you have at compile time is a small subset of
what you have at run time. In order to fill the gap, you must add
type declarations and provide the _complete_ program in order for type
inference to work.
> Imagine for instance that there was a function TYPE-OF-EXPRESSION
> that took the &environment argument to a macro expander and an
> expression and returned some representation of whatever the compiler
> knew about the type of that expression in the lexical environment
> represented by the &environment argument. Then we could write
> for-each:
>
> (defmacro for-each ((var thing) &body body &environment env)
> (let ((thing-type (type-of-expression thing env)))
> (cond
> ((subtypep type-info 'list)
> (generate-list-iterator var thing body))
> ((subtypep type-info 'vector
> (generate-list-iterator var thing body)))
> (t (generate-generic-iterator var thing body)))))
If you go back to the beginning of this thread you will see that your
solution is very similar to what I proposed (in Linj). But Linj is
not Common Lisp, Linj is a statically typed Lisp and the Linj type
inferencer makes several assumptions that, in general, can't be made
in Common Lisp.
I would be glad to see that I am wrong and that it is possible to do
useful static inference at macro expansion time in Common Lisp without
forcing the programmer to include lots of type declarations and
without restricting him of doing what is common to do in Common Lisp.
My claim, from the beginning of this thread, is that if you want to
provide a useful "type-of-expression" operator then you will end up
with a statically typed Common Lisp.
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> I would be glad to see that I am wrong and that it is possible to do
> useful static inference at macro expansion time in Common Lisp
> without forcing the programmer to include lots of type declarations
> and without restricting him of doing what is common to do in Common
> Lisp. My claim, from the beginning of this thread, is that if you
> want to provide a useful "type-of-expression" operator then you will
> end up with a statically typed Common Lisp.
So perhaps the key phrase here is "statically typed Common Lisp". One
possible interpretation of that phrase is Common Lisp that *requires*
the programmer to include static type declarations all the time, a la
Java or C++. Another is a Common Lisp in which the compiler does some
level of type inferencing (with the possibility of "infering" the type
T if all else fails). If you mean the former I'm not sure why you
think that. If you mean the later I think we already have such
implementations--they just don't expose a type-of-expression operator.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <876559tjto.fsf@evaluator.pt>
Peter Seibel <·····@javamonkey.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> I would be glad to see that I am wrong and that it is possible to do
>> useful static inference at macro expansion time in Common Lisp
>> without forcing the programmer to include lots of type declarations
>> and without restricting him of doing what is common to do in Common
>> Lisp. My claim, from the beginning of this thread, is that if you
>> want to provide a useful "type-of-expression" operator then you will
>> end up with a statically typed Common Lisp.
>
> So perhaps the key phrase here is "statically typed Common Lisp". One
> possible interpretation of that phrase is Common Lisp that *requires*
> the programmer to include static type declarations all the time, a la
> Java or C++. Another is a Common Lisp in which the compiler does some
> level of type inferencing (with the possibility of "infering" the type
> T if all else fails). If you mean the former I'm not sure why you
> think that. If you mean the later I think we already have such
> implementations--they just don't expose a type-of-expression operator.
Can you, please, show me just one such implementation?
I ask because all Common Lisp implementations I know do some level of
type inferencing _after_ macro expansion, not _before_. And they do
that for a reason but, as always, I would be happy if you showed me
that I'm wrong.
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>>
>>> I would be glad to see that I am wrong and that it is possible to do
>>> useful static inference at macro expansion time in Common Lisp
>>> without forcing the programmer to include lots of type declarations
>>> and without restricting him of doing what is common to do in Common
>>> Lisp. My claim, from the beginning of this thread, is that if you
>>> want to provide a useful "type-of-expression" operator then you will
>>> end up with a statically typed Common Lisp.
>>
>> So perhaps the key phrase here is "statically typed Common Lisp".
>> One possible interpretation of that phrase is Common Lisp that
>> *requires* the programmer to include static type declarations all
>> the time, a la Java or C++. Another is a Common Lisp in which the
>> compiler does some level of type inferencing (with the possibility
>> of "infering" the type T if all else fails). If you mean the former
>> I'm not sure why you think that. If you mean the later I think we
>> already have such implementations--they just don't expose a
>> type-of-expression operator.
>
> Can you, please, show me just one such implementation?
Nope. If you tell me that they all do type inferencing after macro
expansion, I belive you. But see below.
> I ask because all Common Lisp implementations I know do some level
> of type inferencing _after_ macro expansion, not _before_. And they
> do that for a reason but, as always, I would be happy if you showed
> me that I'm wrong.
So maybe I'm just being thick but I don't see any in principle reason
why a Common Lisp implementation that has a type inferencer couldn't
use it to do type inference before macro expansion in order to supply
a useful TYPE-OF-EXPRESSION function.
I'd imagine that the reason that current implementations do it *after*
all macro expansion is because there's presently no reason to do it
before--since there's no way to expose type information to macro
expanders (i.e. since there *isn't* a TYPE-OF-EXPRESSION function)
there's no point in gathering it in advance. Presently the only reason
for an implementation to do type inference is to improve code
generation. Thus it makes sense to include it in the code generation
phase which happens after macro expansion. But I don't see that an
implementation would become become "a statically typed Common Lisp" if
it ran its type inferencer before macro expansion as well as after.
That's all I'm saying.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <871xfxte06.fsf@evaluator.pt>
Peter Seibel <·····@javamonkey.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> Peter Seibel <·····@javamonkey.com> writes:
>>
>>> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>>>
>>>> I would be glad to see that I am wrong and that it is possible to do
>>>> useful static inference at macro expansion time in Common Lisp
>>>> without forcing the programmer to include lots of type declarations
>>>> and without restricting him of doing what is common to do in Common
>>>> Lisp. My claim, from the beginning of this thread, is that if you
>>>> want to provide a useful "type-of-expression" operator then you will
>>>> end up with a statically typed Common Lisp.
>>>
>>> So perhaps the key phrase here is "statically typed Common Lisp".
>>> One possible interpretation of that phrase is Common Lisp that
>>> *requires* the programmer to include static type declarations all
>>> the time, a la Java or C++. Another is a Common Lisp in which the
>>> compiler does some level of type inferencing (with the possibility
>>> of "infering" the type T if all else fails). If you mean the former
>>> I'm not sure why you think that. If you mean the later I think we
>>> already have such implementations--they just don't expose a
>>> type-of-expression operator.
>>
>> Can you, please, show me just one such implementation?
>
> Nope. If you tell me that they all do type inferencing after macro
> expansion, I belive you. But see below.
I didn't say that. I was only talking about the Common Lisp
implementation that I know.
> So maybe I'm just being thick but I don't see any in principle reason
> why a Common Lisp implementation that has a type inferencer couldn't
> use it to do type inference before macro expansion in order to supply
> a useful TYPE-OF-EXPRESSION function.
>
> I'd imagine that the reason that current implementations do it *after*
> all macro expansion is because there's presently no reason to do it
> before--since there's no way to expose type information to macro
> expanders (i.e. since there *isn't* a TYPE-OF-EXPRESSION function)
> there's no point in gathering it in advance.
I think there's a more important reason: there is not enough type
information available for macro expanders that justifies a much more
complex model of macro expansion.
> Presently the only reason for an implementation to do type inference
> is to improve code generation. Thus it makes sense to include it in
> the code generation phase which happens after macro expansion. But I
> don't see that an implementation would become become "a statically
> typed Common Lisp" if it ran its type inferencer before macro
> expansion as well as after. That's all I'm saying.
I didn't say that. All I'm saying is that if we develop a Common Lisp
implementation that runs a type inferencer before macro expansion, it
will only be useful if the code has enough type information available
at compile time. But having that information available at compile
time is a characteristic of statically typed languages, not
dynamically typed ones.
Ant�nio Leit�o.
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87is99893o.fsf@qrnik.zagroda>
Peter Seibel <·····@javamonkey.com> writes:
> I'd imagine that the reason that current implementations do it *after*
> all macro expansion is because there's presently no reason to do it
> before--since there's no way to expose type information to macro
> expanders (i.e. since there *isn't* a TYPE-OF-EXPRESSION function)
> there's no point in gathering it in advance.
No, the reason is that it's impossible to infer the type of the result
of a call to an unknown macro before expanding it. And if the macro
call is e.g. a toplevel form, it makes no sense to look inside it for
inferring types inside its arguments because it might not treat them
as expressions.
It would be possible to try to do some partial inferencing before
macros, assuming that the code doesn't contain enough macro calls to
interfere with it, but it makes more sense to do it afterwards.
--
__("< Marcin Kowalczyk
\__/ ······@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
>> I'd imagine that the reason that current implementations do it
>> *after* all macro expansion is because there's presently no reason
>> to do it before--since there's no way to expose type information to
>> macro expanders (i.e. since there *isn't* a TYPE-OF-EXPRESSION
>> function) there's no point in gathering it in advance.
>
> No, the reason is that it's impossible to infer the type of the result
> of a call to an unknown macro before expanding it.
I'm not talking about infering the type of the result of the macro
call, I'm talking about type information about the expressions the
macro may use. For instance, consider this code:
(defun foo (x y)
(declare (type integer x y))
(let ((z (+ x y)))
(baz x y z)))
In the body of the LET the environment contains bindings for the names
X, Y, and Z and furthermore knows (or could know) that they are all of
type INTEGER. That information could be made available to a macro that
is called within the body of the LET. I proposed a function
TYPE-OF-EXPRESSION that takes an expression and the &environment
argument to the macro expander. So if BAZ is a macro then I might use TYPE-OF-EXPRESSION within it like this:
something like:
(defmacro baz (a b c)
(if (subtypep (type-of-expression a env) 'number)
(generate-one-expansion a b c)
(generate-different-expansion a b c)))
> It would be possible to try to do some partial inferencing before
> macros, assuming that the code doesn't contain enough macro calls to
> interfere with it, but it makes more sense to do it afterwards.
It also seems that the compiler could intertwingle collecting type
information with expanding macros. For instance the compiler compiling
FOO above would stash away type information about X, Y, and Z in the
environment and then macroexpand any macros within the body of the LET
and then figure out what the types of the resulting expressions are,
etc.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel wrote:
>
> So maybe I'm just being thick but I don't see any in principle reason
> why a Common Lisp implementation that has a type inferencer couldn't
> use it to do type inference before macro expansion in order to supply
> a useful TYPE-OF-EXPRESSION function.
Precisely so. It can do type inference whenever it wants to: it can
do it at read time, at run time, and at all points in between.
> I'd imagine that the reason that current implementations do it
*after*
> all macro expansion is because there's presently no reason to do it
> before--since there's no way to expose type information to macro
> expanders (i.e. since there *isn't* a TYPE-OF-EXPRESSION function)
> there's no point in gathering it in advance. Presently the only
reason
> for an implementation to do type inference is to improve code
> generation. Thus it makes sense to include it in the code generation
> phase which happens after macro expansion. But I don't see that an
> implementation would become become "a statically typed Common Lisp"
if
> it ran its type inferencer before macro expansion as well as after.
> That's all I'm saying.
>
And you are right! This is why I keep going on about `mild
nonportability' and so on. I *know* these things can't be done
portably in common lisp, because the facilities just aren't there. It
may well be that no current implementations can do them either.
However progress is possible.
This underlying discussion that seems to be going on here seems to be
pretty much the classic `one-bit mind' problem described a couple of
years ago by Erik Naggum. To be able to provide some useful type
information to macros, CL does not have to become completely
statically typed, and the assumption that it does is, frankly, silly.
What I want is for progress to happen. Say Allegro CL's implementors
do some clever stuff which allows macros to ask for, and sometimes
get, type and other information which has been derived by the system.
Suddenly people using Allegro can write all sorts of clever macros,
although those macros will have implementation-specific parts. So in
the standard open-source way, the implementors of some open-source
Lisp borrow the Allegro API, soup it up a bit and implement a version
which provides somewhat better than Allegro's. In a few years there's
agreement on what this API should look like, and it becomes a
(sub-)standard.
Incidentally, here's a sketch of how macros might be given type
information that I thought of. This isn't meant to be even plausible:
there are a load of details just omitted. The problem is that type
inference generally happens (and can best be done) *after* macro
expansion. Well, OK, so do it after macro expansion then:
1. Macroexpand everything. Macros that want to make type enquiries
get little or no information.
2. Do some type inference &c
3. `Decorate' the source with the inferences made.
4. Macroexpand *again*. Macros that want to make type enquiries
can now get much better information.
OK, so what's the problem with this? It's fine for macros to get
expanded multiple times, of course, but if they expand to different
things when they have better type information then some of the
inference that was done could now be invalid. Well, there are at
least two answers to that:
1. Part of a type-information-using macro's contract is that it
must behave monotonically (?) with respect to type information.
If it is given information I1, and then later I2 which is a
refinement of I1 (which means something like all the types in I2
are subtypes of those in I1), then any type information it
provides must also be a refinement, so anything else which is
worrying about type information can use the initial information.
2. Alternatively the system will notice if macros have different
expansions, and will invalidate type information for things in
the body of the macro in that case. For added value (and longer
compile times!) it could then redo the inference phase and
iterate.
I'm sure that this sketch won't work (so don't bother picking
elaborate holes in it). But something like it probably could work.
And, as I said, progress is possible: I want to see implementations
competing on this kind of stuff.
--tim
From: Duane Rettig
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4zn2kt5uv.fsf@franz.com>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Peter Seibel wrote:
> >
> > So maybe I'm just being thick but I don't see any in principle reason
> > why a Common Lisp implementation that has a type inferencer couldn't
> > use it to do type inference before macro expansion in order to supply
> > a useful TYPE-OF-EXPRESSION function.
>
> Precisely so. It can do type inference whenever it wants to: it can
> do it at read time, at run time, and at all points in between.
>
> > I'd imagine that the reason that current implementations do it
> *after*
> > all macro expansion is because there's presently no reason to do it
> > before--since there's no way to expose type information to macro
> > expanders (i.e. since there *isn't* a TYPE-OF-EXPRESSION function)
> > there's no point in gathering it in advance. Presently the only
> reason
> > for an implementation to do type inference is to improve code
> > generation. Thus it makes sense to include it in the code generation
> > phase which happens after macro expansion. But I don't see that an
> > implementation would become become "a statically typed Common Lisp"
> if
> > it ran its type inferencer before macro expansion as well as after.
> > That's all I'm saying.
> >
>
> And you are right! This is why I keep going on about `mild
> nonportability' and so on. I *know* these things can't be done
> portably in common lisp, because the facilities just aren't there. It
> may well be that no current implementations can do them either.
> However progress is possible.
Progress is occurring. And I can vouch that at least one
implementation (and probably most others, as well) can infer types
to a certain extent, although not before or after, but _during_
macroexpansion - inferring type before macroexpansion doesn't make
sense, because macros are opaque until macroexpanded, but inferring
type after macroexpansion is too late to allow macros themselves
(especially compiler macros) to make decisions based on the environment
arguments (which might include type info) currently available.
> This underlying discussion that seems to be going on here seems to be
> pretty much the classic `one-bit mind' problem described a couple of
> years ago by Erik Naggum. To be able to provide some useful type
> information to macros, CL does not have to become completely
> statically typed, and the assumption that it does is, frankly, silly.
Right. And the fact that not _all_ type inferrence can be performed
during macroexpansion should not become the basis for rejecting it
out-of-hand.
> What I want is for progress to happen. Say Allegro CL's implementors
> do some clever stuff which allows macros to ask for, and sometimes
> get, type and other information which has been derived by the system.
Funny you should mention Allegro CL specifically :-)
> Suddenly people using Allegro can write all sorts of clever macros,
> although those macros will have implementation-specific parts. So in
> the standard open-source way, the implementors of some open-source
> Lisp borrow the Allegro API, soup it up a bit and implement a version
> which provides somewhat better than Allegro's. In a few years there's
> agreement on what this API should look like, and it becomes a
> (sub-)standard.
As I've mentioned before, my goal is to provide this interface as
open-source, precisely for this purpose. And the nice thing about
the interface is that it is very similar to the CLtL2 system. In
fact, for a while, while we were getting our documentation together,
we would refer to the pages in CLtL2 and then provide a "changes"
document to provide the rest of the descriptions.
> Incidentally, here's a sketch of how macros might be given type
> information that I thought of. This isn't meant to be even plausible:
> there are a load of details just omitted. The problem is that type
> inference generally happens (and can best be done) *after* macro
> expansion. Well, OK, so do it after macro expansion then:
>
> 1. Macroexpand everything. Macros that want to make type enquiries
> get little or no information.
> 2. Do some type inference &c
> 3. `Decorate' the source with the inferences made.
> 4. Macroexpand *again*. Macros that want to make type enquiries
> can now get much better information.
This would be possible for a user to do, but would be highly
inefficient. The level of pre-macroexpansion one does might
gain more in areas where setqs and loops are occurring,
> OK, so what's the problem with this? It's fine for macros to get
> expanded multiple times, of course, but if they expand to different
> things when they have better type information then some of the
> inference that was done could now be invalid. Well, there are at
> least two answers to that:
>
> 1. Part of a type-information-using macro's contract is that it
> must behave monotonically (?) with respect to type information.
> If it is given information I1, and then later I2 which is a
> refinement of I1 (which means something like all the types in I2
> are subtypes of those in I1), then any type information it
> provides must also be a refinement, so anything else which is
> worrying about type information can use the initial information.
What makes such a style so very inefficient is that a macro can
expand to more than one piece of code _based_ on type information
(or the lack thereof). Therefore, the point where a complete
macroexpansion diverges on the second time around may be just one
level down, thus rendering all further macroexpansions useless.
> 2. Alternatively the system will notice if macros have different
> expansions, and will invalidate type information for things in
> the body of the macro in that case. For added value (and longer
> compile times!) it could then redo the inference phase and
> iterate.
Alternatively, type inferences are enclosed lexically so that
de-annotation would not be needed. A potential problem with #2 is
that it might not finish.
> I'm sure that this sketch won't work (so don't bother picking
> elaborate holes in it). But something like it probably could work.
> And, as I said, progress is possible: I want to see implementations
> competing on this kind of stuff.
A better sketch for general puroposes is
1. Macroexpand one form, annotating types that can be inferred at
that point
2. repeat
3. The compiler performs type propagation over the final
macroexpansion, for things it can do itself. Any further rewrites
will not change the basic structure of the resultant code (since
all macros have been expanded by this time) and will instead just be
replacements of more general code (e.g. cl:+ ) to more specific code
(e.g. excl::+_2op) and finally to usage of the type info to generate
the final code and perhaps bum out the call.
Note that in the above sketch the user can exercise control over
steps 1 and 2 by using environment information in user macros and
compiler macros. And your own sketch can be simulated by prepending
a "walk" step beforehand.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Duane Rettig wrote:
> Right. And the fact that not _all_ type inferrence can be performed
> during macroexpansion should not become the basis for rejecting it
> out-of-hand.
Yes.
> What makes such a style so very inefficient is that a macro can
> expand to more than one piece of code _based_ on type information
> (or the lack thereof). Therefore, the point where a complete
> macroexpansion diverges on the second time around may be just one
> level down, thus rendering all further macroexpansions useless.
>
Yes, this could make it absolutely awful. It was really intended as a
demonstration of possibility. I'm reasonably sure implementors will
come up with something better, which seems to be correct!
--tim
Duane Rettig wrote:
> As I've mentioned before, my goal is to provide this interface as
> open-source, precisely for this purpose. And the nice thing about
> the interface is that it is very similar to the CLtL2 system. In
> fact, for a while, while we were getting our documentation together,
> we would refer to the pages in CLtL2 and then provide a "changes"
> document to provide the rest of the descriptions.
A sidequestion here: You don't provide the ENCLOSE function. Is there a
special reason why you don't do that? I think it would have been a very
useful addition.
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
From: Duane Rettig
Subject: Re: enclose, was Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4r7nwt0ul.fsf@franz.com>
Pascal Costanza <········@web.de> writes:
> Duane Rettig wrote:
>
> > As I've mentioned before, my goal is to provide this interface as
> > open-source, precisely for this purpose. And the nice thing about
> > the interface is that it is very similar to the CLtL2 system. In
> > fact, for a while, while we were getting our documentation together,
> > we would refer to the pages in CLtL2 and then provide a "changes"
> > document to provide the rest of the descriptions.
>
> A sidequestion here: You don't provide the ENCLOSE function. Is there
> a special reason why you don't do that? I think it would have been a
> very useful addition.
Well, I'm open to suggestions.
What is troubling about enclose is that it seems like a solution to
a specific problem, where perhaps a more general solution is called
for. Specifically, it only uses the declarations and macro/symbol-macro
definitions in the environment in its operation. This is similar to
the :macros-only environment kind in our module, which we use in the
preparation of a lexical closure for a macrolet. This would make
enclose an implementation tool for compilers, and not necessarily
useful for user code. Of course, I would like our environments access
interface to be useful for compiler implementation, but the main goal
is user access, since compilers already know how to access their own
environments, and I don't want to make any assumptions about how other
compilers implement their environments, nor to force compilers to be
reimplemented if they want to use our environments access package
(indeed, we are still in the process of bootstrapping this system,
and will probably not be fully a pure envirionments-access
implementation until at least 7.1 or later).
So the question is: is there an application where enclose might be
useful in user-code? I'm happy to entertain possibilities.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Duane Rettig wrote:
> So the question is: is there an application where enclose might be
> useful in user-code? I'm happy to entertain possibilities.
I know about one specific case. The MOP specifies that programmatic
creation of methods should be done like this:
(let ((method
(multiple-value-bind
(method-lambda method-initargs)
(make-method-lambda
gf (class-prototype (generic-function-method-class gf))
lambda-expression environment)
(apply #'make-instance
(generic-function-method-class gf)
:function (enclose method-lambda environment)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:lambda-list lambda-list
:qualifiers qualifiers
:specializers specializers
method-initargs))))
(add-method gf method))
Well, it doesn't exactly specify it like that - it doesn't mention
ENCLOSE - but that's the idea AFAICS. Without ENCLOSE I think one can
only programmatically create top-level methods. With ENCLOSE, I could
just use it and forget about the issue.
Of course, Allegro doesn't provide make-method-lambda, but that's
another topic. I think an implementation-independent proposal for
environment objects access should include ENCLOSE. Or am I missing
something?
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
From: Duane Rettig
Subject: Re: enclose, was Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4is97u4vv.fsf@franz.com>
Pascal Costanza <········@web.de> writes:
> Duane Rettig wrote:
>
> > So the question is: is there an application where enclose might be
> > useful in user-code? I'm happy to entertain possibilities.
>
> I know about one specific case. The MOP specifies that programmatic
> creation of methods should be done like this:
>
>
> (let ((method
> (multiple-value-bind
> (method-lambda method-initargs)
> (make-method-lambda
> gf (class-prototype (generic-function-method-class gf))
> lambda-expression environment)
> (apply #'make-instance
> (generic-function-method-class gf)
> :function (enclose method-lambda environment)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> :lambda-list lambda-list
> :qualifiers qualifiers
> :specializers specializers
> method-initargs))))
> (add-method gf method))
>
> Well, it doesn't exactly specify it like that - it doesn't mention
> ENCLOSE - but that's the idea AFAICS. Without ENCLOSE I think one can
> only programmatically create top-level methods. With ENCLOSE, I could
> just use it and forget about the issue.
Note that the above "enclosing" can be done trivially by doing a walk
and coercing the result to function.
It is true that an implementation of enclose might do the job required
here, but note for example that enclose doesn't allow you to specify a
walk-function, and thus it doesn't allow you to handle special cases
that might match what your implementation does with method forms
(consider, for example, optimizations on slot-value, or optimizations
on next-method-p or call-next-method, depending on whether they are used
first-class in the method-function body). Since enclose cannot see these
implementation dependent optimizations, any version of enclose that we
supply would likely not be used at all, unless the interface were
expanded to allow for such specialized uses in general.
If you think of enclose as a walker, it leaves a lot to be desired in
terms of generality.
I have not yet started to consider making walker interfaces general,
and am not opposed to adding such general interfaces to our environments
module in the future, but enclose, as it stands described in CLtL2, does
not have a general enough interface for me to consider it to be useful.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Duane Rettig wrote:
> Pascal Costanza <········@web.de> writes:
>
>>Duane Rettig wrote:
>>
>>>So the question is: is there an application where enclose might be
>>>useful in user-code? I'm happy to entertain possibilities.
>>
>>I know about one specific case. The MOP specifies that programmatic
>>creation of methods should be done like this:
>>
>>(let ((method
>> (multiple-value-bind
>> (method-lambda method-initargs)
>> (make-method-lambda
>> gf (class-prototype (generic-function-method-class gf))
>> lambda-expression environment)
>> (apply #'make-instance
>> (generic-function-method-class gf)
>> :function (enclose method-lambda environment)
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> :lambda-list lambda-list
>> :qualifiers qualifiers
>> :specializers specializers
>> method-initargs))))
>> (add-method gf method))
>>
>>Well, it doesn't exactly specify it like that - it doesn't mention
>>ENCLOSE - but that's the idea AFAICS. Without ENCLOSE I think one can
>>only programmatically create top-level methods. With ENCLOSE, I could
>>just use it and forget about the issue.
>
> Note that the above "enclosing" can be done trivially by doing a walk
> and coercing the result to function.
OK, if you say so. ;) I am not very familiar with walkers, and I can't
imagine how I could transform an s-expression into an equivalent one
that is enclosed in a given enironment. I can imagine how to check
whether something is bound or free in an s-expression and of course, one
can use variable-information and the likes to detect whether they are
defined in an environment, but then what would I do with such an entity?
The documentation mentions "locative conses", and I guess they are
relevant here, but it's not clear how to use them...
> It is true that an implementation of enclose might do the job required
> here, but note for example that enclose doesn't allow you to specify a
> walk-function, and thus it doesn't allow you to handle special cases
> that might match what your implementation does with method forms
> (consider, for example, optimizations on slot-value, or optimizations
> on next-method-p or call-next-method, depending on whether they are used
> first-class in the method-function body). Since enclose cannot see these
> implementation dependent optimizations, any version of enclose that we
> supply would likely not be used at all, unless the interface were
> expanded to allow for such specialized uses in general.
OK, I see.
> If you think of enclose as a walker, it leaves a lot to be desired in
> terms of generality.
...but as a vendor you could provide enclose as a convenience function
so that we "ordinary" programmers don't need to think about specific
walkers, right? Or would that still be too limiting?
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
From: Duane Rettig
Subject: Re: enclose, was Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4zn2jmvhh.fsf@franz.com>
Pascal Costanza <········@web.de> writes:
> Duane Rettig wrote:
> > Pascal Costanza <········@web.de> writes:
> >
>
> >>Duane Rettig wrote:
> >>
> >>>So the question is: is there an application where enclose might be
> >>>useful in user-code? I'm happy to entertain possibilities.
> >>
> >>I know about one specific case. The MOP specifies that programmatic
> >>creation of methods should be done like this:
> >>
> >>(let ((method
> >> (multiple-value-bind
> >> (method-lambda method-initargs)
> >> (make-method-lambda
> >> gf (class-prototype (generic-function-method-class gf))
> >> lambda-expression environment)
> >> (apply #'make-instance
> >> (generic-function-method-class gf)
> >> :function (enclose method-lambda environment)
> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> :lambda-list lambda-list
> >> :qualifiers qualifiers
> >> :specializers specializers
> >> method-initargs))))
> >> (add-method gf method))
> >>
> >>Well, it doesn't exactly specify it like that - it doesn't mention
> >>ENCLOSE - but that's the idea AFAICS. Without ENCLOSE I think one can
> >>only programmatically create top-level methods. With ENCLOSE, I could
> >>just use it and forget about the issue.
> > Note that the above "enclosing" can be done trivially by doing a walk
>
> > and coercing the result to function.
>
> OK, if you say so. ;) I am not very familiar with walkers, and I can't
> imagine how I could transform an s-expression into an equivalent one
> that is enclosed in a given enironment.
Note that enclose doesn't return an s-expression, but a function.
> I can imagine how to check
> whether something is bound or free in an s-expression and of course,
> one can use variable-information and the likes to detect whether they
> are defined in an environment, but then what would I do with such an
> entity?
Precisely. Note that the description of enclose explicitly excludes as
undefined an expression which contains references to variable or function
bindings lexically visible in the environment. So what _would_ you do
with such an entity? What should a portable enclose do with such an
entity? Assuming that the CLtL2 definition represents a spec for considering
"conformance" (though not the Ansi spec), a "conforming" implementation of
enclose could signal an error, ignore the definitions (probably resulting
in unbound variable errors at runtime) or try to encapsulate the binding in
a closure. All would be "conforming", and any definition our environments
access definition might provide would stand a 2/3 chance of being wrong for
an implementation which uses our environments-access module.
> The documentation mentions "locative conses", and I guess they are
> relevant here, but it's not clear how to use them...
The CLtL2 environments access system deals mainly with "presence" of
lexical bindings in an environment; it provides no actual bindings to
values or metavalues (i.e. structures which describe the values).
Also, CLtL2 environments don't allow mutation of environment structure
once created by augment-environment. There are good reasons for this
latter restriction, and our environments-access module maintains that
rule _except_ for these locative cells (implemented as conses simply
because they are the most portable smallest-object available). The
idea is that in a lexical context, either in a compiler or in an
interpreter, a name not only exists in the environment with certain
immutable qualities, but is "bound" environmentally to one of these
locatives, whose car (usually) is defined, used, and modified by the
implementation as it sees fit.
In the case of a macro or symbol-macro, the car of the locative would
hold the actual definition for the macro/symbol-macro, so that
macroexpansion might be performed or even simulated.
For example:
CL-USER(2): (setq sys:*compile-file-environment* (sys:make-compile-file-environment))
#<Augmentable COMPILATION environment @ #x10b839f2>
CL-USER(3): (defmacro foo (x) `(bar ,x))
FOO
CL-USER(4): (sys:function-information 'foo sys::*compile-file-environment* t)
:MACRO
(#<Interpreted Function FOO @ #x10b848a2>)
NIL
CL-USER(5):
> > It is true that an implementation of enclose might do the job required
> > here, but note for example that enclose doesn't allow you to specify a
> > walk-function, and thus it doesn't allow you to handle special cases
> > that might match what your implementation does with method forms
> > (consider, for example, optimizations on slot-value, or optimizations
> > on next-method-p or call-next-method, depending on whether they are used
> > first-class in the method-function body). Since enclose cannot see these
> > implementation dependent optimizations, any version of enclose that we
> > supply would likely not be used at all, unless the interface were
> > expanded to allow for such specialized uses in general.
>
> OK, I see.
>
> > If you think of enclose as a walker, it leaves a lot to be desired in
> > terms of generality.
>
> ...but as a vendor you could provide enclose as a convenience function
> so that we "ordinary" programmers don't need to think about specific
> walkers, right? Or would that still be too limiting?
I'd say if you're going to use enclose, you're not an "ordinary"
programmer :-)
Besides the question of how to handle undefined situations, described
earlier, you would be disappointed with an enclose implementation that
does not provide the same optimizations as, say, a pure defmethod
form. So, for example, if you define an application which makes
extensive use of your enclose/add-method style in your favorite
implementation, and as your favorite implementation ages and optimizes
more method bodies, your own enclose/add-method style will be left
behind, with the rub being that you won't know about it unless they
actually _tell_ you that they have optimized certain kinds of method
forms and you specifically look for and don't find that optimization.
No, an automatic walker which tracks such implementational changes would
at least need hooks into the implementation's walker, to stay up-to-date
with the latest optimizations.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Dirk Gerrits
Subject: Re: enclose, was Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87u0sqzdsx.fsf@dirkgerrits.com>
Duane Rettig <·····@franz.com> writes:
> I'd say if you're going to use enclose, you're not an "ordinary"
> programmer :-)
What is an "out of the ordinary" programmer but tomorrow's "ordinary"
one?
or
One person's "out of the ordinary" is another person's "ordinary".
:)
Perhaps more people would do more interesting things with environments
and code walkers if they had better tools to do so, I don't know. I do
know, that I never did anything interesting with multiple dispatch and
multiple inheritance in C++. ;)
Kind regards,
Dirk Gerrits
Duane Rettig wrote:
> Pascal Costanza <········@web.de> writes:
>
>>OK, if you say so. ;) I am not very familiar with walkers, and I can't
>>imagine how I could transform an s-expression into an equivalent one
>>that is enclosed in a given enironment.
>
> Note that enclose doesn't return an s-expression, but a function.
Yes, of course.
>>I can imagine how to check
>>whether something is bound or free in an s-expression and of course,
>>one can use variable-information and the likes to detect whether they
>>are defined in an environment, but then what would I do with such an
>>entity?
>
> Precisely. Note that the description of enclose explicitly excludes as
> undefined an expression which contains references to variable or function
> bindings lexically visible in the environment.
Damn, I have missed that part of the enclose's description.
> So what _would_ you do
> with such an entity? What should a portable enclose do with such an
> entity? Assuming that the CLtL2 definition represents a spec for considering
> "conformance" (though not the Ansi spec), a "conforming" implementation of
> enclose could signal an error, ignore the definitions (probably resulting
> in unbound variable errors at runtime) or try to encapsulate the binding in
> a closure. All would be "conforming", and any definition our environments
> access definition might provide would stand a 2/3 chance of being wrong for
> an implementation which uses our environments-access module.
OK, I get it.
>>...but as a vendor you could provide enclose as a convenience function
>>so that we "ordinary" programmers don't need to think about specific
>>walkers, right? Or would that still be too limiting?
>
> I'd say if you're going to use enclose, you're not an "ordinary"
> programmer :-)
Sometimes, I feel very ordinary... ;)
Thanks a lot for the clarifications indeed. I have very obviously
overestimated the power and usefulness of an enclose operator.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza wrote:
> I know about one specific case. The MOP specifies that programmatic
> creation of methods should be done like this:
>
> (let ((method
> (multiple-value-bind
> (method-lambda method-initargs)
> (make-method-lambda
> gf (class-prototype (generic-function-method-class gf))
> lambda-expression environment)
> (apply #'make-instance
> (generic-function-method-class gf)
> :function (enclose method-lambda environment)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> :lambda-list lambda-list
> :qualifiers qualifiers
> :specializers specializers
> method-initargs))))
> (add-method gf method))
Duane asked me to look at this thread today. I think the above statement
about enclose and the MOP is incorrect. Not only is enclose _unnecessary_
for method creation, it isn't even _useful_.
There are two kinds of circumstances in which method functions need be
created. make-method-lambda is necessary in both situations since it
is the only portable way to interface to the implementation's
conventions for calling method functions.
One situation is in the processing of the defmethod macro, whether
during file compilation or during execution. That is discussed in
the MOP specification here:
http://www.franz.com/support/documentation/6.2/doc/mop/concepts.html#defmethod
The other situation is at execution time when the application wants
to construct a method and add it to a gf. It is shown here:
http://www.franz.com/support/documentation/6.2/doc/mop/concepts.html#init-generic
It is necessary to examine these situation separately.
In the processing of defmethod, the description in the MOP is explicit that:
"In the second step, the method lambda is converted to a function which
properly captures the lexical scope of the macro form. This is done by having
the method lambda appear in the macro-expansion as the argument of the
function special form. During the subsequent evaluation of the macro-expansion,
the result of the function special form is the method function."
In other words, the capturing of the surrounding environment (both macro and
lexical function/variable bindings) is done in the normal way by the function
special form. So why does make-method-lambda receive an environment argument?
Because some make-method-lambda methods may want to walk the method body
(perhaps, for example, to see whether there are any calls to call-next-method)
and it is impossible to walk the body without knowing macros and symbol macros
defined in the environment. But whether make-method-lambda returns the original
function body, or returns the macroexpanded body, the capturing of the lexical
environment (macros, lexical variables, and lexical functions) is left to the
normal operation of the function special form. Enclose is not necessary, and
in any case wouldn't work at all at compile-file time! (Enclose returns an
actual function object, and functions are not portably-externalizable objects
in the file compiler. See ANS 3.2.4.2.2.)
In the other kind of situation, the run-time creation of a method, observe in
the #init-generic link above the execution-time code passes nil as the
make-method-lambda environment argument:
(let* ((gf (make-instance 'standard-generic-function
:lambda-list '(p l &optional visiblyp &key)))
(method-class (generic-function-method-class gf)))
(multiple-value-bind (lambda initargs)
(make-method-lambda
gf
(class-prototype method-class)
'(lambda (p l &optional (visiblyp t) &key color)
(set-to-origin p)
(when visiblyp (show-move p 0 color)))
>>> nil)
(add-method gf
(apply #'make-instance method-class
:function (compile nil lambda)
:specializers (list (find-class 'position)
(intern-eql-specializer 0))
:qualifiers ()
:lambda-list '(p l &optional (visiblyp t)
&key color)
initargs))))
What _else_ could it pass? There are portably no environment objects around
at execution time; there is no way portably to externalize on from compile-file
time; and in any case the only thing enclose (if it had survived in the ANS)
could portably extract from an environment is the macro/symbol-macro/declaration
info.
> Well, it doesn't exactly specify it like that - it doesn't mention
> ENCLOSE - but that's the idea AFAICS. Without ENCLOSE I think one can
> only programmatically create top-level methods. With ENCLOSE, I could
> just use it and forget about the issue.
Even _with_ enclose one can only programmatically create top-level methods.
Observe that enclose similarly does not allow creation even of non-top-level
_functions_ at execution time. The ANS is careful to draw a line what can
be done in the execution-time promotion of lambda forms to function objects.
Promotion to function never portably allows capturing of lexical variable or
function bindings. If the CLtL2 enclose existed, it would not change that
truth.
Steven M. Haflich wrote:
> Duane asked me to look at this thread today. I think the above statement
> about enclose and the MOP is incorrect. Not only is enclose _unnecessary_
> for method creation, it isn't even _useful_.
Yes, I see that by now. Thanks for making it very clear.
> Even _with_ enclose one can only programmatically create top-level
> methods. Observe that enclose similarly does not allow creation even
> of non-top-level _functions_ at execution time. The ANS is careful
> to draw a line what can be done in the execution-time promotion of
> lambda forms to function objects. Promotion to function never
> portably allows capturing of lexical variable or function bindings.
> If the CLtL2 enclose existed, it would not change that truth.
In the Closette code in AMOP, there is the following comment:
> ;;; Run-time environment hacking (Common Lisp ain't got 'em).
>
> (defun top-level-environment () nil) ; Bogus top level lexical environment
>
> (defvar compile-methods nil) ; by default, run everything interpreted
>
> (defun compile-in-lexical-environment (env lambda-expr)
> (declare (ignore env))
> (if compile-methods
> (compile nil lambda-expr)
> (eval `(function ,lambda-expr))))
The "ain't got 'em" remark seems to indicate that there are Lisp
dialects that have first-class environments. Is there a specific
technical reason why Common Lisp doesn't have them?
It's pretty straightforward to include first-class environments in an
interpreter. Does this have to do with restrictions caused by a compiled
implementation?
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza wrote:
> The "ain't got 'em" remark seems to indicate that there are Lisp
> dialects that have first-class environments. Is there a specific
> technical reason why Common Lisp doesn't have them?
We (X3J13) considered environments to be an important feature in
the language -- without them, you can't easily write walkers which
are compatible and interoperate with the other walkers found in every
implementation (the compiler itself being the most important example).
So IIRC Sandra Loosemore and maybe a couple others drafted the
environment stuff that is captured in CLtL2, but afterwards it became
clear that it had been too ambitious to try to construct this
specification in toto with no experience with either its implementation
or application. It was clear that there were mistakes and any number
missing capabilities. If there had been another year or two to work,
probably the committee would have undertaken to revise and fix it, but
by that time we were feeling time pressure and no one thought we could
get it right in the time remaining. Rather than codify something so
broken, we left it for future work.
> It's pretty straightforward to include first-class environments in an
> interpreter. Does this have to do with restrictions caused by a compiled
> implementation?
There are a couple reasons why the language definition would not want
to require a version of enclose powerful enough to allow the
functionified lambda to access lexical variables and functions in the
environment. Handling macros (macros, compiler macros, and symbol
macros) is easy because they disappear after macroexpand time, but
the compiler needs to make implementation decisions about variables
and functions. A top-level lexical form is entirely analyzed all at
once in CL semantics, and thereafter that analysis cannot change.
Here is a sketch how life would be if this were not so:
(defmacro defunny (name (&rest lambda-list) &body body &environment env)
`(progn (defun ,(intern (symbol-name name) :defunny) (lambda-form)
(enclose lambda-form ',env))
(defun ,name ,lambda-list ,@body)))
The defunny defining macro is just like defun but also defines a
function capable of adding new closures to the surrounding environment.
It might be used this way:
(let ((today "Wednesday")
(month "October"))
(defunny what-day-is-today? ()
today))
Now, a real CL compiler might observe that the value of month is unused
and eliminate it, and that the closed-over variable today cannot be
changed and can be captured as a constant inside the function. But
the defunny maro allows me to violate these observations later, at
execution time:
(funcall (defunny::what-day-is-today '(lambda (x)
(setq today x)
month))
"Thursday")
It would not be impossible to make this kind of thing work, but
compilers would be prevented from doing closure analysis. Most
closures are null closures, and even the ones that aren't null
closures access only a few fragments of the surrounding
environment. Real compilers do more value-flow analysis than
they play with acutal named variables.
Gack -- The sketched code above is so ugly in concept I'm almost
sorry for posting it :-).
So, the reason for not requiring enclose (and compile and coerce)
from adding code to an environment is primarily an efficiency
issue -- it keeps the language consonant with the tradition of
compiled lisp code.
Steven M. Haflich wrote:
> There are a couple reasons why the language definition would not want
> to require a version of enclose powerful enough to allow the
> functionified lambda to access lexical variables and functions in the
> environment. Handling macros (macros, compiler macros, and symbol
> macros) is easy because they disappear after macroexpand time, but
> the compiler needs to make implementation decisions about variables
> and functions. A top-level lexical form is entirely analyzed all at
> once in CL semantics, and thereafter that analysis cannot change.
OK, I see. So what about the following idea: From the examples I have
seen by now, it seems to me that the problem lies in the "automagic"
capture of lexical environments so that optimizations become unreliable.
Instead, Common Lisp could required to explicitly capture environments.
This would be an operator that returns an environment object at runtime,
so one could say
(let ((env (capture-lexical-environment)))
...
(get-variable 'x env)
...
(setf (get-variable 'x env) ...))
Of course, this means that environments that are captured cannot be
optimized anymore. But this is a local effect, so it seems ok...
> Here is a sketch how life would be if this were not so:
>
> (defmacro defunny (name (&rest lambda-list) &body body &environment env)
> `(progn (defun ,(intern (symbol-name name) :defunny) (lambda-form)
> (enclose lambda-form ',env))
> (defun ,name ,lambda-list ,@body)))
This would look as follows:
(defmacro defunny (name (&rest lambda-list) &body body)
`(progn (defun ,(intern (symbol-name name) :defunny) (lambda-form)
(enclose lambda-form (capture-lexical-environment)))
(defun ,name ,lambda-list ,@body)))
Programmatic method creation would look like this:
(let* ((method-class (generic-function-method-class some-gf))
(current-environment (capture-lexical-environment)))
(multiple-value-bind (lambda initargs)
(make-method-lambda
some-gf (class-prototype method-class)
'(lambda (...) ...)
current-environment)
(add-method
some-gf (apply #'make-instance method-class
:function (enclose lambda current-environment)
:specializers (...)
:qualifiers (...)
:lambda-list (...)
initargs))))
The presence of capture-lexical-environment could be detected by the
compiler. There is a possible danger of macros expanding into calls of
capture-lexical-environment, though, which could slow down code with no
apparent reason.
> So, the reason for not requiring enclose (and compile and coerce)
> from adding code to an environment is primarily an efficiency
> issue -- it keeps the language consonant with the tradition of
> compiled lisp code.
Yep, that's clear by now. Thanks a lot for all the helpful background
information!
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
Pascal Costanza wrote:
> Steven M. Haflich wrote:
>
>> There are a couple reasons why the language definition would not want
>> to require a version of enclose powerful enough to allow the
>> functionified lambda to access lexical variables and functions in the
>> environment. Handling macros (macros, compiler macros, and symbol
>> macros) is easy because they disappear after macroexpand time, but
>> the compiler needs to make implementation decisions about variables
>> and functions. A top-level lexical form is entirely analyzed all at
>> once in CL semantics, and thereafter that analysis cannot change.
>
> OK, I see. So what about the following idea: From the examples I have
> seen by now, it seems to me that the problem lies in the "automagic"
> capture of lexical environments so that optimizations become unreliable.
> Instead, Common Lisp could required to explicitly capture environments.
> This would be an operator that returns an environment object at runtime,
> so one could say
>
> (let ((env (capture-lexical-environment)))
> ...
> (get-variable 'x env)
> ...
> (setf (get-variable 'x env) ...))
>
> Of course, this means that environments that are captured cannot be
> optimized anymore. But this is a local effect, so it seems ok...
...but still too gross, after having thought about it. The next step
would be to limit a captured lexical environment to names that you are
actually interested in, but that's already possible:
(lambda (name)
(case name
(x x)
(y y)
...))
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
From: Edi Weitz
Subject: Re: enclose, was Re: Sacla loop: a new loop implementation
Date:
Message-ID: <ur7nwrkui.fsf@agharta.de>
On 18 Oct 2004 10:03:46 -0700, Duane Rettig <·····@franz.com> wrote:
> So the question is: is there an application where enclose might be
> useful in user-code? I'm happy to entertain possibilities.
For my "RDNZL" .NET interface I'm creating new generic functions and
methods at runtime. It looks like for the most general case you'll
need ENCLOSE - see the thread on gmane.org starting with this message:
<http://article.gmane.org/gmane.lisp.lispworks.general/2757>
It turned out that for my specific needs this wasn't needed but at
least I think that might be an application for ENCLOSE. Note that I
didn't know about ENCLOSE and MAKE-METHOD-LAMBDA until I asked on the
mailing list and Pascal Costanza gave this advice.
Cheers,
Edi.
OT: Is there a way to link to a whole thread on Gmane? I really hate
their web interface. It's almost impossible to find a message
until you know in advance where and when it was posted. And if
you've finally found something you can't provide proper links. Or
maybe I'm just too stupid to use it.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
Hi Edi Weitz,
> <http://article.gmane.org/gmane.lisp.lispworks.general/2757>
...
> OT: Is there a way to link to a whole thread on Gmane?
Click on the message's subject link or replace "article" in the root URI
with "thread":
<http://thread.gmane.org/gmane.lisp.lispworks.general/2757>
Regards,
Adam
"Tim Bradshaw" <··········@tfeb.org> a �crit dans le message de
····························@f14g2000cwb.googlegroups.com...
> And you are right! This is why I keep going on about `mild
> nonportability' and so on. I *know* these things can't be done
> portably in common lisp, because the facilities just aren't there. It
> may well be that no current implementations can do them either.
> However progress is possible.
I have the cltl2 in front of me and there is a section 8.5 Environments
describing some functions to help. For example : the function
"variable-information variable & optional env". With it you can have access
to type information.
I have seen no trace of this in hyperspec ? Someone knows what happened to
these functions ?
--
___________________________________________________________
Christophe Turle.
http://perso.wanadoo.fr/turle/lisp/utilities.html
(format nil ···@~a.~a" 'c.turle 'wanadoo 'fr)
From: Duane Rettig
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4vfd8t524.fsf@franz.com>
"Christophe Turle" <······@nospam.com> writes:
> "Tim Bradshaw" <··········@tfeb.org> a �crit dans le message de
> ····························@f14g2000cwb.googlegroups.com...
> > And you are right! This is why I keep going on about `mild
> > nonportability' and so on. I *know* these things can't be done
> > portably in common lisp, because the facilities just aren't there. It
> > may well be that no current implementations can do them either.
> > However progress is possible.
>
>
> I have the cltl2 in front of me and there is a section 8.5 Environments
> describing some functions to help. For example : the function
> "variable-information variable & optional env". With it you can have access
> to type information.
Yes, it was potentially a very useful interface, but one which had
absolutely no implementations at the time.
> I have seen no trace of this in hyperspec ? Someone knows what happened to
> these functions ?
As I understand it, there were continuing semantic problems with it
as the X3J13 meetings progressed, and the committee decided that it
wasn't a good idea to force such an interface on the CL vendors.
The CL philosophy was to incorporate tried and true practice, not
to invent new practice, and this was definitely new practice...
Every CL implementation has environments, and some even have
rudimentary versions of augment-environment, variable-information,
and function information, at least, for interface functionality to
their environments implementations. And there have been various
attempts to resurrect this functionality, but none have gained
wide acceptance. The latest attempt is by us, at Franz Inc., and
we think we have one that will do a good job. Sometime after our 7.0
release, we intend to make the module opensource, so other vendors
can use it if they wish. This of course does not guarantee success,
but I think it will increase the odds, and meanwhile Alelgro CL users
can use it.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Christophe Turle wrote:
> "Tim Bradshaw" <··········@tfeb.org> a �crit dans le message de
> ····························@f14g2000cwb.googlegroups.com...
>
>>And you are right! This is why I keep going on about `mild
>>nonportability' and so on. I *know* these things can't be done
>>portably in common lisp, because the facilities just aren't there. It
>>may well be that no current implementations can do them either.
>>However progress is possible.
>
>
>
> I have the cltl2 in front of me and there is a section 8.5 Environments
> describing some functions to help. For example : the function
> "variable-information variable & optional env". With it you can have access
> to type information.
>
> I have seen no trace of this in hyperspec ? Someone knows what happened to
> these functions ?
They did not make it in the ANS. Allegro has an expanded interface for
these functions. LW does as well. I bet other implementations do too.
--
Marco
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87k6tof4i2.fsf@evaluator.pt>
"Tim Bradshaw" <··········@tfeb.org> writes:
> Peter Seibel wrote:
>>
>> So maybe I'm just being thick but I don't see any in principle reason
>> why a Common Lisp implementation that has a type inferencer couldn't
>> use it to do type inference before macro expansion in order to supply
>> a useful TYPE-OF-EXPRESSION function.
>
>> [...] But I don't see that an implementation would become become "a
>> statically typed Common Lisp" if it ran its type inferencer before
>> macro expansion as well as after. That's all I'm saying.
>>
> And you are right! This is why I keep going on about `mild
> nonportability' and so on. I *know* these things can't be done
> portably in common lisp, because the facilities just aren't there. It
> may well be that no current implementations can do them either.
> However progress is possible.
You are forgetting a fundamental issue. Macro expansion is a
compile-time operation. If you create a Lisp that allows macros that
require type information to compute the expansion then that
information must be produced at compile time. And if you can produce
useful type information at compile time, then you are working with a
statically typed language. That's as simple as that.
Of course, if you say that you are willing to accept that your macros
might need to expand without useful type information available, I
seriously doubt that they will be any useful. And if your macros
_demand_ useful type information then you will have to declare type
information on the code that uses them.
I do think there's a way out, but it is not using macros that are
expanded at compile time. My idea is that you can solve your
performance problems just by stressing your code in every possible way
and then applying a recompilation process that can explore the
statistical information that it gathered about the types of the
function arguments and function results to generate more efficient
code (obviously, using the type inference process again but now with
more refined type information). Note that you would not need to
expand macros again as type inference generally works with an
intermediate form where macro calls no longer exist. Obviously, the
new optimized forms could coexist with the unoptimized versions and
the most appropriate version would be chosen at runtime. That's what
I would like to see in modern Common Lisp implementations.
> This underlying discussion that seems to be going on here seems to be
> pretty much the classic `one-bit mind' problem described a couple of
> years ago by Erik Naggum. To be able to provide some useful type
> information to macros, CL does not have to become completely
> statically typed, and the assumption that it does is, frankly, silly.
What do you mean by "completely statically typed"? I ask because I
never talked about that. I talked about a statically typed language
which is a language where you can get type information at compile
time.
> What I want is for progress to happen. Say Allegro CL's implementors
> do some clever stuff which allows macros to ask for, and sometimes
> get, type and other information which has been derived by the system.
> Suddenly people using Allegro can write all sorts of clever macros,
> although those macros will have implementation-specific parts. So in
> the standard open-source way, the implementors of some open-source
> Lisp borrow the Allegro API, soup it up a bit and implement a version
> which provides somewhat better than Allegro's. In a few years there's
> agreement on what this API should look like, and it becomes a
> (sub-)standard.
What you call "progress" is just making Common Lisp a more statically
typed language. That's not progress, IMHO.
> Incidentally, here's a sketch of how macros might be given type
> information that I thought of. This isn't meant to be even plausible:
> there are a load of details just omitted. The problem is that type
> inference generally happens (and can best be done) *after* macro
> expansion. Well, OK, so do it after macro expansion then:
>
> 1. Macroexpand everything. Macros that want to make type enquiries
> get little or no information.
> 2. Do some type inference &c
> 3. `Decorate' the source with the inferences made.
> 4. Macroexpand *again*. Macros that want to make type enquiries
> can now get much better information.
>
> OK, so what's the problem with this? It's fine for macros to get
> expanded multiple times, of course, but if they expand to different
> things when they have better type information then some of the
> inference that was done could now be invalid. Well, there are at
> least two answers to that:
>
> 1. Part of a type-information-using macro's contract is that it
> must behave monotonically (?) with respect to type information.
> If it is given information I1, and then later I2 which is a
> refinement of I1 (which means something like all the types in I2
> are subtypes of those in I1), then any type information it
> provides must also be a refinement, so anything else which is
> worrying about type information can use the initial information.
> 2. Alternatively the system will notice if macros have different
> expansions, and will invalidate type information for things in
> the body of the macro in that case. For added value (and longer
> compile times!) it could then redo the inference phase and
> iterate.
>
> I'm sure that this sketch won't work (so don't bother picking
> elaborate holes in it). But something like it probably could work.
> And, as I said, progress is possible: I want to see implementations
> competing on this kind of stuff.
(with-shameless-plug
If you want to see implementations competing on this kind of stuff
you can take a look at Linj. It already has something very similar
to what you ask.
In Linj, a macro-expander (but not the traditional Common Lisp macro
expander) can ask about the type of its arguments and the type of
any expression that it decides to build using those arguments. You
compute the macro expansion according to those types, just like you
have been asking all this time.
But these "macro" _require_ type information. Either you provide it
explicitly, or you will depend on the type inferencer that requires
that every reachable point must be properly typed, thus requiring
that you must have the complete program available. Not very Common
Lispy!
)
Ant�nio Leit�o.
From: Duane Rettig
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4mzykszvy.fsf@franz.com>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> "Tim Bradshaw" <··········@tfeb.org> writes:
>
> > Peter Seibel wrote:
> >>
> >> So maybe I'm just being thick but I don't see any in principle reason
> >> why a Common Lisp implementation that has a type inferencer couldn't
> >> use it to do type inference before macro expansion in order to supply
> >> a useful TYPE-OF-EXPRESSION function.
> >
> >> [...] But I don't see that an implementation would become become "a
> >> statically typed Common Lisp" if it ran its type inferencer before
> >> macro expansion as well as after. That's all I'm saying.
> >>
> > And you are right! This is why I keep going on about `mild
> > nonportability' and so on. I *know* these things can't be done
> > portably in common lisp, because the facilities just aren't there. It
> > may well be that no current implementations can do them either.
> > However progress is possible.
>
> You are forgetting a fundamental issue. Macro expansion is a
> compile-time operation.
No, macro expansion is a macroexpand-time operation, which occurs at
compile time _or_ eval time.
> If you create a Lisp that allows macros that
> require type information to compute the expansion then that
> information must be produced at compile time.
With certain exceptions (e.g. special) Common Lisp never requires
type information. Declarations can in general be used or ignored.
And if you can produce
> useful type information at compile time, then you are working with a
> statically typed language. That's as simple as that.
Einstein said "make things as simple as possible, and no simpler".
You've gone beyond the second portion of his maxim.
> Of course, if you say that you are willing to accept that your macros
> might need to expand without useful type information available, I
> seriously doubt that they will be any useful. And if your macros
> _demand_ useful type information then you will have to declare type
> information on the code that uses them.
A macro _must_ either return an expansion, reject the expansion (by
returning &whole) or signal error. All three of these cases can be
results of lack of sufficient type information, based on what the
macro is looking for. A macro that generates incorrect code is
simply wrong, but given a correct macro (in this sense), the returned
code might be as dynamic as the macro is programmed to produce
for the information it receives. "Pure static" and "pure dynamic"
are at extreme ends of the scale; the more likely scenario is something
in the middle.
> I do think there's a way out, but it is not using macros that are
> expanded at compile time. My idea is that you can solve your
> performance problems just by stressing your code in every possible way
> and then applying a recompilation process that can explore the
> statistical information that it gathered about the types of the
> function arguments and function results to generate more efficient
> code (obviously, using the type inference process again but now with
> more refined type information). Note that you would not need to
> expand macros again as type inference generally works with an
> intermediate form where macro calls no longer exist. Obviously, the
> new optimized forms could coexist with the unoptimized versions and
> the most appropriate version would be chosen at runtime. That's what
> I would like to see in modern Common Lisp implementations.
What you're really talking about here is the statification of your
code, not the language. CL has always had the capability to compile
into static code.
> > This underlying discussion that seems to be going on here seems to be
> > pretty much the classic `one-bit mind' problem described a couple of
> > years ago by Erik Naggum. To be able to provide some useful type
> > information to macros, CL does not have to become completely
> > statically typed, and the assumption that it does is, frankly, silly.
>
> What do you mean by "completely statically typed"? I ask because I
> never talked about that. I talked about a statically typed language
> which is a language where you can get type information at compile
> time.
Well, you describe CL, because CL can provide type information at compile
time. Does that make CL statically typed? By your definition, yes. Does
that make it not dynamically typed? Not at all.
> > What I want is for progress to happen. Say Allegro CL's implementors
> > do some clever stuff which allows macros to ask for, and sometimes
> > get, type and other information which has been derived by the system.
> > Suddenly people using Allegro can write all sorts of clever macros,
> > although those macros will have implementation-specific parts. So in
> > the standard open-source way, the implementors of some open-source
> > Lisp borrow the Allegro API, soup it up a bit and implement a version
> > which provides somewhat better than Allegro's. In a few years there's
> > agreement on what this API should look like, and it becomes a
> > (sub-)standard.
>
> What you call "progress" is just making Common Lisp a more statically
> typed language. That's not progress, IMHO.
No, it only allows for more statically typed _code_ portably. The
language doesn't change at all, and is just as static and dynamic
as it always has been.
I think you're making the mistake of "identifying" CL as having static
or dynamic typing, just as people incorrectly misidentify the Functional
nature of CL; CL supports FP, but does not enforce it, which to some
people means that it can't be functional. But the CL philosophy is
not to _be_ any particular thing, but instead to _support_ as many
things as possible. Including dynamic typing. Including static typing.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <874qkrisnx.fsf@evaluator.pt>
Duane Rettig <·····@franz.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> "Tim Bradshaw" <··········@tfeb.org> writes:
>>
>> > Peter Seibel wrote:
>> >>
>> >> So maybe I'm just being thick but I don't see any in principle reason
>> >> why a Common Lisp implementation that has a type inferencer couldn't
>> >> use it to do type inference before macro expansion in order to supply
>> >> a useful TYPE-OF-EXPRESSION function.
>> >
>> >> [...] But I don't see that an implementation would become become "a
>> >> statically typed Common Lisp" if it ran its type inferencer before
>> >> macro expansion as well as after. That's all I'm saying.
>> >>
>> > And you are right! This is why I keep going on about `mild
>> > nonportability' and so on. I *know* these things can't be done
>> > portably in common lisp, because the facilities just aren't there. It
>> > may well be that no current implementations can do them either.
>> > However progress is possible.
>>
>> You are forgetting a fundamental issue. Macro expansion is a
>> compile-time operation.
>
> No, macro expansion is a macroexpand-time operation, which occurs at
> compile time _or_ eval time.
I'm more concerned with the compile time.
>> If you create a Lisp that allows macros that
>> require type information to compute the expansion then that
>> information must be produced at compile time.
>
> With certain exceptions (e.g. special) Common Lisp never requires
> type information. Declarations can in general be used or ignored.
>
> And if you can produce
>> useful type information at compile time, then you are working with a
>> statically typed language. That's as simple as that.
>
> Einstein said "make things as simple as possible, and no simpler".
> You've gone beyond the second portion of his maxim.
>
>> Of course, if you say that you are willing to accept that your macros
>> might need to expand without useful type information available, I
>> seriously doubt that they will be any useful. And if your macros
>> _demand_ useful type information then you will have to declare type
>> information on the code that uses them.
>
> A macro _must_ either return an expansion, reject the expansion (by
> returning &whole) or signal error.
I presume you are talking about compiler macros and not ordinary
macros.
> All three of these cases can be results of lack of sufficient type
> information, based on what the macro is looking for. A macro that
> generates incorrect code is simply wrong, but given a correct macro
> (in this sense), the returned code might be as dynamic as the macro
> is programmed to produce for the information it receives. "Pure
> static" and "pure dynamic" are at extreme ends of the scale; the
> more likely scenario is something in the middle.
I would like to know what's your experience regarding the amount of
type information that is available at macro expansion time. In
practice, is it common to have useful type information? I'm afraid
that even in the case of sequence operations such as remove, position,
etc, it is a rare situation indeed to find enough type information to
generate the most efficient code. But I would like to hear your
opinion on this.
>> I do think there's a way out, but it is not using macros that are
>> expanded at compile time. My idea is that you can solve your
>> performance problems just by stressing your code in every possible way
>> and then applying a recompilation process that can explore the
>> statistical information that it gathered about the types of the
>> function arguments and function results to generate more efficient
>> code (obviously, using the type inference process again but now with
>> more refined type information). Note that you would not need to
>> expand macros again as type inference generally works with an
>> intermediate form where macro calls no longer exist. Obviously, the
>> new optimized forms could coexist with the unoptimized versions and
>> the most appropriate version would be chosen at runtime. That's what
>> I would like to see in modern Common Lisp implementations.
>
> What you're really talking about here is the statification of your
> code, not the language. CL has always had the capability to compile
> into static code.
Sure. I don't want the statification of the language. But I'm afraid
that if we can make macros that require useful type information in
order for its expansion then our programs will need to satisfy a
different set of restrictions. We can say that these restrictions are
on the program but, in practice, they also affect the language or the
programs will not be portable. What's the use of a macro that expands
correctly in, say, Allegro CL but generates an error on a not-so-good
CL implementation that has poor type inference at macro expansion
time?
>> > This underlying discussion that seems to be going on here seems to be
>> > pretty much the classic `one-bit mind' problem described a couple of
>> > years ago by Erik Naggum. To be able to provide some useful type
>> > information to macros, CL does not have to become completely
>> > statically typed, and the assumption that it does is, frankly, silly.
>>
>> What do you mean by "completely statically typed"? I ask because I
>> never talked about that. I talked about a statically typed language
>> which is a language where you can get type information at compile
>> time.
>
> Well, you describe CL, because CL can provide type information at compile
> time. Does that make CL statically typed? By your definition, yes. Does
> that make it not dynamically typed? Not at all.
I'm not talking about "can" but about "must". If I have a macro that
requires type information for its expansion then the macro must have
that type information available. If you're saying that I, macro
writer, must be prepared to write macros that might not obtain the
information they need to compute the expansion, then I ask if it is
not preferable to always expand macros that use typecase forms on the
expansion and leave to the next phases of type inference and
optimization to see if it is possible to remove unneded branches. I'm
really interested in knowing your opinion about this.
>> > What I want is for progress to happen. Say Allegro CL's implementors
>> > do some clever stuff which allows macros to ask for, and sometimes
>> > get, type and other information which has been derived by the system.
>> > Suddenly people using Allegro can write all sorts of clever macros,
>> > although those macros will have implementation-specific parts. So in
>> > the standard open-source way, the implementors of some open-source
>> > Lisp borrow the Allegro API, soup it up a bit and implement a version
>> > which provides somewhat better than Allegro's. In a few years there's
>> > agreement on what this API should look like, and it becomes a
>> > (sub-)standard.
>>
>> What you call "progress" is just making Common Lisp a more statically
>> typed language. That's not progress, IMHO.
>
> No, it only allows for more statically typed _code_ portably. The
> language doesn't change at all, and is just as static and dynamic
> as it always has been.
You are right about the difference between statically typed code and
statically typed language. However, I wonder what do we do about some
of the Common Lisp "restrictions" such as:
3.2.2.3 Semantic Constraints
...
If the compiler processes a function form whose operator is not
defined at compile time, no error is signaled at compile time.
So, during compilation, macro expansion will occur and is possible
(and I also think that is frequent) that some macro call receives a
function form as argument whose operator is completely unknown at
compile time. It seems to me that, in this case, no useful type
information is available regarding that particular argument. What can
my macro expander do in that case?
>
> I think you're making the mistake of "identifying" CL as having static
> or dynamic typing, just as people incorrectly misidentify the Functional
> nature of CL; CL supports FP, but does not enforce it, which to some
> people means that it can't be functional. But the CL philosophy is
> not to _be_ any particular thing, but instead to _support_ as many
> things as possible. Including dynamic typing. Including static typing.
I know that. But I had the interesting experience of writing Linj and
see that, in practice, my type-dependent macros needed lots of precise
type information. In general, this implies that the type inferencer
needs to do whole-program analysis and this introduces several
restrictions in the language. For example, contrary to what I
sometimes do in Common Lisp, I cannot define a function that calls an
(yet) unknown function. But there are other annoying things such as
the fact that normal containers (such as lists, hashtables, etc) don't
keep type information about its contents and this means that you also
have to include type assertions in several places in order for the
type inferencer to produce useful information otherwise it quickly
degenerates. All these restrictions are caused by the need for useful
type information available at compile time and that's what makes Linj
a statically typed language. Can we use it like a dynamically typed
language? Sure we can but then my type-dependent macros will not do
anything useful. But I'm looking forward to see what can we do about
this in Common Lisp.
Ant�nio Leit�o.
From: Duane Rettig
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4pt3fqdyr.fsf@franz.com>
[I think that this is the crux of your issue, and there is
no use answering the rest until we resolve this one thing.]
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > Antonio Menezes Leitao <··············@evaluator.pt> writes:
> >
> >> What do you mean by "completely statically typed"? I ask because I
> >> never talked about that. I talked about a statically typed language
> >> which is a language where you can get type information at compile
> >> time.
> >
> > Well, you describe CL, because CL can provide type information at compile
> > time. Does that make CL statically typed? By your definition, yes. Does
> > that make it not dynamically typed? Not at all.
>
> I'm not talking about "can" but about "must". If I have a macro that
> requires type information for its expansion then the macro must have
> that type information available.
If you have a macro that _requires_ type information for its expansion,
then it is either not a Common Lisp macro or else it is broken. It
is perfectly acceptable for you to write a macro partially implementing
a language where you require your users to declare all types. But Common
Lisp is not one of these languages. Therefore, if your macro is simply
a Common Lisp macro, it must be prepared to deal with the possible
absence of type information.
> If you're saying that I, macro
> writer, must be prepared to write macros that might not obtain the
> information they need to compute the expansion, then I ask if it is
> not preferable to always expand macros that use typecase forms on the
====================^^^^^^
> expansion and leave to the next phases of type inference and
> optimization to see if it is possible to remove unneded branches. I'm
> really interested in knowing your opinion about this.
No. Not always. The nice thing about Common Lisp macros is that they
can provide multiple expansions, depending on what they see in their
environment. If the environment has static type information, then
the returned form can be optimized based on this static information.
Consider this s-expression:
(let ((len (length x)))
(foo len))
where foo is a macro. What is the type of the argument to foo?
The answer is of course implementation-dependent, but in most
implementations the correct answer is
(integer 0 #.most-positive-fixnum)
The reasoning is as follows:
The argument to length is a sequence, and it will ths either be
a vector, which has (1- array-dimension-limit) as max size, or it
is a list, which, if length returns at all, will have as many cons
cells as can be fit into memory (which in most implementations will
be 1/8 of memory in a 32-bit version, or 1/16 of memory in a 64-bit
version. This covers half the fixnum range, or the range from
0 to most-positive-fixnum.
Now, it is true that normally this type information isn't propagated
until after macroexpansion, in an explicit type inference phase.
However, it is perfectly reasonable for this type information to be
inferred at macroexpand time. A simple (but messy) transform
would be to supply compiler macros for functions like length, and the
compiler-macro for length would, if no other expansions were to be done,
return the form (the (integer 0 #.most-positive-fixnum) (length x)),
which can then be stored as the type of len. As long as there are
no intervening setfs/setqs of len, the variable-information for len
provides the required type information.
Now, as I said above, such automatic THE annotations become very
messy, and it would make the macroexpanded code very ugly if we
did this, even if only for all known CL functions. I didn't have time
to do so for 7.0, but will be considering the following for the
future: to add a form-information accessor, as well as a :form
keyword to augment-environment, to allow such annotations to be
done without actually rewriting source code.
Finally, once the foo macro receives its argument, then based on what
it sees in the variable-information for its argument, it can decide
whether to generate code that is generic (e.g. has a type-case) or
code which is specific (e.g. it can assume its argument is a fixnum).
Such code might be completely different for the two situations,
depending on how the macro is defined.
Now, obviously, compiler-macros will benefit most from this kind
of macro-level type inference. But macros can also benefit; they
also can ask to receive an environment argument, and can test
whether the environment is a compilation environment or not. Also,
some of this can be done now, whenever arguments to macros are
either manifest constants or trivially calculable (though as I said
it can sometimes get messy). But it has _always_ been the case
that macros (whether compiler-macros or normal macros) have the
power to make decisions programmatically about what code they want
to expand to, given enough information. What I am working toward
is providing in a portable way more and more of that information,
information that won't be changing,and which is thus in a sense
"static" information, so that the macro has the ability to reason
and pare down its code produced to code that is only as dynamic
as is needed.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87k6tnf3ko.fsf@evaluator.pt>
Duane Rettig <·····@franz.com> writes:
> [I think that this is the crux of your issue, and there is
> no use answering the rest until we resolve this one thing.]
>
> If you have a macro that _requires_ type information for its expansion,
> then it is either not a Common Lisp macro or else it is broken.
That's what I've trying to say all this time (maybe I didn't express
myself clear enough).
> It is perfectly acceptable for you to write a macro partially implementing
> a language where you require your users to declare all types. But Common
> Lisp is not one of these languages. Therefore, if your macro is simply
> a Common Lisp macro, it must be prepared to deal with the possible
> absence of type information.
We agree.
>> If you're saying that I, macro
>> writer, must be prepared to write macros that might not obtain the
>> information they need to compute the expansion, then I ask if it is
>> not preferable to always expand macros that use typecase forms on the
> ====================^^^^^^
>> expansion and leave to the next phases of type inference and
>> optimization to see if it is possible to remove unneded branches. I'm
>> really interested in knowing your opinion about this.
>
> No. Not always. The nice thing about Common Lisp macros is that they
> can provide multiple expansions, depending on what they see in their
> environment. If the environment has static type information, then
> the returned form can be optimized based on this static information.
>
> Consider this s-expression:
>
> (let ((len (length x)))
> (foo len))
>
> where foo is a macro. What is the type of the argument to foo?
>
> The answer is of course implementation-dependent, but in most
> implementations the correct answer is
>
> (integer 0 #.most-positive-fixnum)
>
> The reasoning is as follows:
>
> [nicely explained reasoning]
>
> Finally, once the foo macro receives its argument, then based on what
> it sees in the variable-information for its argument, it can decide
> whether to generate code that is generic (e.g. has a type-case) or
> code which is specific (e.g. it can assume its argument is a fixnum).
> Such code might be completely different for the two situations,
> depending on how the macro is defined.
Maybe I didn't understand your example but it seems to me that that's
another perfect example of a situation where foo can be a normal macro
that can simply expand into a typecase form and that then waits for
the code optimization phase to remove the unneded branches based on
the type information available about the type of the len variable.
This would avoid all the messy transformations that you described.
That's why I ask about what is preferable. I would love to see an
example where it really made a difference to have that information
available at macroexpansion time.
> Now, obviously, compiler-macros will benefit most from this kind
> of macro-level type inference. But macros can also benefit; they
> also can ask to receive an environment argument, and can test
> whether the environment is a compilation environment or not. Also,
> some of this can be done now, whenever arguments to macros are
> either manifest constants or trivially calculable (though as I said
> it can sometimes get messy). But it has _always_ been the case
> that macros (whether compiler-macros or normal macros) have the
> power to make decisions programmatically about what code they want
> to expand to, given enough information. What I am working toward
> is providing in a portable way more and more of that information,
> information that won't be changing,and which is thus in a sense
> "static" information, so that the macro has the ability to reason
> and pare down its code produced to code that is only as dynamic
> as is needed.
That's good, obviously. Do you have any data regarding the amount of
type information that you can provide to macroexpanders, e.g., in the
following situation:
(let ((len (my-length x)))
(foo len))
(defun my-length (x)
(length x))
Thanks for your comments,
Ant�nio Leit�o.
From: Duane Rettig
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4fz4aem50.fsf@franz.com>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Duane Rettig <·····@franz.com> writes:
>
> > Finally, once the foo macro receives its argument, then based on what
> > it sees in the variable-information for its argument, it can decide
> > whether to generate code that is generic (e.g. has a type-case) or
> > code which is specific (e.g. it can assume its argument is a fixnum).
> > Such code might be completely different for the two situations,
> > depending on how the macro is defined.
>
> Maybe I didn't understand your example but it seems to me that that's
> another perfect example of a situation where foo can be a normal macro
> that can simply expand into a typecase form and that then waits for
> the code optimization phase to remove the unneded branches based on
> the type information available about the type of the len variable.
This is what tends to occur now. Some of this is unavoidable. However,
There are two reasons for preferring to do this propagation at
macroexpand time:
1. Rewriting code at the "source" level (i.e. internal representations
understood by all Lisp programmers) eliminates the "magic" of having
transformations and optimizations deep in the bowels of the compiler,
using whatever internal representations the compiler sees fit to
use. In essence, it "opens" the optimization implementation to user
analysis and extension.
2. Sometimes a macroexpansion (which might itself represent an
optimization) removes critical information about the nature of the
source, thus making it hard (sometimes extremely hard) to reason
about types later in the compilation. See my trivial example below.
> This would avoid all the messy transformations that you described.
I used "messy" in a relative sense. It is _much_ more messy to try
to reason about internal compiler structures.
> That's why I ask about what is preferable. I would love to see an
> example where it really made a difference to have that information
> available at macroexpansion time.
Consider the trivial loop
(dotimes (i 10) (foo i))
What is the type of i? The correct answer is (integer 0 10)
[one might actually guess (correctly) that all of the _uses_
of i will only ever see (integer 0 (10)), but in fact i
transiently takes on the value 10 when rounding the last
iteration in expanded code]
Now, based on the CL specification of what components dotimes
must macroexpand into, note that the macroexpansion becomes
much less trivial to reason about:
CL-USER(2): (pprint (macroexpand '(dotimes (i 10) (foo i))))
(BLOCK NIL
(LET ((I 0))
(TAGBODY
#:|Tag2|
(COND ((>= I 10) (RETURN-FROM NIL (PROGN))))
(TAGBODY (FOO I))
(PSETQ I (1+ I))
(GO #:|Tag2|))))
CL-USER(3):
If the type propagation is done after the macroexpansion, then
unless some annotation os carried along with the code, extended
graph theory and type calculus must be done in order to reason
about the type of i (note that it is set once at initialization,
and then again within a loop,but the termination clause must be
examined to know what the upper limit is). If foo had been a
macro and the macroexpansion of foo could have generated better
code if it had known that i is as small as it turns out to be,
then by this time it is too late to rewrite foo's code.
But if we let macroexpansion do this job, using environments:
CL-USER(3): (pprint (macroexpand '(dotimes (i 10) (foo i))
(sys:make-compile-file-environment)))
(BLOCK NIL
(LET ((I 0))
(DECLARE (TYPE (INTEGER 0 10) I))
(TAGBODY
#:|Tag3|
(COND ((>= I 10) (RETURN-FROM NIL (PROGN))))
(TAGBODY (FOO I))
(PSETQ I (THE (INTEGER 0 10) (1+ I)))
(GO #:|Tag3|))))
CL-USER(4):
then we see that the type is deduced, and is even propagated
to the psetq. Note that this macroexpansion is using the
"messy" style of annotating the code with THE forms. This is
still _much_ cleaner than trying to reason about tagbody forms.
The reason I call it messy, and eventually desire to move these
to the compilation environment rather than annotating the code
directly, is because sending this form to an interpreter forces
the interpreter to evaluate (and possibly treat as nops) the THE
forms.
> > Now, obviously, compiler-macros will benefit most from this kind
> > of macro-level type inference. But macros can also benefit; they
> > also can ask to receive an environment argument, and can test
> > whether the environment is a compilation environment or not. Also,
> > some of this can be done now, whenever arguments to macros are
> > either manifest constants or trivially calculable (though as I said
> > it can sometimes get messy). But it has _always_ been the case
> > that macros (whether compiler-macros or normal macros) have the
> > power to make decisions programmatically about what code they want
> > to expand to, given enough information. What I am working toward
> > is providing in a portable way more and more of that information,
> > information that won't be changing,and which is thus in a sense
> > "static" information, so that the macro has the ability to reason
> > and pare down its code produced to code that is only as dynamic
> > as is needed.
>
> That's good, obviously. Do you have any data regarding the amount of
> type information that you can provide to macroexpanders, e.g., in the
> following situation:
>
> (let ((len (my-length x)))
> (foo len))
>
> (defun my-length (x)
> (length x))
Assuming these two forms are in a file to be compiled: the ordering
of these forms disallows any transfer of type information for my-length.
However, if we instead consider a file with:
(defun my-length (x)
(length x))
(let ((len (my-length x)))
(foo len))
then the semantics of compile-file allows the compiler to assume
(unless my-length has been declaed notinline) that the my-length
call is in fact to the definiton one above it, and therefore it
might make sense to establish in the compile-file-environment an
automatic ftype declaration for my-length which would then be used
in the call. So if I understand the direction of your question,
then yes, a foo macro would be able to receive a type declaration
of (integer 0 #.most-positive-fixnum).
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Duane Rettig wrote:
>>(let ((len (my-length x)))
>> (foo len))
>>
>>(defun my-length (x)
>> (length x))
>
>
> Assuming these two forms are in a file to be compiled: the ordering
> of these forms disallows any transfer of type information for my-length.
> However, if we instead consider a file with:
>
> (defun my-length (x)
> (length x))
>
> (let ((len (my-length x)))
> (foo len))
>
> then the semantics of compile-file allows the compiler to assume
> (unless my-length has been declaed notinline) that the my-length
> call is in fact to the definiton one above it, and therefore it
> might make sense to establish in the compile-file-environment an
> automatic ftype declaration for my-length which would then be used
> in the call. So if I understand the direction of your question,
> then yes, a foo macro would be able to receive a type declaration
> of (integer 0 #.most-positive-fixnum).
As an aside. CMUCL/SBCL have a "block compilation" declaration that
eases these inter-function optimizations at compile time. Does ACL have
similar things?
Cheers
--
Marco
From: Duane Rettig
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <47jpmejod.fsf@franz.com>
Marco Antoniotti <·······@cs.nyu.edu> writes:
> Duane Rettig wrote:
>
>
> >>(let ((len (my-length x)))
> >> (foo len))
> >>
> >>(defun my-length (x)
> >> (length x))
> > Assuming these two forms are in a file to be compiled: the ordering
>
> > of these forms disallows any transfer of type information for my-length.
> > However, if we instead consider a file with:
> > (defun my-length (x)
>
> > (length x))
> > (let ((len (my-length x)))
>
> > (foo len))
> > then the semantics of compile-file allows the compiler to assume
>
> > (unless my-length has been declaed notinline) that the my-length
> > call is in fact to the definiton one above it, and therefore it
> > might make sense to establish in the compile-file-environment an
> > automatic ftype declaration for my-length which would then be used
> > in the call. So if I understand the direction of your question,
> > then yes, a foo macro would be able to receive a type declaration
> > of (integer 0 #.most-positive-fixnum).
>
> As an aside. CMUCL/SBCL have a "block compilation" declaration that
> eases these inter-function optimizations at compile time. Does ACL
> have similar things?
No. Block compilation is a useful tool which tends to drive an
implementation toward more static (read: faster) programs. However,
we tend to lean more toward the side of trying to make up that speed
difference while maintaining as full dynamism as possible. And
although we have nothing against block compilation, and talk about
potential implementations, it does have the negative effect of
removing dynamism and making the program more monolithic. So to the
extent that we can get away without providing such block compilation,
but reason about and provide competitive code generation in a more
dynamic setting, we tend to prioritize other enhancements over
block-compilation.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87r7nuseha.fsf@evaluator.pt>
Duane Rettig <·····@franz.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
> [...]
> There are two reasons for preferring to do this propagation at
> macroexpand time:
>
> 1. Rewriting code at the "source" level (i.e. internal representations
> understood by all Lisp programmers) eliminates the "magic" of having
> transformations and optimizations deep in the bowels of the compiler,
> using whatever internal representations the compiler sees fit to
> use. In essence, it "opens" the optimization implementation to user
> analysis and extension.
Ok. I like the "magic" done by the compiler but I understand that
there are programmers that might prefer to do it by hand.
> 2. Sometimes a macroexpansion (which might itself represent an
> optimization) removes critical information about the nature of the
> source, thus making it hard (sometimes extremely hard) to reason
> about types later in the compilation. See my trivial example below.
>
>> This would avoid all the messy transformations that you described.
>
> I used "messy" in a relative sense. It is _much_ more messy to try
> to reason about internal compiler structures.
In my (admitedly limited) experience it works quite well (at least
with CMUCL). You just have to trust it.
>> That's why I ask about what is preferable. I would love to see an
>> example where it really made a difference to have that information
>> available at macroexpansion time.
>
> Consider the trivial loop
>
> (dotimes (i 10) (foo i))
>
> What is the type of i? The correct answer is (integer 0 10)
> [one might actually guess (correctly) that all of the _uses_
> of i will only ever see (integer 0 (10)), but in fact i
> transiently takes on the value 10 when rounding the last
> iteration in expanded code]
> Now, based on the CL specification of what components dotimes
> must macroexpand into, note that the macroexpansion becomes
> much less trivial to reason about:
>
> CL-USER(2): (pprint (macroexpand '(dotimes (i 10) (foo i))))
>
> (BLOCK NIL
> (LET ((I 0))
> (TAGBODY
> #:|Tag2|
> (COND ((>= I 10) (RETURN-FROM NIL (PROGN))))
> (TAGBODY (FOO I))
> (PSETQ I (1+ I))
> (GO #:|Tag2|))))
> CL-USER(3):
>
> If the type propagation is done after the macroexpansion, then
> unless some annotation os carried along with the code, extended
> graph theory and type calculus must be done in order to reason
> about the type of i (note that it is set once at initialization,
> and then again within a loop,but the termination clause must be
> examined to know what the upper limit is). If foo had been a
> macro and the macroexpansion of foo could have generated better
> code if it had known that i is as small as it turns out to be,
> then by this time it is too late to rewrite foo's code.
>
> But if we let macroexpansion do this job, using environments:
>
> CL-USER(3): (pprint (macroexpand '(dotimes (i 10) (foo i))
> (sys:make-compile-file-environment)))
>
> (BLOCK NIL
> (LET ((I 0))
> (DECLARE (TYPE (INTEGER 0 10) I))
> (TAGBODY
> #:|Tag3|
> (COND ((>= I 10) (RETURN-FROM NIL (PROGN))))
> (TAGBODY (FOO I))
> (PSETQ I (THE (INTEGER 0 10) (1+ I)))
> (GO #:|Tag3|))))
> CL-USER(4):
>
> then we see that the type is deduced, and is even propagated
> to the psetq. Note that this macroexpansion is using the
> "messy" style of annotating the code with THE forms. This is
> still _much_ cleaner than trying to reason about tagbody forms.
I don't mind that "messy" style. However, I thought that you were
providing an example where it made a difference to have the type
information available at compile time but, once again, your example
shows that foo doesn't need any kind of type information. Here is one
example of the macro foo:
(defmacro foo (form)
(let ((x (gensym)))
`(let ((,x ,form))
(typecase ,x
(fixnum (+ 1 ,x))
(t (baz ,x))))))
Now, let's use a dotimes:
(defun bar ()
(dotimes (i 10)
(print (foo i))))
and something else:
(defun bar2 (n)
(print (foo n)))
If you compile this with CMUCL (and I presume Allegro will do the
same), you'll notice that
- the function bar doesn't contain the typecase.
- the function bar2 does contain the typecase.
What would be different if foo had access to type information?
>> [...]
>> That's good, obviously. Do you have any data regarding the amount of
>> type information that you can provide to macroexpanders, e.g., in the
>> following situation:
>>
>> (let ((len (my-length x)))
>> (foo len))
>>
>> (defun my-length (x)
>> (length x))
>
> Assuming these two forms are in a file to be compiled: the ordering
> of these forms disallows any transfer of type information for my-length.
> However, if we instead consider a file with:
>
> (defun my-length (x)
> (length x))
>
> (let ((len (my-length x)))
> (foo len))
>
> then the semantics of compile-file allows the compiler to assume
> (unless my-length has been declaed notinline) that the my-length
> call is in fact to the definiton one above it, and therefore it
> might make sense to establish in the compile-file-environment an
> automatic ftype declaration for my-length which would then be used
> in the call. So if I understand the direction of your question,
> then yes, a foo macro would be able to receive a type declaration
> of (integer 0 #.most-positive-fixnum).
I presume you see the implications of what you're saying. In order
for your macros to explore useful type information, you _must_
establish the automatic ftype declaration that you mentioned. But if
my-length also calls other functions then you will have to also
compute the ftype declarations of those functions. In the end, I
suspect that you might have to statically type a significant fragment
of your program.
However, let's wait and see what kind of use people will make with
whatever type information will be available at macroexpansion time.
Thanks a lot for your comments,
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Maybe I didn't understand your example but it seems to me that that's
> another perfect example of a situation where foo can be a normal macro
> that can simply expand into a typecase form and that then waits for
> the code optimization phase to remove the unneded branches based on
> the type information available about the type of the len variable.
> This would avoid all the messy transformations that you described.
> That's why I ask about what is preferable. I would love to see an
> example where it really made a difference to have that information
> available at macroexpansion time.
Using 'typecase' is a good idea. You don't bother about the optimization,
letting this at the compiler discretion.
But compilers only compile CL code. So whenever you build your own language
on lisp, you will need the environment-accessors to make it inference your
OWN types.
For example : CONS string x list-of-string -> list-of-string
So now CONCATENATE list-of-string may be expanded to specific-string-concat.
--
___________________________________________________________
Christophe Turle.
http://perso.wanadoo.fr/turle/lisp/utilities.html
(format nil ···@~a.~a" 'c.turle 'wanadoo 'fr)
Antonio Menezes Leitao wrote:
> You are forgetting a fundamental issue. Macro expansion is a
> compile-time operation. If you create a Lisp that allows macros that
> require type information to compute the expansion then that
> information must be produced at compile time. And if you can produce
> useful type information at compile time, then you are working with a
> statically typed language. That's as simple as that.
>
> Of course, if you say that you are willing to accept that your macros
> might need to expand without useful type information available, I
> seriously doubt that they will be any useful.
Type information is demonstrably useful even if it is not always
available; for example, it can be used for performance improvement
and static error detection. This is the whole point of Lisp type
declarations, after all, which are useful even if the program is
not fully annotated. The same benefits will apply in user macros.
Exposing type information in macros is a very lispy thing to do,
because it allows user access to a mechanism that the lisp is likely
using anyway internally in the compiler.
Paul
Antonio Menezes Leitao wrote:
> You are forgetting a fundamental issue.
No, I'm really not. Trust me, I do know CL fairly well.
> Macro expansion is a
> compile-time operation.
No, it's a macro-expansion time operation, which is subtly but
occasionally importantly different.
> What do you mean by "completely statically typed"? I ask because I
> never talked about that. I talked about a statically typed language
> which is a language where you can get type information at compile
> time.
>
Well, for example, any CL implementation that does not rely on special
purpose Lisp hardware and that generates reasonably decent floating
point code is relying on quite a lot of type information being
available to it at compile time. Part of a compiler's *job* is to
statically type code, and a great chunk of what makes a CL compiler
interesting is that a good one will do this without endless explicit
indications from the user, and without taking away CL's dynamic nature
(in other words, the type information available to it at compile time
is often information it worked out for itself).
... Oh, look, there really is no point carrying on with this. I'm not
sure why you aren't understanding what I'm saying, but I am clearly
making almost no progreess at all getting anything across to you as
you come back with the same arguments again and again. Unless someone
else thinks I'm doign a more than usually bad job of explaining what I
mean I think I'll stop now, and leave it up to Duane &co to carry on,
if they have the energy.
--tim
Antonio Menezes Leitao wrote:
> You are forgetting a fundamental issue. Macro expansion is a
> compile-time operation. If you create a Lisp that allows macros that
> require type information to compute the expansion then that
> information must be produced at compile time. And if you can produce
> useful type information at compile time, then you are working with a
> statically typed language. That's as simple as that.
Why on earth do you believe that "if you can produce useful
type information at compile time, then you are working with
a statically typed language"? That seems to me to be obviously,
ludicrously, false, unless you define some of those terms in
a non-standard way that makes your point uninteresting.
I claim:
- A language in which it is never necessary to issue
explicit type declarations, and in which it is commonplace
for variables to have values of multiple different types,
is not a "statically typed language".
- Such a language may, none the less, have a compiler that
can often deduce useful information about what types some
variables (or other expressions) can have.
- It still isn't a "statically typed language".
- If it has such a compiler, then there's no obvious reason
why that compiler shouldn't (when it can) provide type
information to whatever macro-expansion facility it uses.
- It still isn't a "statically typed language".
I think you do in fact appreciate this, because you said:
> Of course, if you say that you are willing to accept that your macros
> might need to expand without useful type information available, I
> seriously doubt that they will be any useful. And if your macros
> _demand_ useful type information then you will have to declare type
> information on the code that uses them.
Now, obviously you're an authority on the question of what
things you "seriously doubt". But if, as you say, you
seriously doubt that it's possible for compile-time (or,
more exactly, macro-expansion-time) type information to
be useful when it isn't always available, then *I* seriously
doubt that you've thought it through properly.
A good CL compiler (CMU CL, say) can do a *lot* of type
inference. Sometimes it can work out a lot about what type
something has, sometimes not. What it's able to work out
is enough to let it produce some very decent code, even
without explicit type declarations.
So we have an example of a system where not-always-available
type information is helpful. Why do you "seriously doubt"
that there might be other examples?
*
You've used the term "statically typed language" a lot,
and you've defined it in terms that are ambiguous because
a crucial quantifier is missing. Could you please say
which of the following (if any) is your definition?
1 "A language in which type information is always
available at compile time"
2 "A language in which type information is usually
available at compile time"
3 "A language in which type information is often
available at compile time"
4 "A language in which type information is sometimes
available at compile time, no matter how rarely"
Thanks.
--
Gareth McCaughan
.sig under construc
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87oeizf7bz.fsf@evaluator.pt>
Gareth McCaughan <················@pobox.com> writes:
> A good CL compiler (CMU CL, say) can do a *lot* of type
> inference. Sometimes it can work out a lot about what type
> something has, sometimes not. What it's able to work out
> is enough to let it produce some very decent code, even
> without explicit type declarations.
We agree on that.
> So we have an example of a system where not-always-available
> type information is helpful. Why do you "seriously doubt"
> that there might be other examples?
If we return to the original issue that started this subthread, we
will remember that we were looking for a macro that could encapsulate
different iteration forms without requiring the creation of iterator
objects or closures. For this to be possible, I still don't see any
way without demanding that the necessary type information is available
at compile time. Now, given the multitude of different ways that we
can use in Common Lisp to produce an object that we want to iterate
(e.g., on a local variable, from a function parameter, the dynamic
value of a special variable, the result of a yet unknown function
call, the result of a read operation, the result of a generic function
call, etc), I'm afraid that not even the most clever type inference
mechanism available in Common Lisp will be able to always produce the
necessary information for the macro to expand. I suspect (based on my
own, obviously limited, experience) that this will be a frequent
situation.
Now, we all agree that it is possible to expand according to the lack
of type information but that will require that our iterator macro must
be able to expand into something that will decide at run time. If the
macro writer still wants to avoid iterators or closures he might
choose an expansion that uses a typecase form or some such. The
interesting thing is that if there was type information available that
would allow him to not expand into the typecase, that exact same
information is available now to the normal code optimization phases of
compiler and it can achieve the same effect on the typecase form.
Given the fact that code optimization is done at a latter phase than
macro expansion I also suspect (based on my own, obviously limited,
experience) that there will be more type information available that
will allow it to make a better job.
> You've used the term "statically typed language" a lot,
> and you've defined it in terms that are ambiguous because
> a crucial quantifier is missing. Could you please say
> which of the following (if any) is your definition?
>
> 1 "A language in which type information is always
> available at compile time"
>
> 2 "A language in which type information is usually
> available at compile time"
>
> 3 "A language in which type information is often
> available at compile time"
>
> 4 "A language in which type information is sometimes
> available at compile time, no matter how rarely"
>
I guess that point 1 is the most appropriate definition. I do
consider the following definitions:
A statically typed language is a language where types are known at
compile time. A dynamically typed language is a language where types
are discovered at runtime.
Obviously, there are languages that can be both statically and
dynamically typed (Common Lisp is one example). All I'm saying is
that to have useful type information available at macroexpansion time
in general, your language or (as Duane already pointed it out) the
programs that you can write in that language must be statically typed,
at least, in the fragments that use those macros.
I think the best thing I can do now is to shut up and wait until those
macros are available. I'll then see what kind of restrictions they
will entail.
Thanks for your comments,
Ant�nio Leit�o.
Antonio Menezes Leitao wrote:
> If we return to the original issue that started this subthread, we
> will remember that we were looking for a macro that could encapsulate
> different iteration forms without requiring the creation of iterator
> objects or closures. For this to be possible, I still don't see any
> way without demanding that the necessary type information is available
> at compile time.
I see. Macros exploiting type propagation are not useful for this particular
problem (where you have dictated arbitrarily that the generic solution is
not allowed as a fallback), therefore they are not useful at all.
Ridiculous.
Paul
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87ekjug7eq.fsf@evaluator.pt>
"Paul F. Dietz" <·····@dls.net> writes:
> Antonio Menezes Leitao wrote:
>
>> If we return to the original issue that started this subthread, we
>> will remember that we were looking for a macro that could encapsulate
>> different iteration forms without requiring the creation of iterator
>> objects or closures. For this to be possible, I still don't see any
>> way without demanding that the necessary type information is available
>> at compile time.
>
> I see. Macros exploiting type propagation are not useful for this particular
> problem (where you have dictated arbitrarily that the generic solution is
> not allowed as a fallback), therefore they are not useful at all.
>
> Ridiculous.
Can you, please, show me one example where it is useful and that
cannot be done with normal macros and normal code optimization?
Thanks in advance,
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> "Paul F. Dietz" <·····@dls.net> writes:
>
>> Antonio Menezes Leitao wrote:
>>
>>> If we return to the original issue that started this subthread, we
>>> will remember that we were looking for a macro that could
>>> encapsulate different iteration forms without requiring the
>>> creation of iterator objects or closures. For this to be possible,
>>> I still don't see any way without demanding that the necessary
>>> type information is available at compile time.
>>
>> I see. Macros exploiting type propagation are not useful for this
>> particular problem (where you have dictated arbitrarily that the
>> generic solution is not allowed as a fallback), therefore they are
>> not useful at all.
>>
>> Ridiculous.
>
> Can you, please, show me one example where it is useful and that
> cannot be done with normal macros and normal code optimization?
So your argument, as I understand it, is that any macro of the form:
(defmacro foo (x)
(compile-time-typecase x
(this (generate-this-expansion x)
(that (generate-that-expansion x))
(t (generate-generic-expansion x)))))
can be we written instad like this:
(defmacro foo (x)
(once-only (x)
`(typecase ,x
(this ,(generate-this-expansion x))
(that ,(generate-that-expansion x))
(t ,(generate-generic-expansion x)))))
as long as we can count on the compiler to take remove dead code based
on the type information. I agree, that as a matter of logic, there
doesn't seem to be a way to write a macro that depends on
macroexpansion-time type information that can not be transformed this
way.
But what about the practical issue that generating and compiling code
isn't free. Generating all possible expansions for different types
when most of them are (you hope) going to be weeded out by the
optimizer has to be more work than generating just one. And in the
worst case, when the only type the compiler can infer is T, we end up
generating all the different type-specific versions *and* actually
compiling them since the optimizer can't eleminate any of them.
Let's go back one more time to the original example, iterating over an
object of perhaps unknown type:
(defun foo (obj)
(loop for e being the elements of obj do (stuff e)))
Now suppose that our LOOP macro is extensible so users can define
their own ways of handling "being the elements of" for their own
types. There may be hundreds of user defined types it knows about.
Under your proposal LOOP should generate a typecase with hundreds of
arms with a different expansion in each arm. When the optimizer looks
at that typecase it's not going to know anything more about OBJ than
that it is of type T so it is going to have to compile all those arms
and keep them.
On the other hand, if the macro had access to type information about
OBJ it could just generate code to do the less-efficient-at-runtime
general iteration strategy. Then--as is normal in Common Lisp--if the
user is disatisfied with the performance and profiles and sees that
we're spending a bunch of time executing general purpose iteration
code they can change FOO to:
(defun foo (obj)
(declare (type my-thing obj))
(loop for e being the elements of obj do (stuff e)))
or
(defun foo (obj)
(loop for e being the elements of (the my-thing obj) do (stuff e)))
and get the more efficient MY-THING-specific iteration code. Of course
this solution *loses* some runtime performance because it always uses
the general iteration code, even when the dynamic type of OBJ turns
out to be something that we know how to handle more efficiently. But
that seems in line with Common Lisp's general approach to
declarations.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87mzyis3gt.fsf@evaluator.pt>
Peter Seibel <·····@javamonkey.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> "Paul F. Dietz" <·····@dls.net> writes:
>>
>>> Antonio Menezes Leitao wrote:
>>>
>>>> If we return to the original issue that started this subthread, we
>>>> will remember that we were looking for a macro that could
>>>> encapsulate different iteration forms without requiring the
>>>> creation of iterator objects or closures. For this to be possible,
>>>> I still don't see any way without demanding that the necessary
>>>> type information is available at compile time.
>>>
>>> I see. Macros exploiting type propagation are not useful for this
>>> particular problem (where you have dictated arbitrarily that the
>>> generic solution is not allowed as a fallback), therefore they are
>>> not useful at all.
>>>
>>> Ridiculous.
>>
>> Can you, please, show me one example where it is useful and that
>> cannot be done with normal macros and normal code optimization?
>
> So your argument, as I understand it, is that any macro of the form:
>
> (defmacro foo (x)
> (compile-time-typecase x
> (this (generate-this-expansion x)
> (that (generate-that-expansion x))
> (t (generate-generic-expansion x)))))
>
> can be we written instad like this:
>
> (defmacro foo (x)
> (once-only (x)
> `(typecase ,x
> (this ,(generate-this-expansion x))
> (that ,(generate-that-expansion x))
> (t ,(generate-generic-expansion x)))))
>
> as long as we can count on the compiler to take remove dead code based
> on the type information. I agree, that as a matter of logic, there
> doesn't seem to be a way to write a macro that depends on
> macroexpansion-time type information that can not be transformed this
> way.
>
> But what about the practical issue that generating and compiling code
> isn't free. Generating all possible expansions for different types
> when most of them are (you hope) going to be weeded out by the
> optimizer has to be more work than generating just one.
Are you really concerned about that? Usually, the time my programs
take to compile is negligible when compared with the time they take
running. But I accept that others might have different concerns.
> And in the worst case, when the only type the compiler can infer is
> T, we end up generating all the different type-specific versions
> *and* actually compiling them since the optimizer can't eleminate
> any of them.
And what's the problem with that? What can you do to solve that
problem with macros that can explore type information? Will you not
have to produce the typecase and all it's branches? Or will you just
use a generic iteration form in all cases even when a typecase could
choose the most efficient form to do the iteration?
> Let's go back one more time to the original example, iterating over an
> object of perhaps unknown type:
>
> (defun foo (obj)
> (loop for e being the elements of obj do (stuff e)))
>
> Now suppose that our LOOP macro is extensible so users can define
> their own ways of handling "being the elements of" for their own
> types. There may be hundreds of user defined types it knows about.
Let's be pragmatical: did you ever felt that you needed to extend the
(extended) loop macro with hundreds of different iteration forms but
you couldn't do it because you lacked useful type information at
macroexpansion time?
> Under your proposal LOOP should generate a typecase with hundreds of
> arms with a different expansion in each arm. When the optimizer looks
> at that typecase it's not going to know anything more about OBJ than
> that it is of type T so it is going to have to compile all those arms
> and keep them.
I think you are concerned about the code bloat that the simple
typecase version that I showed you might cause in this case. Well,
then create a different version that reduces the code bloat by
factoring out the common parts. Or provide the necessary type
information. Or, if you are not willing to provide it and you do
have, as you say, hundreds of iteration forms then call the generic
iteration form instead of calling the macro.
Even if you decide to use a different macro expansion strategy based
on generic functions, a good CLOS implementation (and some help from
your part in declaring types and other optimization tricks such as
sealing) can generate very efficient code.
> On the other hand, if the macro had access to type information about
> OBJ it could just generate code to do the less-efficient-at-runtime
> general iteration strategy. Then--as is normal in Common Lisp--if the
> user is disatisfied with the performance and profiles and sees that
> we're spending a bunch of time executing general purpose iteration
> code they can change FOO to:
>
> (defun foo (obj)
> (declare (type my-thing obj))
> (loop for e being the elements of obj do (stuff e)))
>
> or
>
> (defun foo (obj)
> (loop for e being the elements of (the my-thing obj) do (stuff e)))
>
> and get the more efficient MY-THING-specific iteration code. Of course
> this solution *loses* some runtime performance because it always uses
> the general iteration code, even when the dynamic type of OBJ turns
> out to be something that we know how to handle more efficiently. But
> that seems in line with Common Lisp's general approach to
> declarations.
As I said above, the typecase approach is just the simplest of the
solutions. You can combine it with a generic function call. However,
I would expect that appropriate type declarations will allow the
compiler to eliminate the generic function call and inline the result,
thus achieving the exact same effect that you described. If you think
that this is way beyond our current compiler technology I suggest that
you take a look at the kind of optimizations that are being included
in compilers such as CMUCL.
Anyway, I think we should not bother comp.lang.lisp readers with this
endless discussion. If you want, we can continue this dicussion
privately.
I just think that this entire discussion could have ended much sooner
if someone had provided a single realistic example that showed that
macros that explore useful type information can make a real
difference.
Thanks a lot for your comments,
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Peter Seibel <·····@javamonkey.com> writes:
>
> > But what about the practical issue that generating and compiling code
> > isn't free. Generating all possible expansions for different types
> > when most of them are (you hope) going to be weeded out by the
> > optimizer has to be more work than generating just one.
>
> Are you really concerned about that? Usually, the time my programs
> take to compile is negligible when compared with the time they take
> running. But I accept that others might have different concerns.
Yes, you really really really should be concerned about this. As I
mention in another post, my system originally had a bug which caused
(effectively) this very thing to happen and the results were horrific.
The combinatorial explosions that can happen with nested cases of
hundreds of iteration schemes can produce phenomenal compilation times
and code bloat.
> > And in the worst case, when the only type the compiler can infer is
> > T, we end up generating all the different type-specific versions
> > *and* actually compiling them since the optimizer can't eleminate
> > any of them.
>
> And what's the problem with that? What can you do to solve that
> problem with macros that can explore type information?
The problem? The results are simply unacceptable, period. The answer
to the second question is obvious: you can completely eliminate the
problem. Again, the proof is by construction. The system I've shown
you _does_ fix this problem by using exactly "macros that can explore
type information". It works. Not using it not only doesn't work, it
results in unacceptable performance and code bloat.
> Will you not have to produce the typecase and all it's branches?
NO! You should know this by now because you've seen some of my
examples!!
> I just think that this entire discussion could have ended much sooner
> if someone had provided a single realistic example that showed that
> macros that explore useful type information can make a real
> difference.
WTF! I provided it, and I know you've seen it because you responded
to it.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
From: Antonio Menezes Leitao
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87d5zd2gt6.fsf@evaluator.pt>
jayessay <······@foo.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> Peter Seibel <·····@javamonkey.com> writes:
>>
>> > But what about the practical issue that generating and compiling code
>> > isn't free. Generating all possible expansions for different types
>> > when most of them are (you hope) going to be weeded out by the
>> > optimizer has to be more work than generating just one.
>>
>> Are you really concerned about that? Usually, the time my programs
>> take to compile is negligible when compared with the time they take
>> running. But I accept that others might have different concerns.
>
> Yes, you really really really should be concerned about this. As I
> mention in another post, my system originally had a bug which caused
> (effectively) this very thing to happen and the results were horrific.
> The combinatorial explosions that can happen with nested cases of
> hundreds of iteration schemes can produce phenomenal compilation times
> and code bloat.
>> [...]
> The problem? The results are simply unacceptable, period.
>
>> I just think that this entire discussion could have ended much sooner
>> if someone had provided a single realistic example that showed that
>> macros that explore useful type information can make a real
>> difference.
>
> WTF! I provided it, and I know you've seen it because you responded
> to it.
Did you? Let me clarify that.
Here is a statement that *I* made previously:
I think you can achieve the exact same effect using normal macros
that expand into a typecase form and then just wait for the normal
code optimization to remove the unneeded branches. Unless I'm missing
something, the final effect would look exactly the same.
*You* answered with this:
These _are_ "normal" macros. In theory, what you are saying _might_
work on some implementations in some cases.
So it seemed to me that the unstated problems that you found in
practice were related to some not-so-good compiler and that, with a
good compiler, you would not see the difference. You didn't mention
that you had performance problems due to combinatorial explosions. If
you did mentioned that, I confess that I didn't see it--in this case,
I apologize for that.
So, apparently, here is one example where it makes sense to explore
useful type information at macro-expansion time.
I'm not sure if it's related to the problem you are mentioning but it
is easy to see that my simple macro (which was done just for
illustrative purposes) also generates a combinatorial explosion, but
it's also easy to provide an alternative one (modulo hygiene, repeated
evaluation, etc):
(defmacro for-each ((x obj) &body body)
`(flet ((f (,x) ,@body))
(typecase ,obj
(list
(dolist (e ,obj)
(f e)))
(vector
(dotimes (i (length ,obj))
(f (aref ,obj i))))
(t
(generic-iter obj f)))))
Note that this version factored out the body of the macro. With this
version, you can write cascaded iterations like this one:
(for-each (x obj1)
(for-each (y x)
(for-each (z y)
(for-each (w z)
(print w)))))
And the expansion will produce
(FLET ((F (X)
(FLET ((F (Y)
(FLET ((F (Z)
(FLET ((F (W)
(PRINT W)))
(TYPECASE Z
(LIST (DOLIST (E Z) (F E)))
(VECTOR (DOTIMES (I (LENGTH Z)) (F (AREF Z I))))
(T (GENERIC-ITER OBJ F))))))
(TYPECASE Y
(LIST (DOLIST (E Y) (F E)))
(VECTOR (DOTIMES (I (LENGTH Y)) (F (AREF Y I))))
(T (GENERIC-ITER OBJ F))))))
(TYPECASE X
(LIST (DOLIST (E X) (F E)))
(VECTOR (DOTIMES (I (LENGTH X)) (F (AREF X I))))
(T (GENERIC-ITER OBJ F))))))
(TYPECASE OBJ1
(LIST (DOLIST (E OBJ1) (F E)))
(VECTOR (DOTIMES (I (LENGTH OBJ1)) (F (AREF OBJ1 I))))
(T (GENERIC-ITER OBJ F))))
[I'm writing this in a hurry and I didn't test it carefully so there
might be errors]
It seems to me that this is a linear "explosion", and not a
combinatorial one. The number of typecases is directly proporcional
to the number of iterations. Now, it's up to the compiler to explore
the available type information (if there is any) and inline the f
local functions if it thinks that it makes a difference. Anyway, it's
a local call so it should run quickly even if it doesn't inline it.
Did you try this? What where the results? Maybe I'm still missing
something so I'm not sure if this was the combinatorial explosion you
were mentioning.
Does this means that I don't think your macro is important? No! It
just means that I trust the compiler more than I trust myself to make
optimized code and that I don't need to resort to what I think are
"extreme measures".
But let's continue with something that you said previously:
For various reasonable examples (certainly the dead simple ones you
give here) an implementation could easily infer the result type of
such functions and tag them as such in the environment.
Asking an implementation to infer the result type of such functions is
what makes me think that you are asking for more and more static type
information. That's why I mentioned that useful type information at
compile time implies a statically typed language. I do understand
that your macros don't strictly depend on that information so let's
not continue with that.
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> jayessay <······@foo.com> writes:
>
> > Yes, you really really really should be concerned about this. As I
> > mention in another post, my system originally had a bug which caused
> > (effectively) this very thing to happen and the results were horrific.
> > The combinatorial explosions that can happen with nested cases of
> > hundreds of iteration schemes can produce phenomenal compilation times
> > and code bloat.
>
> >> [...]
>
> > The problem? The results are simply unacceptable, period.
> >
> >> I just think that this entire discussion could have ended much sooner
> >> if someone had provided a single realistic example that showed that
> >> macros that explore useful type information can make a real
> >> difference.
> >
> > WTF! I provided it, and I know you've seen it because you responded
> > to it.
>
> Did you? Let me clarify that.
Yes. And it appears you even saw the followup.
> Here is a statement that *I* made previously:
>
> I think you can achieve the exact same effect using normal macros
> ...
> *You* answered with this:
>
> These _are_ "normal" macros. In theory, what you are saying _might_
> work on some implementations in some cases.
>
> So it seemed to me that the unstated problems that you found in
> practice were related to some not-so-good compiler and that, with a
> good compiler, you would not see the difference. You didn't mention
> that you had performance problems due to combinatorial explosions. If
> you did mentioned that, I confess that I didn't see it--in this case,
> I apologize for that.
You are beginning to appear to be denser than unobtainium. As several
others have pointed out, there are several areas where no compiler
will be able to do this sort of code sorting and elimination.
Furthermore, even in those cases where it could, the compilation
performance due to the combinatorial explosion effects would still be
unacceptable.
> So, apparently, here is one example where it makes sense to explore
> useful type information at macro-expansion time.
>
> I'm not sure if it's related to the problem you are mentioning but it
> is easy to see that my simple macro (which was done just for
> illustrative purposes) also generates a combinatorial explosion, but
> it's also easy to provide an alternative one (modulo hygiene, repeated
> evaluation, etc):
<example>
> It seems to me that this is a linear "explosion", and not a
Why suggest a half effort? This is close to the worst of all worlds:
you still have to try to sort things out and refactor the code, but
you only go part way there. This results in code bloat and likely
compilicated enough resulting code to derail your supposed optimizer.
And you still are punting off to runtime calls.
> Did you try this?
No, because it is completely _lame_.
> Does this means that I don't think your macro is important? No! It
> just means that I trust the compiler more than I trust myself to make
> optimized code and that I don't need to resort to what I think are
> "extreme measures".
Fine - if you aren't competent enough to do a proper job of this, then
you simply have to hope the compiler will save you [1]. Now, how do
you get from that to suggesting that putting type information in the
environment so that others (such as myself) who _can_ do such a proper
job is somehow the wrong thing and that such people _shouldn't_ have
this and _shouldn't_ use it. That's crazy, and absolutely not the
Lisp way.
> But let's continue with something that you said previously:
>
> For various reasonable examples (certainly the dead simple ones you
> give here) an implementation could easily infer the result type of
> such functions and tag them as such in the environment.
>
> Asking an implementation to infer the result type of such functions is
> what makes me think that you are asking for more and more static type
> information.
I see you still haven't understood a thing that Duane et. al. have
written either. I give up.
/Jon
[1] Having been involved in implementing a few compilers (a couple for
statically typed languages with _extremely_ good optimizers), the sort
of hoping that the optimizer will save you from yourself in these sort
of cases that you are suggesting here will only come to pass in very
rare cases.
--
'j' - a n t h o n y at romeo/charley/november com
From: Antonio Menezes Leitao
Subject: CMUCL optimizer [was Re: Sacla loop: a new loop implementation]
Date:
Message-ID: <87hdoj3s9c.fsf_-_@evaluator.pt>
Hi,
Several people claimed that in order have a generic iterator for-each
macro that generates efficient code it is necessary to allow it to
explore type information at macroexpansion time to decide the macro
expansion. I claimed otherwise and showed an example of a possible
implementation. When used in cascaded iterations my illustrative
solution would obviously produce huge amounts of code bloat and I
proposed a new implementation that was intended to solve this
combinatorial explosions of code that would arise in cascaded
iterations.
Here is the proposed macro:
(defmacro for-each ((var expr) &body body)
(let ((the-expr (gensym)))
`(flet ((f (,var) ,@body))
(let ((,the-expr ,expr))
(typecase ,the-expr
(list
(dolist (e ,the-expr)
(f e)))
(vector
(dotimes (i (length ,the-expr))
(f (aref ,the-expr i))))
(t
(generic-iter ,the-expr #'f)))))))
My proposal was quickly dismissed as being
"close to the worst of all worlds: you still have to try to sort
things out and refactor the code, but you only go part way there.
This results in code bloat and likely compilicated enough resulting
code to derail your supposed optimizer."
It puzzles me that people can be so confident about the (lack of)
intelligence of modern Common Lisp compilers that they don't even
experiment a possible solution, asserting immediately that it is
"completely lame".
In the several years that I have been a faithful CMUCL user I had my
share of occasions where I was happily surprised with the cleverness
of the Python compiler so I thought this would be another opportunity
to show that Python can be as good (or better) at optimizing code than
a human programmer.
Without direct access to the examples that people claim are impossible
for Python to optimize it is obviously difficult to make up a
realistic test case so I had to invent something that, I hope, covers
several extreme cases. If you don't agree, feel free to provide a
realistic example.
Here is my example: we want a macro that shows real code bloat so I
decided to generate a large typecase (due to my lack of time and
imagination, some of the clauses don't make much sense). Note that
the body of the for-each macro calls are factored out in a flet. Note
also that there are a few subtype relationships. Let's call this
macro the "stupid" macro.
(defmacro for-each ((var expr) &body body)
(let ((the-expr (gensym)))
`(flet ((f (,var) ,@body))
(let ((,the-expr ,expr))
(typecase ,the-expr
(character
(f ,the-expr))
(symbol
(f ,the-expr))
(number
(f ,the-expr))
(string
(dotimes (i (length ,the-expr))
(f (char ,the-expr i))))
(list
(dolist (e ,the-expr)
(f e)))
(vector
(dotimes (i (length ,the-expr))
(f (aref ,the-expr i))))
(array
(dotimes (i (array-total-size ,the-expr))
(f (row-major-aref ,the-expr i))))
(t
(generic-iter ,the-expr #'f)))))))
Note that there's a last clause that uses a generic iteration form
whose definition is as follows (this will be necessary in the next
examples):
(defgeneric generic-iter (obj f))
(defmethod generic-iter ((obj symbol) f)
(funcall f obj))
(defmethod generic-iter ((obj number) f)
(funcall f obj))
(defmethod generic-iter ((obj string) f)
(dotimes (i (length obj))
(funcall f (char obj i))))
(defmethod generic-iter ((obj list) f)
(dolist (e obj)
(funcall f e)))
(defmethod generic-iter ((obj vector) f)
(dotimes (i (length obj))
(funcall f (aref obj i))))
(defmethod generic-iter ((obj array) f)
(dotimes (i (array-total-size obj))
(funcall f (row-major-aref obj i))))
To experiment with (what I think are) extreme cases of code bloat,
I created a function with nine cascaded iterations:
(defun test (obj f)
(for-each (i obj)
(for-each (j i)
(for-each (k j)
(for-each (l k)
(for-each (m l)
(for-each (n m)
(for-each (o n)
(for-each (p o)
(for-each (q p)
(funcall f q)))))))))))
Contrary to what others might think, I know that my macro doesn't
generate combinatorial explosions of code. Each for-each macro call
is just one flet plus one typecase and the macroexpansion grows
linearly with the number of cascaded iterations (and with the number
of typecase clauses).
In order to measure the compilation and load time I decided to create
a function defining macro:
(defmacro make-n-test (n)
`(progn
,@(loop for i below n
collect `(defun ,(read-from-string (format nil "~A-~A" 'test i)) (obj f)
(for-each (i obj)
(for-each (j i)
(for-each (k j)
(for-each (l k)
(for-each (m l)
(for-each (n m)
(for-each (o n)
(for-each (p o)
(for-each (q p)
(funcall f q))))))))))))))
and to use it to generate 500 of these 9-level iteration functions:
(make-n-test 500)
The result is a file containing five hundred functions, each with nine
cascaded iterations where each iteration implies a typecase with eight
clauses. If you have even more complex real-life examples, please,
fell free to provide them.
To test the performance of these functions I made a function that
creates a huge structure (although most of it is shared):
(defun make-test (types sizes)
(if (endp types)
42
(let ((type (first types))
(size (first sizes)))
(case type
(string
(subseq "Hi,there!" 0 size))
(list
(make-list size
:initial-element (make-test (rest types) (rest sizes))))
(vector
(apply #'vector (make-test (cons 'list (rest types)) sizes)))
(array
(make-array (list size size)
:initial-element (make-test (rest types) (rest sizes))))))))
And used it to initialize the following variable:
(defparameter *test*
(make-test '(array list vector list array vector list array string)
'(1 2 3 4 5 6 7 8 9)))
If my math is correct, the structure contains 14515200 = (* 1 1 2 3 4
5 5 6 7 8 8 9) leaves of type character. It looks to me that
iterating more than 14 million leaves is going to be a good stress
test (but, as before, if you have more realistic examples I would be
glad to see them). To get more accurate results, I'll make 10 calls
to one of the 500 functions:
(defun test-one (f)
(dotimes (i 10)
(test-0 *test* f)))
To measure the effect of type declarations, I decided to include type
declarations that provided enough precise type information to allow
the compiler to choose just one branch in each typecase. Here is the
new version (due to subtype relationships some of the declarations are
a bit odd):
(defmacro make-n-test (n)
`(progn
,@(loop for i below n
collect `(defun ,(read-from-string (format nil "~A-~A" 'test i)) (obj f)
(declare (type (and array (not vector)) obj))
(for-each (i obj)
(declare (type (and list (not symbol)) i))
(for-each (j i)
(declare (type (and vector (not string)) j))
(for-each (k j)
(declare (type (and list (not symbol)) k))
(for-each (l k)
(declare (type (and array (not vector)) l))
(for-each (m l)
(declare (type (and vector (not string)) m))
(for-each (n m)
(declare (type (and list (not symbol)) n))
(for-each (o n)
(declare (type (and array (not vector)) o))
(for-each (p o)
(declare (string p))
(for-each (q p)
(funcall f q))))))))))))))
What are the contenders to the for-each macro? Assuming that we have
a "clever" macro that explores useful type information at
macroexpansion time, this macro will be able to generate either a
single call for the generic-iter generic function (when there isn't
enough type information at macroexpansion time) or it will be able to
generate a single primitive iteration form for the appropriate type
(when there is enough type information at macroexpansion time).
We can simulate the first situation using the following definition:
(defmacro for-each ((var expr) &body body)
`(generic-iter ,expr #'(lambda (,var) ,@body)))
and we can simulate the second situation using the following
definition (where we extended the macro to accept an extra argument
describing the specific type we expect):
(defmacro for-each ((var expr type) &body body)
(let ((the-expr (gensym))
(i (gensym)))
(ecase type
((character symbol number)
`(let ((,var (the ,type ,expr)))
,@body))
(string
`(let ((,the-expr ,expr))
(declare (,type ,the-expr))
(dotimes (,i (length ,the-expr))
(let ((,var (char ,the-expr ,i)))
,@body))))
(list
`(dolist (,var (the ,type ,expr))
,@body))
(vector
`(let ((,the-expr ,expr))
(declare (,type ,the-expr))
(dotimes (,i (length ,the-expr))
(let ((,var (aref ,the-expr ,i)))
,@body))))
(array
`(let ((,the-expr ,expr))
(declare (,type ,the-expr))
(dotimes (,i (array-total-size ,the-expr))
(let ((,var (row-major-aref ,the-expr ,i)))
,@body)))))))
For this last macro, we need a different example:
(defmacro make-n-test (n)
`(progn
,@(loop for i below n
collect `(defun ,(read-from-string (format nil "~A-~A" 'test i)) (obj f)
(for-each (i obj array)
(for-each (j i list)
(for-each (k j vector)
(for-each (l k list)
(for-each (m l array)
(for-each (n m vector)
(for-each (o n list)
(for-each (p o array)
(for-each (q p string)
(funcall f q))))))))))))))
I also decided to experiment with and without a declamation of the
form:
(declaim (optimize (speed 3) (space 0) (safety 0) (debug 0) (compilation-speed 0)))
[ I also experimented with and without the declamation
(declaim (optimize (speed 0) (space 3) (safety 0) (debug 0) (compilation-speed 0)))
and, in fact, CMUCL produces quite compact code but the performance
becomes terrible. Others might want to explore this in more detail. ]
We finally arrive at the tests. There are four. The first two
compare the "stupid" macro with the "clever" macro when there isn't
type information available. The second two compare the "stupid" macro
with the "clever" macro when there is complete type information
available. My intention is to provide the extreme situations so that
we can interpolate for mixed cases. The four extreme cases are:
Generic: Using the "clever" macro that expands into the single generic
function call generic-iter. This simulates a macro that explores
useful type information at macroexpansion time but that has none.
Typecase: Using the "stupid" macro that expands into a flet plus a
large typecase form without any type declarations.
Specific: Using the "clever" macro that expands into a single
primitive iteration form. This simulates a macro that explores useful
type information at macroexpansion time to generate the most specific
iteration form.
Typecase+types: Using the "stupid" macro that expands into an flet plus
a large typecase form with complete type declarations so that the
compiler can remove unreachable typecase branches.
For each test, I measured the compilation time, the load time, the
size of the compiled file and the time taken by the test form. To
avoid being bothered by the large amount of compilation notes issued
by CMUCL I decided to measure all compilation times as follows:
(let ((*error-output* (make-broadcast-stream)))
(time (compile-file "...")))
It is important to note that the compilation time for the Generic and
Specific cases is a bit unfair (compared with the Typecase and
Typecase+types) because it doesn't include the time that the type
inferencer would spend to provide type information to the "clever"
macro at macroexpansion time. I have my own feelings about this but I
prefer to hear comments from those who should know better.
The running time was measured using the above described test-one
function, as follows:
(time (test-one #'identity))
On each test I made three attempts (for compilation, load, and run)
and used the best results.
Finally, the tests. They were done on a Centrino 1.6 GHz with 512 MB
RAM.
First, without any type information and with CMUCL defaults for the
optimization qualities. All times are in seconds and all sizes are in
bytes.
Declaim Compilation Load Size Run
generic none 7.5 0.51 2553188 17.6
typecase none 56.6 3.38 9680521 14.9
The results show that, without type declarations and without the
((speed 3) (others 0)) declamation, the typecase form produces a file
which is 4 times bigger and that takes 8 times longer to compile and
load. It's an obvious example of code bloat but, given the problem
characteristics, it doesn't look like a combinatorial explosion and
the times and sizes don't seem unreasonable to me. Besides, the whole
idea was to explore type information so it really is an extreme
situation. The good news is that the typecase runs faster than the
generic call so it might payoff to always expand into a (small)
typecase that takes care of the most frequent cases.
In the next test, we explore type declarations:
Declaim Compilation Load Size Run
specific none 13.9 0.41 1828644 14.1
typecase+types none 26.6 0.42 2043929 14.1
Now, the compilation time of the "stupid" macro is the double of the
"clever" macro but it still looks within acceptable limits. The
compiled file size is just a tiny bit bigger so there's just a small
amount of code bloat.
In the next test, we really stress the Python compiler by including
the (declaim (speed 3) (others 0)) declamation. First, let's see what
happens in the case where no type information is available:
Declaim Compilation Load Size Run
generic (speed 3) 6.0 0.36 1314312 11.5
typecase (speed 3) 85.1 2.24 5800717 8.3
It takes 14 times more time to compile the "stupid" macro than it
takes to compile the "clever" macro (let's not forget that I don't
include the time for the hypotetical type inferencer that runs at
macroexpansion time) and the file is 4.4 times bigger and takes 6
times longer to load. The absolute times and sizes still seem under
reasonable limits. The "stupid" macro generates code that runs
definitely faster than the "clever" macro.
Now, we also include type declarations.
Declaim Compilation Load Size Run
specific (speed 3) 21.8 0.24 836601 8.3
typecase+types (speed 3) 33.3 0.23 835655 8.3
This is, IMHO, the most interesting case. The compilation time is 56%
bigger (as always, I'm not including the time it takes for the
"clever" macro to macroexpand using type information available at
macroexpansion time) but the load time, compiled file size and running
times are the same. No code bloat whatsoever. Althought it looked
like a really hard case for CMUCL, the fact is that it could deal with
it and do exactly the same as a "clever" macroexpansion.
Another interesting result shows up when we see that with the (speed
3) declamation, the typecase is equally fast with or without type
declarations and these declarations seem to affect only the
compilation and load times and the size of the compiled file. Python
experts are welcome to explain this effect. Or to show where I got
things wrong. All of this was done in a hurry and late at night so
there might be errors. I would be glad if others could made similar
tests.
The "stupid" vs "clever" macro test looks to me as another example
that shows that the Python compiler is an extremely clever Common Lisp
compiler and that it would be better to experiment different solutions
without dismissing them beforehand as unacceptable. Congratulations to
the CMUCL team.
Ant�nio Leit�o.
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> The "stupid" vs "clever" macro test looks to me as another example
> that shows that the Python compiler is an extremely clever Common Lisp
> compiler and that it would be better to experiment different solutions
> without dismissing them beforehand as unacceptable.
This is just awesome! CMUCL+typecase is now another trick in my
high-performance Lisp bag.
By the way, how does SBCL do in this test? Perhaps you could put the
full code online somewhere so lazy people like myself can try it out
without having to reconstruct it from the snippets you posted?
> Congratulations to the CMUCL team.
Hear hear!
Dirk Gerrits
From: Duane Rettig
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <4breyek70.fsf@franz.com>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Gareth McCaughan <················@pobox.com> writes:
>
> > You've used the term "statically typed language" a lot,
> > and you've defined it in terms that are ambiguous because
> > a crucial quantifier is missing. Could you please say
> > which of the following (if any) is your definition?
> >
> > 1 "A language in which type information is always
> > available at compile time"
> I guess that point 1 is the most appropriate definition. I do
> consider the following definitions:
>
> A statically typed language is a language where types are known at
> compile time. A dynamically typed language is a language where types
> are discovered at runtime.
I think your terminology is suffering from an "all-or-nothing"
flavor. Perhaps this is the basis for us misunderstanding you.
Let me explain: On the one hand, you answer Gareth's question
by choosing #1, which defines a static language as one where
type information is _always_ available at compile time (emphasis
mine). You then define statically typed language, which implies
again option #1, and also define dynamically in a way that is
ambiguous; we might read it as "a language where _all_ types are
discovered at runtime", or as "a language where at least _some_
types are discovered at runtime". At any rate, these form a
dichotomy. However, you then make this statement:
> Obviously, there are languages that can be both statically and
> dynamically typed (Common Lisp is one example).
which must be patently false based on your definition above. I.e.
if I interpret your definition of dynamically typed language as being
the second sense (the "_some_" sense), then Common Lisp is a
dynamically typed language, period. If I take your definition in the
first sense (the "_all_ sense) then Common Lisp is neither a dynamically
typed nor a statically typed language.
All I'm saying is
> that to have useful type information available at macroexpansion time
> in general, your language or (as Duane already pointed it out) the
> programs that you can write in that language must be statically typed,
> at least, in the fragments that use those macros.
There's that "all or nothing" flavor again. There may be languages
out there that purport to be completely statically typed. But
Common Lisp has never claimed to be a purely dynamically typed;
it is a mixture of both (though not with your absolute sense of
the definitions, but instead on a graded scale where staticism
and dynamism can coexist by degree), and although macros can use
any amount of static information in their operation, the whole
program need not be static in order for a macro to benefit from
that static information.
--
Duane Rettig ·····@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
From: Antonio Menezes Leitao
Subject: Static vs Dynamic [was Re: Sacla loop: a new loop implementation]
Date:
Message-ID: <874qkpenqs.fsf_-_@evaluator.pt>
Duane Rettig <·····@franz.com> writes:
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
>> Gareth McCaughan <················@pobox.com> writes:
>>
>> > You've used the term "statically typed language" a lot,
>> > and you've defined it in terms that are ambiguous because
>> > a crucial quantifier is missing. Could you please say
>> > which of the following (if any) is your definition?
>> >
>> > 1 "A language in which type information is always
>> > available at compile time"
>
>
>> I guess that point 1 is the most appropriate definition. I do
>> consider the following definitions:
>>
>> A statically typed language is a language where types are known at
>> compile time. A dynamically typed language is a language where types
>> are discovered at runtime.
>
> I think your terminology is suffering from an "all-or-nothing"
> flavor. Perhaps this is the basis for us misunderstanding you.
> Let me explain: On the one hand, you answer Gareth's question
> by choosing #1, which defines a static language as one where
> type information is _always_ available at compile time (emphasis
> mine). You then define statically typed language, which implies
> again option #1, and also define dynamically in a way that is
> ambiguous; we might read it as "a language where _all_ types are
> discovered at runtime", or as "a language where at least _some_
> types are discovered at runtime". At any rate, these form a
> dichotomy. However, you then make this statement:
>
>> Obviously, there are languages that can be both statically and
>> dynamically typed (Common Lisp is one example).
>
> which must be patently false based on your definition above. I.e.
> if I interpret your definition of dynamically typed language as being
> the second sense (the "_some_" sense), then Common Lisp is a
> dynamically typed language, period. If I take your definition in the
> first sense (the "_all_ sense) then Common Lisp is neither a dynamically
> typed nor a statically typed language.
>
Is Java statically typed or dynamically typed?
In Java, you must declare the type of all variables. The type can be
a primitive type or a reference type. In the case of a primitive type
you must provide precise type information. In the case of a reference
type, you only need to provide the type information necessary to
assure the compiler that the code will run, although it doesn't
exactly know which code. This seems to fit under the definition of a
statically typed language. However, you also know that for the
reference type case, it will be necessary to discover the actual type
at runtime (and the values will carry that information), thus making
it an example of a dynamically typed language. How would you classify
Java?
I think a similar thing can be said about Common Lisp but in a
slightly degenerate form. If we look at the hyperspec, we find:
4.1 Introduction
...
Objects, not variables, have types. Normally, any variable can have
any object as its value.
So, by "definition", the language is dynamically typed. However, the
same section also says:
Normally, any variable can have any object as its value. It is
possible to declare that a variable takes on only values of a given
type by making an explicit type declaration.
So, it seems to me that a variable that lacks a type declaration can
be seen as a variable with a type declaration of type t because t is
the supertype of all types. This means that everything seems to be
typed at compile time and thus it fits the definition of static typed
language that I presented. Is a variable that has type t _usefully_
typed? It depends on the use of the variable but, in general, to be
able to use the type information at compile time you need to find more
interesting types. Thus, the definition of statically typed language
might also require that the types available at compile time (either by
explicit declarations or implicit type inference) must also be non
trivial. So, if the Common Lisp compiler can find non-trivial type
information, then Common Lisp also fits in the statically typed
category. That's why I said it could be both statically and
dynamically typed.
However, I do agree with you that it is a matter of degree.
What are your definitions?
Ant�nio Leit�o.
From: Matthew Danish
Subject: Re: Static vs Dynamic [was Re: Sacla loop: a new loop implementation]
Date:
Message-ID: <87pt3dwfz3.fsf@mapcar.org>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> Is Java statically typed or dynamically typed?
Neither. It has an idiotic type system, designed to be annoying and
pointless.
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote in message news:<··············@qrnik.zagroda>...
> Antonio Menezes Leitao <··············@evaluator.pt> writes:
>
> > Is Java statically typed or dynamically typed?
>
> http://www.pragmaticprogrammer.com/cgi-local/pragprog?JavaIsUntyped
Java is statically typed, but has a means for allowing developers to
operate on data in an untyped manner. That is the use of the Object
class from which all other classes are inherited. The author of the
article is confusing (or at least using) the properties of inheritance
as an argument to show Java as not statically typed. The properties
of inheritance, combined with the rules of static type checking give
you alot of power, but that power has to be used responsibly. For
inexperienced programmers (such as is illustrated in the link), this
can be dangerous.
Java doesn't assume programmers are idiots, therefore if used
incorrectly, it can give the developer as much rope to hang themselves
as an untyped language. The only difference is you have to ask for
the rope.
·········@hotmail.com (Darren) wrote in message news:<···························@posting.google.com>...
> Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote in message news:<··············@qrnik.zagroda>...
> > Antonio Menezes Leitao <··············@evaluator.pt> writes:
> >
> > > Is Java statically typed or dynamically typed?
> >
> > http://www.pragmaticprogrammer.com/cgi-local/pragprog?JavaIsUntyped
>
> Java is statically typed, but has a means for allowing developers to
> operate on data in an untyped manner. That is the use of the Object
> class from which all other classes are inherited. The author of the
> article is confusing (or at least using) the properties of inheritance
> as an argument to show Java as not statically typed. The properties
> of inheritance, combined with the rules of static type checking give
> you alot of power, but that power has to be used responsibly. For
What does ``used responsibly'' mean? It means verify your program to
see that you didn't make any mistake that the language did not catch.
In other words, write bug-free code without assistance from the tool.
Make sure that the type you are casting the reference into matches the
type of the object, by carefully verifying the program.
Experienced, responsible programmers can develop in anything. They can
write an operating system kernel in assembly language.
The problem is that extra responsibility---even if the programmers are
experienced and able to take on that responsibility and
deliver---represents an extra cost. Taking on unnecessary
responsibility imposed by a braindamaged programming language means
that it takes the experienced, responsible programmer longer to
achieve the usual high level of quality.
> inexperienced programmers (such as is illustrated in the link), this
> can be dangerous.
I don't think that the article's author is confused about anything at
all. You are confused.
> Java doesn't assume programmers are idiots, therefore if used
Java assumes (pretty much correctly, I might add) that the language
implementors are idiots, and also (incorrectly) that the programmers
can somehow make up for that.
> incorrectly, it can give the developer as much rope to hang themselves
> as an untyped language. The only difference is you have to ask for
> the rope.
This is a stupid argument, because it may be the case that you have no
choice but to ask for that rope in the vast majority of the typical
applications forwhich the tool is used in the way it was intended, and
not using the rope will cause you to be massively less productive,
leading to time and cost overruns.
The article we are discussing points out that there is a frequent need
to convert from an Object type to the actual type, because of the use
of polymorphic containers.
Containers for aggregating objects are not some esoteric, infrequently
used feature that rarely shows up in real applications.
···@ashi.footprints.net (Kaz Kylheku) wrote in message news:<····························@posting.google.com>...
> What does ``used responsibly'' mean? It means verify your program to
> see that you didn't make any mistake that the language did not catch.
> In other words, write bug-free code without assistance from the tool.
> Make sure that the type you are casting the reference into matches the
> type of the object, by carefully verifying the program.
How about do the cast?? That would have caused a run-time exception
to be thrown. Error found, problem solved. It's not rocket science.
The fact that it took the developer who found this problem any time at
all to recognize it says he is inexperienced.
For that matter he could have simply ran a debugger and seen the
problem in about 30 seconds.
In almost 8 years of doing Java, I have seen that mistake made twice.
Seems pretty low risk compared to the benefit it provides.
> Experienced, responsible programmers can develop in anything. They can
> write an operating system kernel in assembly language.
>
> The problem is that extra responsibility---even if the programmers are
> experienced and able to take on that responsibility and
> deliver---represents an extra cost. Taking on unnecessary
> responsibility imposed by a braindamaged programming language means
> that it takes the experienced, responsible programmer longer to
> achieve the usual high level of quality.
If you are lazy sure. Create a var for the .next() element, or cast
your use of it. Use the API the way it is supposed to be used.
Trashing a language because you can use it incorrectly is just stupid.
Any language can be used incorrectly.
> > inexperienced programmers (such as is illustrated in the link), this
> > can be dangerous.
>
> I don't think that the article's author is confused about anything at
> all. You are confused.
Whatever. The problem with the code would have been pretty obvious.
Compare his fix to even the most basic examples of using an Iterator,
you will see the structure of the code was the first hint he should
have had to the problem. No language can save you from bad
developers.
>
> > Java doesn't assume programmers are idiots, therefore if used
>
> Java assumes (pretty much correctly, I might add) that the language
> implementors are idiots, and also (incorrectly) that the programmers
> can somehow make up for that.
Sorry..I have been assuming that you were educated about Java, it is
clear you aren't. Please read up on Goslings design goals for Java.
It sounds like you have been reading some anti-Java propoganda or
something.
Javas only goal, in terms of simplifiation, was to remove some of the
common bugs found in C/C++ programs (mostly around memory management).
The language itself is hardly any simpler to learn then C++, nor was
it meant to be. Gosling was only trying to make the language easy for
C and C++ developers to learn, not Joe programmer. So, unless C/C++
developers are idiots, your belief is false.
Your misunderstanding of this sample code and the scope of the problem
it illustrates, is proof the language requires thought to use
correctly. The fact the example is using common interfaces
illustrates this as well.
>
> > incorrectly, it can give the developer as much rope to hang themselves
> > as an untyped language. The only difference is you have to ask for
> > the rope.
>
> This is a stupid argument, because it may be the case that you have no
> choice but to ask for that rope in the vast majority of the typical
> applications forwhich the tool is used in the way it was intended, and
> not using the rope will cause you to be massively less productive,
> leading to time and cost overruns.
Then you know where to be careful don't you. Its not "massively less
productive" adding casts. If it is, then typing courses can help with
that.
>
> The article we are discussing points out that there is a frequent need
> to convert from an Object type to the actual type, because of the use
> of polymorphic containers.
>
> Containers for aggregating objects are not some esoteric, infrequently
> used feature that rarely shows up in real applications.
This is the first statement you have made that I agree with.
When using the containers, you are *supposed* to cast (read up on
their usage). The API's can be used inccorrectly, that is all the
example shows. As I said above, no language can save you from bad
developers.
Iterators are just an interface API, not part of the language core.
If you want strongly typed iterators, you can simply make a new one.
There are tons of API's out there that use custom iterators (or
bridged iterators) for strong type checking.
Considering this is the Lisp forum, a language designed around loose
typing, flexibility and the idea of transforming the language to the
problem, the reason and ability to do this in Java, should not been
seen as a weakness of the language. The flexibility of a language can
be seen as a weakness, but it's generally seen as a strength to anyone
that knows it.
Darren wrote:
> Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> wrote in message news:<··············@qrnik.zagroda>...
>
>>Antonio Menezes Leitao <··············@evaluator.pt> writes:
>>
>>>Is Java statically typed or dynamically typed?
>>
>>http://www.pragmaticprogrammer.com/cgi-local/pragprog?JavaIsUntyped
>
> Java is statically typed, but has a means for allowing developers to
> operate on data in an untyped manner. That is the use of the Object
> class from which all other classes are inherited. The author of the
> article is confusing (or at least using) the properties of inheritance
> as an argument to show Java as not statically typed. The properties
> of inheritance, combined with the rules of static type checking give
> you alot of power, but that power has to be used responsibly. For
> inexperienced programmers (such as is illustrated in the link), this
> can be dangerous.
The code at that link just demonstrates an officially recommended
standard Java idiom. Are you saying that the Java designers have been
inexperienced programmers? Not a very good advertisement for that language.
Generic types were supposed to improve the situation, but they actually
seem to make things worse. See http://www.mindview.net/WebLog
> Java doesn't assume programmers are idiots, therefore if used
> incorrectly, it can give the developer as much rope to hang themselves
> as an untyped language. The only difference is you have to ask for
> the rope.
The explicit goal of the Java designers was to make the language usable
for "average" programmers. They may not be idiots, but the specific
example used in the above cited article doesn't require a lot of asking.
Java's type system doesn't really make the language more powerful. Type
systems that allow you to, say, dispatch on the return type of a method
can do a few things more than dynamically typed languages, but not
Java's. Java's type system actually introduces sources for bugs that you
wouldn't have in some dynamically typed language. See
http://prog.vub.ac.be/~wdmeuter/PostJava04/papers/Costanza.pdf
If you know about studies that actually show how Java's type system is
in any way better than others (without resorting to circular reasoning,
for example by giving proofs of soundness), I would be very interested
in knowing about them.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
> The code at that link just demonstrates an officially recommended
> standard Java idiom. Are you saying that the Java designers have been
> inexperienced programmers? Not a very good advertisement for that language.
The example was not standard, it was wrong, the standard use of
iterators is to assign the iterator result to a variable of the type
you expect. The fixed version follows the standard way to do it with
the addition of this code:
Object key = i.next();
Virtually any example of Iterator will do the same thing.
I wasn't saying java developers were inexperienced, just the guy who
created the initial bad code.
>
> Generic types were supposed to improve the situation, but they actually
> seem to make things worse. See http://www.mindview.net/WebLog
I don't feel I can comment on this because I haven't had enough chance
to explore generics yet... stuck in 1.4.2 for a while and no spare
time atm. :(
> > Java doesn't assume programmers are idiots, therefore if used
> > incorrectly, it can give the developer as much rope to hang themselves
> > as an untyped language. The only difference is you have to ask for
> > the rope.
>
> The explicit goal of the Java designers was to make the language usable
> for "average" programmers. They may not be idiots, but the specific
> example used in the above cited article doesn't require a lot of asking.
This is not actually correct. The goal of Java, according to Gosling,
was to make it easy for C/C++ developers to learn, and remove some of
the more common bugs that occur in those languages. You heard some
bad marketing material or something. When asked in an interview,
Gosling admitted he had no idea how hard it would be for a newbie
programmer to learn Java.
> Java's type system doesn't really make the language more powerful. Type
> systems that allow you to, say, dispatch on the return type of a method
> can do a few things more than dynamically typed languages, but not
> Java's. Java's type system actually introduces sources for bugs that you
> wouldn't have in some dynamically typed language. See
> http://prog.vub.ac.be/~wdmeuter/PostJava04/papers/Costanza.pdf
>
In your paper you used this example:
IPerson dilbert = new Person("Dilbert");
Company dogbert = new Company("Dogbert Consulting");
...
dilbert = dogbert.hire(dilbert);
// returns a wrapper of type Employee with new methods
// old methods are forwarded to the original object
...
System.out.println("Name: " + dilbert.getName());
if (dilbert instanceof Employee) {
System.out.println("Employer: " +
((Employee)dilbert).getEmployer().getName();
}
I did find this interesting but it is still not the languages fault,
it is a threading bug. The same kind of bug can happen in just about
any language when using threads. In C you could have a pointer end up
pointing somewhere else, but it is much more difficult to identify
when it happens. At least Java adds key words to make solving a
problem like that pretty easy and exceptions to help you identify
them.
If faced with this in the future, use a synchronzied block:
synchronized(dilbert){
if (dilbert instanceof Employee) {
System.out.println("Employer: " +
((Employee)dilbert).getEmployer().getName();
}
}
> If you know about studies that actually show how Java's type system is
> in any way better than others (without resorting to circular reasoning,
> for example by giving proofs of soundness), I would be very interested
> in knowing about them.
Sorry, I don't. To be honest, I don't really think it is necessarily
better then other systems. It just is what it is and should be used
as such.
Darren wrote:
>>The code at that link just demonstrates an officially recommended
>>standard Java idiom. Are you saying that the Java designers have been
>>inexperienced programmers? Not a very good advertisement for that language.
>
> The example was not standard, it was wrong, the standard use of
> iterators is to assign the iterator result to a variable of the type
> you expect. The fixed version follows the standard way to do it with
> the addition of this code:
> Object key = i.next();
>
> Virtually any example of Iterator will do the same thing.
I didn't say that the example is standard, but that the idiom is
standard. Yes, the example contains a bug that may be very obvious when
you look at the code _in isolation_. When it's part of a much bigger
project, it's easy to miss the bug because the code looks quite correct
from far away. (Java has too much noise in its source code, which makes
it hard to see the essential stuff.)
> I wasn't saying java developers were inexperienced, just the guy who
> created the initial bad code.
As an experienced Java programmer you need to keep lots of code idioms
and patterns in mind and know them by heart which could be be automated
in other languages. In other words, the experience that Java requires
from programmers is a waste of time.
>>Generic types were supposed to improve the situation, but they actually
>>seem to make things worse. See http://www.mindview.net/WebLog
>
> I don't feel I can comment on this because I haven't had enough chance
> to explore generics yet... stuck in 1.4.2 for a while and no spare
> time atm. :(
You should really read Bruce Eckel's blog entries about them. (AFAICT,
Bruce Eckel is quite an experiented Java programmer.)
>>>Java doesn't assume programmers are idiots, therefore if used
>>>incorrectly, it can give the developer as much rope to hang themselves
>>>as an untyped language. The only difference is you have to ask for
>>>the rope.
>>
>>The explicit goal of the Java designers was to make the language usable
>>for "average" programmers. They may not be idiots, but the specific
>>example used in the above cited article doesn't require a lot of asking.
>
> This is not actually correct. The goal of Java, according to Gosling,
> was to make it easy for C/C++ developers to learn, and remove some of
> the more common bugs that occur in those languages. You heard some
> bad marketing material or something. When asked in an interview,
> Gosling admitted he had no idea how hard it would be for a newbie
> programmer to learn Java.
I quote from one of the original Java white papers from 1996, written by
James Gosling and Henry McGilton:
"The system that emerged to meet these needs is simple, so it can be
easily programmed by most developers; familiar, so that current
developers can easily learn the Java programming language; object
oriented, to take advantage of modern software development methodologies
and to fit into distributed client-server applications; multithreaded,
for high performance in applications that need to perform multiple
concurrent activities, such as multimedia; and interpreted, for maximum
portability and dynamic capabilities."
"Primary characteristics of the Java programming language include a
simple language that can be programmed without extensive programmer
training while being attuned to current software practices. The
fundamental concepts of Java technology are grasped quickly; programmers
can be productive from the very beginning."
See http://java.sun.com/docs/white/langenv/index.html
They mention a number of other design criteria, including the one that
you quote. Note that mentioning one goal doesn't mean that it's the
exclusive goal. Further note that there is a fundamental difference in
having a goal and attaining it. (I also doubt that C++ programers find
it easy to switch to Java.)
>>Java's type system doesn't really make the language more powerful. Type
>>systems that allow you to, say, dispatch on the return type of a method
>>can do a few things more than dynamically typed languages, but not
>>Java's. Java's type system actually introduces sources for bugs that you
>>wouldn't have in some dynamically typed language. See
>>http://prog.vub.ac.be/~wdmeuter/PostJava04/papers/Costanza.pdf
>
> In your paper you used this example:
>
> IPerson dilbert = new Person("Dilbert");
> Company dogbert = new Company("Dogbert Consulting");
> ...
> dilbert = dogbert.hire(dilbert);
> // returns a wrapper of type Employee with new methods
> // old methods are forwarded to the original object
> ...
> System.out.println("Name: " + dilbert.getName());
> if (dilbert instanceof Employee) {
> System.out.println("Employer: " +
> ((Employee)dilbert).getEmployer().getName();
> }
>
> I did find this interesting but it is still not the languages fault,
> it is a threading bug. The same kind of bug can happen in just about
> any language when using threads. In C you could have a pointer end up
> pointing somewhere else, but it is much more difficult to identify
> when it happens. At least Java adds key words to make solving a
> problem like that pretty easy and exceptions to help you identify
> them.
First of all, this was not the only example in my paper, but one of
three examples. Do I understand that you agree to the validity of the
other examples? (Most people with whom I have discussed the paper accept
the first two examples but don't agree to the third one.)
Secondly, I still think the third example is valid. I think it's not
primarily about threading but may happen as well in the single-threaded
case. I don't have a good example though.
> If faced with this in the future, use a synchronzied block:
>
> synchronized(dilbert){
> if (dilbert instanceof Employee) {
> System.out.println("Employer: " +
> ((Employee)dilbert).getEmployer().getName();
> }
> }
I know how to program in Java. I have used it exclusively and
extensively for seven years, since the second beta release of JDK 1.0.
Writing correct multi-threaded programs in Java doesn't consist of
arbitrarily placing synchronized statements in several places. Your
suggestion wouldn't work because you are not synchronizing the other
accesses to "dilbert".
But this is besides the point anyway. The gist of that example is that
the bug occurs only after three hours of repeatedly reassigning
"dilbert" every five seconds. That's a very extreme assumption - in
reality the assignments would occur much less often. The Java language
doesn't help you at all in seeing that there might be a problem. To
restate my statement made in the paper, this is the kind of bug you
definitely don't want to have. It's one of the worst because it is not
reproducible. (And it is based on a "check an object's type before
casting it" idiom as described in the Java language specification!)
>>If you know about studies that actually show how Java's type system is
>>in any way better than others (without resorting to circular reasoning,
>>for example by giving proofs of soundness), I would be very interested
>>in knowing about them.
>
> Sorry, I don't. To be honest, I don't really think it is necessarily
> better then other systems. It just is what it is and should be used
> as such.
Here is my summary:
Java was designed as an environment that is exceptionally easy to use by
inexperienced progammers. However in fact, it takes a lot of experience.
Especially, Java's type system was designed to detect bugs and ensure
"robust and secure" programs. To quote from that white paper again: "The
Java programming language is designed for creating highly reliable
software. It provides extensive compile-time checking, followed by a
second level of run-time checking. Language features guide programmers
towards reliable programming habits."
However, the facts are that a) in many cases, the type system lets very
obivous bugs slip through and b) in many cases, the type system doesn't
let you do what you want which makes you work around the type system and
creates additional sources of bugs.
Why should a complete failure taken as "what it is and [...] be used as
such"?
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<cldj43
> I didn't say that the example is standard, but that the idiom is
> standard. Yes, the example contains a bug that may be very obvious when
> you look at the code _in isolation_. When it's part of a much bigger
> project, it's easy to miss the bug because the code looks quite correct
> from far away. (Java has too much noise in its source code, which makes
> it hard to see the essential stuff.)
When debugging, you would be looking at the code in isolation. For
the last 2 years I have been working on a very large Java project and
this still would have been found in minutes, if not be me then the
debugger.
>
> > I wasn't saying java developers were inexperienced, just the guy who
> > created the initial bad code.
>
> As an experienced Java programmer you need to keep lots of code idioms
> and patterns in mind and know them by heart which could be be automated
> in other languages. In other words, the experience that Java requires
> from programmers is a waste of time.
Yes, casting is something to remember to do, but it's certainly not a
lot to ask and is typical in a typed language supporting polymorphism.
> I quote from one of the original Java white papers from 1996, written by
> James Gosling and Henry McGilton:
>
> "The system that emerged to meet these needs is simple, so it can be
> easily programmed by most developers; familiar, so that current
> developers can easily learn the Java programming language; object
> oriented, to take advantage of modern software development methodologies
> and to fit into distributed client-server applications; multithreaded,
> for high performance in applications that need to perform multiple
> concurrent activities, such as multimedia; and interpreted, for maximum
> portability and dynamic capabilities."
>
> "Primary characteristics of the Java programming language include a
> simple language that can be programmed without extensive programmer
> training while being attuned to current software practices. The
> fundamental concepts of Java technology are grasped quickly; programmers
> can be productive from the very beginning."
>
> See http://java.sun.com/docs/white/langenv/index.html
>
> They mention a number of other design criteria, including the one that
> you quote. Note that mentioning one goal doesn't mean that it's the
> exclusive goal. Further note that there is a fundamental difference in
> having a goal and attaining it. (I also doubt that C++ programers find
> it easy to switch to Java.)
Read over that again, he always uses the word *programmer*. He is
talking about making it *easy for programmers* to learn the language,
not someone who doesn't know how to program. Making a language easy
for programmers is entirely different then making a language easy to
learn programming. That is why Java looks alot like C/C++, the syntax
is instantly recognizable to anyone who knows either. C/C++ are not
generally considered easy to learn for non-programmers. In reality
Java is only slightly easier to learn then C/C++ becuase of what it
borrowed from Lisp, garbage collection. BASIC is easy for someone to
learn programming on, not C/++ or Java.
Here is a more direct answer on the topic from Gosling:
"Q: In your experience, how long does it take for a novice programmer
to become a reasonably proficient [C/C++/Java] developer, capable of
writing nontrivial production code? How long for a programmer with
experience in one or more other languages? How can this time be
shortened?
Gosling: I know that for somebody who is a pretty talented C++
programmer, an afternoon pretty much does it for lots of folks. You'll
probably spend a lot of time going through the library manual. The
language itself tends to be a snap to learn; it's all the library
stuff that takes the time, and there the best way to do it is to just
start writing and using it, and when you need something, look for it.
For people who have never written a program before, I don't know."
(http://www.gotw.ca/publications/c_family_interview.htm)
> First of all, this was not the only example in my paper, but one of
> three examples. Do I understand that you agree to the validity of the
> other examples? (Most people with whom I have discussed the paper accept
> the first two examples but don't agree to the third one.)
Well, I actually didn't agree with "Statically Checked Implementation
of Interfaces"
You show how difficult and dangerous it is to have stub methods in
interface implementators, but there is a SIMPLE, commonly used
solution to this problem.
public class FileCharSequence implements CharSequence, Incomplete {
public FileCharSequence() {...}
public char charAt(int index) {...}
public int length() {...}
}
...
public interface Incomplete {};
By having stub implementations implement the "Incomplete" interface,
they are marked as such. Now that you have a marker interface, you
can have the compiler tell you what you need to finish. Your build
scripts do not make the "Incomplete" interface available. Using
marker interfaces is used all the time for problem like this. By
having the build scripts remove access to the marker interfaces you
can get a complete list of what is missing where. We use ones for
Incomplete, Questionable (put in from code inspections), FlawedLogic,
RequiresReview.
Very handy stuff. It not only removes the problems you listed, it
solves some others and helps ensures a quality of production builds.
For this one: "Statically Checked Exceptions"
I do agree with this one. Though it can be dealt with pretty easily
as well and without limitation, unfortunetly it requires more code.
public double[] calcStatistics() throws Exception {
try {
... make call to data abstraction layer ...
}
catch (ExceptionIKnow a) {}
catch (AnotherExceptionIKnow b) {}
catch (Exception e) { throw e }
}
...
try {
double stat[] = calcStatistics();
} catch (Exception e) {
if (e isntanceof SomeNetworkException)
System.out.println("Make sure you are connected to the server");
.etc.
.etc.
}
>
> Secondly, I still think the third example is valid. I think it's not
> primarily about threading but may happen as well in the single-threaded
> case. I don't have a good example though.
>
> > If faced with this in the future, use a synchronzied block:
> >
> > synchronized(dilbert){
> > if (dilbert instanceof Employee) {
> > System.out.println("Employer: " +
> > ((Employee)dilbert).getEmployer().getName();
> > }
> > }
>
> I know how to program in Java. I have used it exclusively and
> extensively for seven years, since the second beta release of JDK 1.0.
> Writing correct multi-threaded programs in Java doesn't consist of
> arbitrarily placing synchronized statements in several places. Your
> suggestion wouldn't work because you are not synchronizing the other
> accesses to "dilbert".
Your right, but to be a fair example you should have syncrhonized the
methods that can change what Dilbert is anyways. Then this simple fix
would always work and you wouldn't even need to worry about an error
or exception handling, etc. But as you said, that is beside the
point.
I also know Java and have used it since beta. I don't expect it to
make up for incorrect usage of its capabilities. Polymorphism is a
powerful capability that I am willing to live with. The problems you
identify all extend from polymorphism, not type checking. They are
conflicting concepts and for Java to support both, there are trade
offs.
To me, what you are really pointing out is that mixing the
polymorphism and type checking within a language can lead to problem
situations that may be tricky to identify. That I would have to agree
with.
>
> But this is besides the point anyway. The gist of that example is that
> the bug occurs only after three hours of repeatedly reassigning
> "dilbert" every five seconds. That's a very extreme assumption - in
> reality the assignments would occur much less often. The Java language
> doesn't help you at all in seeing that there might be a problem. To
> restate my statement made in the paper, this is the kind of bug you
> definitely don't want to have. It's one of the worst because it is not
> reproducible. (And it is based on a "check an object's type before
> casting it" idiom as described in the Java language specification!)
Well..that example is more about bad thread programming in my opinion,
not idioms. Any bad thread programming in any language is troublesome
to diagnose and reproduce. It's a bad example for the problem you are
illustrating IMHO.
> Here is my summary:
>
> Java was designed as an environment that is exceptionally easy to use by
> inexperienced progammers. However in fact, it takes a lot of experience.
>
> Especially, Java's type system was designed to detect bugs and ensure
> "robust and secure" programs. To quote from that white paper again: "The
> Java programming language is designed for creating highly reliable
> software. It provides extensive compile-time checking, followed by a
> second level of run-time checking. Language features guide programmers
> towards reliable programming habits."
>
> However, the facts are that a) in many cases, the type system lets very
> obivous bugs slip through and b) in many cases, the type system doesn't
> let you do what you want which makes you work around the type system and
> creates additional sources of bugs.
a) No language can save you from bad programming. To catch those
bugs, polymorphism would have to be removed.
b) Again, I see polymorphism as the source of those bugs and I am
happy Java has it.
>
> Why should a complete failure taken as "what it is and [...] be used as
> such"?
I'll tell my customers Java is a failure next time they try to write
up another damn success story about their experience wtih our
software...over 3 million lines of Java, developed in 2 years, under
budget and on-time ;)
- Darren
·········@hotmail.com (Darren) writes:
> a) No language can save you from bad programming. To catch those
> bugs, polymorphism would have to be removed.
> b) Again, I see polymorphism as the source of those bugs and I am
> happy Java has it.
If you want to see REAL polymorphism in action in a statically typed
language, see ML or Haskell, not Java. Java is, for a statically
typed language, a VERY BAD example.
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
> ·········@hotmail.com (Darren) writes:
> > a) No language can save you from bad programming. To catch those
> > bugs, polymorphism would have to be removed.
> > b) Again, I see polymorphism as the source of those bugs and I am
> > happy Java has it.
>
> If you want to see REAL polymorphism in action in a statically typed
> language, see ML or Haskell, not Java. Java is, for a statically
> typed language, a VERY BAD example.
Well, this wasn't originally about polymorphism, but I believe that is
at the heart of the problem in all the examples shown. The conflict
between type checking and polymorphism.
The basic problem this entire discussion is about is this:
class A { }
class B extends A implements Z { public void foobar(); }
class C extends A { }
...
someMethod( new C() );
...
public void someMethod(A something) {
// This is the bug all the examples are pointing out happens,
// and the Java compiler has no way to catch this
// At this point we don't know what type "something" really is, we
only
// know it abides by the contract A
((B)something).foobar(); // ClassCastException *may* be comming
(runtime)
// If an instance of C or A were passed, the code would work. If a
B
// instance is passed a ClassCastException will be thrown.
}
My question for the Lisp, Haskell and ML (and any other language you
want to mention) experts:
Could this kind of developer error be made in those languages?
If not, an example of how to do a similar polymorphic call in those
languages would be appreciated.
- Darren
Darren wrote:
> The basic problem this entire discussion is about is this:
>
> class A { }
> class B extends A implements Z { public void foobar(); }
> class C extends A { }
>
> ...
> someMethod( new C() );
> ...
>
> public void someMethod(A something) {
>
> // This is the bug all the examples are pointing out happens,
> // and the Java compiler has no way to catch this
>
> // At this point we don't know what type "something" really is, we
> only
> // know it abides by the contract A
>
> ((B)something).foobar(); // ClassCastException *may* be comming
> (runtime)
>
> // If an instance of C or A were passed, the code would work. If a
> B
> // instance is passed a ClassCastException will be thrown.
> }
>
> My question for the Lisp, Haskell and ML (and any other language you
> want to mention) experts:
>
> Could this kind of developer error be made in those languages?
In Common Lisp, of course:
(defclass a ()
())
(defclass b (a z)
())
(defclass c (a)
())
(defmethod foobar ((object b))
...)
(defmethod some-method ((something A))
(foobar something))
(some-method (make-instance 'c))
...but note that Common Lisp doesn't claim to prevent such errors. Java
claims to do so, but doesn't do it as well.
There are ways to prevent such errors by not allowing any subclasses
anymore at a certain stage. Then you have total knowledge about (part
of) the class hierachy which allows you to do exhaustive checks at
compile time. But this has severe limitations that I don't think of as
useful either.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message
> > Could this kind of developer error be made in those languages?
>
> In Common Lisp, of course:
>
> (defclass a ()
> ())
>
> (defclass b (a z)
> ())
>
> (defclass c (a)
> ())
>
> (defmethod foobar ((object b))
> ...)
>
> (defmethod some-method ((something A))
> (foobar something))
>
> (some-method (make-instance 'c))
>
>
> ...but note that Common Lisp doesn't claim to prevent such errors. Java
> claims to do so, but doesn't do it as well.
>
> There are ways to prevent such errors by not allowing any subclasses
> anymore at a certain stage. Then you have total knowledge about (part
> of) the class hierachy which allows you to do exhaustive checks at
> compile time. But this has severe limitations that I don't think of as
> useful either.
>
>
> Pascal
Well, you can do that in Java also with "final"
final class A {}
class B extends A() // <-- compile time error
Darren wrote:
> Well, you can do that in Java also with "final"
>
> final class A {}
>
> class B extends A() // <-- compile time error
No, you can't:
class A {}
final class B extends A {)
final class C extends A {}
A, B and C should be the only classes in that hierarchy. How do I
prevent the following?
class D extends A {}
(Of course, A should remain a public class because I need it as a type.)
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> Darren wrote:
>
> > Well, you can do that in Java also with "final"
> >
> > final class A {}
> >
> > class B extends A() // <-- compile time error
>
> No, you can't:
>
> class A {}
> final class B extends A {)
> final class C extends A {}
>
> A, B and C should be the only classes in that hierarchy. How do I
> prevent the following?
>
> class D extends A {}
>
>
> (Of course, A should remain a public class because I need it as a type.)
>
>
> Pascal
Java can't stop that at compile time, but you can have it enforced at
run-time very easily.
public class A {
{ // Subclass Enforcment block
if (!((this.getClass() == A.class) || this instanceof B || this
instanceof C))
throw new RuntimeException("No subclassing allowed. Use A, B or C");
}
//.. rest of A class code here
}
final public B extends A {}
final public C extends A {}
You are thinking inside the box..."let go Luke."
Make a subclass D and try to instantiate it. It won't work.
·········@hotmail.com (Darren) writes:
> You are thinking inside the box..."let go Luke."
The box is defined by what the compiler is allowed to do to optimize
your code.
> Make a subclass D and try to instantiate it. It won't work.
And the compiler will have no access to this information.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
Rahul Jain <·····@nyct.net> wrote in message news:<··············@nyct.net>...
>
> The box is defined by what the compiler is allowed to do to optimize
> your code.
>
> > Make a subclass D and try to instantiate it. It won't work.
>
> And the compiler will have no access to this information.
Why does everything needs to be solved at compile time? Isn't it
enough that it can be solved? Java doesn't claim to be able to stop
inheritance of a base class at compile time.
Darren wrote:
> Rahul Jain <·····@nyct.net> wrote in message news:<··············@nyct.net>...
>
>>The box is defined by what the compiler is allowed to do to optimize
>>your code.
>>
>>>Make a subclass D and try to instantiate it. It won't work.
>>
>>And the compiler will have no access to this information.
>
> Why does everything needs to be solved at compile time?
You have claimed that in Java, that specific case can be caught at
compile time (but you misunderstood the original point, which is ok
because it was probably not clear enough).
None of us in this thread claimed that everything should be done at
compile time. It has just been pointed out that that specific case can
be statically checked in certain languages.
My claim is just that Java's type system isn't very helpful because
there is a good chance that type errors that are flagged by the Java
compiler are not really errors and vice versa, there is also a good
chance that code that is accepted by the type checker is still buggy. If
Java's type system wouldn't hinder you to write straightforward code
this wouldn't be a problem, but there is again a good chance that you
need to write more complex code than necessary in order to convince the
type system to accept your code [1]. Taken together, this doesn't make
Java's type system look especially good.
Other aspects of the Java platform are quite good, but this shouldn't
make you blindly ignore the problems that it also factually has.
> Isn't it
> enough that it can be solved? Java doesn't claim to be able to stop
> inheritance of a base class at compile time.
Your solution looks more like a hack to me. I don't think you would
actually do that in production code.
Pascal
[1] Apparently, one can only understand that latter point when one has
actually seriously tried to use other languages that are better in this
regard.
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message
> You have claimed that in Java, that specific case can be caught at
> compile time (but you misunderstood the original point, which is ok
> because it was probably not clear enough).
I claimed B and C, not A.
> None of us in this thread claimed that everything should be done at
> compile time. It has just been pointed out that that specific case can
> be statically checked in certain languages.
>
> My claim is just that Java's type system isn't very helpful because
> there is a good chance that type errors that are flagged by the Java
> compiler are not really errors and vice versa, there is also a good
> chance that code that is accepted by the type checker is still buggy. If
> Java's type system wouldn't hinder you to write straightforward code
> this wouldn't be a problem, but there is again a good chance that you
> need to write more complex code than necessary in order to convince the
> type system to accept your code [1]. Taken together, this doesn't make
> Java's type system look especially good.
It isn't perfect, I agree. But from my perspective it catches more
then it lets by and I don't find it hinders writing straight forward
code.
>
> Other aspects of the Java platform are quite good, but this shouldn't
> make you blindly ignore the problems that it also factually has.
I was never ignoring them, just saying that in real use they aren't as
bad as many people claim they are.
>
> > Isn't it
> > enough that it can be solved? Java doesn't claim to be able to stop
> > inheritance of a base class at compile time.
>
> Your solution looks more like a hack to me. I don't think you would
> actually do that in production code.
Of course not, but I wouldn't use final either and would never right
code that gets me into that situation in the first place. It was just
an exercise not a recommendation.
Gotta admit though, its a neat little hack ;)
Pascal Costanza <········@web.de> wrote in message
Something about your post was bothering me after thinking about it:
> My claim is just that Java's type system isn't very helpful because
> there is a good chance that type errors that are flagged by the Java
> compiler are not really errors
[..]
> Java's type system wouldn't hinder you to write straightforward code
> this wouldn't be a problem, but there is again a good chance that you
> need to write more complex code than necessary in order to convince the
> type system to accept your code
This I really don't understand becuse I never see this problem. All
of the examples you have provided were based on flawed designs and
flawed implementations.
I guess this is why I have been defending Java. I have read many
criticizms of Java but they always seem to use errored code or bad
designs as the proof. To anyone who understands the language the
response is obvious: "you broke it not the language." (Except your
exception example which really does illustrate a weakness and that I
conceded as a valid criticizm.)
Do you have an example of a good OO design that Java forces you to
implement in an ugly way?
I am not talking about having to put in casts or more lines of code
(generally we all type at 30+ words per minute but only generate
200-500 lines of code a day in any language, point being: brevity !=
better), I am talking about something that hinders the developer and
makes the code more complex.
On Mon, 25 Oct 2004 20:15:19 -0700, Darren wrote:
> I am not talking about having to put in casts or more lines of code
> (generally we all type at 30+ words per minute but only generate
> 200-500 lines of code a day in any language, point being: brevity !=
> better)
This is something that really gets on my nerves. People who I work with
often say they don't mind Java's verbosity because either they don't mind
doing lots of typing (so go be a typist, not a programmer!), or
because they can get their IDEs to do the typing for them.
I don't care. The cost is not in the writing. The cost is in
the reading, the understanding, and the modifying.
Brevity == better!
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
William Bland <·······@abstractnonsense.com> writes:
> On Mon, 25 Oct 2004 20:15:19 -0700, Darren wrote:
>
>> I am not talking about having to put in casts or more lines of code
>> (generally we all type at 30+ words per minute but only generate
>> 200-500 lines of code a day in any language, point being: brevity !=
>> better)
>
> This is something that really gets on my nerves. People who I work with
> often say they don't mind Java's verbosity because either they don't mind
> doing lots of typing (so go be a typist, not a programmer!), or
> because they can get their IDEs to do the typing for them.
>
> I don't care. The cost is not in the writing. The cost is in
> the reading, the understanding, and the modifying.
>
> Brevity == better!
2 a pt.
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> wrote in message
> > This is something that really gets on my nerves. People who I work with
> > often say they don't mind Java's verbosity because either they don't mind
> > doing lots of typing (so go be a typist, not a programmer!), or
> > because they can get their IDEs to do the typing for them.
> >
> > I don't care. The cost is not in the writing. The cost is in
> > the reading, the understanding, and the modifying.
> >
> > Brevity == better!
>
> 2 a pt.
>
> -Peter
I used to think brevity was a good thing. Then I had kids.
My daughter once said to me, "Sup dawg? Fix'n meats?"
I replied, "What?"
She replied, "How are you doing? Are you making supper?"
I replied, "Why didn't you just say that then?"
She replied, "Why should I? If you thought about it you could have
figured out what I meant."
I replied, "Because I would have said, Ok and Yes."
She was enlightened.
·········@hotmail.com (Darren) writes:
> Peter Seibel <·····@javamonkey.com> wrote in message
> > > This is something that really gets on my nerves. People who I work with
> > > often say they don't mind Java's verbosity because either they don't mind
> > > doing lots of typing (so go be a typist, not a programmer!), or
> > > because they can get their IDEs to do the typing for them.
> > >
> > > I don't care. The cost is not in the writing. The cost is in
> > > the reading, the understanding, and the modifying.
> > >
> > > Brevity == better!
> >
> > 2 a pt.
> >
> > -Peter
>
> I used to think brevity was a good thing. Then I had kids.
>
> My daughter once said to me, "Sup dawg? Fix'n meats?"
>
> I replied, "What?"
>
> She replied, "How are you doing? Are you making supper?"
>
> I replied, "Why didn't you just say that then?"
>
> She replied, "Why should I? If you thought about it you could have
> figured out what I meant."
>
> I replied, "Because I would have said, Ok and Yes."
>
> She was enlightened.
Nice story. But it has nothing to do with brevity (good, bad, 2 a
pt., or otherwise).
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
On Tue, 26 Oct 2004 04:49:56 -0700, Darren wrote:
> Peter Seibel <·····@javamonkey.com> wrote in message
>> >
>> > Brevity == better!
>>
>> 2 a pt.
>>
>> -Peter
>
> I used to think brevity was a good thing. Then I had kids.
I used to think spelling everything out was good. Then I started working
with people who insist on writing Java like
public int foo(int bar, boolean baz) {
int ret = 0;
if(bar < 0) {
if(baz==true) {
ret = -bar;
} else if(baz==false) {
ret = bar;
}
} else if(bar >= 0) {
if(baz==true) {
ret = bar;
} else if(baz==false) {
ret = -bar;
}
}
return ret;
}
And I'm not kidding. I really do see people write stuff just like that,
every day. And because it "works", they can get away with claiming that
their code is clearer than the obvious alternatives because they've
"spelled everything out". And they do. Bah!
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
Peter Seibel wrote:
> William Bland <·······@abstractnonsense.com> writes:
>
>>On Mon, 25 Oct 2004 20:15:19 -0700, Darren wrote:
>>
>>>I am not talking about having to put in casts or more lines of code
>>>(generally we all type at 30+ words per minute but only generate
>>>200-500 lines of code a day in any language, point being: brevity !=
>>>better)
>>
>>This is something that really gets on my nerves. People who I work with
>>often say they don't mind Java's verbosity because either they don't mind
>>doing lots of typing (so go be a typist, not a programmer!), or
>>because they can get their IDEs to do the typing for them.
>>
>>I don't care. The cost is not in the writing. The cost is in
>>the reading, the understanding, and the modifying.
>>
>>Brevity == better!
>
> 2 a pt.
To paraphrase Albert Einstein: Make it as brief as possible, but not
briefer. ;)
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Peter Seibel wrote:
> William Bland <·······@abstractnonsense.com> writes:
>
>
>>On Mon, 25 Oct 2004 20:15:19 -0700, Darren wrote:
>>
>>
>>>I am not talking about having to put in casts or more lines of code
>>>(generally we all type at 30+ words per minute but only generate
>>>200-500 lines of code a day in any language, point being: brevity !=
>>>better)
>>
>>This is something that really gets on my nerves. People who I work with
>>often say they don't mind Java's verbosity because either they don't mind
>>doing lots of typing (so go be a typist, not a programmer!), or
>>because they can get their IDEs to do the typing for them.
>>
>>I don't care. The cost is not in the writing. The cost is in
>>the reading, the understanding, and the modifying.
>>
>>Brevity == better!
>
> 2 a pt.
How about "directness == better"?
-- MJF
M Jared Finder <·····@hpalace.com> writes:
> How about "directness == better"?
Or expressing your intent as code is better than unintensional
programming. :)
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
·········@hotmail.com (Darren) writes:
> Do you have an example of a good OO design that Java forces you to
> implement in an ugly way?
Java doesn't seem to do mixins very well.
--
An ideal world is left as an excercise to the reader.
--- Paul Graham, On Lisp 8.1
David Steuber <·····@david-steuber.com> wrote in message news:<··············@david-steuber.com>...
> ·········@hotmail.com (Darren) writes:
>
> > Do you have an example of a good OO design that Java forces you to
> > implement in an ugly way?
>
> Java doesn't seem to do mixins very well.
Good point. Ruby makes mixins simple but Java forces you to use
method bridges. Mixin support in Java would be nice.
Darren wrote:
> Do you have an example of a good OO design that Java forces you to
> implement in an ugly way?
Yes, the Visitor pattern is one.
a) Consider the following code:
class C {
static void m(String s) {
out.println("I have got a string.");
}
static void m(Object o) {
out.println("I have got an object.");
}
static void doSomething(Object o) {
m(o);
}
}
...
C.doSomething("test");
...
This doesn't do what you (or at least I ;) intuitively expect. Compare
this to the following equivalent Common Lisp code:
(defmethod m ((s string))
(print "I have got a string."))
(defmethod m ((o standard-object))
(print "I have got an object."))
(defmethod do-something ((o standard-object))
(m o))
...
do-something("test")
...
Because of dynamic type checking, that code does what it should do.
[The class standard-object is usually not used like that, but it makes
that example clearer. The most general type in Common Lisp is t, not
standard-object.]
This example is relevant to the Visitor pattern becaues it requires you
to add accept methods to each class you want to visit. This is necessary
so that the compiler selects the correct visit method at compile time,
whereas in a dynamically typed language the correct method can be
selected at runtime, so you effectively only need to write it once and
it doesn't need to be specialized.
class ANode {
void accept(Visitor v) {
v.visit(this);
}
...
}
b) In order to be able to statically type-check at all, Java requires
you to textually place all the methods that belong to a class inside
that class. This means that methods "belong" to classes / objects.
However, the Visitor pattern needs to dispatch on two objects, the
visitor and the object it visits. This is called double dispatch and
goes via -> accept (dispatch on the visitor) -> visit (dispatch on the
object).
In Common Lisp, because static typing wasn't part of the design goals,
it was possible to allow method definitions to be placed anywhere you
want. Therefore, you don't need the Visitor pattern at all:
(defgeneric some-functionality (o)
(:method ((o a-node))
...)
(:method ((o another-node))
...))
(defgeneric some-other-functionality (o)
(:method ((o a-node))
...)
(:method ((o another-node))
...))
What the Visitor pattern gives you is a way to group methods according
to the functionality they provide instead of the classes to which they
belong. In Common Lisp, this is just a matter of cut'n'paste methods
whereever you want.
(The Visitor pattern also allows you to traverse object structures, but
this is not the main goal as stated in the Design Patterns book. In
Lisp, you traditionally use walkers for that.)
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> Darren wrote:
>
> > Do you have an example of a good OO design that Java forces you to
> > implement in an ugly way?
>
> Yes, the Visitor pattern is one.
>
> a) Consider the following code:
>
> class C {
> static void m(String s) {
> out.println("I have got a string.");
> }
>
> static void m(Object o) {
> out.println("I have got an object.");
> }
>
> static void doSomething(Object o) {
> m(o);
> }
> }
>
> ...
>
> C.doSomething("test");
>
First, that's not an implementation of the Visitor Pattern.
> ...
>
> This doesn't do what you (or at least I ;) intuitively expect. Compare
> this to the following equivalent Common Lisp code:
>
> (defmethod m ((s string))
> (print "I have got a string."))
>
> (defmethod m ((o standard-object))
> (print "I have got an object."))
>
> (defmethod do-something ((o standard-object))
> (m o))
>
> ...
> do-something("test")
>
> ...
>
> Because of dynamic type checking, that code does what it should do.
Second, neither is this, but if your point is you don't need to
implement the visitor pattern in Lisp because of run-time dispatching,
I agree.
You don't need to in Java either by using Reflection. (Don't try this
in C++) ;)
class C {
void visit(String s) {
out.println("I have got a String.");
}
void visit(Integer o) {
out.println("I have got an Integer.");
}
final public void visit(Object o) {
try {
Method v = this.getClass().getDeclaredMethod("visit", new
Class[]{o.getClass)} );
v.invoke(this, new Object[]{o});
} catch (Exception e) {
.. default handling for Object, or anything else
.. you aren't interested in
}
}
}
...
C.visit("test");
This is barely any more code then the Lisp example and it's just as
flexible. If "C" were subclassed and "public void visit(String s)"
was overriden, Java's reflection lookup would call that one instead of
the base one.
It is also *more powerful* then the Lisp version (at least the one you
supplied) because dispatching happens within the only visible method
"public void visit(Object o)" this allows you to apply "Aspect" style
additions (logging, security checks, other lookups, etc) to this
classes visitor methods, any visitor methods you add in the future, as
well as any subclasses visitors, With no additional code or
constraints on them.
Got any more?
·········@hotmail.com (Darren) writes:
[reflection code omitted]
> This is barely any more code then the Lisp example and it's just as
> flexible. If "C" were subclassed and "public void visit(String s)"
> was overriden, Java's reflection lookup would call that one instead of
> the base one.
You're kidding, right? You would take this very heavyweight and VERY
AWKWARD solution and compare it to what Lisp does simply and easily?
> It is also *more powerful* then the Lisp version (at least the one you
> supplied) because dispatching happens within the only visible method
> "public void visit(Object o)" this allows you to apply "Aspect" style
> additions
You have no idea what you are talking about. See Method Selection and
Combination [1].
[1] http://www.lispworks.com/reference/HyperSpec/Body/07_ff.htm
or for an easier to read tutorial:
http://www.gigamonkeys.com/book/object-reorientation-generic-functions.html
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Matthew Danish <··········@cmu.edu> wrote in message
> You're kidding, right? You would take this very heavyweight and VERY
> AWKWARD solution and compare it to what Lisp does simply and easily?
I am sure when you look at Lisp code you can see many wonderful things
hidden as subtle artifacts of the implementation, language design and
capabilities. That makes you a good Lisp programmer, not a good Java
programmer.
There is alot more going on in that code then you realize :
I made Java do the same thing Lisp does and holds as one of its
powerful capabilities, and did it in almost the same amount of code.
I wasn't supposed to be able to do it all without implementing the
full Visitor pattern and without modifying the types being passed in.
Additionally I kept all method names the same (closer to the visitor
pattern) because I am in control of the dispatching (as opposed to the
Lisp example which was automatic), yet still using a single point of
entry.
I can upcast or downcast my calls to vist() if I want to have the
caller determine the type of visit method it would like to be
processed as, without breaking the entry point or changing the name of
the method called (calling m as opposed to do-something in the Lisp
example).
C.visit((SomeOtherType) o);
I threw in scoping, protection, error handling for unknown types and
uncontrolled subclass errors of unknown types, for shits and giggles.
Basically, its not just doing what the Lisp version does, its doing a
helluva lot more.
As you are to Lisp, I am to Java. You see the code as "very
heavyweight and VERY AWKWARD". Beauty is in the eye of the beholder.
I see and elegant (inspired by Lisps beauty, so how you can you
dislike it! ;) ) solution that contains hidden things as subtle
artifacts of the implementation, language design and capabilities.
To answer your question: No, I wasn't kidding.
> > It is also *more powerful* then the Lisp version (at least the one you
> > supplied) because dispatching happens within the only visible method
> > "public void visit(Object o)" this allows you to apply "Aspect" style
> > additions
>
> You have no idea what you are talking about. See Method Selection and
> Combination [1].
I said, quote: "(at least the one you supplied)".
>
> [1] http://www.lispworks.com/reference/HyperSpec/Body/07_ff.htm
> or for an easier to read tutorial:
> http://www.gigamonkeys.com/book/object-reorientation-generic-functions.html
We were talking about the Visitor Pattern, not the Decorator Pattern
which best describes many of those capabilities, as far as I can tell.
Thanks for the links though. (really, no sarcasm) I am learning lots
about Lisp from you guys.
Cheers,
- Darren
Darren wrote:
> I made Java do the same thing Lisp does and holds as one of its
> powerful capabilities, and did it in almost the same amount of code.
[...]
> Basically, its not just doing what the Lisp version does, its doing a
> helluva lot more.
Defeated! See my other posting. You have to work harder. ;)
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message > Defeated! See my other posting. You have to work harder. ;)
>
>
> Pascal
You got me! I knew I should have tried running that code but was in a
rush for work. :)
After a little work...and for my own self respect. I present the new
and improved version of the Visitor with support for inheritance
direction and support for regex expressions to call a pattern matched
visitor method(s). Each method that matches the pattern is called.
I like this because this also means you can get "decorate" the visited
object while it visits or even use it as a mini-language. More could
be done of course, but its a pretty useful piece of code and I will be
bringing it into work with me.
In honour of my defeat, I have named its package: "lispish"
The gist of its usage is here:
public static void main(String[] args) {
// Static usage, no instantiation, visit now, exact regex match
Visitor.visit(new C(), "visit", new D());
// Another static call with regex method lookup
Visitor.visit(new C(), "visit[1234]", new D());
// An instanted resuable version with a regex
Visitor v = new Visitor(new C(), "visit[2,4]", new D());
v.visit(); // do the visit
}
Output:
I have got a D.
I have got a C. (visit1)
I have got a C. (visit2)
I have got a C. (visit3)
I have got a C. (visit4)
I have got a C. (visit2)
I have got a C. (visit4)
Though much more powerful then the previous version, it has a
limitation that the visit methods must be public because the call
happens by another object. It is generic and could be resused
anywhere you need a "lispish" visitor.
Got a Lisp verion of that to show me. Seeing the rebutals is teaching
me alot. I am getting more and more intrigued by Lisp through these
exercises. Hopefully I am teaching people about some of Javas
capabilities as well.
To those that believe the reflection API in Java is only supposed to
be used for meta programming, it isn't. The reflection API is as much
a part of the language as dynamic types are to Lisp. Things like
J2EE, JavaBeans, Beanshell, etc. couldn't work without it.
Gosling, was inspired by Lisp for Garbage collection and reflection.
It is considered one of the most powerful aspects of Java, and also
the least understood by most java programmers.
----- Cut here --------- Visitor.java
// The "Lispish" Visitor, by Darren Pye (java 1.1+ compatible)
package lispish;
import java.lang.reflect.Method;
import java.util.regex.Pattern;
public class Visitor {
private Object m_Visitor;
private Object m_toVisit;
private String m_visitMethodRegex = "visit";
public Visitor(Object visitor, String visitMethodRegex, Object
toVisit) {
m_Visitor = visitor;
m_visitMethodRegex = visitMethodRegex;
m_toVisit = toVisit;
}
public void visit() {
visit(m_Visitor, m_visitMethodRegex, m_toVisit);
}
public void visit(Object toVisit) {
visit(m_Visitor, m_visitMethodRegex, toVisit);
}
static public void visit(Object visitor, String visitMethodRegex,
Object toVisit) {
Class[] params;
Method methods[] = visitor.getClass().getMethods();
Method visitorMethod = null;
Pattern methodPattern = Pattern.compile(visitMethodRegex);
Class lookingFor = null;
while ((lookingFor = (lookingFor == null ? toVisit.getClass() :
lookingFor
.getSuperclass())) != null)
for (int m = 0; m < methods.length; m++)
if (methodPattern.matcher(methods[m].getName()).matches()) {
params = methods[m].getParameterTypes();
if (params.length == 1) // Only one parameter for a visit
if (params[0] == lookingFor) {
visitorMethod = methods[m];
try {
visitorMethod.invoke(visitor, new Object[] { toVisit
});
} catch (Exception e) { // Do nothing on error
}
}
}
}
}
------ End -------------
The test program:
------ Cut here -------- C.java
import lispish.Visitor;
public class C {
public void visit1(C c) {
System.out.println("I have got a C. (visit1)");
}
public void visit2(C c) {
System.out.println("I have got a C. (visit2)");
}
public void visit3(C c) {
System.out.println("I have got a C. (visit3)");
}
public void visit4(C c) {
System.out.println("I have got a C. (visit4)");
}
public void visit(D d) {
System.out.println("I have got a D.");
}
public static void main(String[] args) {
// Static usage, no instantiation, visit now, exact regex match
Visitor.visit(new C(), "visit", new D());
// Another static call with a regex
Visitor.visit(new C(), "visit[1234]", new D());
// An instanted resuable version with a regex
Visitor v = new Visitor(new C(), "visit[2,4]", new D());
v.visit(); // do the visit
}
}
class D extends C {
}
------ End -------------
On Wed, 27 Oct 2004 22:21:48 -0700, Darren wrote:
>
> Though much more powerful then the previous version, it has a
> limitation that the visit methods must be public because the call
> happens by another object.
See AccessibleObject.setAccessible(boolean). Method is a subclass of
AccessibleObject. The methods do not need to be public - Java's
private/protected/public distinction is all just an elaborate illusion.
> To those that believe the reflection API in Java is only supposed to
> be used for meta programming, it isn't. The reflection API is as much
> a part of the language as dynamic types are to Lisp. Things like
> J2EE, JavaBeans, Beanshell, etc. couldn't work without it.
FWIW I know Java's reflection API pretty well[1], but I still prefer Lisp.
Cheers,
Bill.
[1] see http://c2.com/cgi/wiki?MyBestProgrammingMoment for a cute example
of something I did with it recently - search within that page for
WilliamBland to find it.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
William Bland <·······@abstractnonsense.com> wrote in message news:<······························@abstractnonsense.com>...
> On Wed, 27 Oct 2004 22:21:48 -0700, Darren wrote:
> >
> > Though much more powerful then the previous version, it has a
> > limitation that the visit methods must be public because the call
> > happens by another object.
>
> See AccessibleObject.setAccessible(boolean). Method is a subclass of
> AccessibleObject. The methods do not need to be public - Java's
> private/protected/public distinction is all just an elaborate illusion.
In the code comment I said 1.1+ compatible. Method class inheritance
from AccessibleObject was implemented in 1.2.
Also, it only appears to be an illusion. If the calling app is using
a custom security manager or are using a secure class loader then that
won't work without them adding code for the case. As a generic class
(to be called by other peoples applications) it should not be making
those assumptions.
For your own application though, you could subclass it and modify it
with a call to setAccessible if you wanted to do that though, but if
you subclass it there isn't much point because you would have access
to them anyways.
>
> > To those that believe the reflection API in Java is only supposed to
> > be used for meta programming, it isn't. The reflection API is as much
> > a part of the language as dynamic types are to Lisp. Things like
> > J2EE, JavaBeans, Beanshell, etc. couldn't work without it.
>
> FWIW I know Java's reflection API pretty well[1], but I still prefer Lisp.
>
> Cheers,
> Bill.
>
> [1] see http://c2.com/cgi/wiki?MyBestProgrammingMoment for a cute example
> of something I did with it recently - search within that page for
> WilliamBland to find it.
Nifty!
William Bland <·······@abstractnonsense.com> wrote in message >news:<······························@abstractnonsense.com>...
Before someone posts on it (If you haven't already).
I realized that the class is really only 1.4+ compatible (I forgot
about the regex usage.). Either way, it still shouldn't be assuming
it has access to the private/protected methods of the visiting class,
so it shouldn't be making the call to setAccessible(). If someone
used the class and then latter added a security interceptor, they
could break things that worked before and not know why.
Darren wrote:
> After a little work...and for my own self respect. I present the new
> and improved version of the Visitor with support for inheritance
> direction and support for regex expressions to call a pattern matched
> visitor method(s). Each method that matches the pattern is called.
[...]
> Got a Lisp verion of that to show me. Seeing the rebutals is teaching
> me alot. I am getting more and more intrigued by Lisp through these
> exercises. Hopefully I am teaching people about some of Javas
> capabilities as well.
I don't really understand what you are trying to accomplish here. From a
far way away, it looks to me as if you are trying to encode
meta-information in the lexical method names that you want to grab out
again later on. But it's really hard to tell without more detailed
information.
In Lisp, names aren't just names, but they are symbols with identity
(think: object identity) which gives you much more power to deal with
source code. If we want to attach meta-information to code, we actually
attach meta-information to code. ;)
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> I don't really understand what you are trying to accomplish here. From a
> far way away, it looks to me as if you are trying to encode
> meta-information in the lexical method names that you want to grab out
> again later on. But it's really hard to tell without more detailed
> information.
Not quite. Here is an example that illustrates it better. I have two
classes, Dilbert and Catbert extending from Person. I am visiting
them with a few different methods (hello, talkingTo, insult1 and
insult2) to change what they will say next.
This example utilizes there inheritance from person:
public void hello(Person p) // visit method
..
Visitor.visit(c, "hello", dilbert);
Visitor.visit(c, "hello", catbert);
They both say "Hello".
This example utilizes there individual types:
public void talkingTo(Catbert c)
public void talkingTo(Dilbert c)
..
Visitor.visit(c, "talkingTo", dilbert);
Visitor.visit(c, "talkingTo", catbert);
Dilbert says; "Catbert". Catbert says: "Dilbert"
This example utilizes the regex lookup to determine the "visit"
methods to call:
public void insult1(Catbert c) // will say: "Your a shmuck"
public void insult2(Catbert c) // will say: "and your ugly"
..
Visitor.visit(c, "insult[12]", catbert); // Both insults
...Catbert says: "Your a schmuck and your ugly"
Full output from the test (Just a wacky example):
Dilbert: Hello Catbert.
Catbert: <grumbles>
Catbert: Hello Dilbert.
Catbert: You're a shmuck
Dilbert: <grumbles>
Catbert: You're a shmuck and your ugly.
Method ordering could be added to this as well (like next-method in
Lisp), right now it's natural order. Actually tons of things could be
added to this. I was just fixing the original example, and got
carried away because I could see it mixes the visitor*ish pattern with
a decorator*ish capability (within one class with distinct methods).
The effects are very similar to the patterns.
I wasn't trying to out do Lisp, I am sure Lisp could do this pretty
easily since you can evaluate code at run-time. This just illustrates
a way to do a "Lispish" thing in Java. A pretty handy thing for us
type challenged java monkeys. :)
------
import lispish.Visitor;
public class C {
public void newSentence(Person p) {
p.newSentence();
}
public void hello(Person p) {
p.newSentence();
p.willSay("Hello ");
}
public void talkingTo(Catbert c) {
c.willSay( "Dilbert. " );
}
public void talkingTo(Dilbert c) {
c.willSay( "Catbert. ");
}
public void insult1(Catbert c) {
c.newSentence();
c.willSay( "You're a shmuck");
}
public void insult2(Catbert c) {
c.willSay( " and your ugly.");
}
public void annoyed(Person p) {
p.newSentence();
p.willSay("<grumbles>");
}
public static void main(String[] args) {
Dilbert dilbert = new Dilbert();
Catbert catbert = new Catbert();
C c = new C();
Visitor.visit(c, "hello", dilbert);
Visitor.visit(c, "talkingTo", dilbert);
dilbert.speak();
Visitor.visit(c, "annoyed", catbert);
catbert.speak();
Visitor.visit(c, "hello", catbert);
Visitor.visit(c, "talkingTo", catbert);
catbert.speak();
Visitor.visit(c, "insult1", catbert);
catbert.speak();
Visitor.visit(c, "annoyed", dilbert);
dilbert.speak();
Visitor.visit(c, "insult[12]", catbert);
catbert.speak();
}
}
class Person {
protected String personSays = "";
public void willSay(String s) { personSays += s; }
public void newSentence() { personSays = new String(); }
}
class Dilbert extends Person{
public void speak() { System.out.println("Dilbert: " + personSays);
}
}
class Catbert extends Person{
public void speak() { System.out.println("Catbert: " + personSays);
}
}
------
Darren wrote:
> Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
>
>>I don't really understand what you are trying to accomplish here. From a
>>far way away, it looks to me as if you are trying to encode
>>meta-information in the lexical method names that you want to grab out
>>again later on. But it's really hard to tell without more detailed
>>information.
>
> Not quite. Here is an example that illustrates it better. I have two
> classes, Dilbert and Catbert extending from Person. I am visiting
> them with a few different methods (hello, talkingTo, insult1 and
> insult2) to change what they will say next.
I still don't get it:
(defclass person ()
((person-says :accessor person-says :initform "")))
(defvar *dilbert* (make-instance 'person))
(defvar *catbert* (make-instance 'person))
(defmethod will-say ((p person) (s string))
(setf (person-says p)
(concatenate 'string (person-says p) s)))
(defmethod new-sentence ((p person))
(setf (person-says p) ""))
(defmethod speak ((person (eql *dilbert*)))
(format t "Dilbert: ~A~%" (person-says person)))
(defmethod speak ((person (eql *catbert*)))
(format t "Catbert: ~A~%" (person-says person)))
(defmethod hello((p person))
(new-sentence p)
(will-say p "Hello "))
(defmethod talking-to ((c (eql *catbert*)))
(will-say c "Dilbert. "))
(defmethod talking-to ((d (eql *dilbert*)))
(will-say d "Catbert. "))
(defmethod insult ((n (eql 1)) (c (eql *catbert*)))
(new-sentence c)
(will-say c "You're a shmuck"))
(defmethod insult ((n (eql 2)) (c (eql *catbert*)))
(will-say c " and you're ugly."))
(defmethod annoyed ((p person))
(new-sentence p)
(will-say p "<grumbles>"))
(defun test ()
(hello *dilbert*)
(talking-to *dilbert*)
(speak *dilbert*)
(annoyed *catbert*)
(speak *catbert*)
(hello *catbert*)
(talking-to *catbert*)
(speak *catbert*)
(insult 1 *catbert*)
(speak *catbert*)
(annoyed *dilbert*)
(speak *dilbert*)
(insult 1 *catbert*)
(insult 2 *catbert*)
(speak *catbert*))
? (test)
Dilbert: Hello Catbert.
Catbert: <grumbles>
Catbert: Hello Dilbert.
Catbert: You're a shmuck
Dilbert: <grumbles>
Catbert: You're a shmuck and you're ugly.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
[..] (Thanks for the Lisp version BTW)
> I still don't get it:
I thought it was clear, but I see why its not now. The example was
not fully illustrating the key capability of the lispish.Visitor
class:
I have changed the example to better illustrate (full source bottom of
message):
public static void main(String[] args) {
Object dilbert = new Dilbert(); <<-- was: Dilert dilbert =...
Object catbert = new Catbert(); <<-- was: Catbert catbert =...
C c = new C();
...
The program now only knows dilbert/catbert as type Object.
Normally, Java would only dispatch the variables dilbert/catbert to
method signatures with a parameter Object, without a cast to
(Dilbert)/(Catbert). This is why I called it "lispish". The
lispish.Visitor class is doing dynamic dispatching as opposed to
static dispatching. It matches the run-time type to the compiled
method signature.
You get the beneifits of the Visitor Pattern, with much less code and
no changes to those being visited, same as you can in Lisp because of
dynamic dispatching.
I am horrible at explaining things, I appologize. Hopefully you see
what I mean now.
Same output:
-------------------
Dilbert: Hello Catbert.
Catbert: <grumbles>
Catbert: Hello Dilbert.
Catbert: You're a shmuck
Dilbert: <grumbles>
Catbert: You're a shmuck and your ugly.
----------- C.java
import lispish.Visitor;
public class C {
public void newSentence(Person p) {
p.newSentence();
}
public void hello(Person p) {
p.newSentence();
p.willSay("Hello ");
}
public void speak(Person p) {
p.speak();
}
public void talkingTo(Catbert c) {
c.willSay( "Dilbert. " );
}
public void talkingTo(Dilbert c) {
c.willSay( "Catbert. ");
}
public void insult1(Catbert c) {
c.newSentence();
c.willSay( "You're a shmuck");
}
public void insult2(Catbert c) {
c.willSay( " and your ugly.");
}
public void annoyed(Person p) {
p.newSentence();
p.willSay("<grumbles>");
}
public static void main(String[] args) {
Object dilbert = new Dilbert();
Object catbert = new Catbert();
C c = new C();
// Added a visitor for calling their speak capability
Visitor speaker = new Visitor(c, "speak");
Visitor.visit(c, "hello", dilbert);
Visitor.visit(c, "talkingTo", dilbert);
speaker.visit(dilbert);
Visitor.visit(c, "annoyed", catbert);
speaker.visit(catbert);
Visitor.visit(c, "hello", catbert);
Visitor.visit(c, "talkingTo", catbert);
speaker.visit(catbert);
Visitor.visit(c, "insult1", catbert);
speaker.visit(catbert);
Visitor.visit(c, "annoyed", dilbert);
speaker.visit(dilbert);
Visitor.visit(c, "insult[12]", catbert);
speaker.visit(catbert);
}
}
class Person {
protected String personSays = "";
public void willSay(String s) { personSays += s; }
public void newSentence() { personSays = new String(); }
public void speak() {};
}
class Dilbert extends Person{
public void speak() { System.out.println("Dilbert: " + personSays);
}
}
class Catbert extends Person{
public void speak() { System.out.println("Catbert: " + personSays);
}
}
--------- End
--------- Visitor.java (updated)
//The "Lispish" Visitor, by Darren Pye (java 1.4+ compatible)
package lispish;
import java.lang.reflect.Method;
import java.util.regex.Pattern;
public class Visitor {
private Object m_Visitor;
private Object m_toVisit;
private String m_visitMethodRegex = "visit";
public Visitor(Object visitor, String visitMethodRegex, Object
toVisit) {
m_Visitor = visitor;
m_visitMethodRegex = visitMethodRegex;
m_toVisit = toVisit;
}
public Visitor(Object visitor, String visitMethodRegex) {
m_Visitor = visitor;
m_visitMethodRegex = visitMethodRegex;
}
public void visit() {
visit(m_Visitor, m_visitMethodRegex, m_toVisit);
}
public void visit(Object toVisit) {
visit(m_Visitor, m_visitMethodRegex, toVisit);
}
static public void visit(Object visitor, String visitMethodRegex,
Object toVisit) {
Class[] params;
Method methods[] = visitor.getClass().getMethods();
Method visitorMethod = null;
Pattern methodPattern = Pattern.compile(visitMethodRegex);
Class lookingFor = null;
while ((lookingFor = (lookingFor == null ? toVisit.getClass() :
lookingFor.getSuperclass())) != null)
for (int m = 0; m < methods.length; m++)
if (methodPattern.matcher(methods[m].getName()).matches()) {
params = methods[m].getParameterTypes();
if (params.length == 1) // Only one parameter for a visit
if (params[0] == lookingFor) {
visitorMethod = methods[m];
try {
visitorMethod.invoke(visitor, new Object[] { toVisit
});
} catch (Exception e) { // Do nothing on error
}
}
}
}
}
------- End
Darren wrote:
> Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> [..] (Thanks for the Lisp version BTW)
>
>>I still don't get it:
>
>
> I thought it was clear, but I see why its not now. The example was
> not fully illustrating the key capability of the lispish.Visitor
> class:
>
> I have changed the example to better illustrate (full source bottom of
> message):
>
> public static void main(String[] args) {
> Object dilbert = new Dilbert(); <<-- was: Dilert dilbert =...
> Object catbert = new Catbert(); <<-- was: Catbert catbert =...
> C c = new C();
> ...
>
> The program now only knows dilbert/catbert as type Object.
Yet, it seems to me that the Lisp version is still much simpler.
> Normally, Java would only dispatch the variables dilbert/catbert to
> method signatures with a parameter Object, without a cast to
> (Dilbert)/(Catbert). This is why I called it "lispish". The
> lispish.Visitor class is doing dynamic dispatching as opposed to
> static dispatching. It matches the run-time type to the compiled
> method signature.
Yes, you have actually written an interpreter for a Java-like
object-oriented language that traverses a class hierarchy by hand to
find matching methods. Not very impressive. (Note that the Lisp version
can be compiled.)
Please check the original claim: Java's static type systems makes some
coding idioms unnecessarily complicated. You're just circumventing the
type system by implementing your own dispatcher. I could do this as well
in assembly language. That's what I call unnecessarily complicated. Why
doesn't do Java the right thing (tm) from the start?
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> Yes, you have actually written an interpreter for a Java-like
> object-oriented language that traverses a class hierarchy by hand to
> find matching methods. Not very impressive. (Note that the Lisp version
> can be compiled.)
It wasn't supposed to be impressive. It was supposed to illustrate
that Java can easily do what Lisp is doing to solve the Visitor
Pattern without implementing the Visitor Pattern. Your right about it
not being compiled though. The Lisp version would be faster...almost
as fast as if dispatching were done say...oh I don't know....staticly?
;)
> Please check the original claim: Java's static type systems makes some
> coding idioms unnecessarily complicated. You're just circumventing the
> type system by implementing your own dispatcher. I could do this as well
> in assembly language. That's what I call unnecessarily complicated. Why
> doesn't do Java the right thing (tm) from the start?
Ouch, I surely hope you wouldn't have to resort to assembly for that
in Lisp, not when it is so simple to do in Java with the standard
libraries. Surely Lisp can intercept the dispatching and reroute
without going that far.
Why doesn't Java do it that way from the start? It doesn't need to.
It gets the benefits of static types along with the power of dynamic
types. Given a choice between one or the other, they choose both with
a preference for static.
Darren wrote:
> Ouch, I surely hope you wouldn't have to resort to assembly for that
> in Lisp, not when it is so simple to do in Java with the standard
> libraries.
Of course not. It's just to say that you can implement anything in any
language because of Turing equivalence, so providing an implementation
of dynamic dispatch is not the crux of the whole matter.
> Surely Lisp can intercept the dispatching and reroute
> without going that far.
Common Lisp doesn't need to intercept anything for the examples we have
discussed so far. This is one of the problems with you Java people: You
are happy if you can simulate small bits of other languages, but totally
forget the big picture that you are missing something more important
that cannot be captured in little toy examples.
> Why doesn't Java do it that way from the start? It doesn't need to.
> It gets the benefits of static types along with the power of dynamic
> types. Given a choice between one or the other, they choose both with
> a preference for static.
Blech.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
·········@hotmail.com (Darren) writes:
> To those that believe the reflection API in Java is only supposed to
> be used for meta programming, it isn't. The reflection API is as much
> a part of the language as dynamic types are to Lisp. Things like
> J2EE, JavaBeans, Beanshell, etc. couldn't work without it.
And they wouldn't be so slow without it... Seriously, if you need to
resort to using an interpreter at runtime (that is effectively what
reflecting to find the right function to call is), then you're obviously
not doing something that's accepted behavior, expecially in a bondage
and discipline culture like Java (macros are bad, closures around
function locals are bad, etc).
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
Darren wrote:
> First, that's not an implementation of the Visitor Pattern.
I didn't claim that the first part of my posting was an implementation
of the Visitor pattern. It is just one step in explaining the problem
with the Visitor pattern. It would have gotten more clear with the
second part.
Do you have any specific reasons why you haven't read the whole posting?
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Darren wrote:
> [...] if your point is you don't need to
> implement the visitor pattern in Lisp because of run-time dispatching,
> I agree.
>
> You don't need to in Java either by using Reflection. (Don't try this
> in C++) ;)
>
> class C {
> void visit(String s) {
> out.println("I have got a String.");
> }
>
> void visit(Integer o) {
> out.println("I have got an Integer.");
> }
>
> final public void visit(Object o) {
> try {
> Method v = this.getClass().getDeclaredMethod("visit", new
> Class[]{o.getClass)} );
> v.invoke(this, new Object[]{o});
>
> } catch (Exception e) {
> .. default handling for Object, or anything else
> .. you aren't interested in
> }
> }
> }
>
> ...
>
> C.visit("test");
>
>
> This is barely any more code
...except for the exception code that you have left out...
> then the Lisp example and it's just as flexible.
Nope. Consider this variation:
import java.lang.reflect.*;
class C {
void visit(C c) {
System.out.println("I have got a C.");
}
final public void visit(Object o) {
System.out.println("Generic visit called.");
try {
Method v = this.getClass().getDeclaredMethod
("visit", new Class[]{o.getClass()} );
v.invoke(this, new Object[]{o});
} catch (Exception e) {
//.. default handling for Object, or anything else
//.. you aren't interested in
}
}
}
class D extends C {
static void doIt(C c, Object o) {
c.visit(o);
}
public static void main(String[] args) {
doIt(new C(), new D());
}
}
java D
=> Generic visit called.
This means that your code doesn't handle polymorphism correctly - your
meta-level code would have to reimplement Java's inheritance rules.
(That's the whole point of metaprogramming, BTW, to be able to implement
your own rules, not to fix deficiencies of the base language.)
The equivalent Common Lisp code is much simpler and automagically does
the intuitively right thing (tm):
(defclass c ()
())
(defmethod visit ((c c))
(print "I have got a C."))
(defclass d (c)
())
(defun do-it (o)
(visit o))
(visit (make-instance 'd))
=> "I have got a C."
[Note that this is complete working code. I haven't left out any import
statements or exception handling blocks as you did in your previous
posting.]
> Got any more?
Note that this whole exercise is not about being able to find ad-hoc
solutions for toy example problems, but about being able to extrapolate
what effects the deficiencies may have in larger real-world code.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> writes:
> [The class standard-object is usually not used like that, but it makes
> that example clearer. The most general type in Common Lisp is t, not
> standard-object.]
That's because it cannot be used like this.
(typep "foo" 'standard-object) => NIL
But it doesn't detract from the rest of your example.
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Matthew Danish wrote:
> Pascal Costanza <········@web.de> writes:
>
>>[The class standard-object is usually not used like that, but it makes
>>that example clearer. The most general type in Common Lisp is t, not
>>standard-object.]
>
> That's because it cannot be used like this.
>
> (typep "foo" 'standard-object) => NIL
Oh right. I wanted the example to be closer to Java, but I haven't
tested the code. (The only reliable way to see whether code actually
works. (tm) ;)
> But it doesn't detract from the rest of your example.
Yep. But thanks for pointing it out.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
·········@hotmail.com (Darren) wrote in message news:<···························@posting.google.com>...
> Pascal Costanza <········@web.de> wrote in message
>
> Something about your post was bothering me after thinking about it:
>
> > My claim is just that Java's type system isn't very helpful because
> > there is a good chance that type errors that are flagged by the Java
> > compiler are not really errors
> [..]
> > Java's type system wouldn't hinder you to write straightforward code
> > this wouldn't be a problem, but there is again a good chance that you
> > need to write more complex code than necessary in order to convince the
> > type system to accept your code
>
> This I really don't understand becuse I never see this problem. All
> of the examples you have provided were based on flawed designs and
> flawed implementations.
>
> I guess this is why I have been defending Java. I have read many
> criticizms of Java but they always seem to use errored code or bad
> designs as the proof. To anyone who understands the language the
> response is obvious: "you broke it not the language." (Except your
> exception example which really does illustrate a weakness and that I
> conceded as a valid criticizm.)
>
> Do you have an example of a good OO design that Java forces you to
> implement in an ugly way?
I have an abstract syntax tree whose nodes are implemented as class
objects. The class hierarchy follows the phrase structures in the
grammar, and the tree is heterogeneous. One node might be derived from
a Declaration, another from Expression or whatever.
I want to do a particular traversal of the tree and perform some
operation at every node, possibly collecting information (such a a
syntax-directed translation or whatever).
The framework must support the selection of different kinds of
operations, and specializing them to the particular type of node that
is being processed.
E.g. I might want to use this to pretty-print the tree, to evaluate
it, or to translate it to some code. I may need a context object for
the traversal, and it would be nice if the operation also varied based
on the type of that context object.
The Lisp solution looks something like
(traverse #'generic-function tree context)
Inside the bowels of TRAVERSE, what happens is this:
(funcall func node context)
Dispatch takes place based on the type of both NODE and CONTEXT, so
you have total flexibility. Specify any suitable generic function, and
specialize its methods to any context type and node type however you
want.
> I am not talking about having to put in casts or more lines of code
> (generally we all type at 30+ words per minute but only generate
> 200-500 lines of code a day in any language, point being: brevity !=
> better), I am talking about something that hinders the developer and
> makes the code more complex.
What is better is:
- smaller patches to implement a change
- less stuff to *read* and *understand*
- ability to experiment with changes to the program without having to
make irrelevant adjustments just to get it to compile!
This last point is important. I've done some refactorings of Lisp code
where I didn't acually fix the entire program all at once. I
experimented with a risky change and ended up with a half-broken
program. But enough of that program worked that I was able to get it
running and test just the new modifications, being careful not to step
on the broken parts. Once I knew that I was going in the right
direction and thus ``de-risked'' the proposed change, I began to
ripple it through the rest of the program, gaining more and more
confidence as more and more of the program worked with the new data
structures.
On Tue, 26 Oct 2004 15:59:04 -0700, Kaz Kylheku wrote:
> - ability to experiment with changes to the program without having to
> make irrelevant adjustments just to get it to compile!
>
> This last point is important. I've done some refactorings of Lisp code
> where I didn't acually fix the entire program all at once. I
> experimented with a risky change and ended up with a half-broken
> program. But enough of that program worked that I was able to get it
> running and test just the new modifications, being careful not to step
> on the broken parts. Once I knew that I was going in the right
> direction and thus ``de-risked'' the proposed change, I began to
> ripple it through the rest of the program, gaining more and more
> confidence as more and more of the program worked with the new data
> structures.
I haven't actually done so yet, but something else I can see being useful
in this situation is the ability to save an image. I can imagine being in
a tricky half-broken state like you describe above, then being just about
to call a function that *should* work, but then realising that the
function changes a lot of state (and maybe the current state took me a
long time to get to, so I don't really want to go back to the start) so
it might be a good idea to save an image just in case the function is
still broken - that way I can load the saved image, fix the function, and
move on with all that state intact.
Do people do this kind of thing?
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
···@ashi.footprints.net (Kaz Kylheku) wrote in message
[..]
An interesting program. What kind of program is this? AI, a language,
or something else? (just curious)
> - smaller patches to implement a change
> - less stuff to *read* and *understand*
> - ability to experiment with changes to the program without having to
> make irrelevant adjustments just to get it to compile!
>
> This last point is important. I've done some refactorings of Lisp code
> where I didn't acually fix the entire program all at once. I
> experimented with a risky change and ended up with a half-broken
> program. But enough of that program worked that I was able to get it
> running and test just the new modifications, being careful not to step
> on the broken parts. Once I knew that I was going in the right
> direction and thus ``de-risked'' the proposed change, I began to
> ripple it through the rest of the program, gaining more and more
> confidence as more and more of the program worked with the new data
> structures.
I have written a similar kind of application in Java that uses a
context class and a node class which has an implementation of the
Command pattern inside of it. What the program does basically is
reads a chunk of text and through multiple passes (building up a tree
based on groups and previous "knowledge"), recognizes "concepts"
contained within it. Each subsequent pass on the newly formed tree
recognizes higher and higher level "concepts". Its basciallly a
learning program that can identify commonalities in textual structure.
The knowledge and concepts can be serialized and applied during in
future runs. That way you teach it a little bit at a time. Start
with letters, work up to teaching it words, etc.
First lesson might be like this:
H
E
L
O
Now anytime it sees H,E,L,O again, it has knowledge and a concept to
represent them.
Second lesson:
HELLO
First pass recognizes the letters and identifies them as being of the
same concept. The second pass builds a new concept branch with 5
nodes containing the concepts of letters H, E, L, L, O. Each of those
subsequently point to a single "knowledge" that contains (knowledge is
something with concrete data) of a single type Letter (its not named,
but it has the concept of a Letter by now learned in the first lesson
along with the letters). After the second pass it has the new concept
of a word with the knowledge of a word containing the characters HEL(2
knowledge instances, one concept "L" of type letter)O . Its more
complicated then that, but you get the idea.
Further teaching identifies phrases, sentences, etc. Basically
whatever you want to throw at it. You can throw source code at it and
have it learn the structure of a language for example and know and
recognize it in the future.
Evaulations, comparisions and other operations are done on nodes by
executing the "commands" they contain. Knowledge reacts one way,
concepts another (there are also other types of primitive concepts for
finite data identification "Char", etc). The nodes all have the same
base inheritance of "Node" and return command objects.
The application doesn't use casting because it only needs to deal with
subclasses of node objects and command objects for everything and
those objects don't require additional external methods.
My point is just that you can get the same kind of beneifts you
outlined in your explanation by the design. Excessive or unecesary
casting in Java (or any type sensetive OO language) tends to be a
result of bad design in my experience.
·········@hotmail.com (Darren) writes:
> generally we all type at 30+ words per minute but only generate
> 200-500 lines of code a day in any language,
Actually, on a productive day I can _remove_ about 200-500 lines from
a project's code base. (And I can _delete_ code at a much faster rate
than 30 wpm ... I can delete thousands upon thousands of lines per
minute :-)
> point being: brevity != better
Exactly.
> I am talking about something that hinders the developer and makes
> the code more complex.
Indeed: Java's type system.
My ability to reduce the size of the code is severely hampered by type
systems such as Java's.
·········@hotmail.com (Darren) wrote in message news:<···························@posting.google.com>...
> Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
> > ·········@hotmail.com (Darren) writes:
> > > a) No language can save you from bad programming. To catch those
> > > bugs, polymorphism would have to be removed.
> > > b) Again, I see polymorphism as the source of those bugs and I am
> > > happy Java has it.
> >
> > If you want to see REAL polymorphism in action in a statically typed
> > language, see ML or Haskell, not Java. Java is, for a statically
> > typed language, a VERY BAD example.
>
> Well, this wasn't originally about polymorphism, but I believe that is
> at the heart of the problem in all the examples shown. The conflict
> between type checking and polymorphism.
>
> The basic problem this entire discussion is about is this:
>
> class A { }
> class B extends A implements Z { public void foobar(); }
> class C extends A { }
>
> ...
> someMethod( new C() );
> ...
>
> public void someMethod(A something) {
>
> // This is the bug all the examples are pointing out happens,
> // and the Java compiler has no way to catch this
>
> // At this point we don't know what type "something" really is, we
> only
> // know it abides by the contract A
>
> ((B)something).foobar(); // ClassCastException *may* be comming
> (runtime)
>
> // If an instance of C or A were passed, the code would work. If a
> B
> // instance is passed a ClassCastException will be thrown.
> }
>
> My question for the Lisp, Haskell and ML (and any other language you
> want to mention) experts:
>
> Could this kind of developer error be made in those languages?
Of course the error can be made in any language. How can the
programmer be prevented from asking for an operation that doesn't
exist?
The difference is in the level of intelligence in how the situation is
represented and handled in the language.
In Common Lisp, you would just try applying the FOOBAR method to the
object. That could blow up if there is no method specialization for
that object.
The error condition which arises is *specific*. It's not a condition
about a cast failing, because C is not of type B! It's about a method
not being found.
In the language with statically typed references, the error asserts
that you must make the C object support the protocol of B in order to
make this code work. You know nothing about foobar at all because the
error happened even before foobar came into question.
In Lisp, the error tells you that you just have to write a method
specialization to support object C. You can do this simply by writing
that method, not by extending the definition of a class.
You do not have to connect your C class with class B in any way.
Don't you see that it's silly to have type information in two places,
namely in the objects themselves and in the lexical references used to
manipulate them?
In the C language, it makes sense to have type information in the
variables and expressions that refer to data storage, because that
data storage has no type information in it. This is a justifiable
design because programs in that language can then work directly with
types and storage formats imposed by hardware, satisifying the niche
requirement for a high level assembler.
To have the type information in two places is darn stupid.
When type information is contained in objects themselves, it must be
removed from the expressions and lexical references to those objects.
Otherwise you run into completely artificial and unnecessary problems
arising from having to keep the two views in sync.
Think about it, whenever there is a conflict between the two views,
which is taken to be the correct information? It's always the run-time
type of the object which is taken to be correct, and the static
information to be incorrect.
The cast failed not because the object didn't satisfy the static
program, but because the static program made wrong assumptions about
the object!
Why create a whole class of programming errors which have to do with
the redundant, make-believe static type network failing to reflect
what is actually real at run time? I mean, the running software
actually has to look at the run-time types in order to catch these
errors anyway!
In C, the beauty of casting a pointer is that it lets you reinterpret
some object *without generating any additional machine language*.
Converting a pointer from one type to another is almost always a no-op
on the majority of machine architectures, except in such cases where
pointers to bytes might have a slightly different representation (more
bits) than pointers to words and such.
You have the type information in one place, the static program text,
and you trust it. That's what is meant by ``C trusts the programmer''.
Now if you don't trust the programmer, and you peek at the type
information in the objects in order to verify that the static program
is correct, then why require that untrusted information to be there in
the first place?
> If not, an example of how to do a similar polymorphic call in those
> languages would be appreciated.
Why are you asking people to show you how to do something impossible?
You know darn well that the foobar method can't be applied to a C
object, and that this has nothing to do with the programming language.
There are much better ways to handle that situation, however.
(foobar something) ;; works, or else method not found.
Could this be implemented in Java? The problem is that there is more
silliness there than just the static type system. You can't even say
``something.foobar()'' if object is not an instance of a class that
has a foobar function in it.
The static type system is conflated with the lexical resolution of
identifiers!
Hey, why not? If the type system is static, and scopes are static, why
not mix them together in some terrible way, so that we can actually
have the parser itself peek at type information and throw away
semantic errors as if they were lexical errors!
That's why the ((B) something).foobar() cast is written in the Java
program, because you can't write the equivalent something.foobar(),
even though ``something'' might be of type B.
···@ashi.footprints.net (Kaz Kylheku) wrote in message
> > Could this kind of developer error be made in those languages?
>
> Of course the error can be made in any language. How can the
> programmer be prevented from asking for an operation that doesn't
> exist?
That was my point (but see below where you mention this again). All
the errors shown are polymophic problems, not type problems.
> > The difference is in the level of intelligence in how the situation is
> represented and handled in the language.
>
> In Common Lisp, you would just try applying the FOOBAR method to the
> object. That could blow up if there is no method specialization for
> that object.
>
> The error condition which arises is *specific*. It's not a condition
> about a cast failing, because C is not of type B! It's about a method
> not being found.
>
> In the language with statically typed references, the error asserts
> that you must make the C object support the protocol of B in order to
> make this code work. You know nothing about foobar at all because the
> error happened even before foobar came into question.
The error doesn't need to be specific beyond what will be displayed:
java.lang.ClassCastException: B
at Test.someMethod(Test.java:14)
The class, the file, the line and the attempt to cast to B is all you
need. This is just quibbling over the difference between a typed and
untyped language. Just as you can say that, I could say Lisp can't
*really* tell you what is wrong, the problem isn't that the method
"foobar" is missing, it is that you are trying to talk to the wrong
type of object.
In both languages, you get enough information to fix the problem.
> In Lisp, the error tells you that you just have to write a method
> specialization to support object C. You can do this simply by writing
> that method, not by extending the definition of a class.
>
> You do not have to connect your C class with class B in any way.
I would argue that this is actually worse.
What about the next method called on C that isn't a match?
Also, you don't need to connect B to C at all either (In #1-6 below
that isn't even listed because it would be bad OO)
Type checking identifies the real problem in this case and explictly
tells you what is wrong. The problem isn't C doesn't have the method,
the problem is you are doing something wrong with C or have a design
problem.
1) Perhaps C should not be passed in the first place?
2) Perhaps you need to have C implement Z and cast from (B) to (Z)
3) Perhaps you should should change "someMethod" to take a Z as a
paramter instead of an A
4) Perhaps you should perform an instance check and call a method more
suitable to C's purpose.
5) Perhaps you should use the reflection API to inspect "something"
and peek inside of "something" to see what it what methods it has
available.
6) Perhaps you should make A implement Z
> Don't you see that it's silly to have type information in two places,
> namely in the objects themselves and in the lexical references used to
> manipulate them?
>
> In the C language, it makes sense to have type information in the
> variables and expressions that refer to data storage, because that
> data storage has no type information in it. This is a justifiable
> design because programs in that language can then work directly with
> types and storage formats imposed by hardware, satisifying the niche
> requirement for a high level assembler.
>
> To have the type information in two places is darn stupid.
See my previous point. Having type information in the objects
themselves enforces good design, and better object normalization.
> Think about it, whenever there is a conflict between the two views,
> which is taken to be the correct information? It's always the run-time
> type of the object which is taken to be correct, and the static
> information to be incorrect.
>
> The cast failed not because the object didn't satisfy the static
> program, but because the static program made wrong assumptions about
> the object!
err...exactly, the static program is in error and should be fixed.
> Why create a whole class of programming errors which have to do with
> the redundant, make-believe static type network failing to reflect
> what is actually real at run time? I mean, the running software
> actually has to look at the run-time types in order to catch these
> errors anyway!
The reason is the flexibility it provides. Consider the following
possible fix (#4 from above...#5 would be more elaborate and very
generic for a case where you know nothing at all about C):
if (something instanceof B) // Incase of B
((B)something).foobar();
else if (something instanceof C) // Incase of C
((C)something).aMethodOfC();
By testing the type at run-time you can adapt "someMethod" to the case
without changing B or C
> Now if you don't trust the programmer, and you peek at the type
> information in the objects in order to verify that the static program
> is correct, then why require that untrusted information to be there in
> the first place?
Same could be said about Lisp though.
The reason Java does the test is for stability. In C, that kinda cast
could crash the application or worse, in Java (and I assume Lisp) the
method will fail, but the application will continue to execute.
> Why are you asking people to show you how to do something impossible?
> You know darn well that the foobar method can't be applied to a C
> object, and that this has nothing to do with the programming language.
Actually, depending on the compiler, this could be caught. Bug
finders can find this kind of problem and generate a warning,
therefore a compiler could also find it. In the example, there is
enough info:
someMethod( new C() ); // explicit type of C passed to someMethod()
public void someMethod(A something) {
((B)something).foobar(); // unconditional cast to B
}
Warning at : someMethod( new C() ) <<-- Possible cast exception
resulting in someMethod()
I was wondering if any of those languages had a check for this, there
are bug finders for Java that can, but it's not part of the compiler
spec.
> There are much better ways to handle that situation, however.
>
> (foobar something) ;; works, or else method not found.
>
> Could this be implemented in Java? The problem is that there is more
> silliness there than just the static type system. You can't even say
> ``something.foobar()'' if object is not an instance of a class that
> has a foobar function in it.
But why would you want to? It would be incorrect.
If it Java were made to be totally static typing, it could not have
polymorphism. If there is a language that somehow does this, I would
love to see it. In the mean time, I like static typing becuase when
you are sifting through millions of lines of code it makes things a
helluva lot eaiser. It also forces you to think about your design a
bit more and guides you to cleaner object implmentations.
Does it really matter though? It's really just about preference.
Static or dynamic. Both are powerful and both have their uses
I have not seen any argument that has shown me how horrible Java's
type system is. It provides as much checking as can be done without
limiting or basterdizing its implementation of polymorphism.
·········@hotmail.com (Darren) writes:
> Just as you can say that, I could say Lisp can't
> *really* tell you what is wrong, the problem isn't that the method
> "foobar" is missing, it is that you are trying to talk to the wrong
> type of object.
It obviously made sense to try to call foobar on that object in the
code. Therefore, there must be some way to implement foobar so that it
works as intended on the type that the object belongs to. It isn't a
type error, it's simply an incomplete program.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
·········@hotmail.com (Darren) wrote in message news:<···························@posting.google.com>...
> ···@ashi.footprints.net (Kaz Kylheku) wrote in message
> > In the language with statically typed references, the error asserts
> > that you must make the C object support the protocol of B in order to
> > make this code work. You know nothing about foobar at all because the
> > error happened even before foobar came into question.
>
> The error doesn't need to be specific beyond what will be displayed:
> java.lang.ClassCastException: B
> at Test.someMethod(Test.java:14)
>
> The class, the file, the line and the attempt to cast to B is all you
> need. This is just quibbling over the difference between a typed and
> untyped language. Just as you can say that, I could say Lisp can't
> *really* tell you what is wrong, the problem isn't that the method
> "foobar" is missing, it is that you are trying to talk to the wrong
> type of object.
>
> In both languages, you get enough information to fix the problem.
That information comes from the dynamic types of the object, and in
the Java case, it is revealed when the static type system is defeated
with a cast. The static system brought nothing to the table.
The added static layer provides no new information; it's just extra
baggage which tries to tell a fictional story about what is likely
happening at run-time.
Don't get me wrong, once in a while it's good to put an assertion into
the program (not just regarding type, but about any other state
information).
But there are several things wrong in the Java design.
Firstly, it retains the idea that an object's type comes from the
expression which refers to it. Thus in the expression ((B)
something).foobar() the subexpression ((B) something) produces a
reference of type B, to which you can now apply the member selection
.foobar. This language feature was inherited from C by way of C++.
Secondly, symbolic references are conflated with the type system. In
order to resolve the meaning of something.foobar(), the compiler has
to look up semantic information about the something identifier, and
verify that it refers to an object of a class. Associated with that
class is a symbol namespace in which foobar() can be found.
Thus, it's impossible to remove declarations from the language;
without those declarations, an expression like something.foobar()
cannot be semantically analyzed. The static type information is needed
to translate the program, even though objects have perfectly good type
information in them at run time.
Lisp does not share this design feature with Java at all; it has a
completely different approach. (foobar something) is an ordinary
function call, which can be translated without knowing anything about
the SOMETHING symbol. It is not the case that the definition of FOOBAR
is sought after in some special namespace associated with the class to
which SOMETHING belongs. There is no such thing as class scope.
This is why we don't have to have anything resembling a cast operator:
we do not have to override the static type system in order to create
an expression of some desired type, merely in order to gain the right
to be able express the call to the foobar() function.
I suspect that you are laboring under some misconception that static
typing is needed in order to resolve these expressions---which is true
in a design like Java, but is not a mathematical necessity.
> > In Lisp, the error tells you that you just have to write a method
> > specialization to support object C. You can do this simply by writing
> > that method, not by extending the definition of a class.
> >
> > You do not have to connect your C class with class B in any way.
>
> I would argue that this is actually worse.
>
> What about the next method called on C that isn't a match?
You can deal with the problem one method at a time, or just read the
program. There is probably a set of related methods, all of which you
have to implement so that the object can participate in the protocol
involving these methods.
> Also, you don't need to connect B to C at all either (In #1-6 below
> that isn't even listed because it would be bad OO)
If some code wants my object to be of type B, and I cannot change that
code, then I must inherit from that base.
> Type checking identifies the real problem in this case and explictly
> tells you what is wrong.
Yes, RUN TIME type checking! Not static type checking. You defeated
static type checking with the cast, remember? Static type checking
would just tell you that class A doesn't have the foobar() method.
Okay, put in the cast, and so now the static type system has been shut
up and there is a potential run-time error that the object is not of
type B.
> See my previous point. Having type information in the objects
> themselves enforces good design, and better object normalization.
That is not static typing though, which I'm arguing against. I want
type in objects. Just not in the reference variables that refer to
them.
> > Think about it, whenever there is a conflict between the two views,
> > which is taken to be the correct information? It's always the run-time
> > type of the object which is taken to be correct, and the static
> > information to be incorrect.
> >
> > The cast failed not because the object didn't satisfy the static
> > program, but because the static program made wrong assumptions about
> > the object!
>
> err...exactly, the static program is in error and should be fixed.
Not necessarily. Like Rahul said, perhaps the program is merely
incomplete. Without recompiling that module with its erroneous
line---no, indeed, without even having source code to it, we can fix
the program by writing the additional methods and loading them into
the image.
> > Why create a whole class of programming errors which have to do with
> > the redundant, make-believe static type network failing to reflect
> > what is actually real at run time? I mean, the running software
> > actually has to look at the run-time types in order to catch these
> > errors anyway!
>
> The reason is the flexibility it provides.
Whaaat? How do static anything provide flexibility?
> Consider the following
> possible fix (#4 from above...#5 would be more elaborate and very
> generic for a case where you know nothing at all about C):
>
> if (something instanceof B) // Incase of B
> ((B)something).foobar();
> else if (something instanceof C) // Incase of C
> ((C)something).aMethodOfC();
>
> By testing the type at run-time you can adapt "someMethod" to the case
> without changing B or C
Yes, by implementing a run-time check, coupled with casts which defeat
the obtuse static type system.
All you are doing here is chores that the machine really ought to be
doing. The ((B) something) cast exists only so that you can write ((B)
something).foobar(), because if you write something.foobar(), then
foobar is not found, or the wrong foobar is found.
We have a sane object system in Lisp which doesn't use classes as
symbol tables.
Thus we can just write:
(typecase something
(b (foobar something))
(c (a-method-on-c something))
(otherwise ... handle ...)))
Same idea, heck of a lot less typing, thanks to no casts, and
TYPECASE.
Same flexibility, no static typing.
> > Now if you don't trust the programmer, and you peek at the type
> > information in the objects in order to verify that the static program
> > is correct, then why require that untrusted information to be there in
> > the first place?
>
> Same could be said about Lisp though.
How so? Lisp doesn't require declarations. And when they are
optionally specified, and the right optimization and safety tradeoffs
are asked for in that block of code, the compiler *will* trust them.
In Lisp, declarations really are for performance. Not for establishing
the meaning of expressions.
> > There are much better ways to handle that situation, however.
> >
> > (foobar something) ;; works, or else method not found.
> >
> > Could this be implemented in Java? The problem is that there is more
> > silliness there than just the static type system. You can't even say
> > ``something.foobar()'' if object is not an instance of a class that
> > has a foobar function in it.
>
> But why would you want to? It would be incorrect.
It's not incorrect if that object actually is an instance of a class
that has a foobar() method. It's just that the static type of the
reference says that the object is of some other type.
Suppose that methods were not confined to class scopes, but were
global. The compiler would then generate code which handles the
``something.foobar()'' dispatch, with a built-in run-time check.
Bingo, you don't have to write a cast at all. Does the foobar method
apply to the something object, that's the only relevant question.
> If it Java were made to be totally static typing, it could not have
> polymorphism. If there is a language that somehow does this, I would
> love to see it. In the mean time, I like static typing becuase when
> you are sifting through millions of lines of code it makes things a
> helluva lot eaiser. It also forces you to think about your design a
> bit more and guides you to cleaner object implmentations.
Cleaner, as in:
if (something instanceof B)
((B)something).foobar();
else if (something instanceof C)
((C)something).aMethodOfC();
versus
(typecase something
(b (foobar something))
(c (a-method-on-c something)))
Ironically, count the parentheses and compare. :)
You're just making work for yourself with all these stupid
declarations which, because they are wrong half the time, you have to
override with casts. And then when you change something in the
program, you have tons of extra editing to do.
Suppose you do a simple class renaming. Firstly, stupid Java will
force you to rename a file, but let's skip that issue. Because you
have all these casts throughout the code to that class type, you now
have to edit all those casts to use the new class name.
> Does it really matter though?
Yes, it does!
> It's really just about preference.
> Static or dynamic. Both are powerful and both have their uses
But not stupid mixtures of the two, in which we always trust the
dynamic type, and often have to defeat the static type in order to get
dynamically correct things to compile!
Static type by itself: useful tool of the low level programmer.
Dynamic type by itself: useful high level programming tool.
Combining them in a C-like language? Beyond stupid.
> I have not seen any argument that has shown me how horrible Java's
> type system is. It provides as much checking as can be done without
> limiting or basterdizing its implementation of polymorphism.
Only to the extent that you can write a cast anywhere you want to turn
it off.
The problem is that in some cases, your program will end up being
littered with casts.
In Java you can't even write something like Lisp's FUNCALL for
indirecting upon an arbitrary function, e.g.
(let ((method #'some-method))
...
(funcall method object-1 object-2 object-3))
FUNCALL takes arguments of any type whatsoever, and calls the method.
The correct dispatch takes place, as if the method were directly
called like this:
(some-method object-1 object-2 object-3)
How would the Java version of FUNCALL deal with the problem that
everything passing thorugh it is just an Object, but the target
function takes references to specific classes like A, B or C?
Where will you put the casts? Do you write a shim function that takes
Object references, casts them, and then calls the function you really
want?
···@ashi.footprints.net (Kaz Kylheku) wrote in message
You make some good points, but they really only apply for bad designs.
A rule of thumb is: If you find yourself making lots of casts you
probably have a bad OO model.
> But not stupid mixtures of the two, in which we always trust the
> dynamic type, and often have to defeat the static type in order to get
> dynamically correct things to compile!
>
> Static type by itself: useful tool of the low level programmer.
>
> Dynamic type by itself: useful high level programming tool.
>
> Combining them in a C-like language? Beyond stupid.
>
> > I have not seen any argument that has shown me how horrible Java's
> > type system is. It provides as much checking as can be done without
> > limiting or basterdizing its implementation of polymorphism.
>
> Only to the extent that you can write a cast anywhere you want to turn
> it off.
>
> The problem is that in some cases, your program will end up being
> littered with casts.
If you are a bad Java OO designer, yes. You gain nothing particularly
useful from either. If you have a good design, you have scoping and
checking when you should and you dont rely on casts to fix your lack
of design. Casting in the examples we have been evaluating is only
necessary because the designs are bad.
> In Java you can't even write something like Lisp's FUNCALL for
> indirecting upon an arbitrary function, e.g.
>
> (let ((method #'some-method))
> ...
> (funcall method object-1 object-2 object-3))
>
> FUNCALL takes arguments of any type whatsoever, and calls the method.
> The correct dispatch takes place, as if the method were directly
> called like this:
>
> (some-method object-1 object-2 object-3)
>
> How would the Java version of FUNCALL deal with the problem that
> everything passing thorugh it is just an Object, but the target
> function takes references to specific classes like A, B or C?
>
> Where will you put the casts? Do you write a shim function that takes
> Object references, casts them, and then calls the function you really
> want?
You can very easily using reflection without casts. See my solution
to Pascal's Visitor Pattern problem. When using reflection properly
you don't even need casts...and its the static types that allow you to
do it.
·········@hotmail.com (Darren) writes:
> If you are a bad Java OO designer, yes. You gain nothing particularly
> useful from either. If you have a good design, you have scoping and
> checking when you should and you dont rely on casts to fix your lack
> of design. Casting in the examples we have been evaluating is only
> necessary because the designs are bad.
No; casting is necessary in order to have polymorphism in Java. If
you have been paying attention to my other responses, you would
realize that Java's way of doing polymorphism is inferior and badly
designed. It is Java's fault.
> You can very easily using reflection without casts. See my solution
> to Pascal's Visitor Pattern problem. When using reflection properly
> you don't even need casts...and its the static types that allow you to
> do it.
This is funny. Do you know where the ideas behind Java's "reflection"
API come from? Take a wild guess.
(And using reflection in such a manner is still quite ridiculous,
remember what reflection is for: inquiring about the system, and
perform operations upon the system itself; not for manipulating
objects within the system!)
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
> ·········@hotmail.com (Darren) writes:
> > If you are a bad Java OO designer, yes. You gain nothing particularly
> > useful from either. If you have a good design, you have scoping and
> > checking when you should and you dont rely on casts to fix your lack
> > of design. Casting in the examples we have been evaluating is only
> > necessary because the designs are bad.
>
> No; casting is necessary in order to have polymorphism in Java. If
> you have been paying attention to my other responses, you would
> realize that Java's way of doing polymorphism is inferior and badly
> designed. It is Java's fault.
It is necessary to a degree yes, but if you have to cast excessively
you have an inefficient and poor design and that's not Java's fault.
>
> > You can very easily using reflection without casts. See my solution
> > to Pascal's Visitor Pattern problem. When using reflection properly
> > you don't even need casts...and its the static types that allow you to
> > do it.
>
> This is funny. Do you know where the ideas behind Java's "reflection"
> API come from? Take a wild guess.
Yes I know where it came from. Many languages have capabilities that
were inspired by Lisp. Lisp has many powerful ideas. I use Linux
from time to time, maybe Linux is funny because it was inspired by
unix? Maybe Lisp is funny because it was inspired by machine code?
What's your point?
>
> (And using reflection in such a manner is still quite ridiculous,
> remember what reflection is for: inquiring about the system, and
> perform operations upon the system itself; not for manipulating
> objects within the system!)
That's *one* use of reflection. It is intended for MUCH more then
that. Using reflection like I have is done all the time. Probably
not by Joe IT programmer, but by people that know the language. Ever
hear of J2EE, Javabeans, IDE's, debuggers, Beanshell...(insert
hundreds of other applications here) take a guess how they work.
From your "funny" response, I am sure you have no idea about Java or
how to properly use it. Which might explain why you end up with so
many casts and see it as a huge problem.
·········@hotmail.com (Darren) writes:
> That's *one* use of reflection. It is intended for MUCH more then
> that. Using reflection like I have is done all the time. Probably
> not by Joe IT programmer, but by people that know the language. Ever
> hear of J2EE, Javabeans, IDE's, debuggers, Beanshell...(insert
> hundreds of other applications here) take a guess how they work.
When you use reflection like this, it tells me that the base language
was not suitable. Because reflection is for creating new languages.
Hence "meta-programming." Certainly there are problems which will
require this sort of meddling. But you seem to be proposing that the
Reflection API should be used in some fashion for all problems! That
indicates a fundamental problem with the original Java model.
I do not make use of the CLOS Meta-Object Protocol very often. When I
do, I cover up the dirty details with macros and functions so that the
real code does not have to be peppered with MOPology.
> From your "funny" response, I am sure you have no idea about Java or
> how to properly use it. Which might explain why you end up with so
> many casts and see it as a huge problem.
Perhaps, since universities are well known for their poor teaching of
programming languages...
The problem with casts is that Java requires downcasting for its form
of polymorphism. Are you denying that? Any casts, or any time I have
to write down the name of a type explicitly, is bad. Because that
causes a "type lock-in" which hurts later on, especially when you are
trying to program generically.
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
>
> When you use reflection like this, it tells me that the base language
> was not suitable.
If you evaluate code at run-time in Lisp, is it meddling? No, its
part of the language. This is no different, it is part of the
language. Reflection as a base API distinguishes Java from its C++
origin and was intended to do so.
http://java.sun.com/developer/technicalArticles/ALT/Reflection/
www-106.ibm.com/developerworks/java/library/j-dyn0603/
> Because reflection is for creating new languages.
> Hence "meta-programming." Certainly there are problems which will
> require this sort of meddling. But you seem to be proposing that the
> Reflection API should be used in some fashion for all problems! That
> indicates a fundamental problem with the original Java model.
I don't suggest people use it for all problems. You normally only use
it when other language capabilities won't suffice, like what I was
doing.
> I do not make use of the CLOS Meta-Object Protocol very often. When I
> do, I cover up the dirty details with macros and functions so that the
> real code does not have to be peppered with MOPology.
>
> > From your "funny" response, I am sure you have no idea about Java or
> > how to properly use it. Which might explain why you end up with so
> > many casts and see it as a huge problem.
>
> Perhaps, since universities are well known for their poor teaching of
> programming languages...
>
> The problem with casts is that Java requires downcasting for its form
> of polymorphism. Are you denying that? Any casts, or any time I have
> to write down the name of a type explicitly, is bad. Because that
> causes a "type lock-in" which hurts later on, especially when you are
> trying to program generically.
I agree it requires downcasts for polymorphism. But I don't agree on
the amount that will hurt later on. Use interfaces instead of class
inheritance where you want to be generic and nimble. Marker
interfaces are great for stabalizing types if you need to. Like the
Serializable interface is used in Java. Markers won't lock you into a
contract that you many not want to, or be able to keep later on. They
also allow you to group objects of similar types and pass them around
as such. Inheritance locks you into one hiarchy, but interfaces
don't.
interface Something ; // marker interface
class Person implements Something {..}
class GeorgeBush extends Person implements Something {..}
class Dog implements Something {..}
class Blackhole implements Something { public void
removeFromUniverse(Something s);}
Blackhole blackHole = new BlackHole();
GeorgeBush idiot = new GeorgeBush();
Dog dog = new Dog();
blackHole.removeFromUniverse(idiot);
blackHole.removeFromUniverse(dog);
See what I mean? Marker interfaces don't lock you into a contract. I
made Person, GeorgeBush and Dog all of a type Something (an object
group) but they didn't have to have specific methods on them that tied
them to the type, becuase the interface has no methods. No casting
required and no lock-in, generic.
I apologize for the typos on several of my last posts (they are
getting worse).
I have been running on 2-3 hours sleep per night, for the last 4 days
and my brain is starting to seize up. My son has been sick and needs
to be watched at night: Asthma + a cold = bad.
I'll try to proof things better.
·········@hotmail.com (Darren) writes:
> Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
>>
>> When you use reflection like this, it tells me that the base
>> language was not suitable.
>
> If you evaluate code at run-time in Lisp, is it meddling? No, its
> part of the language. This is no different, it is part of the
> language. Reflection as a base API distinguishes Java from its C++
> origin and was intended to do so.
Actually, it's quite rare to need to evaluate code at runtime in Lisp,
if you mean use EVAL. Between closures and macros, there's actually
little need to ever use EVAL. So I actually think your analogy is more
apt than you meant--EVAL in Lisp should only be used when it is the
*only* way to do something. And if you've put yourself in a situation
where EVAL *is* the only way to do something, chances are you missed a
turn a few steps back.
FWIW, I worked from 1997-2003 as a Java programmer including as one of
the early developers at Weblogic. Unless things have changed quite
radically since I was doing it, there are a lot of Java programmers
who do not share your enthusiasm for reflection. Reflection is a okay
for certain kinds of meta programming and tool development--I wrote
several JUnit like test frameworks that use reflection. But it is--in
my experience--not an every day tool. For one thing, it's such a
godawful clumsy way to do anything and there are no macros to ease the
pain. Obviously, YMMV. (I actually dug reflection a lot when I was a
Java programmer but I'm the kind of guy who likes to push languages
around a little bit; I knew that overuse of it was both probably a bad
idea and non-idiomatic. The nice thing about Common Lisp is that all
the kinds of programming that you can sort of get a glimpse of using
things like reflection are, in fact, natural and idiomatic uses of the
language.)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> wrote in message
>
> Reflection is a okay
> for certain kinds of meta programming and tool development--I wrote
> several JUnit like test frameworks that use reflection. But it is--in
> my experience--not an every day tool. For one thing, it's such a
> godawful clumsy way to do anything and there are no macros to ease the
> pain. Obviously, YMMV. (I actually dug reflection a lot when I was a
> Java programmer but I'm the kind of guy who likes to push languages
> around a little bit; I knew that overuse of it was both probably a bad
> idea and non-idiomatic.
I agree, you should only use reflection when necessary. Java has many
other ways to skin that cat, just as Lisp does. But there is nothing
wrong with using it when it is the best tool for the problem. A
problem like the one Pascal put before me is a good example of when to
use it.
> The nice thing about Common Lisp is that all
> the kinds of programming that you can sort of get a glimpse of using
> things like reflection are, in fact, natural and idiomatic uses of the
> language.)
>
> -Peter
I am begining to see. :)
·········@hotmail.com (Darren) writes:
> Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
>>
>> When you use reflection like this, it tells me that the base language
>> was not suitable.
>
> If you evaluate code at run-time in Lisp, is it meddling?
Reflection isn't evaluating code. It's analyzing the properties of
various internal bits of the language and being able to call them more
abstractly than the base language allows. Good reflection APIs (unlike
Java's) should be able to allow for you to also intercede -- you should
be able to change the way the functionality of the base language works
for your specific functions/classes/whatever-is-being-reflected-upon.
Evaluating code at run-time (which is actually the _definition_ of
run-time: when the code is evaluated. I assume you mean evaluating some
passed-in expression using EVAL) is usually an indication that you used
a macro where a function would be more appropriate or at least some
functional interface would be more appropriate. However, if that code
comes from the end-user or from something else outside of the current
application, then you're merely providing a powerful interface.
A few reflection operators in Lisp are GET-SETF-EXPANSION, MACROEXPAND,
SPECIAL-OPERATOR-P, and (given AMOP) CLASS-SLOTS.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
Darren wrote:
> [...] if you have to cast excessively you have an inefficient and
> poor design [...]
Says who?
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
Pascal Costanza <········@web.de> wrote in message news:<············@f1node01.rhrz.uni-bonn.de>...
> Darren wrote:
>
> > [...] if you have to cast excessively you have an inefficient and
> > poor design [...]
>
> Says who?
>
>
> Pascal
I have 12 developers working for me. They vary in experience and
skills from university grad newbies up to minor-deties. When I do
code inspections the difference is blatantly obvious. The
inexperienced guys are casting all over the place because their design
(or lack of it) is forcing them to. The experienced OO developers are
completely different. They cast, but it is always obvious why it was
necessary and it is typically just when using things like iterators or
collections (though 1.5 will minimize that).
Just as an example: Java internally implements many design patterns
and has a huge code base, it is flexible (obviously it is the
language) and it is complete. How many casts do you find in a typical
java source files? They are hard to find.
I have been doing OO for about 16 years and in my experience the most
common problem found in any OO programming is the OO design. It is to
easy to fix a bad design by casting, so people do it. Things aren't
as bad as they used to be because of refactoring IDE's. Which make
refactoring simple. Hopefully more people will start using them.
C++ has the same problem. People tend do the same thing instead of
doing it right.
Gosling and Stroustrup said basically the same thing in the interview
I pointed you at earlier:
http://www.gotw.ca/publications/c_family_interview.htm
I suspect, though have no proof, that Lisp allows you to get away with
a bad design easier then a typed language (or at least its harder to
see if its a bad design). Though, I don't know that is necessarily a
bad thing since the problem is so common.
·········@hotmail.com (Darren) wrote in message news:<···························@posting.google.com>...
> Pascal Costanza <········@web.de> wrote in message news:<············@f1node01.rhrz.uni-bonn.de>...
> > Darren wrote:
> >
> > > [...] if you have to cast excessively you have an inefficient and
> > > poor design [...]
> >
> > Says who?
> >
> >
> > Pascal
>
> I have 12 developers working for me. They vary in experience and
> skills from university grad newbies up to minor-deties. When I do
> code inspections the difference is blatantly obvious. The
> inexperienced guys are casting all over the place because their design
> (or lack of it) is forcing them to. The experienced OO developers are
> completely different. They cast, but it is always obvious why it was
> necessary and it is typically just when using things like iterators or
> collections (though 1.5 will minimize that).
That simply because the experienced programmers have enough brain
cycles to spare that they can reject the obvious, good designs that
are peppered with casts, and search for ones that will draw less
criticism. Not only that, but it creates an extra challenge which
alleviates boredom. Okay, I know how to solve this problem in fifty
different ways. *Yawn*. How about I not only solve it, but to make it
interesting for myself, I will do it with the fewest possible number
of casts?
Luckily, I discovered Common Lisp right around the time I was
experiencing this kind of boredom, which enabled me to move on to
higher level challenges. :)
How do you know that the design with minimal casts is the best one?
Maybe it needs extra framework to support it, for instance. For
instance, one object ends up being wrapped in another so that its type
information can be preserved as it travels through some pathway in the
program. The wrapper's job is purely to handle some tricky dispatch,
it has no role in the problem domain being solved and obscures the
code.
Or maybe you have to push methods too far into base classes so that
they are available for you when you don't know that an object of type
Base is really of type Derived. That could be bad design, just for the
sake of eliminating casts!
Why do you think there is a tendency to push features into bases? It's
because of the static type systems and their upcasts.
I have new method to add. It really only applies to Derived objects,
but if I put it into that class, I won't be able to use it in all the
places where Derived objects are robbed of their manifest type are
just known as Base. One deceptively attractive solution is to just put
the method into Base anyway. Everything derived from Base gets some
default implementation, and so the calls will work silently, even if
they don't make sense. Look, Ma! No casts. But is it good design?
In Lisp, you never have to unnecessarily specialize a generic function
too deeply into the class hierarchy. If some two-argument method only
makes sense over a widget and grommet combination, then just write it
for that.
How about that Null Object design pattern? In Lisp, just specialize
for the NULL type, to match the NIL value.
> Just as an example: Java internally implements many design patterns
> and has a huge code base, it is flexible (obviously it is the
> language) and it is complete. How many casts do you find in a typical
> java source files? They are hard to find.
> I have been doing OO for about 16 years and in my experience the most
> common problem found in any OO programming is the OO design. It is to
> easy to fix a bad design by casting, so people do it.
So your doctrine is that every good OO design can be fit into a
statically typed program with no casts.
From what branch of computing research does this magic result come
from?
> I suspect, though have no proof, that Lisp allows you to get away with
> a bad design easier then a typed language (or at least its harder to
You mean *manifestly* typed language.
Lisp is a *latently* typed language.
This discussion revolves around the realization that Java is both at
the same time.
> see if its a bad design).
If you can't see it no matter how hard you look, it doesn't exist.
If your design goal is to implement your latently typed program such
that the number of wrinkles in its parallel manifest typing is
minimized, then that can turn into a measure of the success of your
design.
There is no need to have that as a measure of success when the
manifest type
system doesn't exist in the tools that you are currently using.
There are probably still assembly language programmers who define
their good designs in terms of minimized execution cycles or code
size. They look at the assembly language programs written by their
colleagues and note that the experienced ones write smaller, faster
programs. They are better designers, so there must be a correlation
between that and good design!
There is a human tendency to identify the accomplishment of anything
that requires effort as a success. When humans work with tools that
require extra effort that is not related to the task achieved with
those tools, they regard it as a success when they overcome those
intrinsic difficulties in those tools. Sometimes that definition of
success clouds the real objective. Thus for instance the computer
programmer is happy when the program has next to no casts in it, even
though it's flopping in the marketplace and the company is going down
the tubes. Over years, it becomes a matter of personal pride, leading
to the rejection of tools that don't have those intrinsic
difficulties, and disdain for the work achieved with those easier
tools, even though it may be of better quality.
Say, as a Java user, what would you say to a C++ programmer who
claimed this: In a good program design, the programmer always knows
where and when it is safe to delete an object, so there is no need for
garbage collection. Some of those Java programs that rely on garbage
collection are actually bad design, but it's just hard to tell it
because both good designs and bad designs are garbage collected. If
you just go through the exercise of sticking in the explicit
deletions, you will be able to tell: in the good design, this will be
easy to do! But under the nasty, bad design, it won't be obvious where
the object lifetimes end. Down with garbage collection, phooey!
···@ashi.footprints.net (Kaz Kylheku) wrote in message news:<····························@posting.google.com>...
>
> That simply because the experienced programmers have enough brain
> cycles to spare that they can reject the obvious, good designs that
> are peppered with casts, and search for ones that will draw less
> criticism. Not only that, but it creates an extra challenge which
> alleviates boredom. Okay, I know how to solve this problem in fifty
> different ways. *Yawn*. How about I not only solve it, but to make it
> interesting for myself, I will do it with the fewest possible number
> of casts?
Trust me, we had NO spare cycles to play with code. The application
is gigantic and we built our own framework to support it as well.
Hell, getting the software done on time was a minor miracle ;)
>
> Luckily, I discovered Common Lisp right around the time I was
> experiencing this kind of boredom, which enabled me to move on to
> higher level challenges. :)
>
> How do you know that the design with minimal casts is the best one?
> Maybe it needs extra framework to support it, for instance. For
> instance, one object ends up being wrapped in another so that its type
> information can be preserved as it travels through some pathway in the
> program. The wrapper's job is purely to handle some tricky dispatch,
> it has no role in the problem domain being solved and obscures the
> code.
There are definetly times when casting comes into play in a good
design. The piont is that in a bad design casts pop-up all over the
place. With proper object decomposition you end up with the right
amount of objects with the right inheritance hierarchy. Then objects
flow through the code rather then get mashed through with casts.
>
> Or maybe you have to push methods too far into base classes so that
> they are available for you when you don't know that an object of type
> Base is really of type Derived. That could be bad design, just for the
> sake of eliminating casts!
True, it can work the opposite way.
> Why do you think there is a tendency to push features into bases? It's
> because of the static type systems and their upcasts.
>
> I have new method to add. It really only applies to Derived objects,
> but if I put it into that class, I won't be able to use it in all the
> places where Derived objects are robbed of their manifest type are
> just known as Base. One deceptively attractive solution is to just put
> the method into Base anyway. Everything derived from Base gets some
> default implementation, and so the calls will work silently, even if
> they don't make sense. Look, Ma! No casts. But is it good design?
>
> In Lisp, you never have to unnecessarily specialize a generic function
> too deeply into the class hierarchy. If some two-argument method only
> makes sense over a widget and grommet combination, then just write it
> for that.
That is where in a statically typed OO language you would have the
widget and grommet implement an inteface. Calling "instanceof" for a
specialization check like that is not bad OO and still doesn't mean
you need casting (though it would be justified if you did).
I agree, throwing it into the base is BAD. Here is an example of such
as specialization done without casting:
class Person { }
class StephenHawking extends Person {}
class GeorgeBushJr extends Person implements Dumbass {}
class GeorgeBushSr extends Person implements Dumbass {}
interface Dumbass { public void makeFoolOfSelf(); }
..
void putOnTv(Dumbass dumbass) {
dumbass.makeFoolOfSelf();
}
..
Person people[] = { new GeorgeBushSr(), new StephenHawking(), new
GeorgeBushJr() };
for (int i : people)
if (people[i] instanceof DumbAss)
putOnTv( people[i] );
..
output:
----
"For seven and a half years I've worked alongside President Reagan.
We've had triumphs. Made some mistakes. We've had some sex ...
uh...setbacks." - George Bush senior
"Thank you all. Thanks for coming out to say hello. I got to tell
*me*, you have lifted my spirits, for which I am grateful. " - George
Bush Jr.
----
No casting necessary.
>
> How about that Null Object design pattern? In Lisp, just specialize
> for the NULL type, to match the NIL value.
>
> > Just as an example: Java internally implements many design patterns
> > and has a huge code base, it is flexible (obviously it is the
> > language) and it is complete. How many casts do you find in a typical
> > java source files? They are hard to find.
>
> > I have been doing OO for about 16 years and in my experience the most
> > common problem found in any OO programming is the OO design. It is to
> > easy to fix a bad design by casting, so people do it.
>
> So your doctrine is that every good OO design can be fit into a
> statically typed program with no casts.
>
> From what branch of computing research does this magic result come
> from?
No that's not my doctrine. My doctrine is good design requires
casting only when it should. In the GeorgeBush(es) example above
there are many ways to do a similar thing wrong, and inexperienced OO
designers often do.
>
> > I suspect, though have no proof, that Lisp allows you to get away with
> > a bad design easier then a typed language (or at least its harder to
>
> You mean *manifestly* typed language.
>
> Lisp is a *latently* typed language.
>
> This discussion revolves around the realization that Java is both at
> the same time.
True.
> If you can't see it no matter how hard you look, it doesn't exist.
I only ment you may have to look harder.
>
> If your design goal is to implement your latently typed program such
> that the number of wrinkles in its parallel manifest typing is
> minimized, then that can turn into a measure of the success of your
> design.
>
> There is no need to have that as a measure of success when the
> manifest type
> system doesn't exist in the tools that you are currently using.
Ok, that I agree with..kinda. The different tools of the language
will affect what is good in implementation, but good design is good
design.
>
> There are probably still assembly language programmers who define
> their good designs in terms of minimized execution cycles or code
> size. They look at the assembly language programs written by their
> colleagues and note that the experienced ones write smaller, faster
> programs. They are better designers, so there must be a correlation
> between that and good design!
>
> There is a human tendency to identify the accomplishment of anything
> that requires effort as a success. When humans work with tools that
> require extra effort that is not related to the task achieved with
> those tools, they regard it as a success when they overcome those
> intrinsic difficulties in those tools. Sometimes that definition of
> success clouds the real objective. Thus for instance the computer
> programmer is happy when the program has next to no casts in it, even
> though it's flopping in the marketplace and the company is going down
> the tubes. Over years, it becomes a matter of personal pride, leading
> to the rejection of tools that don't have those intrinsic
> difficulties, and disdain for the work achieved with those easier
> tools, even though it may be of better quality.
Good design in Java usually requires less effort to code. Again, the
Bush example above, if designed improperly would probably take more
code to write and be more difficult to read.
>
> Say, as a Java user, what would you say to a C++ programmer who
> claimed this: In a good program design, the programmer always knows
> where and when it is safe to delete an object, so there is no need for
> garbage collection. Some of those Java programs that rely on garbage
> collection are actually bad design, but it's just hard to tell it
> because both good designs and bad designs are garbage collected. If
> you just go through the exercise of sticking in the explicit
> deletions, you will be able to tell: in the good design, this will be
> easy to do! But under the nasty, bad design, it won't be obvious where
> the object lifetimes end. Down with garbage collection, phooey!
I totally agree. That supports the belief that good design transcends
language. Good design in Lisp, Java or C++, or any OO language should
all be structurally similar, though the details of their
implementation might be quite different. That's why design patterns
work.
·········@hotmail.com (Darren) writes:
> That is where in a statically typed OO language you would have the
> widget and grommet implement an inteface. Calling "instanceof" for a
> specialization check like that is not bad OO and still doesn't mean
> you need casting (though it would be justified if you did).
No, it is bad OO, because it is "outside" of the OO system. This
reminds me of a presentation on Slate that demonstrated what this means:
http://tunes.org/~eihrul/talk.pdf (see also http://slate.tunes.org/).
> I totally agree. That supports the belief that good design transcends
> language. Good design in Lisp, Java or C++, or any OO language should
> all be structurally similar, though the details of their
> implementation might be quite different. That's why design patterns
> work.
Hah, you just opened up another can of worms! =) Design patterns as
realized in software architecture today are bull; they are just
published and accepted ways of working around the limitations of C++
and Java. There is nothing fundamental about most of them that makes
them transcend language. Especially since the paradigms of different
languages really do encourage different solutions to the same problem.
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
> ·········@hotmail.com (Darren) writes:
> > That is where in a statically typed OO language you would have the
> > widget and grommet implement an inteface. Calling "instanceof" for a
> > specialization check like that is not bad OO and still doesn't mean
> > you need casting (though it would be justified if you did).
>
> No, it is bad OO, because it is "outside" of the OO system. This
> reminds me of a presentation on Slate that demonstrated what this means:
> http://tunes.org/~eihrul/talk.pdf (see also http://slate.tunes.org/).
Interesting PDF but far from convincing. The Mainstreem OO example
can't even be qualified as OO in Java. It doesn't leverage the fact
that sharks inherit from fish, nor does it acknowledge/use its own
encapsulation. It then goes on to claim "Mainstream OO is not
expressive enough" and says it has "A Brittle Program Structure".
Those would be interesting claims if the example even approached a
good design. I agree that *that* example is brittle and unexpressive
but *that* example is garbage.
Example:
Shark encounter pseudo code method from the PDF presentation (really
ugly):
method: encounter object
if self has state Healthy
if object is in class Shark
then fight object
otherwise
if object is in class Fish
then eat object
otherwise
if self has state Injured
and object is in class Shark
and object has state Healthy
then swim away
My Shark's encounter method:
public void encounter(Fish fish) {
if (fish instanceof Shark && isHealthy)
fight((Shark)fish);
else if (isHealthy)
eat(fish);
else
super.encounter(fish);
}
Plenty expressive, not brittle, less code and resloves 6 encounter
scenarios for sharks.
"Resolving Ambiguities: Ordering On The Fly"
Hmm...they reinvented what happens naturally in good OO. Notice in my
shark's encounter method, I don't test to see if it is a fish
encounter. Shark's don't worry about fish, they only worry about
other sharks. Sharks also only attack other sharks if they themselves
are healthy. The pseudo code is wasteful and perfoming checks which
should be resolved naturally (on the fly) by it's inheritance from
fish and it's health sate.
Through the same mechanism, my example covers two scenarios more then
the original without any more code:
Other circumstances:
Unhealthy Shark ignores any fish it meets
Unhealthy Shark ignores any unhealthy shark it meets
-------
Implementation notes:
The presentation didn't handle all cases (for any of the examples):
Undefined if fish encounters fish
Undefined if unhealthy shark encounters unhealthy shark
Possible logic problem
If healthy shark fights healthy or unhealthy shark it is
injured in either case.
Implemented fixes:
defined fish encounters fish
defined fish encounters unhealthy shark
Healthy shark only gets injured if it fights another healthy shark. To
restore original examples logic, remove the following line in void
fight(Shark shark)
if (shark.isHealthy())
Output from Encounters.java :
-----------------------------------------------
a Fish ignores a Fish.... 1 = 1
a Fish ignores an Unhealthy Shark.... 1 = 1
a Fish swims away from a Healthy Shark.... 1 < 0
an Unhealthy Shark ignores a Fish.... 1 = 1
an Unhealthy Shark ignores an Unhealthy Shark.... 1 = 1
an Unhealthy Shark swims away from a Healthy Shark.... 1 < 0
a Healthy Shark eats a Fish.... 0 < 1
a Healthy Shark fights an Unhealthy Shark.... 0 = 0
a Healthy Shark fights a Healthy Shark.... 0 = 0
------------------------------------------------
------- Encounters.java
public class Encounters {
public static void main(String[] args) {
Fish fish = new Fish();
Fish healthyShark = new Shark(true);
Fish unhealthyShark = new Shark(false);
// Fish encounters...
fish.encounter(new Fish()); // Another fish
fish.encounter(unhealthyShark);
fish.encounter(healthyShark);
// Unhealthy shark encounters...
unhealthyShark.encounter(fish);
unhealthyShark.encounter(new Shark(false)); // Another unhealthy
shark
unhealthyShark.encounter(healthyShark);
// Healthy shark encounters...
healthyShark.encounter(fish) ;
healthyShark.encounter(unhealthyShark);
healthyShark.encounter(new Shark(true)) ;// Another healthy shark
}
static void explain(String p) { System.out.println(p); }
}
class Fish {
public void encounter(Fish fish) {
if (fish instanceof Shark && ((Shark)fish).isHealthy())
swimaway(fish);
else
Encounters.explain(this +" ignores " + fish + ".... 1 = 1");
}
void swimaway(Fish fish) {
Encounters.explain(this + " swims away from "+fish +".... 1 < 0");
}
public String toString() { return "a Fish"; }
}
class Shark extends Fish {
private boolean isHealthy = true ;
public Shark(boolean healthy) {
isHealthy = healthy;
}
public boolean isHealthy() { return isHealthy; }
public String toString() {
return (isHealthy ? "a Healthy" : "an Unhealthy")+" Shark";
}
public void encounter(Fish fish) {
if (fish instanceof Shark && isHealthy)
fight((Shark)fish);
else if (isHealthy)
eat(fish);
else
super.encounter(fish);
}
void fight(Shark shark) {
Encounters.explain(this +" fights "+shark + ".... 0 = 0");
if (shark.isHealthy())
isHealthy = false;
}
void eat(Fish fish) {
Encounters.explain(this +" eats " + fish + ".... 0 < 1");
}
}
------- End ----
> > I totally agree. That supports the belief that good design transcends
> > language. Good design in Lisp, Java or C++, or any OO language should
> > all be structurally similar, though the details of their
> > implementation might be quite different. That's why design patterns
> > work.
>
> Hah, you just opened up another can of worms! =) Design patterns as
> realized in software architecture today are bull; they are just
> published and accepted ways of working around the limitations of C++
> and Java. There is nothing fundamental about most of them that makes
> them transcend language. Especially since the paradigms of different
> languages really do encourage different solutions to the same problem.
Well, that's a whole other religious debate... Let's not go there ;)
·········@hotmail.com (Darren) writes:
> class Person { }
> class StephenHawking extends Person {}
> class GeorgeBushJr extends Person implements Dumbass {}
> class GeorgeBushSr extends Person implements Dumbass {}
Hawking made a fool of himself when he was presenting that paper in
Dublin, tho. :)
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
·········@hotmail.com (Darren) writes:
> ..
> void putOnTv(Dumbass dumbass) {
> dumbass.makeFoolOfSelf();
> }
> ..
>
> Person people[] = { new GeorgeBushSr(), new StephenHawking(), new
> GeorgeBushJr() };
>
> for (int i : people)
> if (people[i] instanceof DumbAss)
> putOnTv( people[i] );
dumbass.makeFoolOfSelf() will never be called here. Your unfamiliarity
with Java is really showing here.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
·········@hotmail.com (Darren) writes:
> Rahul Jain <·····@nyct.net> wrote in message news:<··············@nyct.net>...
>> ·········@hotmail.com (Darren) writes:
>>
>> > ..
>> > void putOnTv(Dumbass dumbass) {
>> > dumbass.makeFoolOfSelf();
>> > }
>> > ..
>> >
>> > Person people[] = { new GeorgeBushSr(), new StephenHawking(), new
>> > GeorgeBushJr() };
>> >
>> > for (int i : people)
>> > if (people[i] instanceof DumbAss)
>> > putOnTv( people[i] );
>>
>> dumbass.makeFoolOfSelf() will never be called here. Your unfamiliarity
>> with Java is really showing here.
>
>>
>> > ..
>> > void putOnTv(Dumbass dumbass) {
>> > dumbass.makeFoolOfSelf();
>> > }
>> > ..
>> >
>> > Person people[] = { new GeorgeBushSr(), new StephenHawking(), new
>> > GeorgeBushJr() };
>> >
>> > for (int i : people)
>> > if (people[i] instanceof DumbAss)
>> > putOnTv( people[i] );
>>
>> dumbass.makeFoolOfSelf() will never be called here. Your unfamiliarity
>> with Java is really showing here.
>
>
> ..
> void putOnTv(Dumbass dumbass) {
> dumbass.makeFoolOfSelf();
> }
> ..
>
> Person people[] = { new GeorgeBushSr(), new StephenHawking(), new
> GeorgeBushJr() };
>
> for (int i : people)
> if (people[i] instanceof DumbAss)
> putOnTv( (Dumbass)people[i] );
>
> Happy now?
Why are we only putting dumbasses on TV? Been listening to Jon Stewart's
complaints? ;)
In any case, if you did
for (int i : people)
if (people[i] instanceof DumbAss)
putOnTv( (Dumbass)people[i] );
else
putOnTv(people[i]);
That would be more complete. The equivalent Lisp code would be:
(mapcar #'put-on-tv people)
Look ma, no casts! ... And it's a small fraction of the amount of code
you needed in Java.
The key issue is that in Java putOnTv(Dumbass) is a different method
name than putOnTv(Person). (Technically "putOnTv(LDumbass;)" vs.
"putOnTv(LPerson;)".) The name when calling the method in Java code is
usually scattered throughout the function calling it, and sometimes
throughout the entire class or even the entire application (for example,
when passing in the value of some constant defined in a library).
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
···@ashi.footprints.net (Kaz Kylheku) wrote in message news:<····························@posting.google.com>...
> ·········@hotmail.com (Darren) wrote in message
I have got to stop trying to rebutt this stuff till I get a good night
sleep.
My example *does* require a cast. It is the correct way to solve your
example though. My only saving grace was that I said a good design
may still need casting and that's ok.
I concede to everything you guys are saying... (until I get at least 6
hours sleep in a row and 2 litres of coffee!)
:)
·········@hotmail.com (Darren) writes:
> I suspect, though have no proof, that Lisp allows you to get away with
> a bad design easier then a typed language (or at least its harder to
> see if its a bad design). Though, I don't know that is necessarily a
> bad thing since the problem is so common.
I don't think any language, not even Lisp, can save you from a badly
designed architecture. Nor can any language turn someone into a great
programmer. Bad languages can make becoming a great programmer
harder though. In this context, I would define a language as bad if
it doesn't allow you to create abstractions appropriate to the problem
domain.
--
An ideal world is left as an excercise to the reader.
--- Paul Graham, On Lisp 8.1
Darren wrote:
> I have been doing OO for about 16 years and in my experience the most
> common problem found in any OO programming is the OO design.
Common Lisp is a multi-paradigm language. We don't try to fit everything
into class-based OO designs, only those things that lend themselves to
those. What looks like a bad OO design might just as well be a good
imperative or functional design.
And again, just because the type checker doesn't complain and you are
using design pattern doesn't make your code a good OO design. See
http://csis.pace.edu/~bergin/patterns/ppoop.html
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
On Sat, 30 Oct 2004 11:07:43 +0200, Pascal Costanza wrote:
>
> And again, just because the type checker doesn't complain and you are
> using design pattern doesn't make your code a good OO design. See
> http://csis.pace.edu/~bergin/patterns/ppoop.html
That's an interesting page. I would write:
import java.util.*;
public class PrintOS
{
static HashMap messages;
static
{
messages = new HashMap();
String unix = "This is a UNIX box and therefore good.";
messages.put( "SunOS", unix);
messages.put( "Linux", unix);
String windows = "This is a Windows box and therefore bad.";
messages.put( "Windows NT", windows);
messages.put( "Windows 95", windows);
}
public static void main(final String[] args)
{
String message = (String)messages.get(System.getProperty("os.name"));
if( message != null )
{
System.out.println(message);
}
else
{
System.out.println("This is not a box.") ;
}
}
}
rather than any of the solutions on that page. Does that make me a
"hacker"?
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
William Bland wrote:
> Does that make me a "hacker"?
Yep. Aren't you happy? ;-)
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
On Sat, 30 Oct 2004 19:39:33 +0200, Pascal Costanza wrote:
>
> William Bland wrote:
>
>> Does that make me a "hacker"?
>
> Yep. Aren't you happy? ;-)
>
> Pascal
I guess I am, yes...
It makes me a little nervous though, to see the number of times the
authors of that web-page use the word "sophisticated" for their final
solution of the problem... kinda makes me feel I must be missing
something. Bah, when I read that code though it just makes me want to
throw up - what an over-engineered and tasteless solution to a simple
problem!
Yup, ok, I'm a happy hacker :-)
(If the web page was intended as a joke that was too subtle for me, I'm
suitably embarrassed).
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
William Bland wrote:
> On Sat, 30 Oct 2004 19:39:33 +0200, Pascal Costanza wrote:
>
>
>>William Bland wrote:
>>
>>
>>>Does that make me a "hacker"?
>>
>>Yep. Aren't you happy? ;-)
>>
>>Pascal
>
>
> I guess I am, yes...
>
> It makes me a little nervous though, to see the number of times the
> authors of that web-page use the word "sophisticated" for their final
> solution of the problem... kinda makes me feel I must be missing
> something. Bah, when I read that code though it just makes me want to
> throw up - what an over-engineered and tasteless solution to a simple
> problem!
>
> Yup, ok, I'm a happy hacker :-)
>
> (If the web page was intended as a joke that was too subtle for me, I'm
> suitably embarrassed).
>
> Cheers,
> Bill.
Sadly I don't think the authors were joking, in fact the line
...
The object-oriented programmers see the nature of computation as a
swarm of interacting agents that provide services for other objects.
...
makes me think the authors imagine their approach to be 'clever'.
They seem to think the last solution will make future development
easier, I assume their reasoning essentially goes like this...
There is a set of 'things'; If we partition this set by different
implementations of some interface A, then future work will be made
easier as we will only have to add new methods to the interface and its
implementations.
The problem I often come across with OO in java is that a slightly (or
very) different partition of the set is needed, in which case interface
A is useless, and I need a new interface B, with implementations, to
represent this second partition. All of this adds up to a lot of code,
and each interface is another concept the developer needs to remember.
cheers,
Alex
William Bland wrote:
> On Sat, 30 Oct 2004 19:39:33 +0200, Pascal Costanza wrote:
>
>>William Bland wrote:
>>
>>>Does that make me a "hacker"?
>>
>>Yep. Aren't you happy? ;-)
>>
>>Pascal
>
> I guess I am, yes...
>
> It makes me a little nervous though, to see the number of times the
> authors of that web-page use the word "sophisticated" for their final
> solution of the problem... kinda makes me feel I must be missing
> something. Bah, when I read that code though it just makes me want to
> throw up - what an over-engineered and tasteless solution to a simple
> problem!
>
> Yup, ok, I'm a happy hacker :-)
>
> (If the web page was intended as a joke that was too subtle for me, I'm
> suitably embarrassed).
I understand the page as a good introduction to what it means to be
_really_ object-oriented. OOP means that autonomous objects interact
with each other solely by sending messages. Each object can determine
itself how to react to a message, there is no behavior forced upon it.
Your code is just imperative. Darren's code is a dirty hack that turns a
system identification string into a class name, a solution that probably
breaks when the conversion routine fails to create a class name that is
acceptable for Java.
Note that I don't think that imperative programmming or dirty hacks are
necessarily bad ideas. It always depends on the concrete situation what
is best for you. However, that webpage is intended as a pedagogical
resource to teach something about OOP. (Toy examples always have the
disadvantage that they are toy examples.)
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> Your code is just imperative. Darren's code is a dirty hack that turns a
> system identification string into a class name, a solution that probably
> breaks when the conversion routine fails to create a class name that is
> acceptable for Java.
I beg your pardon? That wasn't a dirty hack at all! True, it wasn't
solid and would only work with the examples given, but dirty? No.
Suppose I made it rock solid with a 12 character addition to the code.
Will you still think it's a dirty hack?
Class.forName(System.getProperty("os.name").replaceAll("\\p{Punct}|[
]", "")).newInstance();
The change is: "\\p{Punct}|[ ]". With that change any occurences of:
!\"#$%&'()*+,-./:;<=>·@[\\]^_`{|}~ will be removed from the os.name
result.
Lets evaluate:
- The code is easy to read
- There is no adhoc-polymorphism anywhere
- There is no object registration
- There is no singleton usage
- All OS's are represented as objects
- The OS inheritence hierarchy represents thier RL historical lineage
- The code doesn't use any if conditions
- To add new OS's, you create a class using its normal name minus any
punctuation
- New OS classes can be added without ANY code change or even
recompiling
What criteria are you judging it on? According to that link, I met
every possible criteria they had and threw in a few more! Seriously,
why is it a "dirty hack"?
I am starting to see a trend here. Every time I do something with
Java that you hadn't thought of, you call it a hack. What I did there
is called "factory loading" and it's done all the time in java
software!!
---- Updated OSDetector.java
import java.util.regex.Matcher;
import java.util.regex.Pattern;
class Unix {
public Unix() {
System.out.println("This is a UNIX based box and therefore
good.");
}
}
class Windows {
public Windows() {
System.out.println("This is a Windows based box and therefore
bad.");
}
}
class Linux extends Unix {}
class SunOS extends Unix {}
class Windows95 extends Windows {}
class WindowsXP extends Windows {}
public class OSDetector {
public static void main(String[] args) {
try {
String os = System.getProperty("os.name").replaceAll("\\p{Punct}|[
]", "");
Class.forName(os).newInstance();
} catch (Exception e) {
System.out.println("This is not a box.");
}
}
}
Darren wrote:
> Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
>
>>Your code is just imperative. Darren's code is a dirty hack that turns a
>>system identification string into a class name, a solution that probably
>>breaks when the conversion routine fails to create a class name that is
>>acceptable for Java.
>
> I beg your pardon? That wasn't a dirty hack at all! True, it wasn't
> solid and would only work with the examples given, but dirty? No.
>
> Suppose I made it rock solid with a 12 character addition to the code.
> Will you still think it's a dirty hack?
Imagine OS names that start with numbers.
> Class.forName(System.getProperty("os.name").replaceAll("\\p{Punct}|[
> ]", "")).newInstance();
>
> The change is: "\\p{Punct}|[ ]". With that change any occurences of:
> !\"#$%&'()*+,-./:;<=>·@[\\]^_`{|}~ will be removed from the os.name
> result.
Imagine names that only differ in their use of punctuation but otherwise
become ambiguous.
> What criteria are you judging it on? According to that link, I met
> every possible criteria they had and threw in a few more! Seriously,
> why is it a "dirty hack"?
It wouldn't stand the test of time. Today's operating system names may
work in your solution, but you don't know what names vendors will come
up with in the future.
> I am starting to see a trend here. Every time I do something with
> Java that you hadn't thought of, you call it a hack. What I did there
> is called "factory loading" and it's done all the time in java
> software!!
Too bad for Java software. The trend I see is that you are only
correcting bugs when someone points them out to you.
I'll stop now.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
* Pascal Costanza (········@web.de)
...
>> Suppose I made it rock solid with a 12 character addition to the code.
>> Will you still think it's a dirty hack?
>
> Imagine OS names that start with numbers.
>
...
>> !\"#$%&'()*+,-./:;<=>·@[\\]^_`{|}~ will be removed from the os.name
>> result.
>
> Imagine names that only differ in their use of punctuation but otherwise
> become ambiguous.
>
Both problems can be trivially solved by using some suitable
form of quoting -- and using reflection is the only way to get
this problem solved witout having to modify some global class
when new os arrive.
...
> Too bad for Java software. The trend I see is that you are only
> correcting bugs when someone points them out to you.
>
You are getting a little mean. You can't fix a bug until you
either notice it or get it shown. A mediocre implementation
does not weigh against using the reflection API as method
of polymorphism.
Andreas
--
np: 4'33
Andreas Krey wrote:
> * Pascal Costanza (········@web.de)
> ...
>
>>>Suppose I made it rock solid with a 12 character addition to the code.
>>>Will you still think it's a dirty hack?
>>
>>Imagine OS names that start with numbers.
>
> ...
>
>>>!\"#$%&'()*+,-./:;<=>·@[\\]^_`{|}~ will be removed from the os.name
>>>result.
>>
>>Imagine names that only differ in their use of punctuation but otherwise
>>become ambiguous.
>
> Both problems can be trivially solved by using some suitable
> form of quoting -- and using reflection is the only way to get
> this problem solved witout having to modify some global class
> when new os arrive.
...but slowly you are approaching a solution that looks much less simple
than it looked first.
>>Too bad for Java software. The trend I see is that you are only
>>correcting bugs when someone points them out to you.
>
> You are getting a little mean.
OK, sorry.
> You can't fix a bug until you
> either notice it or get it shown.
...but a more principled solution doesn't need to be overly complex but
can probably be more robust.
But to stress this once again: The article I have cited is _not_ about
finding the best way to determine the OS a program is running on. It is
_solely_ about illustrating an important aspect of object-oriented
programming, and the fact that using inheritance and design patterns
alone doesn't make you an object-oriented programmer.
The only criticism of that paper I have heard so far seems to be that
OOP is sometimes overkill, and especially in the example given in that
paper. But that's besides the point. Even if OOP sometimes is too much
(I agree), the paper is still a worthwhile illustration of OOP.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
* Pascal Costanza (········@web.de)
...
>> Both problems can be trivially solved by using some suitable
>> form of quoting -- and using reflection is the only way to get
>> this problem solved witout having to modify some global class
>> when new os arrive.
>
> ...but slowly you are approaching a solution that looks much less simple
> than it looked first.
>
Actually, I would have immediately arrived at a proper f for
classForName (f (osname)), and used that for evaluation of
beauty (and the classnames will look ugly).
...
> ...but a more principled solution doesn't need to be overly complex but
> can probably be more robust.
Yes. The specific problem of C++ is that you need quite some complexity
of problem before C++ actually starts to pay off.
...
> The only criticism of that paper I have heard so far seems to be that
> OOP is sometimes overkill, and especially in the example given in that
> paper. But that's besides the point. Even if OOP sometimes is too much
> (I agree), the paper is still a worthwhile illustration of OOP.
>
After reading it shortly, I found it could be clearer on why having
a representative object for $someting may be a good idea. Factoring
out either the procedures or the classes from the if chain looked
silly to me for some time, and that I finally got it wasn't entirely
the paper's merit.
Andreas
--
np: 4'33
Andreas Krey wrote:
> Yes. The specific problem of C++ is that you need quite some complexity
> of problem before C++ actually starts to pay off.
Same here in Lispland. ;)
(Although I wouldn't hesitate anymore to use Lisp even for simple problems.)
>>The only criticism of that paper I have heard so far seems to be that
>>OOP is sometimes overkill, and especially in the example given in that
>>paper. But that's besides the point. Even if OOP sometimes is too much
>>(I agree), the paper is still a worthwhile illustration of OOP.
>
> After reading it shortly, I found it could be clearer on why having
> a representative object for $someting may be a good idea. Factoring
> out either the procedures or the classes from the if chain looked
> silly to me for some time, and that I finally got it wasn't entirely
> the paper's merit.
I think the author is very open to constructive criticism.
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
* Pascal Costanza (········@web.de)
> Andreas Krey wrote:
>
>> Yes. The specific problem of C++ is that you need quite some complexity
>> of problem before C++ actually starts to pay off.
>
> Same here in Lispland. ;)
>
More, actually. I made a comparison and came out with half the lines
of code for lisp as opposed to C++ for the same small GUI program.
Now if only the lisp toolkit binding I assumed would exist. :-(
Now some of the savings stem from C++ being a typed language,
but even for those C++ is needlessly talkative. No anonymous
nothing, for example.
Andreas
--
np: 4'33
Andreas Krey <··········@gmx.de> wrote in message news:<·····················@inner.h.uberluser.org>...
> * Pascal Costanza (········@web.de)
> ...
> >> Suppose I made it rock solid with a 12 character addition to the code.
> >> Will you still think it's a dirty hack?
> >
> > Imagine OS names that start with numbers.
> >
> ...
> >> !\"#$%&'()*+,-./:;<=>·@[\\]^_`{|}~ will be removed from the os.name
> >> result.
> >
> > Imagine names that only differ in their use of punctuation but otherwise
> > become ambiguous.
> >
>
> Both problems can be trivially solved by using some suitable
> form of quoting -- and using reflection is the only way to get
> this problem solved witout having to modify some global class
> when new os arrive.
>
> ...
> > Too bad for Java software. The trend I see is that you are only
> > correcting bugs when someone points them out to you.
> >
> You are getting a little mean. You can't fix a bug until you
> either notice it or get it shown. A mediocre implementation
> does not weigh against using the reflection API as method
> of polymorphism.
>
> Andreas
Exactly.
- Darren
On Tue, 02 Nov 2004 09:26:07 +0000, Andreas Krey wrote:
> * Pascal Costanza (········@web.de)
> ...
>>> Suppose I made it rock solid with a 12 character addition to the code.
>>> Will you still think it's a dirty hack?
>>
>> Imagine OS names that start with numbers.
>>
> ...
>>> !\"#$%&'()*+,-./:;<=>·@[\\]^_`{|}~ will be removed from the os.name
>>> result.
>>
>> Imagine names that only differ in their use of punctuation but otherwise
>> become ambiguous.
>
> Both problems can be trivially solved by using some suitable
> form of quoting -- and using reflection is the only way to get
> this problem solved witout having to modify some global class
> when new os arrive.
The only way? I disagree. I wrote my solution (see earlier in this
thread) thinking "ah, this is just right - now I can make it extensible by
reading the HashMap entries from a text file if I want to". No need to
modify any class file when a new os arrives.
I still don't see that object orientation (by which I mean representing
each os by a class) is necessary, or even desirable, in a solution to this
problem (as Pascal pointed out though, it is only a toy problem).
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
* William Bland (·······@abstractnonsense.com)
...
> The only way? I disagree. I wrote my solution (see earlier in this
> thread) thinking "ah, this is just right - now I can make it extensible by
> reading the HashMap entries from a text file if I want to". No need to
> modify any class file when a new os arrives.
Unfortunately, this just forms another central point to modify.
You cannot *just* add the new class file, although the text file
has potential in the direction that you could use rexexps, for example.
> I still don't see that object orientation (by which I mean representing
> each os by a class) is necessary, or even desirable, in a solution to this
> problem (as Pascal pointed out though, it is only a toy problem).
>
The example is (as most examples are) silly, but the mechanism isn't.
We used classForName to load dialog objects into an application which
are named in a config file. For a new customization you need only the
new class(es) and a new config, but no change to pre-existing code.
Of course, using classForName means that you cannot statically
determine which classes you are going to need.
Andreas
--
np: 4'33
On Tue, 02 Nov 2004 18:55:59 +0000, Andreas Krey wrote:
> * William Bland (·······@abstractnonsense.com)
> ...
>> The only way? I disagree. I wrote my solution (see earlier in this
>> thread) thinking "ah, this is just right - now I can make it extensible by
>> reading the HashMap entries from a text file if I want to". No need to
>> modify any class file when a new os arrives.
>
> Unfortunately, this just forms another central point to modify.
> You cannot *just* add the new class file, although the text file
> has potential in the direction that you could use rexexps, for example.
Well, yeah, *something* has to be modified in order to inform the system
of a new os and message. Given the choice between it being a class or a
text file, I would choose the text file. Hell, if a new os suddenly gets
released at 3 in the morning and I'm called out by a customer to fix this
code, and I don't have access to my development environment but I can find
some kind of text editor, I think I'd be glad of this choice.
>> I still don't see that object orientation (by which I mean representing
>> each os by a class) is necessary, or even desirable, in a solution to this
>> problem (as Pascal pointed out though, it is only a toy problem).
>>
> The example is (as most examples are) silly, but the mechanism isn't.
Sure, I never said classForName was silly. Anti-tank weapons aren't silly
either, but I wouldn't use one to hang a picture.
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
William Bland <·······@abstractnonsense.com> writes:
> Well, yeah, *something* has to be modified in order to inform the system
> of a new os and message. Given the choice between it being a class or a
> text file, I would choose the text file. Hell, if a new os suddenly gets
> released at 3 in the morning and I'm called out by a customer to fix this
> code, and I don't have access to my development environment but I can find
> some kind of text editor, I think I'd be glad of this choice.
Why do you even need a text editor? Just connect to the process and add
another definition. :)
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
William Bland wrote:
> I still don't see that object orientation (by which I mean representing
> each os by a class) is necessary, or even desirable, in a solution to this
> problem (as Pascal pointed out though, it is only a toy problem).
It's probably not.
Pascal
--
Pascal Costanza University of Bonn
···············@web.de Institute of Computer Science III
http://www.pascalcostanza.de R�merstr. 164, D-53117 Bonn (Germany)
Andreas Krey <··········@gmx.de> writes:
> You are getting a little mean. You can't fix a bug until you
> either notice it or get it shown. A mediocre implementation
> does not weigh against using the reflection API as method
> of polymorphism.
A mediocre implementation does not support using the reflection API as a
method of polymorphism, either.
The fact that his implementation is bad is _because_ the reflection API
is a bad method of polymorphism. If it were good, he would be able to
implement exactly what he wanted easily instead of having to half-ass a
solution because getting it right is so much effort.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
Rahul Jain <·····@nyct.net> wrote in message news:<··············@nyct.net>...
> The fact that his implementation is bad is _because_ the reflection API
> is a bad method of polymorphism.
"Class loaders, one of the cornerstones of Java dynamics, determine
when and how classes can be added to a running Java environment. This
article helps you learn about class loaders in general and network
class loaders in particular...as well as the security issues that
surround them." - java.sun.com (November 2, 2004)
Educate yourself:
http://java.sun.com/developer/technicalArticles/Networking/classloaders/
> If it were good, he would be able to
> implement exactly what he wanted easily instead of having to half-ass a
> solution because getting it right is so much effort.
Or I was just trying to solve the problem presented and didn't feel it
required anything more to prove the point. If you *really* believe
solving that trival problem "is so much effort", you shouldn't be
commenting.
This was a toy problem exercise intended to illustrate a point, I
presented a toy implementation to illustrate the power of class
loading. Quibbling about the implementation does not diminish the
point it illustrates.
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
>
> Imagine names that only differ in their use of punctuation but otherwise
> become ambiguous.
I don't have to imagine it. Research what are the known values
returned by "os.name". They already ARE ambiguous. That is what the
companion property "os.version" is for. All of the solutions on the
page would have the same problem. A combination of "os.name" and
"os.version" as part of the class naming convention would easily solve
this.
>
> It wouldn't stand the test of time. Today's operating system names may
> work in your solution, but you don't know what names vendors will come
> up with in the future.
quib�ble ( P ) Pronunciation Key (kwbl)
intr.v. quib�bled, quib�bling, quib�bles
To evade the truth or importance of an issue by raising trivial
distinctions and objections.
To find fault or criticize for petty reasons; cavil.
n.
A petty distinction or an irrelevant objection.
Archaic. A pun.
Come on, you are quibbling. You know (at least I hope you do) as well
as I that that could easily be solved by prepending something like
"OSj" to the class naming convention and using a double transformation
on the "os.name" result.
Example:
String osName = System.getProperty("os.name").replaceAll("\\p{Punct}|[
]", "_");
String os = "OSj"+osName.replaceAll("_", "");
Wow...that was complictated. Though I never saw such a requirement
expressed in the example. Will your next complaint be that I didn't
implement all known operating systems as objects and therefore it will
break if moved to a Mac box or an AS/400?
>
> > I am starting to see a trend here. Every time I do something with
> > Java that you hadn't thought of, you call it a hack. What I did there
> > is called "factory loading" and it's done all the time in java
> > software!!
>
> Too bad for Java software.
Whatever. Factory loading, incase you are unfamiliar, is typically
done using a hash table, the Factory Pattern and class loading. The
Factory loads the class on demand or extracts it from a cache and
returns... Never mind, if you thought I was implementing a "dirty
hack", you wouldn't get it. It's obvious to me you don't do real
programming and probably couldn't see the value of such a thing in a
real system.
>The trend I see is that you are only correcting bugs when someone
points them out to you.
Ya, I have better things to do with my time then turn a toy example
into production quality code, especially when you keep changing the
parameters by which the code is judged.
> I'll stop now.
It would be appreciated.
Darren wrote:
> Come on, you are quibbling.
No, I am not.
> Will your next complaint be that I didn't implement all known
> operating systems as objects and therefore it will break if moved to
> a Mac box or an AS/400?
No, my next complaint will be the same as before: Your code is not
object-oriented. I still think it's a hack. Recall that I've said that
hacks aren't necessarily bad ideas, so I wonder what you are currently
fighting against.
The paper I have referred to is an illustration of OO principles and of
the fact that using inheritance and design patterns doesn't make you an
OO programmer. That's the only thing I am interested in here.
If I were really, really, really concerned about finding out about the
operating system my program is running on, I would check the *features*
list. ;)
To stress this the last time: _This is not the topic of our discussion._
The topic is: What does static typing, especially Java's type system,
bring to the table? That's where we started from.
> It's obvious to me you don't do real programming and probably
> couldn't see the value of such a thing in a real system.
I have insulted you, now you have insulted me. Can we call it quits?
>> The trend I see is that you are only correcting bugs when someone
>> points them out to you.
>
> Ya, I have better things to do with my time then turn a toy example
> into production quality code, especially when you keep changing the
> parameters by which the code is judged.
I have never changed them.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza wrote:
>
> Darren wrote:
>
>> Come on, you are quibbling.
>
>
> No, I am not.
>
>> Will your next complaint be that I didn't implement all known
>> operating systems as objects and therefore it will break if moved to
>> a Mac box or an AS/400?
>
>
> No, my next complaint will be the same as before: Your code is not
> object-oriented. I still think it's a hack. Recall that I've said that
> hacks aren't necessarily bad ideas, so I wonder what you are currently
> fighting against.
>
> The paper I have referred to is an illustration of OO principles and of
> the fact that using inheritance and design patterns doesn't make you an
> OO programmer. That's the only thing I am interested in here.
>
> If I were really, really, really concerned about finding out about the
> operating system my program is running on, I would check the *features*
> list. ;)
Which gives me a very nice handle for self-aggrandizing selfish
advertisment: check out the CL-ENVIRONMENT library in the CLOCC :)
Cheers
--
Marco
>
> To stress this the last time: _This is not the topic of our discussion._
>
> The topic is: What does static typing, especially Java's type system,
> bring to the table? That's where we started from.
>
>> It's obvious to me you don't do real programming and probably
>> couldn't see the value of such a thing in a real system.
>
>
> I have insulted you, now you have insulted me. Can we call it quits?
>
>>> The trend I see is that you are only correcting bugs when someone
>>> points them out to you.
>>
>>
>> Ya, I have better things to do with my time then turn a toy example
>> into production quality code, especially when you keep changing the
>> parameters by which the code is judged.
>
>
> I have never changed them.
>
>
>
> Pascal
>
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> No, my next complaint will be the same as before: Your code is not
> object-oriented. I still think it's a hack. Recall that I've said that
> hacks aren't necessarily bad ideas, so I wonder what you are currently
> fighting against.
To me it's a commonly used, clean form of dynamic polymorphism, to you
it's a hack. Lets agree to disagree on this.
> I have insulted you, now you have insulted me. Can we call it quits?
Happily. :)
·········@hotmail.com (Darren) writes:
> Whatever. Factory loading, incase you are unfamiliar, is typically
> done using a hash table, the Factory Pattern and class loading. The
> Factory loads the class on demand or extracts it from a cache and
> returns... Never mind, if you thought I was implementing a "dirty
> hack", you wouldn't get it. It's obvious to me you don't do real
> programming and probably couldn't see the value of such a thing in a
> real system.
It's obvious that you know nothing about Lisp, since you are pretending
that the "Factory pattern" is actually something novel to us.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
Rahul Jain <·····@nyct.net> wrote in message news:<··············@nyct.net>...
> ·········@hotmail.com (Darren) writes:
>
> > Whatever. Factory loading, incase you are unfamiliar, is typically
> > done using a hash table, the Factory Pattern and class loading. The
> > Factory loads the class on demand or extracts it from a cache and
> > returns... Never mind, if you thought I was implementing a "dirty
> > hack", you wouldn't get it. It's obvious to me you don't do real
> > programming and probably couldn't see the value of such a thing in a
> > real system.
>
> It's obvious that you know nothing about Lisp, since you are pretending
> that the "Factory pattern" is actually something novel to us.
"Factory Loading" != "Factory Pattern"
"Facotry Loading" composed of "Factory Pattern"
I wasn't explaining what the factory pattern is, I was explaining what
factory loading in Java is. You should read before commenting.
On Mon, 01 Nov 2004 15:25:55 +0100, Pascal Costanza wrote:
>
> I understand the page as a good introduction to what it means to be
> _really_ object-oriented. OOP means that autonomous objects interact
> with each other solely by sending messages. Each object can determine
> itself how to react to a message, there is no behavior forced upon it.
>
OK, I think I understand the idea behind the page now. It's not saying
"look, here's how to do an OO solution to this problem, isn't is great?".
Instead it is trying to define what "pure Object Orientation" really
means. Just like you sometimes see pages that give pure functional
solutions to problems just to illustrate what it means to be "purely
functional".
Yup, I get it now... but I still think they should have used an example
where you can *see* the benefits of Object Orientation - otherwise the
message gets lost in the noise (well, it did for me anyway).
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
* William Bland (·······@abstractnonsense.com)
...
> Yup, I get it now... but I still think they should have used an example
> where you can *see* the benefits of Object Orientation - otherwise the
> message gets lost in the noise (well, it did for me anyway).
>
Unfortunately, that is next to impossible. The problems is that
either a sample is so small that OO does not perceivably help,
or it is so big that it is not easily perceivable.
Just like the two classes of programs: The ones that obviously
contain no bugs and those that contain no obvious bugs.
Andreas
--
np: 4'33
On Wed, 03 Nov 2004 10:15:10 +0000, Andreas Krey wrote:
> * William Bland (·······@abstractnonsense.com)
> ...
>> Yup, I get it now... but I still think they should have used an example
>> where you can *see* the benefits of Object Orientation - otherwise the
>> message gets lost in the noise (well, it did for me anyway).
>>
> Unfortunately, that is next to impossible. The problems is that
> either a sample is so small that OO does not perceivably help,
> or it is so big that it is not easily perceivable.
>
I disagree. Using OO to model a mapping from os names to messages is
clearly silly. Using it to model objects (gasp!) would make much more
sense. If the example was some kind of hugely simplified physics problem,
with a bunch of particles represented by Objects, I think the benefits of
OO would be blindingly obvious.
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
William Bland wrote:
> It makes me a little nervous though, to see the number of times the
> authors of that web-page use the word "sophisticated" for their final
> solution of the problem... kinda makes me feel I must be missing
> something. Bah, when I read that code though it just makes me want to
> throw up - what an over-engineered and tasteless solution to a simple
> problem!
>
> Yup, ok, I'm a happy hacker :-)
>
> (If the web page was intended as a joke that was too subtle for me, I'm
> suitably embarrassed).
>
> Cheers,
> Bill.
Having pretty done all the forms on that page, I'm back in the happy
hacker camp too, simply for the sake of most problems I deal with, if
you don't 'build the least complicated thing that could possibly work'
then they are intractable.
Pete
William Bland <·······@abstractnonsense.com> wrote in message news:<·····························@abstractnonsense.com>...
> On Sat, 30 Oct 2004 11:07:43 +0200, Pascal Costanza wrote:
> >
> > And again, just because the type checker doesn't complain and you are
> > using design pattern doesn't make your code a good OO design. See
> > http://csis.pace.edu/~bergin/patterns/ppoop.html
>
> That's an interesting page. I would write:
>
> import java.util.*;
> public class PrintOS
> {
> static HashMap messages;
>
> static
> {
> messages = new HashMap();
> String unix = "This is a UNIX box and therefore good.";
> messages.put( "SunOS", unix);
> messages.put( "Linux", unix);
> String windows = "This is a Windows box and therefore bad.";
> messages.put( "Windows NT", windows);
> messages.put( "Windows 95", windows);
> }
>
> public static void main(final String[] args)
> {
> String message = (String)messages.get(System.getProperty("os.name"));
> if( message != null )
> {
> System.out.println(message);
> }
> else
> {
> System.out.println("This is not a box.") ;
> }
> }
> }
>
> rather than any of the solutions on that page. Does that make me a
> "hacker"?
I think it does, but I like it better then the ones on the page. It's
easy to maintain, fast and doesn't cluttering things up with 'if'
condidtions . Thumbs up!
On Sun, 31 Oct 2004 07:03:44 -0800, Darren wrote:
> William Bland <·······@abstractnonsense.com> wrote in message news:<·····························@abstractnonsense.com>...
>> On Sat, 30 Oct 2004 11:07:43 +0200, Pascal Costanza wrote:
>> >
>> > And again, just because the type checker doesn't complain and you are
>> > using design pattern doesn't make your code a good OO design. See
>> > http://csis.pace.edu/~bergin/patterns/ppoop.html
>>
>> That's an interesting page. I would write:
>>
>> import java.util.*;
>> public class PrintOS
>> {
>> static HashMap messages;
>>
>> static
>> {
>> messages = new HashMap();
>> String unix = "This is a UNIX box and therefore good.";
>> messages.put( "SunOS", unix);
>> messages.put( "Linux", unix);
>> String windows = "This is a Windows box and therefore bad.";
>> messages.put( "Windows NT", windows);
>> messages.put( "Windows 95", windows);
>> }
>>
>> public static void main(final String[] args)
>> {
>> String message = (String)messages.get(System.getProperty("os.name"));
>> if( message != null )
>> {
>> System.out.println(message);
>> }
>> else
>> {
>> System.out.println("This is not a box.") ;
>> }
>> }
>> }
>>
>> rather than any of the solutions on that page. Does that make me a
>> "hacker"?
>
> I think it does, but I like it better then the ones on the page. It's
> easy to maintain, fast and doesn't cluttering things up with 'if'
> condidtions . Thumbs up!
Thanks!
Of course, having used Lisp in my spare time now for several months, I
look at the above code and think it's still rather verbose ;-)
Lisp really is an incredible sweet-spot of language design.
Cheers,
Bill.
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
>
> Common Lisp is a multi-paradigm language. We don't try to fit everything
> into class-based OO designs, only those things that lend themselves to
> those. What looks like a bad OO design might just as well be a good
> imperative or functional design.
I agree with that.
>
> And again, just because the type checker doesn't complain and you are
> using design pattern doesn't make your code a good OO design. See
> http://csis.pace.edu/~bergin/patterns/ppoop.html
I agree with that too. There are some pretty ugly solutions on that
page.
Here's the way an Java, OO hacker does things:
-------- OSDetector.java
class Unix {
public Unix() {
System.out.println("This is a UNIX based box and therefore
good.");
}
}
class Windows {
public Windows() {
System.out.println("This is a Windows based box and therefore
bad.");
}
}
class Linux extends Unix {}
class SuuOS extends Unix {}
class Windows95 extends Windows {}
class WindowsXP extends Windows {}
public class OSDetector {
public static void main(String[] args) {
try {
Class.forName(System.getProperty("os.name").replaceAll(" ",
"")).newInstance();
} catch (Exception e) {
System.out.println("This is not a box.");
}
}
}
------ End
What do you think? 2 points for originality?
·········@hotmail.com (Darren) wrote in message news:<··························@posting.google.com>...
> ···@ashi.footprints.net (Kaz Kylheku) wrote in message
>
> You make some good points, but they really only apply for bad designs.
Bad Java designs are not bad designs.
> A rule of thumb is: If you find yourself making lots of casts you
> probably have a bad OO model.
No, I probably have a language that's getting in the way of my very
good OO model, thank you very much.
If my model leads to beautiful Lisp code, it's good.
The language should rise to the design; I should not have to cripple
my design to the language.
·········@hotmail.com (Darren) writes:
> ···@ashi.footprints.net (Kaz Kylheku) wrote in message
>
> You make some good points, but they really only apply for bad designs.
Only to the bad design of languages such as Java, you mean?
Agreed.
···@ashi.footprints.net (Kaz Kylheku) writes:
> ·········@hotmail.com (Darren) wrote in message news:<···························@posting.google.com>...
> > Matthew Danish <··········@cmu.edu> wrote in message news:<··············@mapcar.org>...
> > > ·········@hotmail.com (Darren) writes:
> > > > a) No language can save you from bad programming. To catch those
> > > > bugs, polymorphism would have to be removed.
> > > > b) Again, I see polymorphism as the source of those bugs and I am
> > > > happy Java has it.
> > >
> > > If you want to see REAL polymorphism in action in a statically typed
> > > language, see ML or Haskell, not Java. Java is, for a statically
> > > typed language, a VERY BAD example.
> >
> > Well, this wasn't originally about polymorphism, but I believe that is
> > at the heart of the problem in all the examples shown. The conflict
> > between type checking and polymorphism.
> >
> > The basic problem this entire discussion is about is this:
> >
> > class A { }
> > class B extends A implements Z { public void foobar(); }
> > class C extends A { }
> >
> > ...
> > someMethod( new C() );
> > ...
> >
> > public void someMethod(A something) {
> >
> > // This is the bug all the examples are pointing out happens,
> > // and the Java compiler has no way to catch this
> >
> > // At this point we don't know what type "something" really is, we
> > only
> > // know it abides by the contract A
> >
> > ((B)something).foobar(); // ClassCastException *may* be comming
> > (runtime)
> >
> > // If an instance of C or A were passed, the code would work. If a
> > B
> > // instance is passed a ClassCastException will be thrown.
> > }
> >
> > My question for the Lisp, Haskell and ML (and any other language you
> > want to mention) experts:
> >
> > Could this kind of developer error be made in those languages?
>
> Of course the error can be made in any language. How can the
> programmer be prevented from asking for an operation that doesn't
> exist?
>
> The difference is in the level of intelligence in how the situation is
> represented and handled in the language.
>
> In Common Lisp, you would just try applying the FOOBAR method to the
> object. That could blow up if there is no method specialization for
> that object.
This suggests a kind of interactivity and experimental development
technique that is a fundamentally different mindset from the approaches
used in other language environments.
On Sun, 24 Oct 2004 19:48:47 +0000, Christopher C. Stacy wrote:
>> In Common Lisp, you would just try applying the FOOBAR method to the
>> object. That could blow up if there is no method specialization for
>> that object.
>
> This suggests a kind of interactivity and experimental development
> technique that is a fundamentally different mindset from the approaches
> used in other language environments.
Actually this is exactly how I develop programs at work, in Java. I write
code in one Emacs buffer, and in another Emacs buffer I have the Java
interpreter (beanshell) ready to evaluate snippets of code. I find it
*almost* as nice as using SLIME.
Responses from coworkers when I show them the way I write Java vary from
"why would anyone ever want to do that?"[1] to "wow, I need to learn Emacs!".
Cheers,
Bill.
[1] My reply is "because I've found when I write code interactively I have
far fewer bugs, more modular code, and much better APIs" (I find that if
an API is clumsy, trying to use it from a REPL makes that clumsiness clear
*very* quickly).
--
"If you give someone Fortran, he has Fortran. If you give someone Lisp,
he has any language he pleases." -- Guy Steele
William Bland <·······@abstractnonsense.com> wrote in message
> Actually this is exactly how I develop programs at work, in Java.
Me too.
> I write code in one Emacs buffer, and in another Emacs buffer I have the Java
> interpreter (beanshell) ready to evaluate snippets of code. I find it
> *almost* as nice as using SLIME.
Me too! Bean shell rocks. We use it many places in the system for
run-time scripting so we can tune the app after its installed andin
other places for generating code that generates code.
>
> Responses from coworkers when I show them the way I write Java vary from
> "why would anyone ever want to do that?"[1] to "wow, I need to learn Emacs!".
There was a guy at work who reacted the same way when he first ran
into the bean scripts running on the server side. Then he saw he
could edit his program while it was executing ..."Sweeet!" :)
I imagine any Lisper would feel right at home with Beanshell.
·········@hotmail.com (Darren) writes:
> If not, an example of how to do a similar polymorphic call in those
> languages would be appreciated.
This is how a functional statically typed language would do things
(Haskell, various MLs, but specifically Standard ML here):
The type system in SML has the notion of ``parametric polymorphism''
where it is possible to have types with unknowns, with variables. The
polymorphism comes into play when you define functions using these
parameterized types. Consider a basic type, the list.
list is defined to be either [], the empty list, or X::list, in Lisp
terms a cons.
Without parameterized types, you would have to define separately
an int_list, a char_list, etc, for example:
datatype int_list = INT_NIL | INT_CONS of int * int_list
This defines what is called a disjoint type that has two constructors.
INT_NIL constructs a value of type int_list. INT_CONS also constructs
a value of type int_list, for which it requires two constructor
parameters; an int, and an int_list. Then you could construct such an
int_list by doing this:
INT_CONS(1, INT_CONS(2, INT_CONS(3, INT_NIL)))
The real definition of list looks something more like this:
(pseudo-code) datatype 'a list = [] | :: of int * 'a list
where 'a (pronounced: alpha) stands for the type parameter. This is
pseudo-code because you (the programmer) couldn't really define a
constructor named [] or the operator :: in SML, but other names would
be acceptable.
With such a system, it is easy to write statically-typed polymorphic
functions that operate on lists in general, or specific kinds of
lists.
fun length [] = 0
| length (first::rest) = 1 + (length rest)
That is the simple recursive definition of the length function on
lists. It makes use of some pattern matching functionality of SML, as
you can see the arguments to the function are deconstructed
automatically. The type system of SML will determine that the type of
the above function is
length : 'a list -> int
Which means that length is of the function type with parameter 'a list and
return value int. So
length [1,2,3] (* translates to length (1::(2::(3::[]))) *)
is a valid call and so is
length ["a","b","c"]
What the typechecker does is called Unification of types. In this
example, it would take the following types:
length : 'a list -> int
["a","b","c"] : string list
and determine that
length ["a","b","c"] : int
by substituting string for 'a and applying the function.
But suppose you wrote a function like this:
fun sum [] = 0
| sum (first::rest) = first + (sum rest)
Now the unifier has to consider that you are using the + operator
op+ : int * int -> int
Therefore, first must be an int, and rest must be an int list. So,
sum : int list -> int
This makes
sum [1,2,3]
a valid, well formed, typed expression (with result 6) and
sum ["a","b","c"]
a static type error.
Standard ML's type system is based closely on what is called the
Hindley-Milner type system. It is designed for parametric
polymorphism, and for ease of type inference. SML adds a module
system. Other languages with similar goals include Objective CAML,
which adds an object system, and Haskell, which makes use of an
interesting category theory concept called Monads. [1]
As you can see, the goals of these languages are fundamentally
different from Common Lisp or Java. In Common Lisp, it is acceptable
for type errors to be found at run-time if they cannot be found by the
compiler, and there is a robust condition system for handling such
errors (and others). Java seems to be designed with the goal of being
simple, and thus leaves out both the rigor of a proper static type
system, and the robustness of a good condition system.
[1] More reading:
http://www.standardml.org/ -- SML
http://www.smlnj.org/ -- SML/NJ, an implementation with a top-level
http://caml.inria.fr/ -- OCAML
http://www.haskell.org/ -- Haskell
--
;; Matthew Danish -- user: mrd domain: cmu.edu
;; OpenPGP public key: C24B6010 on keyring.debian.org
Darren wrote:
> Pascal Costanza <········@web.de> wrote in message news:<cldj43
>
>>I didn't say that the example is standard, but that the idiom is
>>standard. Yes, the example contains a bug that may be very obvious when
>>you look at the code _in isolation_. When it's part of a much bigger
>>project, it's easy to miss the bug because the code looks quite correct
>>from far away. (Java has too much noise in its source code, which makes
>>it hard to see the essential stuff.)
>
> When debugging, you would be looking at the code in isolation.
..._after_ you have found out where the bug is.
>>>I wasn't saying java developers were inexperienced, just the guy who
>>>created the initial bad code.
>>
>>As an experienced Java programmer you need to keep lots of code idioms
>>and patterns in mind and know them by heart which could be be automated
>>in other languages. In other words, the experience that Java requires
>>from programmers is a waste of time.
>
> Yes, casting is something to remember to do, but it's certainly not a
> lot to ask and is typical in a typed language supporting polymorphism.
I've been referring to the "iterate through this bunch of stuff" idiom,
not to type casts per se. If iteration were no issue they wouldn't have
added a new for construct in JDK 5.
> Read over that again, he always uses the word *programmer*.
Read my postings again, I have also always used the word "programmer".
>>First of all, this was not the only example in my paper, but one of
>>three examples. Do I understand that you agree to the validity of the
>>other examples? (Most people with whom I have discussed the paper accept
>>the first two examples but don't agree to the third one.)
>
> Well, I actually didn't agree with "Statically Checked Implementation
> of Interfaces"
[...]
> By having stub implementations implement the "Incomplete" interface,
> they are marked as such. Now that you have a marker interface, you
> can have the compiler tell you what you need to finish. Your build
> scripts do not make the "Incomplete" interface available. Using
> marker interfaces is used all the time for problem like this. By
> having the build scripts remove access to the marker interfaces you
> can get a complete list of what is missing where. We use ones for
> Incomplete, Questionable (put in from code inspections), FlawedLogic,
> RequiresReview.
>
> Very handy stuff. It not only removes the problems you listed, it
> solves some others and helps ensures a quality of production builds.
That's a neat idea I haven't thought of before. I have to think about
the implications, but thanks anyway.
> For this one: "Statically Checked Exceptions"
>
> I do agree with this one. Though it can be dealt with pretty easily
> as well and without limitation, unfortunetly it requires more code.
Yep, and more code implies more potential sources of bugs.
>>Writing correct multi-threaded programs in Java doesn't consist of
>>arbitrarily placing synchronized statements in several places. Your
>>suggestion wouldn't work because you are not synchronizing the other
>>accesses to "dilbert".
>
> Your right, but to be a fair example you should have syncrhonized the
> methods that can change what Dilbert is anyways. Then this simple fix
> would always work and you wouldn't even need to worry about an error
> or exception handling, etc. But as you said, that is beside the
> point.
Usually, you only want write accesses block other accesses to a
variable. That's harder.
> I also know Java and have used it since beta. I don't expect it to
> make up for incorrect usage of its capabilities. Polymorphism is a
> powerful capability that I am willing to live with. The problems you
> identify all extend from polymorphism, not type checking. They are
> conflicting concepts and for Java to support both, there are trade
> offs.
These problems don't exist in a dynamically typed language.
> To me, what you are really pointing out is that mixing the
> polymorphism and type checking within a language can lead to problem
> situations that may be tricky to identify. That I would have to agree
> with.
Exactly.
>>But this is besides the point anyway. The gist of that example is that
>>the bug occurs only after three hours of repeatedly reassigning
>>"dilbert" every five seconds. That's a very extreme assumption - in
>>reality the assignments would occur much less often. The Java language
>>doesn't help you at all in seeing that there might be a problem. To
>>restate my statement made in the paper, this is the kind of bug you
>>definitely don't want to have. It's one of the worst because it is not
>>reproducible. (And it is based on a "check an object's type before
>>casting it" idiom as described in the Java language specification!)
>
> Well..that example is more about bad thread programming in my opinion,
> not idioms. Any bad thread programming in any language is troublesome
> to diagnose and reproduce. It's a bad example for the problem you are
> illustrating IMHO.
Maybe.
> a) No language can save you from bad programming.
But some make it easier than others.
> To catch those
> bugs, polymorphism would have to be removed.
No, not at all.
>>Why should a complete failure taken as "what it is and [...] be used as
>>such"?
>
> I'll tell my customers Java is a failure next time they try to write
> up another damn success story about their experience wtih our
> software...over 3 million lines of Java, developed in 2 years, under
> budget and on-time ;)
You have put a smiley here, so I take this as an ironic comment. I still
think it should be noted that my "complete failure" remark was about
Java's type system, not about Java in general. Some of Java's aspects
are very good, like its excellent JVM implementation, and it's good to
hear that it works well for you. In large projects the choice of
programming language is but one aspect of many issues one has to take
into account. IMHO, social issues by far dominate technical ones. [1]
However from a programming language perspective, Java sucks big time: It
was a nice little language at first, but has clearly failed to meet its
original design criteria, and the incomplete patches that are applied to
it over the years make it continually worse (-> incomplete closures for
inner classes, "autoboxing" instead of turning primitive types into real
objects, incomplete generic types, etc.)
Pascal
[1] Lispers usually have to work against "social limitations", not
technical ones. For instance, many people believe that Common Lisp is
"just" a functional language while it can be any language you like, and
it already incorporates one of the most powerful object-oriented
language around.
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Darren wrote:
> If faced with this in the future, use a synchronzied block:
>
> synchronized(dilbert){
> if (dilbert instanceof Employee) {
> System.out.println("Employer: " +
> ((Employee)dilbert).getEmployer().getName();
> }
> }
or, if you want some procedural code that will 'work':
final Object dilbert = TheClass.dilbert; // or this.dilbert
if (dilbert instanceof Employee) {
System.out.println("Employer: " +
((Employee)dilbert).getEmployer().getName();
}
}
as you are synchronizing on the monitor of the object that the value of
the dilbert field is a reference to when the synchronization is
executed, and the bug is that the field value may be changed between the
execution of the instanceof and the cast, so it equally may be changed
between the execution of the synchronization and the synchronized block.
Though the above has slightly different semantics, namely that it is
solely concerned with the object that was referred to at the start of
the block, which can become significant if the concurrent processing
takes longer.
Java is an object oriented not a type based language, so you should use
encapsulation and not have exposed global state or explicit instanceof
checks. A method getEmploymentDetails() that returns a map would be
better design, and extensible to other types:
for (Map.Entry detail : dilbert.getEmploymentDetails().entrySet()) {
System.out.println(detail.getKey() + ": " + detail.getValue());
}
This also doesn't require synchronization[2]. Use of either
single-assignment or OO generally helps with concurrency problems.
Being an employee is a role that a person is decorated with, not a class
of person[1]. The distinction is perhaps more blurred in dynamic
languages, but has to be managed explicitly in static typed ones. The
same object should map to the same person through their employment
history; the identity of the dilbert concept should rest in the object
that models dilbert not in the name of the field.
Pete
[1] http://ootips.org/role-object.html and other sites elaborate on this
[2] assuming either the map is a copy not live, or that the keys are
single assignment value objects
Pete Kirkham <·················@cafemosaic.co.uk> wrote in message
> as you are synchronizing on the monitor of the object that the value of
> the dilbert field is a reference to when the synchronization is
> executed, and the bug is that the field value may be changed between the
> execution of the instanceof and the cast, so it equally may be changed
> between the execution of the synchronization and the synchronized block.
> Though the above has slightly different semantics, namely that it is
> solely concerned with the object that was referred to at the start of
> the block, which can become significant if the concurrent processing
> takes longer.
>
> Java is an object oriented not a type based language, so you should use
> encapsulation and not have exposed global state or explicit instanceof
> checks. A method getEmploymentDetails() that returns a map would be
> better design, and extensible to other types:
> for (Map.Entry detail : dilbert.getEmploymentDetails().entrySet()) {
> System.out.println(detail.getKey() + ": " + detail.getValue());
> }
>
> This also doesn't require synchronization[2]. Use of either
> single-assignment or OO generally helps with concurrency problems.
>
> Being an employee is a role that a person is decorated with, not a class
> of person[1]. The distinction is perhaps more blurred in dynamic
> languages, but has to be managed explicitly in static typed ones. The
> same object should map to the same person through their employment
> history; the identity of the dilbert concept should rest in the object
> that models dilbert not in the name of the field.
>
>
> Pete
>
>
> [1] http://ootips.org/role-object.html and other sites elaborate on this
> [2] assuming either the map is a copy not live, or that the keys are
> single assignment value objects
I totally agree with your post, the designs are flawed. But I left
that out of the discussion (as much as I could) because the point
being made was more around people make bad code and type checking is
supposed to help you avoid that (adding they make bad designs, would
just distract from the discussion).
I think we sorta came to an agreement that its not so much Java type
checking as it is Javas mix of type checking and polymorphism that
causes problems.
Darren wrote:
> I think we sorta came to an agreement that its not so much Java type
> checking as it is Javas mix of type checking and polymorphism that
> causes problems.
In a sense, yes. Consider a class hierarchy:
C -+
|} inheritance
D extends C -+
|} implicit conversion
E extends D -+
Both static typing and polymorphism are inclusive, but in opposing
directions. The type D includes instances of type E, but not of type C.
However, the inherited definitions in D are those of class C, but not of
class E. The problems of Java's type system stem from the fact that
these two directions "contradict" each other: The new definitions in D
hurt static checkability of type C, whereas disallowing new definitions
in D would render object-oriented inheritance useless.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza <········@web.de> wrote in message news:<············@newsreader2.netcologne.de>...
> Darren wrote:
>
> > I think we sorta came to an agreement that its not so much Java type
> > checking as it is Javas mix of type checking and polymorphism that
> > causes problems.
>
> In a sense, yes. Consider a class hierarchy:
>
> C -+
> |} inheritance
> D extends C -+
> |} implicit conversion
> E extends D -+
>
>
> Both static typing and polymorphism are inclusive, but in opposing
> directions. The type D includes instances of type E, but not of type C.
> However, the inherited definitions in D are those of class C, but not of
> class E. The problems of Java's type system stem from the fact that
> these two directions "contradict" each other: The new definitions in D
> hurt static checkability of type C, whereas disallowing new definitions
> in D would render object-oriented inheritance useless.
>
>
> Pascal
Agreed, they are an odd mix because they contradict each other. I
consider it a strength not a weakness. It's a dichotomy you can hack
with. As long as developers are aware of it, they can keep out of
trouble and use it to their advantage.
Pete Kirkham wrote:
> Being an employee is a role that a person is decorated with, not a class
> of person[1].
(change-class dilbert 'employee :employer "some company")
Definitely not a full-fledged solution because change-class has issues,
but a hint that things could be done differently.
> The distinction is perhaps more blurred in dynamic
> languages, but has to be managed explicitly in static typed ones. The
> same object should map to the same person through their employment
> history; the identity of the dilbert concept should rest in the object
> that models dilbert not in the name of the field.
Which, I think, supports my point: Getting such models is difficult
enough, having to fight a pointless type system at the same time is a waste.
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Pascal Costanza wrote:
> (change-class dilbert 'employee :employer "some company")
>
> Definitely not a full-fledged solution because change-class has issues,
> but a hint that things could be done differently.
I think I'd prefer either a fully fledged description logic, or stay in
message-passing OO and use an association - either could then include
start and end dates, and model people with multiple jobs - rather than
using intensional class to model employment, even though some
intensional class based object systems do allow changes of class to
preserve object identity.
>> The distinction is perhaps more blurred in dynamic languages, but has
>> to be managed explicitly in static typed ones. The same object should
>> map to the same person through their employment history; the identity
>> of the dilbert concept should rest in the object that models dilbert
>> not in the name of the field.
>
>
> Which, I think, supports my point: Getting such models is difficult
> enough, having to fight a pointless type system at the same time is a
> waste.
Yes, if you're fighting the language rather than using it, you've done
something wrong, and that might well be that you're using the wrong
language for the job. But some things, like understanding that
employment is a transient association between a person and an employer,
is not a startling revelation to most Java monkeys.
Pete
Pete Kirkham wrote:
> I think I'd prefer either a fully fledged description logic, or stay in
> message-passing OO and use an association - either could then include
> start and end dates, and model people with multiple jobs - rather than
> using intensional class to model employment, even though some
> intensional class based object systems do allow changes of class to
> preserve object identity.
To be more specific, in my experience systems suffer requirements drift,
so as well as modelling 'an employee is a person who is employed', you
end up modelling 'a manager is an employee who manages other employees',
'a mentor mentors other employees' and 'a married person is a person
with a living spouse' etc.
If you use description logics, where it is possible for an instance to
be of more than one type at once without declaring an intensional class
- you are a married-employee-manager automatically if you satisfy the
classifier predicates for each- then the system scales linearly with
requirements.
In the same way, if you use associations the system scales linearly,
though usually with a higher constant factor.
If you use intensional types, you need a type for MarriedEmployee,
MarriedManagerCurrentlyInEmployment, etc, and the system scales
exponentially with requirements increments. You might get away with it
for very small, static systems, but in larger evolving systems this is a
problem. Even if you were to use macros to generate the possible cross
product of the classifiers as intentional types, you'll hit machine
limits before you have 32 or 64 classifiers, and human comprehensibility
limits well before then.
Of course, you could use CL and change-class to build the DL, and
dynamically create the convolved types on demand, but then you're
outside the normal use of that language's type system too.
Pete
Pete Kirkham wrote:
> Yes, if you're fighting the language rather than using it, you've done
> something wrong, and that might well be that you're using the wrong
> language for the job. But some things, like understanding that
> employment is a transient association between a person and an employer,
> is not a startling revelation to most Java monkeys.
OK, your point is well taken.
Thanks,
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
From: Lars Brinkhoff
Subject: Re: Static vs Dynamic [was Re: Sacla loop: a new loop implementation]
Date:
Message-ID: <85fz49bo50.fsf@junk.nocrew.org>
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> So, it seems to me that a variable that lacks a type declaration can
> be seen as a variable with a type declaration of type t because t is
> the supertype of all types. This means that everything seems to be
> typed at compile time and thus it fits the definition of static typed
> language that I presented.
"Common Lisp is already statically, implicitly typed. All objects are
of type T, and this is very rigourously enforced." -- Erik Naggum
--
Lars Brinkhoff, Services for Unix, Linux, GCC, HTTP
Brinkhoff Consulting http://www.brinkhoff.se/
Antonio Menezes Leitao <··············@evaluator.pt> writes:
> If we return to the original issue that started this subthread, we
> will remember that we were looking for a macro that could encapsulate
> different iteration forms without requiring the creation of iterator
> objects or closures.
I think this is where you got it wrong. The original issue was to be
able to not resort to the use of such objects/closures when possible.
Never requiring their creation or use under any circumstances is not
possible - even in statically typed languages.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
Antonio Menezes Leitao wrote:
[SNIP: we agree that a CL compiler can do type inference
and make use of the results, even though it sometimes
can't deduce anything useful about types.]
>> So we have an example of a system where not-always-available
>> type information is helpful. Why do you "seriously doubt"
>> that there might be other examples?
>
> If we return to the original issue that started this subthread, we
> will remember that we were looking for a macro that could encapsulate
> different iteration forms without requiring the creation of iterator
> objects or closures. For this to be possible, I still don't see any
> way without demanding that the necessary type information is available
> at compile time.
We were looking for something that could, ***in cases where
the compiler is able to deduce enough about the type of thing
being iterated over***, do that job.
That requires only that type information is *sometimes*
available. It will be useful only if type information is
*often* available. But, as it happens, we know that type
information is often available, because good CL compilers
exist.
> Now, given the multitude of different ways that we
> can use in Common Lisp to produce an object that we want to iterate
> (e.g., on a local variable, from a function parameter, the dynamic
> value of a special variable, the result of a yet unknown function
> call, the result of a read operation, the result of a generic function
> call, etc), I'm afraid that not even the most clever type inference
> mechanism available in Common Lisp will be able to always produce the
> necessary information for the macro to expand. I suspect (based on my
> own, obviously limited, experience) that this will be a frequent
> situation.
No one (except maybe you) was demanding that enough information
to avoid inefficiency would *always* be available.
> Now, we all agree that it is possible to expand according to the lack
> of type information but that will require that our iterator macro must
> be able to expand into something that will decide at run time.
Correct.
> If the
> macro writer still wants to avoid iterators or closures he might
> choose an expansion that uses a typecase form or some such. The
> interesting thing is that if there was type information available that
> would allow him to not expand into the typecase, that exact same
> information is available now to the normal code optimization phases of
> compiler and it can achieve the same effect on the typecase form.
Quite right ... when the complete set of possibilities is
known to the writer of the macro, or at least is available
to the macro expander itself. But there's a compile-time
efficiency issue here, and also something possibly more
fundamental.
Suppose we have an extensible system for defining iterable
types. Then, in a world where type information is available
at macro-expansion time, you can write a macro that turns
(let (...)
(declare (type something-iterable x))
(loop for y over x do ...))
into, say,
(let (...)
(declare (type something-iterable x))
(loop for :foo1 = +iteration-start+ then (grab-next :foo1) do
(let ((y (extract x :foo1)))
...)))
Well, of course it wouldn't be quite that, but you get the idea.
The macro-expander might do this by some sort of lookup in a
table of iteration protocols. It could be cheap.
But in a world where macros have no access to such information
and the optimization must be done by the compiler, we get instead
(let (...)
(declare (type something-iterable x))
(typecase x
((simple-vector) ...)
((list) ...)
((string) ...)
...
((something-iterable) ...)
...))
which is a potentially very large blow-up of code. Now imagine
that we have nested loops. Yow!
This all assumes that the type information potentially available
at macroexpansion time is such as to be completely captured by
that sort of expansion. It's not clear to me that it is. I don't
have any entirely convincing examples to hand, but do you in fact
disagree?
Oh, another thing. This "solution" to the problem differs
from the sort of thing Tim Bradshaw was talking about in another
important way: what happens when the type information is not
available at macro expansion time. The enormous typecase
turns into an enormous explicit typecase. Code bloat.
(Again, especially when there are nested loops.) That
might sometimes be what you want, but I bet it usually
isn't. The approach Tim was suggesting produces *different
and simpler code* when no useful type information is
available at macro expansion time: a few generic function
calls, perhaps.
> Given the fact that code optimization is done at a latter phase than
> macro expansion I also suspect (based on my own, obviously limited,
> experience) that there will be more type information available that
> will allow it to make a better job.
There's no reason why both can't happen. Sophisticated
optimizing compilers tend not to like dealing with
exponentially huge amounts of code, so you may get
better optimization by avoiding the sort of blowup
I mentioned above.
> > You've used the term "statically typed language" a lot,
> > and you've defined it in terms that are ambiguous because
> > a crucial quantifier is missing. Could you please say
> > which of the following (if any) is your definition?
> >
> > 1 "A language in which type information is always
> > available at compile time"
...
> I guess that point 1 is the most appropriate definition.
Splendid. Then it is obviously not true that making macro
expanders able to use whatever type information happens
to be available when they run would turn Lisp into a
"statically typed language".
> I do consider the following definitions:
>
> A statically typed language is a language where types are known at
> compile time. A dynamically typed language is a language where types
> are discovered at runtime.
Those are not definitions, because once again crucial quantifiers
are missing: sometimes known at compile time, or always?
> Obviously, there are languages that can be both statically and
> dynamically typed (Common Lisp is one example). All I'm saying is
> that to have useful type information available at macroexpansion time
> in general, your language or (as Duane already pointed it out) the
> programs that you can write in that language must be statically typed,
> at least, in the fragments that use those macros.
A better way of putting it: when your code happens to be
(to some extent) statically typeable, those macros can
exploit the fact. That is the exact same principle on
which type-inferencing compilers rely.
--
Gareth McCaughan
.sig under construc
Gareth McCaughan wrote:
>
> No one (except maybe you) was demanding that enough information
> to avoid inefficiency would *always* be available.
I certainly was not.
> Oh, another thing. This "solution" to the problem differs
> from the sort of thing Tim Bradshaw was talking about in another
> important way: what happens when the type information is not
> available at macro expansion time. The enormous typecase
> turns into an enormous explicit typecase. Code bloat.
> (Again, especially when there are nested loops.) That
> might sometimes be what you want, but I bet it usually
> isn't. The approach Tim was suggesting produces *different
> and simpler code* when no useful type information is
> available at macro expansion time: a few generic function
> calls, perhaps.
>
I have, in fact, written macros (not for iteration) which did the
TYPECASE-and-assume-the-compiler-will-elide-code trick (they were
never intended to work with other than CMUCL), and I made the crucial
mistake of forgetting that:
(locally (declare (type t x)) ; this declaration was not literally
present
(typecase x
...
(t ...)))
is not at all the same thing as
(locally (declare (type good-type x))
(typecase x
...
(good-type ...)
(t ...)))
because in the latter case the compiler can elide the unreachable
branches (possibly inserting a check), but in the former case it can
not elide anything: in particular it can't just take the T branch.
These macros were in fact being wrapped around things by other macros
to slightly more depth than I'd expected (but not very high, and
anyway, since these were not nested loops but expansions of things in
a little language there should not have been any real issue).
The end result was that if there were enough type declarations
compilation was very painful but usually possible, but if there
weren't it was a hideous catastrophe.
--tim
Tim Bradshaw wrote:
> I have, in fact, written macros (not for iteration) which did the
> TYPECASE-and-assume-the-compiler-will-elide-code trick (they were
> never intended to work with other than CMUCL), and I made the crucial
> mistake of forgetting that:
Maybe it would be useful to have a new special operator,
COMPILER-TYPECASE. It would have the syntax:
(compiler-typecase <form> { (<type> <forms>) } )
The meaning would be this: if the compiler can determine
that the type of <form> is a subtype of some clause's <type>
(but not of any earlier <type> in that list of clauses) then
this becomes equivalent to (progn <form> (progn <forms>)), where
<forms> is the list following that <type>. Typically the last
clause would have <type> being T.
This would defer the type propagation to compile time,
but still let you avoid the code explosion.
Paul
From: Rahul Jain
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <87hdooloh5.fsf@nyct.net>
"Paul F. Dietz" <·····@dls.net> writes:
> Maybe it would be useful to have a new special operator,
> COMPILER-TYPECASE. It would have the syntax:
>
> (compiler-typecase <form> { (<type> <forms>) } )
I second the proposal!
Sheeh. All this complicated discussion and all we needed was something
as simple as that.
Other than that, the only place that type information is really used and
can be optimized at compile time would involve sealing of GFs and
classes so that the effective method can be inlined. This probably would
be better implemented within the compiler guts than in some portable
manner anyway.
--
Rahul Jain
·····@nyct.net
Professional Software Developer, Amateur Quantum Mechanicist
From: Christophe Rhodes
Subject: Re: Sacla loop: a new loop implementation
Date:
Message-ID: <sqr7ntot7v.fsf@cam.ac.uk>
"Tim Bradshaw" <··········@tfeb.org> writes:
> I have, in fact, written macros (not for iteration) which did the
> TYPECASE-and-assume-the-compiler-will-elide-code trick (they were
> never intended to work with other than CMUCL), and I made the crucial
> mistake of forgetting that:
>
> (locally (declare (type t x)) ; this declaration was not literally
> present
> (typecase x
> ...
> (t ...)))
>
> is not at all the same thing as
>
> (locally (declare (type good-type x))
> (typecase x
> ...
> (good-type ...)
> (t ...)))
(locally (declare (type (not good-type) x))
(typecase x
...
(good-type ...)
(t ...)))
should allow the system to delete the good-type branch, though. Would
that have helped?
Christophe
Christophe Rhodes wrote:
>
> (locally (declare (type (not good-type) x))
> (typecase x
> ...
> (good-type ...)
> (t ...)))
>
> should allow the system to delete the good-type branch, though.
Would
> that have helped?
>
Yes, except that the types I was dispatching on where things that (I
hoped that) the compiler had inferred: I didn't actually have any type
declarations (or very few) in the code.
--tim
Gareth McCaughan <················@pobox.com> writes:
> Antonio Menezes Leitao wrote:
>
> > If we return to the original issue that started this subthread, we
> > ...
>
> We were looking for something that could, ***in cases where
> the compiler is able to deduce enough about the type of thing
> being iterated over***, do that job.
Exactly.
> > [the idea of generating jumbo typecases and hoping the
> > implementation's optimizer will somehow elide the unnecessary bits
> But in a world where macros have no access to such information
> and the optimization must be done by the compiler, we get instead
>
> (let (...)
> (declare (type something-iterable x))
> (typecase x
> ((simple-vector) ...)
> ((list) ...)
> ((string) ...)
> ...
> ((something-iterable) ...)
> ...))
>
> which is a potentially very large blow-up of code. Now imagine
> that we have nested loops. Yow!
This is exactly true. Originally, the expansions of the system I've
discussed a bit had a bug in them which caused this very thing to
actually happen. The results were indeed horrific and the
implementation did not sort things out and save me from my bug (i.e.,
Antonio's requested design). The compilations were indeed phenomenal
at times due to combinatorial explosions as you hint at. The
resulting code bloat was equally "impressive".
> Oh, another thing. This "solution" to the problem differs
> from the sort of thing Tim Bradshaw was talking about in another
> important way: what happens when the type information is not
> available at macro expansion time. The enormous typecase
> turns into an enormous explicit typecase. Code bloat.
> (Again, especially when there are nested loops.)
It is hard to overstate this: in practice this "solution" (aka bug)
does indeed produce horrific results.
> A better way of putting it: when your code happens to be
> (to some extent) statically typeable, those macros can
> exploit the fact. That is the exact same principle on
> which type-inferencing compilers rely.
Exactly correct.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
On Tue, 28 Sep 2004 20:33:00 +0000, Gareth McCaughan wrote:
> I'd love to see the following design goal:
>
> - A flexible and clearly documented extension mechanism,
> enabling programmers to make LOOP loop in new ways
> efficiently and without great pain.
>
> As I understand it, the MIT implementation manages
> at least "flexible". I don't know how it does in other
> respects :-).
Thank you for the comment.
Hmm. I intentionally ignored the extensibility, and called the
functionality in MIT implementation part of historical code. I'm afraid
that the wording was misleading. I was thinking stepping out of the
standard was a bad habbit. But since Sacla is a testbed for experiments, I
might reconsider.
I think adding an ad hoc mechanism to this implementation, in order to
allow users to extend for-as-clause is relatively easy, (it's just a
matter of adding a subclause processor function) although that means
letting the structure of internal variables (*loop-components* and
*for-as-components*) be opened up for users, which I don't think is
perticulary clean.
I heard iterate macro has an extension mechanism. I think I'm gonna study
it.
Yuji.
Yuji Minejima <········@nifty.ne.jp> writes:
> I heard iterate macro has an extension mechanism. I think I'm gonna study
> it.
The MIT LOOP macro (including the version with refinements done by
Symbolics) has always had an extension mechanism; that would be a good
place to begin your studies.
On Wed, 29 Sep 2004 00:36:15 -0400, Carl Shapiro wrote:
> The MIT LOOP macro (including the version with refinements done by
> Symbolics) has always had an extension mechanism; that would be a good
> place to begin your studies.
Yes, I've noticed that. I think you mean the loop-universe structure and
add-loop-path and the like. I guess you need to know too many
implementation details to extend the loop using these things. (Or am I
making a gross mistake?) I think adding something like that to my
implementation is relatively easy, it's just a matter of preparing a
mechanism to register clause processor functions to some table (and
document it).
In order to let the users to extend the Loop facility, I admit that
the users probably need to know some kind of internal workings of the
facility. I think the key is how to abstract the internal mechanisms and
present that to the users.
I quickly checked iterate's extension facility, it seems to have
defmacro-clause and defmacro-driver which let the users to write a new
clause in terms of already defined ones (plus ordinaly code).
I think defining a mini language which can be used in a clause defining
form is important.
Actually my implementation already has some thing like this.
The count clause is implemented like the following.
(lp :if form :do `(incf ,var)) ;; lp behaves like normal loop macro.
Regards,
Yuji.
Yuji Minejima <········@nifty.ne.jp> writes:
> Yes, I've noticed that. I think you mean the loop-universe structure and
> add-loop-path and the like.
ADD-LOOP-PATH and LOOP-UNIVERSE do not represent the public interface
for writing LOOP extensions. The documented LOOP extension interface
is DEFINE-LOOP-SEQUENCE-PATH which wraps ADD-LOOP-PATH and rebinds the
resident LOOP-UNIVERSE. Another elaboration of ADD-LOOP-PATH is the
definer DEFINE-LOOP-ITERATION-PATH, whose purpose is easily inferred
but not generally useful.
Lucid documented writing extensions to the LOOP macro. If you do not
have a copy of their The Loop Facility manual handy, ask Google about
"liquid common lisp loop facility" for a pointer to an on-line
version. Next, look out for the DEFLOOP macro which is more or less a
renamed version of DEFINE-LOOP-SEQUENCE-PATH.
On Wed, 29 Sep 2004 04:09:57 -0400, Carl Shapiro wrote:
> Lucid documented writing extensions to the LOOP macro. If you do not
> have a copy of their The Loop Facility manual handy, ask Google about
> "liquid common lisp loop facility" for a pointer to an on-line
> version. Next, look out for the DEFLOOP macro which is more or less a
> renamed version of DEFINE-LOOP-SEQUENCE-PATH.
Thank you for the information. I didn't have the manual. I just found a
Loop manual at LispWorks site. I should have searched for it first.
I'm surprised that they are actually opening up the internal template
filler variables (init-bindings, prologue-forms, pre-step-tests, etc) to
the users.
Now things are getting interesting. I'm considering taking the same
approach (let the users touch the template filler variables) for the
lowest layer, then add another abstraction which let the users to define
their own loop clause in terms of already defined one (with something like
the lp macro in my implementation). But I think I should peruse the
manual more.
Regards,
Yuji.
Yuji Minejima <········@nifty.ne.jp> writes:
> I'm surprised that they are actually opening up the internal template
> filler variables (init-bindings, prologue-forms, pre-step-tests, etc) to
> the users.
There are two approaches taken: one which enables the user to pass a
function to return values needed to drive and terminate a loop, and
the other which provides a wrapper around numerically keyed sequences.
> Now things are getting interesting. I'm considering taking the same
> approach (let the users touch the template filler variables) for the
> lowest layer, then add another abstraction which let the users to define
> their own loop clause in terms of already defined one (with something like
> the lp macro in my implementation). But I think I should peruse the
> manual more.
Your lowest layer interface is analogous to DEFINE-LOOP-ITERATION-PATH
or the Lucid DEFLOOP. DEFINE-LOOP-SEQUENCE-PATH is one example of the
abstraction you would like to add on top of that lowest layer. (As an
academic exercise you could probably write the current sequence path
definer in terms of the iteration path definer.)
On Thu, 30 Sep 2004 00:52:03 -0400, Carl Shapiro wrote:
> Your lowest layer interface is analogous to DEFINE-LOOP-ITERATION-PATH
> or the Lucid DEFLOOP. DEFINE-LOOP-SEQUENCE-PATH is one example of the
> abstraction you would like to add on top of that lowest layer. (As an
> academic exercise you could probably write the current sequence path
> definer in terms of the iteration path definer.)
I fully agree with you.
Let me present an example to illustrate what I want to do in the
end. Let's consider a driver that iterates over all the elements of a
vector, ignoring its fill-pointer. (FOR-AS-ACROSS subcause respects
the fill-pointer of its argument vector, so it doesn't cut it.)
With this, you can write something like the following in a loop,
(loop for n of-type fixnum in-whole-vector some-vector
...)
I'd like to let the users be able to define this subclause like this.
(define-for-as-subclause in-whole-vector (var type)
(let ((vector (gensym))
(end (gensym))
(index (gensym)))
(lp :with var :of-type type
:with vector := (form1)
:with end := `(array-dimension ,vector 0)
:for index :from 0 :below end
:doing `(dsetq ,var (aref ,vector ,index)))))
This example is actually taken from ITERATE manual.
In iterate, you can write,
(defmacro-clause (FOR var IN-WHOLE-VECTOR v)
"All the elements of a vector (disregards fill-pointer)"
(let ((vect (gensym))
(end (gensym))
(index (gensym)))
(progn
(with ,vect = ,v)
(with ,end = (array-dimension ,vect 0))
(for ,index from 0 below ,end)
(dsetq ,var (aref ,vect ,index)))))
Thank you again, things are getting clearer after talking with you.
Yuji.
Yuji Minejima wrote:
> On Thu, 30 Sep 2004 00:52:03 -0400, Carl Shapiro wrote:
>
>
>>Your lowest layer interface is analogous to DEFINE-LOOP-ITERATION-PATH
>>or the Lucid DEFLOOP. DEFINE-LOOP-SEQUENCE-PATH is one example of the
>>abstraction you would like to add on top of that lowest layer. (As an
>>academic exercise you could probably write the current sequence path
>>definer in terms of the iteration path definer.)
>
>
> I fully agree with you.
>
> Let me present an example to illustrate what I want to do in the
> end. Let's consider a driver that iterates over all the elements of a
> vector, ignoring its fill-pointer. (FOR-AS-ACROSS subcause respects
> the fill-pointer of its argument vector, so it doesn't cut it.)
>
> With this, you can write something like the following in a loop,
> (loop for n of-type fixnum in-whole-vector some-vector
> ...)
What is wrong with
(loop for n of-type fixnum across some-vector ...)
?
I do agree that the IN, ON, and ACROSS clauses could be made more
useful, e.g. by allowing something like
(loop for n across some-vector from start to end ...)
and
(loop for n across some-vector from end downto start ...)
Also, although I may need to do some RTFM before, the DEFINE-LOOP-PATH
and friends allow you to play with the BEING EACH <x> clause in LOOP.
They do not allow you to introduce a new, possibly less verbose clause.
Cheers
--
Marco
Yuji Minejima <········@nifty.ne.jp> writes:
> Let me present an example to illustrate what I want to do in the
> end. Let's consider a driver that iterates over all the elements of a
> vector, ignoring its fill-pointer. (FOR-AS-ACROSS subcause respects
> the fill-pointer of its argument vector, so it doesn't cut it.)
>
> With this, you can write something like the following in a loop,
> (loop for n of-type fixnum in-whole-vector some-vector
> ...)
Such a LOOP path is trivial to author since we are operating across a
numerically-keyed collection. The extant LOOP sequence path definer
directly accommodates this pattern.
> (defloop (total-element total-elements) row-major-aref array-total-size)
T
> (setq array (make-array 4 :fill-pointer 4))
#<Vector T 4 FE0A476>
> (loop for i from 0 below (length array) do (setf (aref array i) i))
NIL
> (setf (fill-pointer array) 2)
2
> (loop for i from 0 below (length array) do (print i))
0
1
NIL
> (loop for i being the total-elements of array do (print i))
0
1
2
3
NIL
On Thu, 30 Sep 2004 14:49:03 -0400, Carl Shapiro wrote:
> Such a LOOP path is trivial to author since we are operating across a
> numerically-keyed collection. The extant LOOP sequence path definer
> directly accommodates this pattern.
> ...
Oh, you're right.
I just wanted to express having a general extension facility which let
you define a new clause in terms of loop clauses themselves is nice.
I guess most of the cases might be covered by more specific extension
facilities (in this case, the sequence path definer).
Yuji.
Yuji Minejima <········@nifty.ne.jp> writes:
> Oh, you're right.
> I just wanted to express having a general extension facility which let
> you define a new clause in terms of loop clauses themselves is nice.
This seems like a solution looking for a problem. Without the
addition of new clauses LOOP already has a vocabulary to express
iteration over virtually all sequenceable collections.
> I guess most of the cases might be covered by more specific extension
> facilities (in this case, the sequence path definer).
Each collection with a unique sequencing protocol will need such a
definer. Fortunately, the number of sequencing protocols is quite
limited as can be seen in languages implementing generic collections
and iteration protocols.
On Fri, 01 Oct 2004 03:30:26 -0400, Carl Shapiro wrote:
> Yuji Minejima <········@nifty.ne.jp> writes:
>
>> Oh, you're right.
>> I just wanted to express having a general extension facility which let
>> you define a new clause in terms of loop clauses themselves is nice.
>
> This seems like a solution looking for a problem. Without the
> addition of new clauses LOOP already has a vocabulary to express
> iteration over virtually all sequenceable collections.
Could you elaborate on what kind of a problem you have in mind?
Are you saying that `extending' already existing clauses (like you did
with for-as clause by defining for-as-total-elements subclause in your
previous article) is enough? If that's the case, I haven't given enough
thoughts on that issue yet.
>> I guess most of the cases might be covered by more specific extension
>> facilities (in this case, the sequence path definer).
>
> Each collection with a unique sequencing protocol will need such a
> definer. Fortunately, the number of sequencing protocols is quite
> limited as can be seen in languages implementing generic collections and
> iteration protocols.
I envision the use of a general extension facility like `lp' not only for
defining iteration clauses but for defining other kinds of clauses.
I admit I haven't thought things out yet if there are room for new kinds
of clauses.
I think I want something analogous to `defmacro' for LOOP clause.
(But a clause defining code just returning multiple template filler values
seems very hard to understand, instead using already understood loop
syntax seems very neat). I got this idea looking at the ITERATE manual. In
ITERATE, a clause is just a normal macro form.
Regards,
Yuji.