From: Christian Jullien
Subject: Performance of generic functions
Date: 
Message-ID: <43cb55af$0$20184$8fcfb975@news.wanadoo.fr>
Hi all,

I wonder what is considered as a "reasonnable" cost of a generic function 
V.S standard call

Given the following code (please adapt for CLtL):

(defgeneric gfun (a))
(defmethod  gfun ((a <integer>)) a)

(defun fun (a) a)

(defun test-fun (n)
   (do ((i 0 (1+ i)))
       ((= i n))
       (fun 0)))

(defun test-gfun (n)
   (do ((i 0 (1+ i)))
       ((= i n))
       (gfun 0)))

My own ISLISP (OpenLisp) implementation gives me for 10M iterations:

(test-fun 10000000) -> 0.038s
(test-gfun 10000000) -> 1.387s

So, generic function call is about 40 times slower that standard call.
Beside my (perhaps) poor implementation, I would like to know what is the 
ratio for other implementations (compiled code).

Thanks for your replies.

Christian 

From: Marc Battyani
Subject: Re: Performance of generic functions
Date: 
Message-ID: <6-ydnTm_r_iTxFbeRVnyig@giganews.com>
"Christian Jullien" <········@hotmail.com> wrote
> Hi all,
>
> I wonder what is considered as a "reasonnable" cost of a generic function
> V.S standard call
>
> Given the following code (please adapt for CLtL):
>
> (defgeneric gfun (a))
> (defmethod  gfun ((a <integer>)) a)
>
> (defun fun (a) a)
>
> (defun test-fun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (fun 0)))
>
> (defun test-gfun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (gfun 0)))
>
> My own ISLISP (OpenLisp) implementation gives me for 10M iterations:
>
> (test-fun 10000000) -> 0.038s
> (test-gfun 10000000) -> 1.387s
>
> So, generic function call is about 40 times slower that standard call.
> Beside my (perhaps) poor implementation, I would like to know what is the
> ratio for other implementations (compiled code).

100 000 000 calls LWW4.4.6:

gfun:
user time    =      6.218
system time  =      0.000
Elapsed time =   0:00:06
Allocation   = 5632 bytes standard / 3696 bytes conses
0 Page faults

fun:
user time    =      1.862
system time  =      0.000
Elapsed time =   0:00:02
Allocation   = 2880 bytes standard / 3630 bytes conses
0 Page faults

So a 3.33 ratio.
62 ns and 18.6 ns !

Marc
From: André Thieme
Subject: Re: Performance of generic functions
Date: 
Message-ID: <1137401843.186291.164290@g49g2000cwa.googlegroups.com>
SBCL 0.9.8, Gentoo, P4 2800, 32-Bit-System.

Ten million calls:

CL-USER> (time (test-fun 10000000))
Evaluation took:
  0.185 seconds of real time
  0.184971 seconds of user run time
  0.0 seconds of system run time
  0 page faults and
  0 bytes consed.

CL-USER> (time (test-gfun 10000000))
Evaluation took:
  0.307 seconds of real time
  0.306954 seconds of user run time
  0.0 seconds of system run time
  0 page faults and
  0 bytes consed.

CL-USER> (/ 0.307 0.185)
1.6594595



One hundred million calls:

CL-USER> (time (test-fun 100000000))
Evaluation took:
  1.885 seconds of real time
  1.835721 seconds of user run time
  0.0 seconds of system run time
  0 page faults and
  0 bytes consed.

CL-USER> (time (test-gfun 100000000))
Evaluation took:
  3.073 seconds of real time
  3.072533 seconds of user run time
  0.0 seconds of system run time
  0 page faults and
  0 bytes consed.

CL-USER> (/ 3.073 1.885)
1.6302387


André
--
From: Christian Jullien
Subject: Re: Performance of generic functions
Date: 
Message-ID: <43cb60e6$0$19713$8fcfb975@news.wanadoo.fr>
Thanks for the reply,

I'm not sure my test is good because (gfun 0) can be detected by good 
compilers as a call to gfun with integer.
In that case, it avoids the tampoline call to generic function and it can 
make a direct call to the specific integer method.

Could you please change the test to

(setq nb 0)

(defun test-gfun (n)
    (do ((i 0 (1+ i)))
        ((= i n))
        (gfun nb)))


"Marc Battyani" <·············@fractalconcept.com> wrote in message 
···························@giganews.com...
>
> "Christian Jullien" <········@hotmail.com> wrote
>> Hi all,
>>
>> I wonder what is considered as a "reasonnable" cost of a generic function
>> V.S standard call
>>
>> Given the following code (please adapt for CLtL):
>>
>> (defgeneric gfun (a))
>> (defmethod  gfun ((a <integer>)) a)
>>
>> (defun fun (a) a)
>>
>> (defun test-fun (n)
>>    (do ((i 0 (1+ i)))
>>        ((= i n))
>>        (fun 0)))
>>
>> (defun test-gfun (n)
>>    (do ((i 0 (1+ i)))
>>        ((= i n))
>>        (gfun 0)))
>>
>> My own ISLISP (OpenLisp) implementation gives me for 10M iterations:
>>
>> (test-fun 10000000) -> 0.038s
>> (test-gfun 10000000) -> 1.387s
>>
>> So, generic function call is about 40 times slower that standard call.
>> Beside my (perhaps) poor implementation, I would like to know what is the
>> ratio for other implementations (compiled code).
>
> 100 000 000 calls LWW4.4.6:
>
> gfun:
> user time    =      6.218
> system time  =      0.000
> Elapsed time =   0:00:06
> Allocation   = 5632 bytes standard / 3696 bytes conses
> 0 Page faults
>
> fun:
> user time    =      1.862
> system time  =      0.000
> Elapsed time =   0:00:02
> Allocation   = 2880 bytes standard / 3630 bytes conses
> 0 Page faults
>
> So a 3.33 ratio.
> 62 ns and 18.6 ns !
>
> Marc
>
> 
From: Marc Battyani
Subject: Re: Performance of generic functions
Date: 
Message-ID: <4pednRByyeo5_lbeRVnyrQ@giganews.com>
"Christian Jullien" <········@hotmail.com> wrote
> Thanks for the reply,
>
> I'm not sure my test is good because (gfun 0) can be detected by good
> compilers as a call to gfun with integer.
> In that case, it avoids the tampoline call to generic function and it can
> make a direct call to the specific integer method.
>
> Could you please change the test to
>
> (setq nb 0)
>
> (defun test-gfun (n)
>     (do ((i 0 (1+ i)))
>         ((= i n))
>         (gfun nb)))

Same timings for LWW. It was not optimized out.

if I use (declare (optimize (speed 3)(debug 0)(safety 0))) the timings are:
1.422 s and 5.908 s

Marc
From: Ivan Boldyrev
Subject: Re: Performance of generic functions
Date: 
Message-ID: <fpmu93-7dm.ln1@ibhome.cgitftp.uiggm.nsc.ru>
On 9356 day of my life Christian Jullien wrote:
> I'm not sure my test is good because (gfun 0) can be detected by good 
> compilers as a call to gfun with integer.
> In that case, it avoids the tampoline call to generic function and it can 
> make a direct call to the specific integer method.

It can't avoid trampoline call because set of methods of generic
function can be changed at runtime.  What if (GFUN INTEGER) is
removed?  Will compiler recompile everything?

-- 
Ivan Boldyrev

                                              I live in interesting time.
From: Christophe Rhodes
Subject: Re: Performance of generic functions
Date: 
Message-ID: <sqzmlw34gj.fsf@cam.ac.uk>
"Christian Jullien" <········@hotmail.com> writes:

> I wonder what is considered as a "reasonnable" cost of a generic function 
> V.S standard call

You should probably read 
  Gregor J. Kiczales and Luis H. Rodriguez Jr., _Efficient Method
  Dispatch in PCL_, 1990
and you might also be interested in
  <http://www.sbcl.org/sbcl-internals/Discriminating-Functions.html#Discriminating-Functions>

The executive summary: a generic function call with only one
applicable method should probably have an amortized cost of no more
than a small constant factor: for the sake of argument, plucking a
number out of the air, a factor of 5.

Christophe
From: Kenny Tilton
Subject: Re: Performance of generic functions
Date: 
Message-ID: <vBMyf.2273$cj3.1516@news-wrt-01.rdc-nyc.rr.com>
Christophe Rhodes wrote:
> "Christian Jullien" <········@hotmail.com> writes:
> 
> 
>>I wonder what is considered as a "reasonnable" cost of a generic function 
>>V.S standard call
> 
> 
> You should probably read 
>   Gregor J. Kiczales and Luis H. Rodriguez Jr., _Efficient Method
>   Dispatch in PCL_, 1990
> and you might also be interested in
>   <http://www.sbcl.org/sbcl-internals/Discriminating-Functions.html#Discriminating-Functions>
> 
> The executive summary: a generic function call with only one
> applicable method should probably have an amortized cost of no more
> than a small constant factor: for the sake of argument, plucking a
> number out of the air, a factor of 5.

Nice pluck. ACL7, 328ms vs 78, or a factor of 4.2

kt
From: Pieter Breed
Subject: Re: Performance of generic functions
Date: 
Message-ID: <1137436478.042112.105730@f14g2000cwb.googlegroups.com>
I realise that this is mostly about back-of-envelope stats, but with
reference to this
(http://www.zedshaw.com/blog/programming/programmer_stats.html) what
are the standard deviations, the required certainty &c?
From: Christophe Rhodes
Subject: Re: Performance of generic functions
Date: 
Message-ID: <sqlkxg0x69.fsf@cam.ac.uk>
"Pieter Breed" <············@gmail.com> writes:

> I realise that this is mostly about back-of-envelope stats, but with
> reference to this
> (http://www.zedshaw.com/blog/programming/programmer_stats.html) what
> are the standard deviations, the required certainty &c?

When I say "a small constant factor ... of 5", what could I mean?
Well, given that I didn't say "... of 5.0", it would be fair (assuming
you know that I'm a better statistician than I am a programmer ;-) to
infer that 5 is accurate to no more than one significant figure; that
is, my back of the envelope number is claiming no more than a standard
deviation of about 0.3.  (You want about 30% of measurements to be
outside one standard deviation, don't forget!)

But since I explicitly also said in that elision "plucking a number
out of the air", you probably don't want to attach much certainty at
all even to the first significant figure, and indeed since we're
talking a factor which involves scaling, you probably don't want to
apply a Laplacian uniform prior but a Jeffreys prior (or a
modification, to take account of the fact that that the expected range
isn't 0->\infty but 1->\infty).

As for those actually taking measurements, I'd actually be far more
worried that they're not measuring what they think they're measuring
or would like to measure; some of their measurements may be setup
costs to populate tables; others might be peculiar optimizations for
special cases of method dispatch.  After these systematic errors are
removed or accounted for, it becomes fairly straightforward to infer
(even from one measurement!) properties of the system.

Christophe

(I've been reading Edwin Jaynes recently.  Can anyone tell?  The stuff
at that weblog is OK as far as it goes, but I think Programmers, And
Everyone Else, Need To Learn Probability Theory, Not Just Statistics)
From: Pascal Costanza
Subject: Re: Performance of generic functions
Date: 
Message-ID: <4316ieF1klmfiU1@individual.net>
Christian Jullien wrote:
> Hi all,
> 
> I wonder what is considered as a "reasonnable" cost of a generic function 
> V.S standard call
> 
> Given the following code (please adapt for CLtL):
> 
> (defgeneric gfun (a))
> (defmethod  gfun ((a <integer>)) a)
> 
> (defun fun (a) a)
> 
> (defun test-fun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (fun 0)))
> 
> (defun test-gfun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (gfun 0)))
> 
> My own ISLISP (OpenLisp) implementation gives me for 10M iterations:
> 
> (test-fun 10000000) -> 0.038s
> (test-gfun 10000000) -> 1.387s
> 
> So, generic function call is about 40 times slower that standard call.
> Beside my (perhaps) poor implementation, I would like to know what is the 
> ratio for other implementations (compiled code).

40 times slower is not reasonable, especially not for such a simple case.

A few remarks:

- You are not comparing exactly the same thing. If you define a method 
that is specialized on some type, then this implicitly creates a type 
test. So your (defmethod gfun ((a <integer>)) a) corresponds more 
correctly to the following function definition:

(defun (a)
   (unless (typep a '<integer>)
     (error "This function is not applicable."))
   a)


- Generic functions in CLOS are implemented in a "curried" fashion: 
Whenever a new method is defined on a generic function, 
compute-discriminating-function is called to determine the actual 
function to be called when the generic function is invoked (see the CLOS 
MOP specification). The (actual) discriminating function takes all 
parameters and performs method selection, combination and invocation. 
This approach has the advantage that it can introduce a number of 
special cases:

+ If all the methods that are defined on a generic function are not 
specialized on any argument, then the discriminating function is simply 
just a precomputed combination of all those methods, because those 
methods can just be blindly invoked. No type tests are necessary.

+ If there is only one method (or only few methods) with specializers, 
then the discriminating function can be a simple if/cond form that 
selects precomputed combinations of the appropriate methods, or raises 
an error if none of the methods are applicable.

+ Only if there are a lot of methods, the selection of methods is better 
performed through a special-purpose hash table. See 
http://www2.parc.com/csl/groups/sda/publications/papers/Kiczales-Andreas-PCL/ 
for more details on one specific approach.

It's also worthwhile to note that method combination (that is, 
determining the effective method that is composed from all the 
applicable methods for a given set of arguments) is explicitly specified 
to be a macro expansion process. See the documentation for 
define-method-combination in ANSI Common Lisp. This means that the code 
that is effectively executed when a generic function is called is 
(theoretically) at least as good as comparable hand-written code.

Some time ago I have made some measurements, and it was indeed the case 
that generic functions with just one primary unspecialized method had 
the same performance numbers as regular functions across several Common 
Lisp implementations.


Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: André Thieme
Subject: Re: Performance of generic functions
Date: 
Message-ID: <1137406745.779485.230290@g43g2000cwa.googlegroups.com>
Pascal Costanza schrieb:

> Some time ago I have made some measurements, and it was indeed the case
> that generic functions with just one primary unspecialized method had
> the same performance numbers as regular functions across several Common
> Lisp implementations.

How did you do these measurements?
Motivated by Marcs comparison (62 ns and 18.6 ns) I wanted to find out
how much time it takes to call a function. But (get-internal-real-time)
is by far not accurate enough.
Under SBCL: internal-time-units-per-second => 1000
Is there a way to measure differences of millionths of seconds or
possibly even more accurate?

In Python I could do this:
import time

def foo():
  t = time.time()
  return time.time()-t

foo()  =>  4.0531158447265625e-06


André
--
From: Pascal Costanza
Subject: Re: Performance of generic functions
Date: 
Message-ID: <431bjsF1lk4heU1@individual.net>
Andr� Thieme wrote:
> Pascal Costanza schrieb:
> 
> 
>>Some time ago I have made some measurements, and it was indeed the case
>>that generic functions with just one primary unspecialized method had
>>the same performance numbers as regular functions across several Common
>>Lisp implementations.
> 
> 
> How did you do these measurements?
> Motivated by Marcs comparison (62 ns and 18.6 ns) I wanted to find out
> how much time it takes to call a function. But (get-internal-real-time)
> is by far not accurate enough.
> Under SBCL: internal-time-units-per-second => 1000
> Is there a way to measure differences of millionths of seconds or
> possibly even more accurate?

I don't know the details anymore. It was probably not very accurate - 
although the numbers were indeed exactly the same.

I was experimenting with my own implementation of generic functions, and 
wanted to see in what ballpark I was. I wasn't even near the performance 
of regular functions, while the CLOS implementations were almost the 
same. That was all the information I needed at that time, so I didn't 
delve more deeply into it...


Cheers,
Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: André Thieme
Subject: Re: Performance of generic functions
Date: 
Message-ID: <1137408842.125025.250390@g49g2000cwa.googlegroups.com>
Is there really no timer that is more accurate than
(get-internal-real-time)?

I already tried to do
CL-USER> (defmacro voodoo (n)
	   (let ((calls nil))
	   (dotimes (i n)
	     (push `(identity ,i) calls))
	   (setf calls (reverse calls))
	   `(defun foo ()
	     (time (progn ,@calls)))))

But it seems SBCL 0.9.8 can't compile a function with more than 50,000
LOC on my machine, at least not the way I tried it.
To get accurate timing I wanted to do: (voodoo (expt 10 8))
The strategy that gets used at the moment includes measuring something
else too, like the do loop.


André
--
From: Christian Jullien
Subject: Re: Performance of generic functions
Date: 
Message-ID: <43cb6ace$0$21306$8fcfb975@news.wanadoo.fr>
Thanks, you give me many ways to improve GF calls.
My current implementation makes only the general case with hash tables.

Christian

"Pascal Costanza" <··@p-cos.net> wrote in message 
····················@individual.net...
> Christian Jullien wrote:
>> Hi all,
>>
>> I wonder what is considered as a "reasonnable" cost of a generic function 
>> V.S standard call
>>
>> Given the following code (please adapt for CLtL):
>>
>> (defgeneric gfun (a))
>> (defmethod  gfun ((a <integer>)) a)
>>
>> (defun fun (a) a)
>>
>> (defun test-fun (n)
>>    (do ((i 0 (1+ i)))
>>        ((= i n))
>>        (fun 0)))
>>
>> (defun test-gfun (n)
>>    (do ((i 0 (1+ i)))
>>        ((= i n))
>>        (gfun 0)))
>>
>> My own ISLISP (OpenLisp) implementation gives me for 10M iterations:
>>
>> (test-fun 10000000) -> 0.038s
>> (test-gfun 10000000) -> 1.387s
>>
>> So, generic function call is about 40 times slower that standard call.
>> Beside my (perhaps) poor implementation, I would like to know what is the 
>> ratio for other implementations (compiled code).
>
> 40 times slower is not reasonable, especially not for such a simple case.
>
> A few remarks:
>
> - You are not comparing exactly the same thing. If you define a method 
> that is specialized on some type, then this implicitly creates a type 
> test. So your (defmethod gfun ((a <integer>)) a) corresponds more 
> correctly to the following function definition:
>
> (defun (a)
>   (unless (typep a '<integer>)
>     (error "This function is not applicable."))
>   a)
>
>
> - Generic functions in CLOS are implemented in a "curried" fashion: 
> Whenever a new method is defined on a generic function, 
> compute-discriminating-function is called to determine the actual function 
> to be called when the generic function is invoked (see the CLOS MOP 
> specification). The (actual) discriminating function takes all parameters 
> and performs method selection, combination and invocation. This approach 
> has the advantage that it can introduce a number of special cases:
>
> + If all the methods that are defined on a generic function are not 
> specialized on any argument, then the discriminating function is simply 
> just a precomputed combination of all those methods, because those methods 
> can just be blindly invoked. No type tests are necessary.
>
> + If there is only one method (or only few methods) with specializers, 
> then the discriminating function can be a simple if/cond form that selects 
> precomputed combinations of the appropriate methods, or raises an error if 
> none of the methods are applicable.
>
> + Only if there are a lot of methods, the selection of methods is better 
> performed through a special-purpose hash table. See 
> http://www2.parc.com/csl/groups/sda/publications/papers/Kiczales-Andreas-PCL/ 
> for more details on one specific approach.
>
> It's also worthwhile to note that method combination (that is, determining 
> the effective method that is composed from all the applicable methods for 
> a given set of arguments) is explicitly specified to be a macro expansion 
> process. See the documentation for define-method-combination in ANSI 
> Common Lisp. This means that the code that is effectively executed when a 
> generic function is called is (theoretically) at least as good as 
> comparable hand-written code.
>
> Some time ago I have made some measurements, and it was indeed the case 
> that generic functions with just one primary unspecialized method had the 
> same performance numbers as regular functions across several Common Lisp 
> implementations.
>
>
> Pascal
>
> -- 
> My website: http://p-cos.net
> Closer to MOP & ContextL:
> http://common-lisp.net/project/closer/ 
From: Pascal Costanza
Subject: Re: Performance of generic functions
Date: 
Message-ID: <432o6iF1l6loqU1@individual.net>
Christian Jullien wrote:
> Thanks, you give me many ways to improve GF calls.

If that's your goal, you may definitely want to look at 
http://people.csail.mit.edu/jrb/Projects/partial-dispatch.htm - it also 
includes a pretty good overview of related work.


Cheers,
Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Didier Verna
Subject: Re: Performance of generic functions
Date: 
Message-ID: <mux64nqw52t.fsf@uzeb.lrde.epita.fr>
Pascal Costanza <··@p-cos.net> wrote:

> - Generic functions in CLOS are implemented in a "curried" fashion: Whenever
> a new method is defined on a generic function,
> compute-discriminating-function is called to determine the actual function
> to be called when the generic function is invoked (see the CLOS MOP
> specification). The (actual) discriminating function takes all parameters
> and performs method selection, combination and invocation. This approach has
> the advantage that it can introduce a number of special cases:

> + If there is only one method (or only few methods) with specializers, then
> the discriminating function can be a simple if/cond form that selects
> precomputed combinations of the appropriate methods, or raises an error if
> none of the methods are applicable.

        In such a case, is there a precise specification of the order of the
type tests (like, the order in which specializations were defined), or is it
left to the implementation ?

-- 
Didier Verna, ······@lrde.epita.fr, http://www.lrde.epita.fr/~didier

EPITA / LRDE, 14-16 rue Voltaire   Tel.+33 (1) 44 08 01 85
94276 Le Kremlin-Bic�tre, France   Fax.+33 (1) 53 14 59 22   ······@xemacs.org
From: Pascal Costanza
Subject: Re: Performance of generic functions
Date: 
Message-ID: <44tv8cF3up1bU1@individual.net>
Didier Verna wrote:
> Pascal Costanza <··@p-cos.net> wrote:
> 
>>- Generic functions in CLOS are implemented in a "curried" fashion: Whenever
>>a new method is defined on a generic function,
>>compute-discriminating-function is called to determine the actual function
>>to be called when the generic function is invoked (see the CLOS MOP
>>specification). The (actual) discriminating function takes all parameters
>>and performs method selection, combination and invocation. This approach has
>>the advantage that it can introduce a number of special cases:
> 
>>+ If there is only one method (or only few methods) with specializers, then
>>the discriminating function can be a simple if/cond form that selects
>>precomputed combinations of the appropriate methods, or raises an error if
>>none of the methods are applicable.
> 
>         In such a case, is there a precise specification of the order of the
> type tests (like, the order in which specializations were defined), or is it
> left to the implementation ?

It's an optimization, and with all optimizations, the behavior should be 
the same as in the non-optimized case. CLOS specifies what checks are 
performed when, and what kinds of errors are signalled in erroneous 
situations. If the optimized version doesn't adhere to this 
specification, it's a bug.

Does this answer your question?


Cheers,
Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Didier Verna
Subject: Re: Performance of generic functions
Date: 
Message-ID: <muxwtg6ufr2.fsf@uzeb.lrde.epita.fr>
Pascal Costanza <··@p-cos.net> wrote:

>>         In such a case, is there a precise specification of the order of
>> the type tests (like, the order in which specializations were defined), or
>> is it left to the implementation ?
>
> It's an optimization, and with all optimizations, the behavior should be the
> same as in the non-optimized case. CLOS specifies what checks are performed
> when, and what kinds of errors are signalled in erroneous situations. If the
> optimized version doesn't adhere to this specification, it's a bug.
>
> Does this answer your question?

Not really, but if you tell me to go read the f*ing CLOS spec, I'll accept the
answer ... :-)


What I meant was this. Consider the following code:

(defun fn (a)
  (etypecase a
    (single-float a)
    (fixnum a)))

(defgeneric gn (a))
(defmethod gn ((a fixnum)) a)
(defmethod gn ((a single-float)) a)


If I want to compare the performances of standard vs. generic funcalls, I'd be
interested to know in what order the types are checked in the generic
function, assuming the implementation goes indeed through an if/cond test (and
not a hashtable or what not). Indeed, benches could be biased by the fact that
the type tests in the generic call would be in the opposite order from the
standard function.

In other words, my question was: is this order deterministic, and what is it.

-- 
Didier Verna, ······@lrde.epita.fr, http://www.lrde.epita.fr/~didier

EPITA / LRDE, 14-16 rue Voltaire   Tel.+33 (1) 44 08 01 85
94276 Le Kremlin-Bic�tre, France   Fax.+33 (1) 53 14 59 22   ······@xemacs.org
From: Pascal Costanza
Subject: Re: Performance of generic functions
Date: 
Message-ID: <44ujm2F433qtU1@individual.net>
Didier Verna wrote:
> Pascal Costanza <··@p-cos.net> wrote:
> 
>>>        In such a case, is there a precise specification of the order of
>>>the type tests (like, the order in which specializations were defined), or
>>>is it left to the implementation ?
>>
>>It's an optimization, and with all optimizations, the behavior should be the
>>same as in the non-optimized case. CLOS specifies what checks are performed
>>when, and what kinds of errors are signalled in erroneous situations. If the
>>optimized version doesn't adhere to this specification, it's a bug.
>>
>>Does this answer your question?
> 
> Not really, but if you tell me to go read the f*ing CLOS spec, I'll accept the
> answer ... :-)
> 
> What I meant was this. Consider the following code:
> 
> (defun fn (a)
>   (etypecase a
>     (single-float a)
>     (fixnum a)))
> 
> (defgeneric gn (a))
> (defmethod gn ((a fixnum)) a)
> (defmethod gn ((a single-float)) a)
> 
> 
> If I want to compare the performances of standard vs. generic funcalls, I'd be
> interested to know in what order the types are checked in the generic
> function, assuming the implementation goes indeed through an if/cond test (and
> not a hashtable or what not). Indeed, benches could be biased by the fact that
> the type tests in the generic call would be in the opposite order from the
> standard function.
> 
> In other words, my question was: is this order deterministic, and what is it.

No, the order is not deterministic. ANSI Common Lisp specifies the 
following error situations:

- The number of arguments don't match and/or keyword arguments don't 
match. This is specified in 3.5 of the HyperSpec.

- For the standard method combination, if there is no primary method, an 
error will be signalled. See 7.6.6.2. It's indeed not specified in what 
way methods are selected and how the type checks are performed. In the 
general case, effective methods which have been cached in previous calls 
to a generic function are simply reused, so that no type check (in the 
usual sense of the term) is actually performed at all.

You can find more details in 7.6 and especially 7.6.6.


Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Ray Dillinger
Subject: Re: Performance of generic functions
Date: 
Message-ID: <43fba941$0$58045$742ec2ed@news.sonic.net>
Pascal Costanza wrote:

> No, the order is not deterministic. ANSI Common Lisp specifies the 
> following error situations:
> 
> - The number of arguments don't match and/or keyword arguments don't 
> match. This is specified in 3.5 of the HyperSpec.

This is something about CLOS generic functions I don't get; why don't
they dispatch on number of arguments??

				Bear
From: Thomas A. Russ
Subject: Re: Performance of generic functions
Date: 
Message-ID: <ymiwtfnb7xc.fsf@sevak.isi.edu>
Ray Dillinger <····@sonic.net> writes:

> This is something about CLOS generic functions I don't get; why don't
> they dispatch on number of arguments??

(defmethod M1 ((x INTEGER) &optional y)
  (print "First"))

(defmethod M1 ((x INTEGER) y)
  (print "Second"))

(defmethod M1 ((x INTEGER) &rest z)
  (print "Third"))

(defmethod M1 ((x INTEGER) y &key a b c)
  (print "Fourth"))

(M1 5 10)  ==>  ???

-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Ray Dillinger
Subject: Re: Performance of generic functions
Date: 
Message-ID: <43fd0ab7$0$58050$742ec2ed@news.sonic.net>
Thomas A. Russ wrote:
> Ray Dillinger <····@sonic.net> writes:
> 
> 
>>This is something about CLOS generic functions I don't get; why don't
>>they dispatch on number of arguments??
> 
> 
> (defmethod M1 ((x INTEGER) &optional y)
>   (print "First"))
==> T

> (defmethod M1 ((x INTEGER) y)
>   (print "Second"))
==> Error: Multimethod arglist conflicts with extant multimethod arglist.


> (defmethod M1 ((x INTEGER) &rest z)
>   (print "Third"))
==> Error: Multimethod arglist conflicts with extant multimethod arglist.

> (defmethod M1 ((x INTEGER) y &key a b c)
>   (print "Fourth"))
==> Error: Multimethod arglist conflicts with extant multimethod arglist.

> (M1 5 10)  
==> First
From: Coby Beck
Subject: Re: Performance of generic functions
Date: 
Message-ID: <lVbLf.88$dg.3@clgrps13>
"Ray Dillinger" <····@sonic.net> wrote in message 
······························@news.sonic.net...
> Thomas A. Russ wrote:
>> Ray Dillinger <····@sonic.net> writes:
>>
>>
>>>This is something about CLOS generic functions I don't get; why don't
>>>they dispatch on number of arguments??
>>
>>
>> (defmethod M1 ((x INTEGER) &optional y)
>>   (print "First"))
> ==> T
>
>> (defmethod M1 ((x INTEGER) y)
>>   (print "Second"))
> ==> Error: Multimethod arglist conflicts with extant multimethod arglist.
>
>
>> (defmethod M1 ((x INTEGER) &rest z)
>>   (print "Third"))
> ==> Error: Multimethod arglist conflicts with extant multimethod arglist.
>
>> (defmethod M1 ((x INTEGER) y &key a b c)
>>   (print "Fourth"))
> ==> Error: Multimethod arglist conflicts with extant multimethod arglist.
>
>> (M1 5 10)
> ==> First

I'm sure Thomas knows you can't do what he presented, it was to make the 
point.  You were asking why not dispatch on number of args.  This would 
presumably allow the selection of definitions above.  Now, if they were all 
defined as presented, then which M1 would be called for (M1 5 10)?

-- 
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
From: Ray Dillinger
Subject: Re: Performance of generic functions
Date: 
Message-ID: <43fd4f89$0$58064$742ec2ed@news.sonic.net>
Coby Beck wrote:

> 
> I'm sure Thomas knows you can't do what he presented, it was to make the 
> point.  You were asking why not dispatch on number of args.  This would 
> presumably allow the selection of definitions above.  Now, if they were all 
> defined as presented, then which M1 would be called for (M1 5 10)?
> 

"presumably allow" isn't presumable here.  When you can't construct a
method discriminator, it is perfectly reasonable to disallow the
addition of the method that creates the conflict.  With these semantics,
they cannot all be defined as presented.

Alternatively, you could use a more permissive semantics and define
the correct behavior as calling *any* matching method - which would
select one of the methods presented and call it, possibly a different
method on different calls, runs, versions, or implementations.  In
that case, conflicting definitions becomes bad taste and may generate
a warning, but needn't be disallowed.

Either would be more useful than generic methods being fixed in arity.

					Bear
From: Thomas A. Russ
Subject: Re: Performance of generic functions
Date: 
Message-ID: <ymilkw1c0y8.fsf@sevak.isi.edu>
Ray Dillinger <····@sonic.net> writes:

> Coby Beck wrote:
> 
> > I'm sure Thomas knows you can't do what he presented, it was to make
> > the point.  You were asking why not dispatch on number of args.  This
> > would presumably allow the selection of definitions above.  Now, if
> > they were all defined as presented, then which M1 would be called for
> > (M1 5 10)?
> 
> "presumably allow" isn't presumable here.  When you can't construct a
> method discriminator, it is perfectly reasonable to disallow the
> addition of the method that creates the conflict.  With these semantics,
> they cannot all be defined as presented.

Well, that really makes things unpredictable, especially when you
consider having independent parties defining new methods for generic
functions.  Who is to say which one creates a conflict?  And, in
general, since these wouldn't be compiled at the same time, you would
have to make this a run or load time error.  You really wouldn't be able
to tell for certain before then.

So that transforms a rule that is easily understood into one that
requires you to know about every other method defined for that generic
function.  I think that would violate my sense of elegance of design.

It also violates part of the idea of what a generic function is supposed
to represent, namely a function which is generically applicable and
which then determines its method based on its arguments.  Having
different numbers of arguments means that it isn't really the same
function, in that you are now considering additional information.

> Alternatively, you could use a more permissive semantics and define
> the correct behavior as calling *any* matching method - which would
> select one of the methods presented and call it, possibly a different
> method on different calls, runs, versions, or implementations.  In
> that case, conflicting definitions becomes bad taste and may generate
> a warning, but needn't be disallowed.

Wow!  Talk about unpredictable.  I like to be able to predict what my
software will do when I write it.  You are correct in noting that this
will discourage people from trying to exploit this.  But now we are
introducin a mechanism to discourage people from using a feature that
you claim you want in the language.  This is madness.

> Either would be more useful than generic methods being fixed in arity.

I disagree (obviously).

Initially, I was also disturbed by the congruence requirements of
generic functions, but with time and thought have come to appreciate the
value of it.  This is from practical, philosophical and aesthetic points
of view.  

It means you always know how to call a generic function without having
to know the precise particulars of its argument types.  After all, one
of the points of using a generic function rather than an ordinary
function is that you get (slightly) different behaviors based on the
argument types -- but you aren't required to know this when you call it,
because the generic dispatch handles that for you.

If you have to know the precise type of the arguments, so that you can
decide how to formulate the generic function, I don't see why you are
bothering with a generic function in the first place.  You want a
somewhat different, specialized function.  So why not use an ordinary
non-generic function in its place?  It isn't performing the same
operation, since there are a lot more arguments to whatever is being
done.  That is also why is suggested that methods should really only do
slightly different things from one another.  If you want to do greatly
different things, then you really want a different function.  A single
function should perform a particular operation, and the different
methods on a generic function really should implement semantically
similar operations, as appropriate to their arguments.

From a purist and philosophical point of view, that argues for exactly
the same argument list in each method.  Common Lisp is a bit more
flexible than that.  Lambda-list congruence applies only to the required
arguments.  If you want to have other arguments, they can be optional,
rest or keyword arguments.

Perhaps the closest to what you seem to want are the keyword arguments,
which can be different for each method.  Now that possibility has to be
planned for in advance(*) when you write the generic function signature
and the other methods.  Then you can have things like

(defmethod m ((a INTEGER) &key b c &allow-other-keys) ...)
(defmethod m ((a SYMBOL)  &key x y &allow-other-keys) ...)
(defmethod m ((a CLASS-C) &key c y j &allow-other-keys) ...)

And then you can actually call things like

  (m 3 :b 8 :c 'fred)
  (m 'george :y 10)

But you also still have some functionality by not having to know exactly
which of the keywords are there:

  (defun call-it (arg)
     (m arg :a 2 :b 8 :x 10 :y 'foo :j "Help!"))

But I think that this is getting a bit out of hand, as far as figuring
out what the semantics of M is supposed to be.  I favor a cleaner
semantics.


(*) OK, you don't really have to plan it in advance, since you can
either be really careful about which method gets invoked (raising again
the question of why bother with methods if you have to already know
exactly which one is used), or you use :allow-other-keys t in the method
call.  But I would call that bad design.


-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Ray Dillinger
Subject: Re: Performance of generic functions
Date: 
Message-ID: <43ff7730$0$58098$742ec2ed@news.sonic.net>
Thomas A. Russ wrote:
> Ray Dillinger <····@sonic.net> writes:
> 
> 
>>Coby Beck wrote:
>>
>>
>>>I'm sure Thomas knows you can't do what he presented, it was to make
>>>the point.  You were asking why not dispatch on number of args.  This
>>>would presumably allow the selection of definitions above.  Now, if
>>>they were all defined as presented, then which M1 would be called for
>>>(M1 5 10)?
>>
>>"presumably allow" isn't presumable here.  When you can't construct a
>>method discriminator, it is perfectly reasonable to disallow the
>>addition of the method that creates the conflict.  With these semantics,
>>they cannot all be defined as presented.
> 
> 
> Well, that really makes things unpredictable, especially when you
> consider having independent parties defining new methods for generic
> functions.  Who is to say which one creates a conflict?  And, in
> general, since these wouldn't be compiled at the same time, you would
> have to make this a run or load time error.  You really wouldn't be able
> to tell for certain before then.

 > So that transforms a rule that is easily understood into one that
 > requires you to know about every other method defined for that generic
 > function.  I think that would violate my sense of elegance of design.
 >

And this is different from the current design how?  I mean seriously,
Who says now which definition creates a conflict?  If two different
people specialize the same generic function for integer first
arguments, they're gonna conflict, even at the same arity.  Arguing
against dispatch-on-arity on this ground without also arguing against
the status quo seems silly, doesn't it?


> Initially, I was also disturbed by the congruence requirements of
> generic functions, but with time and thought have come to appreciate the
> value of it.  This is from practical, philosophical and aesthetic points
> of view.  
> 
> It means you always know how to call a generic function without having
> to know the precise particulars of its argument types.  After all, one
> of the points of using a generic function rather than an ordinary
> function is that you get (slightly) different behaviors based on the
> argument types -- but you aren't required to know this when you call it,
> because the generic dispatch handles that for you.

I have grown wary of all arguments of the form "It's good to follow
this rule so we'll disallow all other ways of doing it..."  Surely
if it's good to follow a rule you can go on following it even if
other options are available.  I assume coders are smart and have good
taste.  If something doesn't work with their design, they should know
it and not use it.  And I'm chary of our ability to guess in advance
that there is only one way of doing things that must perforce work with
all designs.

> If you have to know the precise type of the arguments, so that you can
> decide how to formulate the generic function, I don't see why you are
> bothering with a generic function in the first place.  You want a
> somewhat different, specialized function.  So why not use an ordinary
> non-generic function in its place?  It isn't performing the same
> operation, since there are a lot more arguments to whatever is being
> done.  

I disagree.  Consider print methods that take an variable-size
list of things to print or have the output port as an optional
argument.  Consider eval methods with a lexical environment as
an optional argument.  Consider HTML formatters that take varying
numbers of HTML fields and options via keyword args...  All of
these families are "the same function" conceptually but have
different arities.

> Perhaps the closest to what you seem to want are the keyword arguments,
> which can be different for each method.  Now that possibility has to be
> planned for in advance(*) when you write the generic function signature
> and the other methods.  Then you can have things like
> 
> (defmethod m ((a INTEGER) &key b c &allow-other-keys) ...)
> (defmethod m ((a SYMBOL)  &key x y &allow-other-keys) ...)
> (defmethod m ((a CLASS-C) &key c y j &allow-other-keys) ...)

Ah!  Okay, yes, that definitely goes a long way toward answering
the need.  'Scuse my ignorance, I didn't realize there was a way
to make specialized keyword arguments with lambda-lists that didn't
conflict.

				Bear
From: Thomas A. Russ
Subject: Re: Performance of generic functions
Date: 
Message-ID: <ymioe0sabnc.fsf@sevak.isi.edu>
Ray Dillinger <····@sonic.net> writes:

> Thomas A. Russ wrote:
> > Ray Dillinger <····@sonic.net> writes:
> >
> 
> >>Coby Beck wrote:
> >>
> >>
> >>>I'm sure Thomas knows you can't do what he presented, it was to make
> >>>the point.  You were asking why not dispatch on number of args.  This
> >>>would presumably allow the selection of definitions above.  Now, if
> >>>they were all defined as presented, then which M1 would be called for
> >>>(M1 5 10)?
> >>
> >>"presumably allow" isn't presumable here.  When you can't construct a
> >>method discriminator, it is perfectly reasonable to disallow the
> >>addition of the method that creates the conflict.  With these semantics,
> >>they cannot all be defined as presented.
> > Well, that really makes things unpredictable, especially when you
> 
> > consider having independent parties defining new methods for generic
> > functions.  Who is to say which one creates a conflict?  And, in
> > general, since these wouldn't be compiled at the same time, you would
> > have to make this a run or load time error.  You really wouldn't be able
> > to tell for certain before then.
> 
>  > So that transforms a rule that is easily understood into one that
>  > requires you to know about every other method defined for that generic
>  > function.  I think that would violate my sense of elegance of design.
>  >
> 
> And this is different from the current design how?  I mean seriously,
> Who says now which definition creates a conflict?  If two different
> people specialize the same generic function for integer first
> arguments, they're gonna conflict, even at the same arity.  Arguing
> against dispatch-on-arity on this ground without also arguing against
> the status quo seems silly, doesn't it?

This argument is silly.  If you redefine EXACTLY the same function, then
of course you get a conflict, regardless of whether the function is
generic or not.  Now, for generic functions, the definition of what it
means to be EXACTLY the same is relaxed just a bit:  A GF is exactly the
same if all of the required arguments are the same and have the same
types.

Conflicts caused by redefining the same function are unavoidable.  But
making the definition of exactly the same more complicated does not, in
my opinion improve the software engineering environment.

> > Initially, I was also disturbed by the congruence requirements of
> > generic functions, but with time and thought have come to appreciate the
> > value of it.  This is from practical, philosophical and aesthetic points
> > of view.  It means you always know how to call a generic function
> > without having
> 
> > to know the precise particulars of its argument types.  After all, one
> > of the points of using a generic function rather than an ordinary
> > function is that you get (slightly) different behaviors based on the
> > argument types -- but you aren't required to know this when you call it,
> > because the generic dispatch handles that for you.
> 
> I have grown wary of all arguments of the form "It's good to follow
> this rule so we'll disallow all other ways of doing it..."  Surely
> if it's good to follow a rule you can go on following it even if
> other options are available.  I assume coders are smart and have good
> taste.  If something doesn't work with their design, they should know
> it and not use it.  And I'm chary of our ability to guess in advance
> that there is only one way of doing things that must perforce work with
> all designs.
> 
> > If you have to know the precise type of the arguments, so that you can
> > decide how to formulate the generic function, I don't see why you are
> > bothering with a generic function in the first place.  You want a
> > somewhat different, specialized function.  So why not use an ordinary
> > non-generic function in its place?  It isn't performing the same
> > operation, since there are a lot more arguments to whatever is being
> > done.
> 
> 
> I disagree.  Consider print methods that take an variable-size
> list of things to print or have the output port as an optional
> argument.  Consider eval methods with a lexical environment as
> an optional argument.  Consider HTML formatters that take varying
> numbers of HTML fields and options via keyword args...  All of
> these families are "the same function" conceptually but have
> different arities.

Yes, but there is nothing to stop you from writing these functions under
the CLOS model.  And I'm sure that one could conceivably imagine trying
to use some sort of dispatch to do things with the variable-size print,
but that would seem not to really be something one would want to do.
Would you reall want to write something like:

(defmethod multi-print ((s output-stream) (arg1 integer)) ...)
(defmethod multi-print ((s output-stream) (arg1 float)) ...)

(defmethod multi-print ((s output-stream) (arg1 integer) (arg2 integer))
  ...)
(defmethod multi-print ((s output-stream) (arg1 integer) (arg2 float))
  ...)
(defmethod multi-print ((s output-stream) (arg1 integer) (arg2 string))
  ...)
etc.

Presumably what one would do in an actual design would be to have the
top-level, presumably even non-generic function multi-print take the
variable length argument list and then iterate over it calling the
generic forms that match up printing of individual argument types.

The general notion is that the &... argument list options provide ways
of specifying different argument lists to the same function.  But in all
of those cases, there is actually only a single function.  There doesn't
have to be some dispatch decision, so the different shapes of the
argument list can be resolved in a reasonable fashion, since the shape
of that arglist is known.

You get the same thing with the current CLOS method system, in that once
you have resolved the dispatch by looking at the required arguments, you
are then given the grammar for parsing the entire argument list.  But to
have to figure out the dispatch by building a discrimination test that
has optional elements to it sounds like an implementation nightmare.

So, the real question why all of those elements should go into the
dispatch decision.  IMHO it would not be possible to design an
unambiguous or even efficient dispatch method for such an open-ended
problem.  Of course, since this is lisp, anyone who feels strongly the
other way can easily write their own method dispatching system and use
that one instead.  I don't think the MOP is quite flexible enough to do
that inside CLOS, [but I'm not certain about this -- I'm not a MOP
expert].  But you could use the Macro system to build your own object
system.


> > Perhaps the closest to what you seem to want are the keyword arguments,
> > which can be different for each method.  Now that possibility has to be
> > planned for in advance(*) when you write the generic function signature
> > and the other methods.  Then you can have things like
> > (defmethod m ((a INTEGER) &key b c &allow-other-keys) ...)
> 
> > (defmethod m ((a SYMBOL)  &key x y &allow-other-keys) ...)
> > (defmethod m ((a CLASS-C) &key c y j &allow-other-keys) ...)
> 
> Ah!  Okay, yes, that definitely goes a long way toward answering
> the need.  'Scuse my ignorance, I didn't realize there was a way
> to make specialized keyword arguments with lambda-lists that didn't
> conflict.

Ah.
The benefits of having a well thought out system.

You can usually manage to achieve what you want within the constraints
of the language, because it was designed to be useful and support
a reasonable design space.


-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Pascal Costanza
Subject: Re: Performance of generic functions
Date: 
Message-ID: <465sddF9c48tU1@individual.net>
Ray Dillinger wrote:
> Coby Beck wrote:
> 
>>
>> I'm sure Thomas knows you can't do what he presented, it was to make 
>> the point.  You were asking why not dispatch on number of args.  This 
>> would presumably allow the selection of definitions above.  Now, if 
>> they were all defined as presented, then which M1 would be called for 
>> (M1 5 10)?
>>
> 
> "presumably allow" isn't presumable here.  When you can't construct a
> method discriminator, it is perfectly reasonable to disallow the
> addition of the method that creates the conflict.  With these semantics,
> they cannot all be defined as presented.
> 
> Alternatively, you could use a more permissive semantics and define
> the correct behavior as calling *any* matching method - which would
> select one of the methods presented and call it, possibly a different
> method on different calls, runs, versions, or implementations.  In
> that case, conflicting definitions becomes bad taste and may generate
> a warning, but needn't be disallowed.
> 
> Either would be more useful than generic methods being fixed in arity.

It's debatable whether fixing the semantics to be non-deterministic is a 
good idea. ;)

But anyway, your sketch is very far from a specification-quality 
description of semantics for this.

You may want to take a look at the paper "CLOS in Context: The Shape of 
the Design Space" at http://www.dreamsongs.com/CLOS.html - I don't think 
it contains a discussion of how to handle dispatch for non-required 
args, but it discusses a few other possible extensions that they weren't 
able to nail down. It may give you an idea of the complexities of 
designing such language extensions.


Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Thomas A. Russ
Subject: Re: Performance of generic functions
Date: 
Message-ID: <ymipsldc1zt.fsf@sevak.isi.edu>
Ray Dillinger <····@sonic.net> writes:

> Thomas A. Russ wrote:
> > Ray Dillinger <····@sonic.net> writes:
> >
> 
> >>This is something about CLOS generic functions I don't get; why don't
> >>they dispatch on number of arguments??
> > (defmethod M1 ((x INTEGER) &optional y)
> 
> >   (print "First"))
> ==> T
> 
> > (defmethod M1 ((x INTEGER) y)
> >   (print "Second"))
> ==> Error: Multimethod arglist conflicts with extant multimethod arglist.
> 
> 
> > (defmethod M1 ((x INTEGER) &rest z)
> >   (print "Third"))
> ==> Error: Multimethod arglist conflicts with extant multimethod arglist.
> 
> > (defmethod M1 ((x INTEGER) y &key a b c)
> >   (print "Fourth"))
> ==> Error: Multimethod arglist conflicts with extant multimethod arglist.
> 
> > (M1 5 10)
> 
> ==> First

Sorry, I guess I was a bit too cryptic in my post.

It wasn't meant to demonstrate the Common Lisp DOES this type of
dispatch, but rather to be a thought experiment showing one reason why
the designers may have decided NOT to support such dispatch.

It's clear that CL doesn't allow these methods, and the examples were
meant to be an argument for why that isn't done.

If you assume the various method definitions were legal, could you come
up with a principled answer to what the dispatcher should do?

-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Ray Dillinger
Subject: Re: Performance of generic functions
Date: 
Message-ID: <43ff78cd$0$58089$742ec2ed@news.sonic.net>
Thomas A. Russ wrote:
> Ray Dillinger <····@sonic.net> writes:
>>Thomas A. Russ wrote:
>>>Ray Dillinger <····@sonic.net> writes:

>>>>why don't[CLOS generic functions] dispatch on number of arguments??

>>>(defmethod M1 ((x INTEGER) &optional y)
>>>  (print "First"))
>>==> T
<clip>

>>>(defmethod M1 ((x INTEGER) y &key a b c)
>>>  (print "Fourth"))
>>==> Error: Multimethod arglist conflicts with extant multimethod arglist.
>>>(M1 5 10)
>>==> First

> Sorry, I guess I was a bit too cryptic in my post.
> 
> It wasn't meant to demonstrate the Common Lisp DOES this type of
> dispatch, but rather to be a thought experiment showing one reason why
> the designers may have decided NOT to support such dispatch.
> 
> It's clear that CL doesn't allow these methods, and the examples were
> meant to be an argument for why that isn't done.
> 
> If you assume the various method definitions were legal, could you come
> up with a principled answer to what the dispatcher should do?
> 

I'm sorry, I gues I was a bit too cryptic in my response.  It wasn't
meant to demonstrate that Common Lisp DOESN'T allow these method
definitions, but rather to be a reminder that conflicting definitions
ought to be handled exactly the same way they are now.

It's clear that CL doesn't allow these methods, and the responses were
meant to be an argument that the reason for not allowing them is
utterly unchanged by having methods of differing arity.

I assume the various method definitions are illegal, because they are
conflicting, and my principled answer to what the dispatcher should do
is exactly what it does in the case of conflicting definitions now.

				Bear
From: Arthur Smyles
Subject: Re: Performance of generic functions
Date: 
Message-ID: <1140894201.069212.14740@i40g2000cwc.googlegroups.com>
How about choosing the most specific match:

(M1 10) -> "First"
(M1 10 2) -> "Second"
(M1 10 2 3) -> "Third"
(M1 10 2 :a 3) -> "Fourth"
(M1 10 2 :e 3) -> "Third"

Thomas A. Russ wrote:
> Ray Dillinger <····@sonic.net> writes:
>
> > This is something about CLOS generic functions I don't get; why don't
> > they dispatch on number of arguments??
>
> (defmethod M1 ((x INTEGER) &optional y)
>   (print "First"))
>
> (defmethod M1 ((x INTEGER) y)
>   (print "Second"))
>
> (defmethod M1 ((x INTEGER) &rest z)
>   (print "Third"))
>
> (defmethod M1 ((x INTEGER) y &key a b c)
>   (print "Fourth"))
>
> (M1 5 10)  ==>  ???
> 
> -- 
> Thomas A. Russ,  USC/Information Sciences Institute
From: Thomas A. Russ
Subject: Re: Performance of generic functions
Date: 
Message-ID: <ymivev0ad1r.fsf@sevak.isi.edu>
"Arthur Smyles" <········@earthlink.net> writes:

> How about choosing the most specific match:
> 
> (M1 10) -> "First"
> (M1 10 2) -> "Second"
> (M1 10 2 3) -> "Third"
> (M1 10 2 :a 3) -> "Fourth"
> (M1 10 2 :e 3) -> "Third"

And what are the rules on specificity?

What about 

(defmethod M1 ((x INTEGER) y &key a b c)
   (print "Fourth"))

(defmethod M1 ((x INTEGER) y &key a c d)
   (print "Fifth"))

(M1 10 2 :a 3)  ==> ?


Now, it may be possible to specify some precedence relationship amongs
&optional, &rest, &key, but can it ever be completely unambiguous?

The current set of rules has the benefit that they don't have any cases
where you can't tell.  I don't see how you can avoid that with certain
combinations of arguments.  Would you say that the "Fourth" and "Fifth"
versions above are intrinsically ambiguous?  Or only if the caller
doesn't specify either :b or :c?

And what is the point of having optional arguments to a function if the
fact that they are optional may never have any impact.  Using the
original examples, the only way you could even get to the "First" M1
method when its optional argument is specified is via a
(call-next-method) in the "Second" or "Third", etc. methods.

At some point you have to acknowledge that the specification and
implementation of all of this is getting so arcane that it isn't worth
the effort.  And in the final analysis, what does one gain?  Where is
the compelling example that shows that the CL-style of method dispatch
really gets in the way of writing useful, maintainable and
understandable code?

> Thomas A. Russ wrote:
> > Ray Dillinger <····@sonic.net> writes:
> >
> > > This is something about CLOS generic functions I don't get; why don't
> > > they dispatch on number of arguments??
> >
> > (defmethod M1 ((x INTEGER) &optional y)
> >   (print "First"))
> >
> > (defmethod M1 ((x INTEGER) y)
> >   (print "Second"))
> >
> > (defmethod M1 ((x INTEGER) &rest z)
> >   (print "Third"))
> >
> > (defmethod M1 ((x INTEGER) y &key a b c)
> >   (print "Fourth"))
> >
> > (M1 5 10)  ==>  ???
> > 
> > -- 
> > Thomas A. Russ,  USC/Information Sciences Institute
> 

-- 
Thomas A. Russ,  USC/Information Sciences Institute
From: Arthur Smyles
Subject: Re: Performance of generic functions
Date: 
Message-ID: <1141096871.992560.325210@j33g2000cwa.googlegroups.com>
Thomas A. Russ wrote:
> "Arthur Smyles" <········@earthlink.net> writes:
>
> > How about choosing the most specific match:
> >
> > (M1 10) -> "First"
> > (M1 10 2) -> "Second"
> > (M1 10 2 3) -> "Third"
> > (M1 10 2 :a 3) -> "Fourth"
> > (M1 10 2 :e 3) -> "Third"
>
> And what are the rules on specificity?

For number of arguments in general:
required > optional
opional > key
key > rest

>
> What about
>
> (defmethod M1 ((x INTEGER) y &key a b c)
>    (print "Fourth"))
>
> (defmethod M1 ((x INTEGER) y &key a c d)
>    (print "Fifth"))
>
> (M1 10 2 :a 3)  ==> ?
>
>
> Now, it may be possible to specify some precedence relationship amongs
> &optional, &rest, &key, but can it ever be completely unambiguous?

Yes. If you gave the situation above, one way to solve the problem is
to throw an error (at declaration time) and ask the user to decide
which method should take precedence. It would be like adding a new
method without b or d which could be considered more specific. Then
when the generic method is called it will always call the right one in
case of a tie.

> The current set of rules has the benefit that they don't have any cases
> where you can't tell.  I don't see how you can avoid that with certain
> combinations of arguments.  Would you say that the "Fourth" and "Fifth"
> versions above are intrinsically ambiguous?  Or only if the caller
> doesn't specify either :b or :c?
>
> And what is the point of having optional arguments to a function if the
> fact that they are optional may never have any impact. Using the
> original examples, the only way you could even get to the "First" M1
> method when its optional argument is specified is via a
> (call-next-method) in the "Second" or "Third", etc. methods.

Lisp is a dynamic language. Maybe one person wrote that as the original
function. And  then another person created a new method latter to
override the default behavior. I don't see a problem if logically, a
piece of functionality never gets used in the program.

> At some point you have to acknowledge that the specification and
> implementation of all of this is getting so arcane that it isn't worth
> the effort.  And in the final analysis, what does one gain?  Where is
> the compelling example that shows that the CL-style of method dispatch
> really gets in the way of writing useful, maintainable and
> understandable code?

Ultimately, what I would like to see is the application of a function
being completely determined by pattern matching, whether it is in the
number of arguments or their types.

> > Thomas A. Russ wrote:
> > > Ray Dillinger <····@sonic.net> writes:
> > >
> > > > This is something about CLOS generic functions I don't get; why don't
> > > > they dispatch on number of arguments??
> > >
> > > (defmethod M1 ((x INTEGER) &optional y)
> > >   (print "First"))
> > >
> > > (defmethod M1 ((x INTEGER) y)
> > >   (print "Second"))
> > >
> > > (defmethod M1 ((x INTEGER) &rest z)
> > >   (print "Third"))
> > >
> > > (defmethod M1 ((x INTEGER) y &key a b c)
> > >   (print "Fourth"))
> > >
> > > (M1 5 10)  ==>  ???
> > >
> > > --
> > > Thomas A. Russ,  USC/Information Sciences Institute
> > 
> 
> -- 
> Thomas A. Russ,  USC/Information Sciences Institute
From: Pascal Costanza
Subject: Re: Performance of generic functions
Date: 
Message-ID: <462pj4F8ncboU1@individual.net>
Ray Dillinger wrote:
> Pascal Costanza wrote:
> 
>> No, the order is not deterministic. ANSI Common Lisp specifies the 
>> following error situations:
>>
>> - The number of arguments don't match and/or keyword arguments don't 
>> match. This is specified in 3.5 of the HyperSpec.
> 
> This is something about CLOS generic functions I don't get; why don't
> they dispatch on number of arguments??

...because it's hard to come up with reasonable semantics for this. Try 
to define some as an exercise. ;) Note that when you define semantics 
for language constructs, you have to make sure that you cover all corner 
cases and give them sane semantics as well. So this means that you have 
to consider all kinds of possible interactions with lambda list 
keywords, like &rest, &optional, &key, &aux, etc.

Note that there is a relatively simple workaround: Write a (plain, not 
generic) wrapper function that deals with the number of arguments 
manually, and then call the appropriate generic function(s) in there.


Pascal

-- 
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
From: Kenny Tilton
Subject: Re: Performance of generic functions
Date: 
Message-ID: <Pd2Mf.6452$cF5.5799@news-wrt-01.rdc-nyc.rr.com>
Ray Dillinger wrote:
> Pascal Costanza wrote:
> 
>> No, the order is not deterministic. ANSI Common Lisp specifies the 
>> following error situations:
>>
>> - The number of arguments don't match and/or keyword arguments don't 
>> match. This is specified in 3.5 of the HyperSpec.
> 
> 
> This is something about CLOS generic functions I don't get; why don't
> they dispatch on number of arguments??

Kind of useless, isn't it? (I have not about if it is even possible 
given the rest of GF dispatch.)

When would I ever want different behavior based on how /many/ arguments 
I supply to a function? One wants different behavior with different 
types of arguments, or different argument values as when we do an EQL 
specialization.

But: (foo 1 2) vs (foo 1 2 3)?

I mean, sure, we want to be able to code that, but not to get different 
behavior. We want the same behavior for a variable number of values 
(without having to code up (list 1 2 3) to get it). And of course Lisp 
has &rest and &optional for that.

Now conceivably one wants:

(foo 'x)
(foo 3)
(foo "abc")

and then for a list one wants to get into some extended specialization 
not needed by the other case:

(foo '(1 2) 42)
(foo '(1 2) "abc")

But then I think one has probably actually started a new GF, or is 
collapsing what should be two GFs into one.

kenny
From: Kenny Tilton
Subject: Re: Performance of generic functions
Date: 
Message-ID: <YY5Mf.6461$cF5.2640@news-wrt-01.rdc-nyc.rr.com>
Kenny Tilton wrote:
> Ray Dillinger wrote:
> 
>> Pascal Costanza wrote:
>>
>>> No, the order is not deterministic. ANSI Common Lisp specifies the 
>>> following error situations:
>>>
>>> - The number of arguments don't match and/or keyword arguments don't 
>>> match. This is specified in 3.5 of the HyperSpec.
>>
>>
>>
>> This is something about CLOS generic functions I don't get; why don't
>> they dispatch on number of arguments??
> 
> 
> Kind of useless, isn't it? (I have not about if it is even possible 
> given the rest of GF dispatch.)
> 
> When would I ever want different behavior based on how /many/ arguments 
> I supply to a function? One wants different behavior with different 
> types of arguments, or different argument values as when we do an EQL 
> specialization.
> 
> But: (foo 1 2) vs (foo 1 2 3)?
> 
> I mean, sure, we want to be able to code that, but not to get different 
> behavior. We want the same behavior for a variable number of values 
> (without having to code up (list 1 2 3) to get it). And of course Lisp 
> has &rest and &optional for that.
> 
> Now conceivably one wants:
> 
> (foo 'x)
> (foo 3)
> (foo "abc")
> 
> and then for a list one wants to get into some extended specialization 
> not needed by the other case:
> 
> (foo '(1 2) 42)
> (foo '(1 2) "abc")
> 
> But then I think one has probably actually started a new GF, or is 
> collapsing what should be two GFs into one.
> 
> kenny

I should have just said, This is one of the nice things about using a 
language that (a) we know is generally brilliant and (b) is going on 
fifty years old and (c) has been through a standrads process; they 
probably got it right.

kenny
From: Joerg Hoehle
Subject: Re: Performance of generic functions
Date: 
Message-ID: <u64n1kuvn.fsf@users.sourceforge.net>
Ray Dillinger <····@sonic.net> writes:

> Pascal Costanza wrote:
> This is something about CLOS generic functions I don't get; why don't
> they dispatch on number of arguments??

Because language design is about choosing a set of designs among
possible ones.  And some decisions may preclude other decisions.

For example, in the Erlang language, you can have multiple instances
(definitions) of the foo functions, differing in their arity. In fact,
they are completely different functions: foo/1, foo/2, foo/3 etc.

However, CL has variable length argument lists, optionals and
keywords.  Choosing to have these [*] precludes going the Erlang way.
I bet you won't be able to provide a satisfactory definition for the
behaviour of a dispatcher on the number of arguments *in the presence*
of &optional, &rest and &keys (not to forget &allow-other-keys).

So what's fine in e.g. Erlang is not applicable in another language.

[*] Old Lisps knew only (arg1 ... argn [ . arg-rest ])
You can still see traces of the explicit dot notation in the macro
DESTRUCTURING-BIND.  BTW, Scheme also uses this notation.

Regards,
	Jorg Hohle
Telekom/T-Systems Technology Center
From: Bruce Hoult
Subject: Re: Performance of generic functions
Date: 
Message-ID: <bruce-F9BE35.00381817012006@news.clear.net.nz>
In article <·························@news.wanadoo.fr>,
 "Christian Jullien" <········@hotmail.com> wrote:

> Hi all,
> 
> I wonder what is considered as a "reasonnable" cost of a generic function 
> V.S standard call
> 
> Given the following code (please adapt for CLtL):
> 
> (defgeneric gfun (a))
> (defmethod  gfun ((a <integer>)) a)
> 
> (defun fun (a) a)
> 
> (defun test-fun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (fun 0)))
> 
> (defun test-gfun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (gfun 0)))
> 
> My own ISLISP (OpenLisp) implementation gives me for 10M iterations:
> 
> (test-fun 10000000) -> 0.038s
> (test-gfun 10000000) -> 1.387s
> 
> So, generic function call is about 40 times slower that standard call.
> Beside my (perhaps) poor implementation, I would like to know what is the 
> ratio for other implementations (compiled code).
> 
> Thanks for your replies.

Dylan d2c code:

module: gf-cost

define generic gfun(a);
define method gfun(a ::<integer>) a end;

define function fun(a) a end;

define function test-fun(n :: <integer>)
  for (i from 0 below n)
    fun(0);
  end;
end;

define function test-gfun(n :: <integer>)
  for (i from 0 below n)
    gfun(0);
  end;
end;

test-fun(1000000000);    // 1.525 
test-gfun(1000000000);   // 1.525


Note that I ran the test 100 times more than you did.

As it happens, the code generated is a little different for the two 
tests -- because the argument for gfun is declared as <integer> while 
the type of the argument for fun is not known -- but it evens out.  Here 
is the intermediate C code generated:

/* gfun{<integer>} */
descriptor_t * gf_costZgf_costZgfun_METH(descriptor_t *orig_sp, long A_a 
/* a */, heapptr_t A1)
{
    descriptor_t L_temp;
    L_temp.heapptr = gf_costZliteral.heapptr;
    L_temp.dataword.l = A_a;
    orig_sp[0] = L_temp;
    return orig_sp + 1;
}


/* fun */
descriptor_t * gf_costZgf_costZfun_FUN(descriptor_t *orig_sp, 
descriptor_t A_a /* a */)
{
    orig_sp[0] = A_a;
    return orig_sp + 1;
}


/* test-fun */
descriptor_t * gf_costZgf_costZtest_fun_FUN(descriptor_t *orig_sp, long 
A_n /* n */)
{
    descriptor_t *cluster_0_top;
    long L_i; /* i */
    long L_i_2; /* i */
    descriptor_t L_temp;
    L_i = 0;
    while (1) {
       L_i_2 = L_i;
       if ((L_i_2 < A_n)) {
           L_temp.heapptr = gf_costZliteral.heapptr;
           L_temp.dataword.l = 0;
           /* fun */
           gf_costZgf_costZfun_FUN(orig_sp, L_temp);
           L_i = (L_i_2 + 1);
       }
       else {
           goto block0;
       }
    }
  block0:;
    orig_sp[0] = dylanZfalse;
    return orig_sp + 1;
}


/* test-gfun */
descriptor_t * gf_costZgf_costZtest_gfun_FUN(descriptor_t *orig_sp, long 
A_n /* n */)
{
    descriptor_t *cluster_0_top;
    long L_i; /* i */
    long L_i_2; /* i */
    L_i = 0;
    while (1) {
        L_i_2 = L_i;
        if ((L_i_2 < A_n)) {
            /* gfun{<integer>} */
            gf_costZgf_costZgfun_METH(orig_sp, 0,
                                      &dylanZempty_list_ROOT);
            L_i = (L_i_2 + 1);
        }
        else {
            goto block0;
        }
    }
  block0:;
    orig_sp[0] = dylanZfalse;
    return orig_sp + 1;
}

-- 
Bruce |  41.1670S | \  spoken |          -+-
Hoult | 174.8263E | /\ here.  | ----------O----------
From: Thomas A. Russ
Subject: Re: Performance of generic functions
Date: 
Message-ID: <ymiwtgyh9pf.fsf@sevak.isi.edu>
"Christian Jullien" <········@hotmail.com> writes:

> 
> Hi all,
> 
> I wonder what is considered as a "reasonnable" cost of a generic function 
> V.S standard call
> 
> Given the following code (please adapt for CLtL):
> 
> (defgeneric gfun (a))
> (defmethod  gfun ((a <integer>)) a)
> 
> (defun fun (a) a)
> 
> (defun test-fun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (fun 0)))
> 
> (defun test-gfun (n)
>    (do ((i 0 (1+ i)))
>        ((= i n))
>        (gfun 0)))
> 
> So, generic function call is about 40 times slower that standard call.
> Beside my (perhaps) poor implementation, I would like to know what is the 
> ratio for other implementations (compiled code).

No special optimization settings:

MCL 5.0 on TiBook 800 MHz G4:

(test-fun 10000000) => 1.35s
(test-fun 10000000) => 3.00s

Ratio: 2.2


Allegro CL 5.0 on Sun Ultra-2 Sparc 2x 168 Mhz

(test-fun 10000000) => 2.75s
(test-fun 10000000) => 9.22s

Ratio: 3.34

-- 
Thomas A. Russ,  USC/Information Sciences Institute