From: Wendall Marvel
Subject: define-method-combination question
Date: 
Message-ID: <3FCAD7C4.3060907@speakeasy.net>
(define-method-combination whatever ()
   ((primary (*) :required t))
   (flet ((call-methods (methods)
	   (mapcar
	    #'(lambda (method)
		`(when (find
			(first (method-qualifiers ,method))
			hooks)
		   (setq result (cons (call-method ,method ()) result))))
	    methods)))
     `(progn
        ,@(call-methods primary))))

(defgeneric foo (bar baz) (:method-combination whatever))

(defmethod foo one (bar baz)
   (format T "foo one ~A ~A~%" bar baz))

(defmethod foo two (bar baz)
   (format T "foo two ~A ~A~%" bar baz))

(foo 'a 'b)

I get errors in both ACL and CMUCL saying

"More than one method of type METHODS with the same specializers."

However, I'm confused. The Hyperspec has this to say:

In the section on define-method-combination ->

   Note that two methods with identical specializers, but with different
   qualifiers, are not ordered by the algorithm described in Step 2 of
   the method selection and combination process described in Section
   7.6.6 (Method Selection and Combination). Normally the two methods
   play different roles in the effective method because they have
   different qualifiers, and no matter how they are ordered in the result
   of Step 2, the effective method is the same. If the two methods play
   the same role and their order matters, an error is signaled. This
   happens as part of the qualifier pattern matching in
   define-method-combination.

That would seem to indicate that the error is correct, but it references 
section 7.6.6, 'Step 2', which looks to me like it means 7.6.6.1.2, 
'Sorting the Applicable Methods by Precedence Order', since 7.6.6.1 is 
the only place I see 'steps' under 7.6.6 in the hyperspec. 7.6.6.1.2 has 
this to say:

  The corresponding parameter specializers from each method are compared.
  When a pair of parameter specializers agree, the next pair are compared
  for agreement. If all corresponding parameter specializers agree, the
  two methods must have different qualifiers; in this case, either method
  can be selected to precede the other. For information about agreement,
  see Section 7.6.3 (Agreement on Parameter Specializers and Qualifiers).


Am I missing something here? The behavior in 7.6.6.1.2 is what I want, 
for what I'm doing I don't particularly care what the order is, I know 
the methods have the same specializers, that's why I gave them a 
qualifier, and for my purposes it doesn't matter what order they get 
called in if they have the same specializers. However, it seems like 
whether or not an error occurs depends entirely on if the methods get 
selected for the same method group.

I can't predeclare all method groups because I do not know all potential 
qualifiers, and I don't want to go rewriting my method combination every 
time I add one.

Is there a way to get what I want to work?

From: Wendall Marvel
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCADCFB.7020008@speakeasy.net>
Aaagh.

The code in my last post is wrong, it errors where expected when both 
methods are defined but doesn't work when only one is defined.

So allow me to substitute an example from the hyperspec.

;Order methods by positive integer qualifiers
;:around methods are disallowed to keep the example small
(define-method-combination example-method-combination ()
   ((methods positive-integer-qualifier-p))
   `(progn ,@(mapcar #'(lambda (method)
                         `(call-method ,method))
                (stable-sort methods #'<
                    :key  #'(lambda (method)
                              (first (method-qualifiers method)))))))

(defun positive-integer-qualifier-p (method-qualifiers)
   (and (= (length method-qualifiers) 1)
        (typep (first method-qualifiers) '(integer 0 *))))

(defgeneric foo (bar baz) (:method-combination example-method-combination))

(defmethod foo 1 (bar baz)
   (format T "foo one ~A ~A~%" bar baz))


(foo 'a 'b) => prints "foo one a b"

(defmethod foo 2 (bar baz)
   (format T "foo two ~A ~A~%" bar baz))

(foo 'a 'b) => errors with "More than one method of type METHODS with 
the same specializers."

I'd expect it to first print "foo one a b" then "foo two a b", but 
again, maybe I'm missing something.
From: james anderson
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCB08BD.A3BD0DA3@setf.de>
the error which you observe is the consequence of one possible interpretation
of the spec. the error message is a bit misleading. i suggest that it is not
the only interpretation and is neither necessary nor useful. as an additional
data point, mcl does what i believe you would have expected. the issue is not
explained entirely by 7.6.*, as it depends on the term "role", which is
described in the definition for define-method-combination. 

the clause in 7.6.6.1.2

"If all corresponding parameter specializers agree, the two methods must have
different qualifiers;"

is necessarily a constraint on a correct mop implementation rather than on
correct programs. otherwise, to redefine a method onbe would have to
explicitly delete any existing definition.

this behaviour is also stated explicitly in the definition of defmethod. this
means that, no two methods which "agree on parameter specializers and
qualifiers" can contribute to "step 2". that is none of the methods can
actually have "the same specializers". in which sense the error message is misleading.

the message is, i suspect, actually calling attention to the case, that it is
not possible to order two methods which it asserts have been specified by the
combination to play the same "role". that is, in the sense use in the
define-method-combination definition, they match the same qualifier pattern.

the definition does however also say

"It is also possible to bypass the method group specifier mechanism and do
everything in the body forms."

in its description of the interpretation for the * pattern.

there is some sense in which this contradicts the uniqueness rules for
matching, but the only useful interpretation is that it is a special case, in
which the uniqueness constraint does not apply. in any other interpretation,
where one identifies * with a single "role", the qualifiers no longer have any
significance and the provision, that the method combination can "do everything
in the body forms" is vacuous. i would not think that this was the intent.

...
From: Wendall Marvel
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCB77B8.20006@speakeasy.net>
Thanks for the lengthy reply :)

james anderson wrote:
 > the message is, i suspect, actually calling attention to the case, 
that it is
 > not possible to order two methods which it asserts have been 
specified by the
 > combination to play the same "role". that is, in the sense use in the
 > define-method-combination definition, they match the same qualifier
 > pattern.

Well, it says "If the two methods play the same role and their order 
matters, an error is signaled."

I don't see any way to tell it that the order does not matter. I also 
don't see anything clearly stating what it means for the order to 
matter, so I'd agree with you, that there are two different 
interpretations of the spec, now that I know that MCL behaves as I would 
expect.

> the definition does however also say
> 
> "It is also possible to bypass the method group specifier mechanism and do
> everything in the body forms."
> 
> in its description of the interpretation for the * pattern.
> 

Right after that, it says

"This is accomplished by writing a single method group with * as its 
only qualifier-pattern; the variable is then bound to a list of all of 
the applicable methods, in most-specific-first order."

So, you have to put all your methods in the one group.  Both of the 
lisps I'm using (CMUCL and ACL) error out if any single group has more 
than one method with the same specializers. Whether the single method
group specifier is (methods *) or (methods (*)) - I'm not quite sure
the first is allowed but it seems to work (at least, I fail in the
expected ways :))

This means you can't bypass the method group specifier and do
everything yourself in the body forms, because you get an error before you
ever get the chance.
From: Wendall Marvel
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCB8045.9060909@speakeasy.net>
Just to clarify, I'm in agreement that 'the qualifiers no longer have 
any significance and the provision, that the method combination can "do 
everything in the body forms" is vacuous.'


My last post didn't really make that clear, I reread it and thought that 
it sounded somewhat like I was disagreeing, but we're saying the same thing.
From: james anderson
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCB8D57.EE71892B@setf.de>
Wendall Marvel wrote:
> 
> ...
> 
> I don't see any way to tell it that the order does not matter.

i do not think that it is possible for the order to "not matter". the only
options permitted in a method group specifier are :most-specific-first and
:most-specific-last, the former of which is the default. i have understond
that passage to refer to the "unordered" relation between methods in different
groups. 

>   I also
> don't see anything clearly stating what it means for the order to
> matter,

it matters for delivery to the method combination computation as the contract
is "according to the order in the group specifier">

>   so I'd agree with you, that there are two different
> interpretations of the spec, now that I know that MCL behaves as I would
> expect.
> 
> > the definition does however also say
> >
> > "It is also possible to bypass the method group specifier mechanism and do
> > everything in the body forms."
> >
> > in its description of the interpretation for the * pattern.
> >
> 
> Right after that, it says
> 
> "This is accomplished by writing a single method group with * as its
> only qualifier-pattern; the variable is then bound to a list of all of
> the applicable methods, in most-specific-first order."
> 
> So, you have to put all your methods in the one group.  Both of the
> lisps I'm using (CMUCL and ACL) error out if any single group has more
> than one method with the same specializers. Whether the single method
> group specifier is (methods *) or (methods (*)) - I'm not quite sure
> the first is allowed but it seems to work (at least, I fail in the
> expected ways :))

cltl2 reads as if second should match the single * qualifier.

> 
> This means you can't bypass the method group specifier and do
> everything yourself in the body forms, because you get an error before you
> ever get the chance.

yes. as i said, not very useful. the effect is of one role only and one need
not have bothered with the qualifiers. that cannot have been the intended effect.

...
From: Wendall Marvel
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCC2B94.7090906@speakeasy.net>
james anderson wrote:
> 
> i do not think that it is possible for the order to "not matter". the only
> options permitted in a method group specifier are :most-specific-first and
> :most-specific-last, the former of which is the default. i have understond
> that passage to refer to the "unordered" relation between methods in different
> groups. 
> 
> 

...

> 
> 
> it matters for delivery to the method combination computation as the contract
> is "according to the order in the group specifier">
> 

Here are two cases which, imho, the order does not matter. However, I'm 
not going to be pretentious enough to say that this is what the spec 
means when it says 'If the two methods play the same role and their 
order matters'

In the example, which orders the method calls by an integer qualifier, 
the order in which two methods with the same specializers and differing 
qualifiers appear in the list that is bound to methods does not matter, 
because the methods are explicitly ordered by the value of the qualifier 
using stable-sort.

For the purpose that I'm using define-method-combination, the order that 
  the methods appear in the list that is bound to methods does not 
matter, because I'm explicitly saying that methods using that 
combination should not have side effects that would cause the order to 
matter. Each method is supposed to get called, but not depend on whether 
some other method gets called first, whether it's more specific or less 
specific.

In other words, I think whether the order the methods appear in the list 
matters or not is actually determined by what the method combination 
does, not whether there's ambiguity in how to order the methods in the 
list, if there happen to be two methods with the same specializers. If 
the order of two methods with the same specializers but different 
qualifiers matters for what you're doing, you ought write the method 
combination to explicitly order them. Sure, it's nice to have them in 
most-specific-first or most-specific-last order for a lot of things, so 
it makes sense to order them that way, but if I'm defining a method 
combination, and there's possibly more than one method with the same 
specializers but different qualifiers, it should be my job to explicitly 
order them if I need to, and not the implementation's job to signal an 
error.

The logic behind this is the fact that with the standard method 
combination, it can never happen - you can't have qualifiers other than 
:before, :after, and :around, or you'll end up with an invalid method error:

"If there is an applicable method that does not fall into any method 
group, the function invalid-method-error is called."

The other built in method combination types are likewise restricted in 
what is allowed to be a qualifier:

"* An around method has the keyword symbol :around as its sole 
qualifier. The meaning of :around methods is the same as in standard 
method combination. Use of the functions call-next-method and 
next-method-p is supported in around methods.

* A primary method has the name of the method combination type as its 
sole qualifier. For example, the built-in method combination type and 
recognizes methods whose sole qualifier is and; these are primary 
methods. Use of the functions call-next-method and next-method-p is not 
supported in primary methods."

I think these restrictions prevent this from ever occurring with the 
built-in method combinations, since any method defined has to have a 
qualifier that is a member of some set of qualifiers. For the built in 
method combinations, any two methods with the same specializers but 
different qualifiers always end up in different groups. If you use a 
method qualifier that's not a member of the set for that method 
combination, invalid-method-error will get called.

Since you can't cause this with the built in method combinations, it 
only affects people who are defining their own method combinations.

If I'm defining my own method combination, I'd expect to be allowed to 
resolve order issues myself, instead of having the implementation tell 
me that I can't, that the order the methods appear in the list is more 
important than what I do with the list. It seems much saner to just 
decide to put one before the other, and let it be known that if two 
methods with the same specializers end up in the same group, it's up to 
the definer of the method combination to resolve it.

Of course, I could be wrong, but that's basically why I think it 
shouldn't error, along with the fact that, as you noted, it makes the 
provision that the method combination can "do everything
in the body forms" vacuous.

Hmm. This started out as an explanation of why I think it can be said 
that 'order does not matter', at least in regard to the order that two 
methods with the same specializers appear in the list - but seems to 
have turned into a rant. I'll stop now. :)
From: james anderson
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCC5FAC.59CE278B@setf.de>
Wendall Marvel wrote:
> 
> james anderson wrote:
> >
> > i do not think that it is possible for the order to "not matter". the only
> > options permitted in a method group specifier are :most-specific-first and
> > :most-specific-last, the former of which is the default. i have understond
> > that passage to refer to the "unordered" relation between methods in different
> > groups.
> ...
> > it matters for delivery to the method combination computation as the contract
> > is "according to the order in the group specifier">
> >
> 
> Here are two cases which, imho, the order does not matter. However, I'm
> not going to be pretentious enough to say that this is what the spec
> means when it says 'If the two methods play the same role and their
> order matters'
> 
> ...
> 
> In other words, I think whether the order the methods appear in the list
> matters or not is actually determined by what the method combination
> does, not whether there's ambiguity in how to order the methods in the
> list, if there happen to be two methods with the same specializers.

not entirely. the combination form has to fulfil at least the constract, that,
if it supports call-next-method the respective constraints apply. namely, that
the ordered set of methods does not change where call-next-method is invoked
on a explicit argument list. so there is an issue, that something outside of
the method combination computation will be computing an ordered list of
methods. i have not had occasion to try to understand this interaction, but
without having done so, i would not claim that one can ignore it.

...
> 
> Hmm. This started out as an explanation of why I think it can be said
> that 'order does not matter', at least in regard to the order that two
> methods with the same specializers appear in the list - but seems to
> have turned into a rant. I'll stop now. :)

no need to rant. franz has a bug list and cmucl's pcl developer will certainly
entertain a patch. that's the best way to find out where this leads.

...
From: Wendall Marvel
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCC7CC0.6040008@speakeasy.net>
james anderson wrote:
 >
 >
 > not entirely. the combination form has to fulfil at least the 
constract, that,
 > if it supports call-next-method the respective constraints apply. 
namely, that
 > the ordered set of methods does not change where call-next-method is 
invoked
 > on a explicit argument list. so there is an issue, that something 
outside of
 > the method combination computation will be computing an ordered list of
 > methods. i have not had occasion to try to understand this 
interaction, but
 > without having done so, i would not claim that one can ignore it.
 >

Ahh, now I grok.

I don't think it would cause a problem, since the relevant bit of the 
description says 'the ordered set of applicable methods', and 
'applicable methods' has a fairly specific definition given in the spec, 
which is definitely -not- guaranteed to be the same as the list of 
methods in any method group - since the list of methods in any method 
group may be that ordered set, that ordered set reversed, or a subset.

Since it's not guaranteed, I don't see why an implementation would jump 
through hoops to use the list of methods in a method group for that 
check when it does happen to match, but I can at least look to make sure 
while I'm poking around PCL.


 > no need to rant. franz has a bug list and cmucl's pcl developer will 
certainly
 > entertain a patch. that's the best way to find out where this leads.

Heh, I was kidding about it being a 'rant'.
I just got tad bit more verbose than I'd originally intended.

I've actually already modified it in PCL, at least the PCL that is in 
the CMUCL 18e source tar, and it's working.

It was fairly easy, actually, which leaves me worried that I've missed 
something.
From: james anderson
Subject: Re: define-method-combination question
Date: 
Message-ID: <3FCC8ED2.109B5C28@setf.de>
Wendall Marvel wrote:
> 
> james anderson wrote:
>  >
>  >
>  > not entirely. the combination form has to fulfil at least the
> constract, that,
>  > if it supports call-next-method the respective constraints apply.
> namely, that
>  > the ordered set of methods does not change where call-next-method is
> invoked
>  > on a explicit argument list. so there is an issue, that something
> outside of
>  > the method combination computation will be computing an ordered list of
>  > methods. i have not had occasion to try to understand this
> interaction, but
>  > without having done so, i would not claim that one can ignore it.
>  >
> 
> Ahh, now I grok.
> 
> I don't think it would cause a problem, since the relevant bit of the
> description says 'the ordered set of applicable methods', and
> 'applicable methods' has a fairly specific definition given in the spec,
> which is definitely -not- guaranteed to be the same as the list of
> methods in any method group - since the list of methods in any method
> group may be that ordered set, that ordered set reversed, or a subset.
> 
> Since it's not guaranteed, I don't see why an implementation would jump
> through hoops to use the list of methods in a method group for that
> check when it does happen to match, but I can at least look to make sure
> while I'm poking around PCL.

now that i've reread the cltl2 passages on call-method and call-next-method,
the only way to understand them is that the constraints apply to the method
set and ordering supplied as the second argument to call-method. which would
mean that the method combination computation should be able to do anything it
wants independent of the method groups supplied to it. even reshuffle the
roles. which also makes sense since, given make-method, the combination can
fabricate methods which are otherwise unknown. so, i now think my concern wrt
ordering is groundless.

> ...
> 
> It was fairly easy, actually, which leaves me worried that I've missed
> something.

complements to the cmucl pcl implementation.

...