From: Steven M. Haflich
Subject: A little, tiny crack in the language
Date: 
Message-ID: <tfnV9.223$0S1.16973058@newssvr17.news.prodigy.com>
ANS 11.1.2.1.2.1 "Some Exceptions to Constraints on the COMMON-LISP Package for
Conforming Programs" explicitly allows defining lexical functions on CL-package
symbols:

   If an external symbol of the common-lisp package is not defined as a
   standardized function, macro, or special operator, it is allowed to
   lexically bind it as a function (e.g., with flet), to declare the ftype
   of that binding, and (in implementations which provide the ability to do
   so) to trace that binding.

The prohibition against shadowing CL operators is to protect macroexpansions
that might refer to the global definitions.

Now what about the following little gem:

  (defun foo ()
    (flet ((declare (x) (format t "~&~a!!!!!~%" x)))
      (macrolet ((optimize (&rest rest) (declare (ignore rest)) ''YesSir))
        nil      ; <<<<<===
        (declare (optimize (speed 3)))
        'done)))

Now, the issue is not with cl:speed but with cl:declare.  Neither has any
definition as an operator, but whereas speed cannot appear in a place where
an operator could appear, declare most definitely can appear in such a place.

Unaesthetic as it is, the above form may be legal, portable CL.  Ugh!  The only
possible prohibition is the statement in the dictionary page for declare that
declare may _only_ be used at the head of certain form bodies.  But the passage
above implies that it _is_ legal for make lexical operator definitions on it,
and by implication, to use such a definition.

Now, what about is the flagged nil subform is removed.  Things get even uglier.
Now the "call" to declare occurs in a place where it could be a legitimate
declaration.  The ANS does not prohibit this definition, but SFAICT leaves
its interpretation ambiguous.

I think this unpleasant (but UNimportant) glitch in the specification couls have
been avoided if declarfe were specifically prohibited in the 11.1.2.1.2.1 passage
above.

[Pondering things like this is more fun than working.]

From: Steven M. Haflich
Subject: Re: A little, tiny crack in the language
Date: 
Message-ID: <R5acnRAMs_glQLujXTWc2Q@speakeasy.net>
I'm correcting a little tiny error in my previous post:

Steven M. Haflich wrote:

> Now, the issue is not with cl:speed but with cl:declare.  Neither has any
> definition as an operator, but whereas speed cannot appear in a place where
> an operator could appear, declare most definitely can appear in such a 
> place.

In place of "speed" I meant to write "optimize".
From: Kaz Kylheku
Subject: Re: A little, tiny crack in the language
Date: 
Message-ID: <cf333042.0301170649.22d19193@posting.google.com>
"Steven M. Haflich" <·················@franz.com> wrote in message news:<······················@newssvr17.news.prodigy.com>...
> I think this unpleasant (but UNimportant) glitch in the specification couls have
> been avoided if declarfe were specifically prohibited in the 11.1.2.1.2.1 passage
> above

What programs could you write under the added prohibition that you
cannot write now? Why obsess over prohibitions? If some possible
programs are displeasing to you, don't write them; but don't ask that
others be forcibly prevented from writing them, or worse, that the
well-defined meaning should be removed from their programs.

A prohibition must exist for some kind of reason other than simply the
arbitrary imposition of a designer's will on the user.

When a user and implementor agree that the user shall not do
something, or else the implementor's product can respond in arbitrary
ways, it must be because the designer envisioned the possibility for
some mutual profit between those two, arising from the restriction.

> [Pondering things like this is more fun than working.]

I.e. pondering things like this is something other than working.
From: Barry Margolin
Subject: Re: A little, tiny crack in the language
Date: 
Message-ID: <puVV9.8$f83.410@paloalto-snr1.gtei.net>
In article <····························@posting.google.com>,
Kaz Kylheku <···@ashi.footprints.net> wrote:
>"Steven M. Haflich" <·················@franz.com> wrote in message
>news:<······················@newssvr17.news.prodigy.com>...
>> I think this unpleasant (but UNimportant) glitch in the specification
>couls have
>> been avoided if declarfe were specifically prohibited in the
>11.1.2.1.2.1 passage
>> above
>
>What programs could you write under the added prohibition that you
>cannot write now? Why obsess over prohibitions?

The lack of a prohibition allows him to write programs that may behave
differently under different implementations, because the implementors may
resolve the ambiguity differently.  It's *supposed* to be the case that if
your program doesn't do anything that the standard prohibits, it should be
portable.  Any time that the standard allows something that can have widely
varying results in different implementations, it's a bug in the standard
unless it's explicitly stated that it's implementation-dependent.

-- 
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Steven M. Haflich
Subject: Re: A little, tiny crack in the language
Date: 
Message-ID: <5YecnSy6toFWyLWjXTWcoA@speakeasy.net>
Barry Margolin wrote:
  Kaz Kylheku <···@ashi.footprints.net> wrote:
>>What programs could you write under the added prohibition that you
>>cannot write now? Why obsess over prohibitions?
> 
> The lack of a prohibition allows him to write programs that may behave
> differently under different implementations, because the implementors may
> resolve the ambiguity differently.  It's *supposed* to be the case that if
> your program doesn't do anything that the standard prohibits, it should be
> portable.  Any time that the standard allows something that can have widely
> varying results in different implementations, it's a bug in the standard
> unless it's explicitly stated that it's implementation-dependent.

I thank Barry for his clear and concise summary of the issue, with which
I concur.  The clause "_anything_ [not prohibited by] the standard" might
extend the position a little too far (e.g. consider heap size limitations)
but there is little question about the basic premise.

The two ANS sections restricting the kinds of definitions either the
implementation or the application can make on exported CL symbols were
motivated by the need to draw a line between these two bodies of code.
Otherwise one might trample the other unintentionally.  Speaking as an
implementor, there are all sorts  of gross and subtle problems that happen
when user code wants to redefine something in system code in a way that
was intended to be harmless or helpful.

We (X3J13) drew that bright line in these sections, and it only now occurred
to me that we got it slightly wrong.  Of course, no one in his right mind
would intentionally write code exploiting the liberty to define cl:declare.
This is merely a minor glitch in the ANS, and one that could in principle
easily be fixed.  My thinking is that cl:declare is the only symbol that
would need to be added to the prohibition list.  I'm wondering if Barry or
anyone wants to think through the issue to concur with or refute my
analysis.
From: Barry Margolin
Subject: Re: A little, tiny crack in the language
Date: 
Message-ID: <1%YV9.30$f83.899@paloalto-snr1.gtei.net>
In article <······················@speakeasy.net>,
Steven M. Haflich <·················@alum.mit.edu> wrote:
>We (X3J13) drew that bright line in these sections, and it only now occurred
>to me that we got it slightly wrong.  Of course, no one in his right mind
>would intentionally write code exploiting the liberty to define cl:declare.
>This is merely a minor glitch in the ANS, and one that could in principle
>easily be fixed.  My thinking is that cl:declare is the only symbol that
>would need to be added to the prohibition list.  I'm wondering if Barry or
>anyone wants to think through the issue to concur with or refute my
>analysis.

Actually, I'm not completely sure that the OP's complaint is really a
problem with the standard.

Sure, we don't prohibit defining DECLARE as a function.  But I think the
way that we specified that bodies are parsed makes it clear that if a
(declare ...) list appears in a place where a declaration is permitted, it
will be treated as such.  The fact that DECLARE also has a function or
macro definition should not change this -- it's part of the way bodies are
parsed, which precedes interpretation of the individual expressions as
operator invocations.

I was thinking that this obscurity could result in bugs in macros that
don't do their own body parsing to look for declarations.  But I've
convinced myself that they would be buggy regardless of this issue.  If a
macro doesn't allow declarations, and its expansion uses its body wholesale
as the body in another form, it must make sure that the expansion form
won't try to look for declarations in it.  That's easily done by preceding
the body with a NIL, i.e.

(defmacro with-foo ((...) &body body)
  (let (...)
    nil
    ,@body))

Basically, the idea is that if WITH-FOO allows declarations in its body,
you leave out the NIL; if it doesn't allow them, you include the NIL.  This
is necessary whether or not DECLARE is allowed to be defined.

-- 
Barry Margolin, ······@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Christophe Rhodes
Subject: Re: A little, tiny crack in the language
Date: 
Message-ID: <sqsmvrtsyj.fsf@lambda.jcn.srcf.net>
Barry Margolin <······@genuity.net> writes:

> (defmacro with-foo ((...) &body body)
>   (let (...)
>     nil
>     ,@body))
> 
> Basically, the idea is that if WITH-FOO allows declarations in its body,
> you leave out the NIL; if it doesn't allow them, you include the NIL.  This
> is necessary whether or not DECLARE is allowed to be defined.

You need to be slightly careful, I think -- it's fine in this case,
but for macros that end up as TAGBODYs, it might be better to have the
first form as (PROGN NIL), not just a bare NIL, or else you could end
up with a non-compound form in a context which requires a compound one.

The issue over DECLARE had occurred to me -- I was surprised to find
that it wasn't in fact defined as a special operator.  But I'm glad to
say that I've only seen a relatively few of these corner cases (or
maybe that just means that there's trouble in store, later :-)

Cheers,

Christophe
-- 
http://www-jcsu.jesus.cam.ac.uk/~csr21/       +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%")    (pprint #36rJesusCollegeCambridge)