From: Hidayet Tunc Simsek
Subject: [non-descriptive optimization]
Date: 
Message-ID: <38360145.E0A3ECF4@EECS.Berkeley.Edu>
The CMU manual says:

--
Non-Descriptor Representations

      

From the discussion above, we can see that the standard descriptor
representation has many problems, the worst being number consing. Common
Lisp compilers try to avoid
these descriptor efficiency problems by using non-descriptor
representations. A compiler that uses non-descriptor representations can
compile this function so that it does no
number consing: 

       (defun multby (vec n)
         (declare (type (simple-array single-float (*)) vec)
                  (single-float n))
         (dotimes (i (length vec))
           (setf (aref vec i)
                 (* n (aref vec i)))))
--

I try this on my CMUCL 18b on Solaris ultra 1 and here is what I get:

(defvar vec (make-array 100 :element-type 'single-float
			:initial-element 0.0s0))

(defun multby (vec n)
  (declare (type (simple-array single-float (*)) vec)
	   (single-float n))
  (dotimes (i (length vec))
    (setf (aref vec i)
      (* n (aref vec i)))))

(eval-when (load)
  (profile:profile multby)
  
  (dotimes (i 10000)
    (multby vec 2.0s0))
  
  (profile:report-time multby))


when I load:

  Seconds  |  Consed   |  Calls  |  Sec/Call  |  Name:
------------------------------------------------------
    10.130 | 24,031,200 |  10,000 |    0.00101 | MULTBY
------------------------------------------------------
    10.130 | 24,031,200 |  10,000 |            | Total
 
Estimated total profiling overhead: 0.16 seconds


Am I missing something, are there any other statements that we have to
make so that
this code stops consing?

Thanks,
Tunc

From: Pierre R. Mai
Subject: Re: [non-descriptive optimization]
Date: 
Message-ID: <87d7t5i9nh.fsf@orion.dent.isdn.cs.tu-berlin.de>
Hidayet Tunc Simsek <······@EECS.Berkeley.Edu> writes:

> I try this on my CMUCL 18b on Solaris ultra 1 and here is what I get:
> 
> (defvar vec (make-array 100 :element-type 'single-float
> 			:initial-element 0.0s0))
> 
> (defun multby (vec n)
>   (declare (type (simple-array single-float (*)) vec)
> 	   (single-float n))
>   (dotimes (i (length vec))
>     (setf (aref vec i)
>       (* n (aref vec i)))))
> 
> (eval-when (load)
>   (profile:profile multby)
>   
>   (dotimes (i 10000)
>     (multby vec 2.0s0))
>   
>   (profile:report-time multby))
> 
> 
> when I load:
> 
>   Seconds  |  Consed   |  Calls  |  Sec/Call  |  Name:
> ------------------------------------------------------
>     10.130 | 24,031,200 |  10,000 |    0.00101 | MULTBY
> ------------------------------------------------------
>     10.130 | 24,031,200 |  10,000 |            | Total
>  
> Estimated total profiling overhead: 0.16 seconds
> 
> 
> Am I missing something, are there any other statements that we have to
> make so that
> this code stops consing?

First of all you should set your speed-optimization levels to 3.  This 
will a) cause CMUCL to optimize better, and b) cause CMUCL to tell you 
where your code exhibits problems that make it hard for CMUCL to
optimize.  So with an added (optimize (speed 3)) declaration I get
this:

Python version 1.0, VM version Intel x86 on 20 NOV 99 01:53:47 pm.
Compiling: /home/dent/src/lang/lisp/consing2.cl 20 NOV 99 01:52:31 pm

Converted MULTBY.
Compiling DEFUN MULTBY: 

File: /home/dent/src/lang/lisp/consing2.cl

In: DEFUN MULTBY
  (LENGTH VEC)
Note: Unable to optimize due to type uncertainty:
      The first argument is a (OR VECTOR CONS NULL), not a (SIMPLE-ARRAY * (*)).
Note: Unable to optimize due to type uncertainty:
      The first argument is a (OR VECTOR CONS NULL), not a VECTOR.

[...]

Byte Compiling Top-Level Form: 

Compilation unit finished.
  13 notes


/home/dent/src/lang/lisp/consing2.x86f written.
Compilation finished in 0:00:00.

So this makes it clear that your declaration of the type of vec
doesn't seem to have the desired effect.  Why?  Because you first
declared vec to be special (with defvar), which means that the lambda
parameter vec is also special (and doesn't quite behave like one would 
expect a parameter to behave), which throws CMU CL, and probably isn't 
what you wanted to do.  If you rename the global vec to *vec* (which
it should have been called anyway), then you will get cons-less
performance...

Regs, Pierre.

-- 
Pierre Mai <····@acm.org>         PGP and GPG keys at your nearest Keyserver
  "One smaller motivation which, in part, stems from altruism is Microsoft-
   bashing." [Microsoft memo, see http://www.opensource.org/halloween1.html]
From: Tim Bradshaw
Subject: Re: [non-descriptive optimization]
Date: 
Message-ID: <ey3bt8p5j2l.fsf@lostwithiel.tfeb.org>
* Pierre R Mai wrote:
> So this makes it clear that your declaration of the type of vec
> doesn't seem to have the desired effect.  Why?  Because you first
> declared vec to be special (with defvar), which means that the lambda
> parameter vec is also special (and doesn't quite behave like one would 
> expect a parameter to behave), which throws CMU CL, and probably isn't 
> what you wanted to do.  If you rename the global vec to *vec* (which
> it should have been called anyway), then you will get cons-less
> performance...

Damn! that's the obvious answer, I'm all embarassed now (:-).

Of course the lesson from this is: make sure your special variables
have *stars* around them!

--tim
From: Marco Antoniotti
Subject: Re: [non-descriptive optimization]
Date: 
Message-ID: <lwaeo7aq73.fsf@parades.rm.cnr.it>
Tim Bradshaw <···@tfeb.org> writes:

> * Pierre R Mai wrote:
> > So this makes it clear that your declaration of the type of vec
> > doesn't seem to have the desired effect.  Why?  Because you first
> > declared vec to be special (with defvar), which means that the lambda
> > parameter vec is also special (and doesn't quite behave like one would 
> > expect a parameter to behave), which throws CMU CL, and probably isn't 
> > what you wanted to do.  If you rename the global vec to *vec* (which
> > it should have been called anyway), then you will get cons-less
> > performance...
> 
> Damn! that's the obvious answer, I'm all embarassed now (:-).
> 
> Of course the lesson from this is: make sure your special variables
> have *stars* around them!

However, there is a question raised by the above behavior.  Does the
compiler do "the right thing"?  I.e. why should the compiler not
shadow the 'specialness' of 'vec' when it finds it declared as a
lambda list variable?  What kind of worms are in the can I just
opened?  :)

Cheers

-- 
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa
From: Pierre R. Mai
Subject: Re: [non-descriptive optimization]
Date: 
Message-ID: <87u2mexs61.fsf@orion.dent.isdn.cs.tu-berlin.de>
Marco Antoniotti <·······@parades.rm.cnr.it> writes:

> However, there is a question raised by the above behavior.  Does the
> compiler do "the right thing"?  I.e. why should the compiler not
> shadow the 'specialness' of 'vec' when it finds it declared as a
> lambda list variable?  What kind of worms are in the can I just
> opened?  :)

I'm too lazy just now to to check the HyperSpec, but IIRC then the
compiler would be breaking the Spec if it shadowed the specialness of
vec.  There is nothing "special" (no pun intended) about lambda
variables.  With your intended behaviour the following "equivalence"[1]
would not hold anymore:

((lambda (x y) ...) A B) == (let ((x A) (y B)) ...)

IMHO the behaviour you want would be an evil case of DWIM behaviour...

What I find more questionable is the fact that the compiler gets
confused about the type declaration on the binding of the special
variable.  I think the inferred type of the new binding of the
special variable should either be the declared type, or t, and not
(or vector cons null), or am I missing something here?

Regs, Pierre.

Footnotes: 
[1]  It isn't really a strict equivalence because of a number of
things, but special variables aren't currently one of them ;)

-- 
Pierre Mai <····@acm.org>         PGP and GPG keys at your nearest Keyserver
  "One smaller motivation which, in part, stems from altruism is Microsoft-
   bashing." [Microsoft memo, see http://www.opensource.org/halloween1.html]
From: Jeff Dalton
Subject: defvar pervasiveness, was Re: [non-descriptive optimization]
Date: 
Message-ID: <x2r9hhdpxo.fsf_-_@todday.aiai.ed.ac.uk>
Marco Antoniotti <·······@parades.rm.cnr.it> writes:

> Tim Bradshaw <···@tfeb.org> writes:
> 
> > * Pierre R Mai wrote:
> > > So this makes it clear that your declaration of the type of vec
> > > doesn't seem to have the desired effect.  Why?  Because you first
> > > declared vec to be special (with defvar), which means that the lambda
> > > parameter vec is also special (and doesn't quite behave like one would 
> > > expect a parameter to behave), which throws CMU CL, and probably isn't 
> > > what you wanted to do.  If you rename the global vec to *vec* (which
> > > it should have been called anyway), then you will get cons-less
> > > performance...

> > Damn! that's the obvious answer, I'm all embarassed now (:-).

> > Of course the lesson from this is: make sure your special variables
> > have *stars* around them!

> However, there is a question raised by the above behavior.  Does the
> compiler do "the right thing"?  I.e. why should the compiler not
> shadow the 'specialness' of 'vec' when it finds it declared as a
> lambda list variable?  What kind of worms are in the can I just
> opened?  :)

Defvar proclaims / declaims the variable special which has a
"pervasive" effect.  It basically makes the name always be
treated as the name of a special variable.  There is no way
to override this in Common Lisp, although a way was considered
during standardization.

Merely declaring (with declare) a variable special does not have this
effect.

Some people this this effect of defvar is very bad.

Others think that it makes it much more convenient to use dynamic /
special variables and that any confusion can be removed by following
the star-convention.

And some people once thought that if that's the way you're going
to see it, maybe stars alone should have that effect automatically.

CMU CL is I think unique in proclaiming a free variable special if the
user hasn't said anything one way or another.  All the other Lisps
I've used (so far as I can recall) just treat it as special in that
context.  You've not mentioned "x" before, and you type "(defun f () x)",
for instance, and the compiler or interpreter notices that x is not
bound in that definition.  Does it (a) warn you and treat that
instance of "x" as special?, or (b) warn you and make "x" globally,
pervasively special thereafter?  So far as I know, only CMU CL does
(b).  If CMUCL sees something like the definition above, it pretends
you did something like "(defvar x)".

One more thing.  People will sometimes say global vars in Common Lisp
are just like having a big "let" around your code.  They are wrong.

I think that pretty much covers this issue, modulo details.

-- jeff
From: Tim Bradshaw
Subject: Re: [non-descriptive optimization]
Date: 
Message-ID: <ey3emdl5p43.fsf@lostwithiel.tfeb.org>
* Hidayet Tunc Simsek wrote:

> I try this on my CMUCL 18b on Solaris ultra 1 and here is what I get:


>   Seconds  |  Consed   |  Calls  |  Sec/Call  |  Name:
> ------------------------------------------------------
>     10.130 | 24,031,200 |  10,000 |    0.00101 | MULTBY
> ------------------------------------------------------
>     10.130 | 24,031,200 |  10,000 |            | Total
 

> Am I missing something, are there any other statements that we have to
> make so that
> this code stops consing?


I've no idea why it is so bad if you run it when loading, but if you
run it after loading (without the profiling stuff in the file) it
doesn't cons at all.  It may be that the use of the profiler at load
time is causing it to catch an interpreted definition of the function
or something.

--tim