From: Tamas Papp
Subject: calling compiled lambda forms
Date: 
Message-ID: <87ps3dbomu.fsf@pu100877.student.princeton.edu>
I am trying to call the result of compile.  It works fine when I pass
it as a parameter to mapcar:

CL-USER> (mapcar (lambda (x) (1+ x)) '(1 2 3))
(2 3 4)
CL-USER> (mapcar (compile nil (lambda (x) (1+ x))) '(1 2 3))
(2 3 4)

But not when I want to call it directly:

CL-USER> ((lambda (x) (1+ x)) 5)
6
CL-USER> ((compile nil (lambda (x) (1+ x))) 5)
; in: LAMBDA NIL
;     ((COMPILE NIL (LAMBDA (X) (1+ X))) 5)
; 
; caught ERROR:
;   illegal function call
; 
; compilation unit finished
;   caught 1 ERROR condition
; Evaluation aborted
CL-USER> (#'(compile nil (lambda (x) (1+ x))) 5)
; in: LAMBDA NIL
;     (#'(COMPILE NIL (LAMBDA (X) (1+ X))) 5)
; 
; caught ERROR:
;   illegal function call
; 
; compilation unit finished
;   caught 1 ERROR condition
; Evaluation aborted

I thought compile returned the compiled function -- I must be missing
something.

Tamas

From: Pascal Bourguignon
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <87hcopmvy9.fsf@informatimago.com>
Tamas Papp <······@gmail.com> writes:

> I am trying to call the result of compile.  It works fine when I pass
> it as a parameter to mapcar:
>
> CL-USER> (mapcar (lambda (x) (1+ x)) '(1 2 3))
> (2 3 4)
> CL-USER> (mapcar (compile nil (lambda (x) (1+ x))) '(1 2 3))
> (2 3 4)
>
> But not when I want to call it directly:
>
> CL-USER> ((lambda (x) (1+ x)) 5)
> 6
> CL-USER> ((compile nil (lambda (x) (1+ x))) 5)
> ; in: LAMBDA NIL
> ;     ((COMPILE NIL (LAMBDA (X) (1+ X))) 5)
> ; 
> ; caught ERROR:
> ;   illegal function call
> ; 
> ; compilation unit finished
> ;   caught 1 ERROR condition
> ; Evaluation aborted
> CL-USER> (#'(compile nil (lambda (x) (1+ x))) 5)
> ; in: LAMBDA NIL
> ;     (#'(COMPILE NIL (LAMBDA (X) (1+ X))) 5)
> ; 
> ; caught ERROR:
> ;   illegal function call
> ; 
> ; compilation unit finished
> ;   caught 1 ERROR condition
> ; Evaluation aborted
>
> I thought compile returned the compiled function -- I must be missing
> something.

Yes.  The syntax (f args...) doesn't accept a function for f.  Only a
function name or a lambda expression.

To call a function, you must use either FUNCALL or APPLY.

Note also that using a lambda expression in a function call
automatically compiles the function when you compile the expression.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.
From: Matthias Buelow
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <5enmj5F39hj7lU1@mid.dfncis.de>
Pascal Bourguignon wrote:

> Yes.  The syntax (f args...) doesn't accept a function for f.  Only a
> function name or a lambda expression.

Just out of curiosity.. does anyone know why lambda-expressions are
treated specially here? Historical reasons?
From: Kent M Pitman
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <u6455i1o2.fsf@nhplace.com>
Matthias Buelow <···@incubus.de> writes:

> Pascal Bourguignon wrote:
> 
> > Yes.  The syntax (f args...) doesn't accept a function for f.  

I don't Pascal really means what he wrote.  Accepting "a function" for f
would mean that 
 (#.#'car '(a b))
would be defined to work, returning A.  The reason that's not allowed has
to do with the fact that it would force a file compiler to handle a lot
more cases of externalizing data (and for anonymous functions, this would
be problematic).

I assume he means "a form", as in 
 (let ((x (list #'cons)))
   ((car x) (car x) (car x)))
which of course we write in CL as 
 (let ((x (list #'cons)))
   (funcall (car x) (car x) (car x)))

> > Only a function name or a lambda expression.
>
> Just out of curiosity.. does anyone know why lambda-expressions are
> treated specially here? Historical reasons?

Well, I was taught, back in the 70's, which wasn't exactly at the
birth of Lisp, but was early enough that the insight might help, that
a lambda expression IS a function, so that the sentence Pascal offered
is redundant, and lambda expressions are not a special case.  That is,
the name of the function that adds 1 to a number IS
(lambda (x) (+ x 1)), and that you can attach a symbolic name by using
defun, but that anonymous functions are just functions named by their
program.  This is the same reason that #'foo and #'(lambda (x) (+ x 1))
makes sense, since the FUNCTION special form takes function names.

Also, I'm pretty sure it's a latter-day contrivance to actually model
the first element of a lambda combination as evaluating to a function,
rather than as just representing the notion of a definition/call as
almost primitive itself.  No one ever talked about that being an
object without cringing at the efficiency loss that would imply.  (We
used to have discussions about which was "faster"--LET or LAMBDA--as
if one should choose these syntaxes for speed reasons, not for
expressional reasons.  That probably seems laughable now, but on a
machine with millisecond instruction times and 30 logged in users and
256K words [18bits] of addressable memory), speed and space mattered.

Another part of the puzzle is that even in Maclisp, there was no first
class datum for "function".  There were lists starting with LAMBDA.
And there were objects of type RANDOM (machine addresses,
effectively), but they were not typed in a way that was callable.  You
had to know they were callable and trust you were right and then use
SUBRCALL [call with the register protocol of a fixed number of
arguments] or LSUBRCALL [call with the protocol of a variable number
of arguments] on an object of type RANDOM.  That's why the symbol kept
its function value in myriad different properties, like SUBR and LSUBR
and FSUBR, and why there was no function cell but rather something
that searched the plist for the first of a zillion different
properties (such that the order of the properties on the property list
mattered a lot).  But all this to say that if a LAMBDA had evaluated
to an object in Maclisp, at least, the system would have had no idea
how to call it since the object itself was not tagged with enough
information to know.

It didn't matter anyway about evaluating to an object, though, because
in early Lisps, it was pragmatically undesirable because it would have
been slow to create and GC merely for the purpose of executing.  It
was (and I think continues to be in many implementations) just a piece
of special notation.  LAMBDA was not talked about as an operator but
rather as a special marker or token; the notion of LAMBDA as an
operator itself is a retroactively imposed fiction that papers over
the fact that it was not defined that way.  FUNCTION was the operator,
and not even powerful enough to return an upward closure until 1984
when the first CLTL forced it.  [At least in the Maclisp family; there
were other major branches of Lisp that I didn't follow as closely and
I might not be summarizing well, including Interlisp and PSL.  I don't
know if they agreed on these points but surely would have confronted
these issues.]  So, historically, lambda combinations were primitive
syntax one learned separately from lambda expressions, and the fact of
syntactic agreement between them was largely ceremonial, not
operational.

The notation originates, of course, with the lambda calculus, and
surely represented a function but the notion of notations, values, and
objects were blurred somewhat in the lambda calculus.  And anyway,
neither McCarthy's first stabs at Lisp nor subsequent dialects that
followed were literal implementations of the lambda calculus--they
were all just vaguely inspired by it.  In fact, it was not until
Scheme that Sussman/Steele tried to revive the notion of the lambda
calculus (and to merge it with ALGOL-style block structure and lexical
variables) that I think using Lisp to literally address issues in a
manner actively consistent with the lambda calculus was really
achieved.

I recall using some earlier interpreters for Lisp we used on a PDP-11
for class in one of Sussman's earlier courses, before Scheme, in which
he attempted to teach us issues like "normal order evaluation".  But I
don't recall how well those interpreters really got across the idea.

Or such is my recollection.  Others who have different recollections
should, as usual, feel free to correct me or add embellishment.
From: Barry Margolin
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <barmar-98CD4E.20134330062007@comcast.dca.giganews.com>
In article <···············@mid.dfncis.de>,
 Matthias Buelow <···@incubus.de> wrote:

> Pascal Bourguignon wrote:
> 
> > Yes.  The syntax (f args...) doesn't accept a function for f.  Only a
> > function name or a lambda expression.
> 
> Just out of curiosity.. does anyone know why lambda-expressions are
> treated specially here? Historical reasons?

Yes.  Before LET was invented, lambda expressions were the primary way 
to get local variables.  So the equivalent of

(let ((a 1))
  <body>)

was

((lambda (a) <body>) 1)

Another way to look at it is that the car of a function-call expression 
is the name of the function to call.  For functions defined with DEFUN 
it's the symbol that was defined; for anonymous functions, it's the 
lambda expression itself.

-- 
Barry Margolin, ······@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
From: Pascal Costanza
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <5en31jF399o2mU1@mid.individual.net>
Tamas Papp wrote:
> I am trying to call the result of compile.  It works fine when I pass
> it as a parameter to mapcar:
> 
> CL-USER> (mapcar (lambda (x) (1+ x)) '(1 2 3))
> (2 3 4)
> CL-USER> (mapcar (compile nil (lambda (x) (1+ x))) '(1 2 3))
> (2 3 4)
> 
> But not when I want to call it directly:
> 
> CL-USER> ((lambda (x) (1+ x)) 5)
> 6
> CL-USER> ((compile nil (lambda (x) (1+ x))) 5)
> ; in: LAMBDA NIL
> ;     ((COMPILE NIL (LAMBDA (X) (1+ X))) 5)
> ; 
> ; caught ERROR:
> ;   illegal function call

((lambda (x) (1+ x)) 5) is a very special way of defining a function and 
calling it at the same time. Common Lisp recognizes and handles it as a 
special case. (lambda (x) (1+ x)) may give you a function which you can 
then call, but this is not what happens here. Something like ((lambda 
(x) (1+ x)) 5) should better be restricted to internal uses (for 
example, as the result of macro expansion).

In general, the first position of a list form, like (foo 1 2 3), is 
always handled specially. It is checked whether it names a function, a 
macro or a special form, or whether it is a lambda form like in your 
example above. However, it is _not_ evaluated in any way like any of the 
other positions in a list form.

((compile ...) ...) is simply not among the kinds of things that are 
specially recognized when evaluating a list form.

The analogy you are looking for is rather (funcall (lambda ...) ...) vs. 
(funcall (compile nil (lambda ...)) ...). This should give you the 
semantics that you want.

HOWEVER:

There is no good reason to do what you seem to want to do. In your code 
examples (if they are expressed correctly), compilations will happen on 
each and every invocation of (compile nil ...). You typically want to 
compile code to get better efficiency out of it, but you have to keep in 
mind that compilation itself is a costly process. So you only benefit 
from compilation when you ensure that a piece of code is compiled far 
less often than it is actually invoked. In the usual edit-compile-run 
cycle, this is quite straightforward because there, compilation isn't 
part of program execution. But as soon as you make compilation part of 
the actual program execution itself, that is you compile during runtime, 
you have to invest a lot more into the design of your program so that 
you actually benefit from this. If you don't do this properly, there is 
a relatively high degree of likeliness that your program will actually 
become slower and even consume more memory, because compilation takes 
time and memory.

So don't try to be smarter than the Common Lisp compiler here. Just use 
(lambda ...) instead of (compile nil (lambda ...)), and you'll be fine. ;)


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Tamas Papp
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <87d4zdfseu.fsf@pu100877.student.princeton.edu>
Pascal Costanza <··@p-cos.net> writes:

> HOWEVER:
>
> There is no good reason to do what you seem to want to do. In your
> code examples (if they are expressed correctly), compilations will
> happen on each and every invocation of (compile nil ...). You
> typically want to compile code to get better efficiency out of it, but
> you have to keep in mind that compilation itself is a costly
> process. So you only benefit from compilation when you ensure that a
> piece of code is compiled far less often than it is actually
> invoked. In the usual edit-compile-run cycle, this is quite
> straightforward because there, compilation isn't part of program
> execution. But as soon as you make compilation part of the actual
> program execution itself, that is you compile during runtime, you have
> to invest a lot more into the design of your program so that you
> actually benefit from this. If you don't do this properly, there is a
> relatively high degree of likeliness that your program will actually
> become slower and even consume more memory, because compilation takes
> time and memory.
>
> So don't try to be smarter than the Common Lisp compiler here. Just
> use (lambda ...) instead of (compile nil (lambda ...)), and you'll be
> fine. ;)

I am not writing any program, just going through On Lisp and playing
around in the REPL.

Thanks for the advice,

Tamas
From: Pillsy
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <1183222468.422588.67280@m36g2000hse.googlegroups.com>
On Jun 30, 10:06 am, Tamas Papp <······@gmail.com> wrote:
[...]
> I am not writing any program, just going through On Lisp and playing
> around in the REPL.

Which implementation are you using? Two of the more popular free
implementations (SBCL and OpenMCL) just compile everything, at least
by default.

Cheers,
Pillsy
From: Tamas Papp
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <871wftfgfu.fsf@pu100877.student.princeton.edu>
Pillsy <·········@gmail.com> writes:

> On Jun 30, 10:06 am, Tamas Papp <······@gmail.com> wrote:
> [...]
>> I am not writing any program, just going through On Lisp and playing
>> around in the REPL.
>
> Which implementation are you using? Two of the more popular free
> implementations (SBCL and OpenMCL) just compile everything, at least
> by default.

I am using SBCL.  While the compilation of defuns is indicated, the
fact that lambda forms are complied was not apparent so I didn't know.

Thanks for the clarification,

Tamas
From: Rob Warnock
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <oOWdnYYc59KtgBrbnZ2dnUVZ_rDinZ2d@speakeasy.net>
Tamas Papp <······@gmail.com> wrote:
+---------------
| Pascal Costanza <··@p-cos.net> writes:
| > There is no good reason to do what you seem to want to do. ...
| 
| I am not writing any program, just going through On Lisp and playing
| around in the REPL.
+---------------

O.k., in that case others have answered you -- FUNCALL is your friend:

    > (funcall (compile nil (lambda (x y) (1- (expt x y))))
	       2
	       100)
    ; Compiling LAMBDA (X Y): 
    ; Compiling Top-Level Form: 

    1267650600228229401496703205375
    > 


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Kent M Pitman
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <uk5tkg4kw.fsf@nhplace.com>
····@rpw3.org (Rob Warnock) writes:

> Tamas Papp <······@gmail.com> wrote:
> +---------------
> | Pascal Costanza <··@p-cos.net> writes:
> | > There is no good reason to do what you seem to want to do. ...
> | 
> | I am not writing any program, just going through On Lisp and playing
> | around in the REPL.
> +---------------
> 
> O.k., in that case others have answered you -- FUNCALL is your friend:
> 
>     > (funcall (compile nil (lambda (x y) (1- (expt x y))))
> 	       2
> 	       100)
>     ; Compiling LAMBDA (X Y): 
>     ; Compiling Top-Level Form: 
> 
>     1267650600228229401496703205375

With the caveat that compiling just to call something once is horribly 
expensive.  At least TRY to do

 (defvar *my-function* (compile nil '(lambda (x y) ...)))

and then later when you need it, just call up the memory of that compiled
function.

 (funcall *my-function* ...)

so that you can use the function's compilation a second time. :)
Moreover, this use is something you might sometimes see in automatic
programming situations--where you explicitly invoke the compiler after
constructing a function from data and then store the compiled function
someplace for faster access.  But USUALLY you know the information at
an earlier time and should do a compilation once before the function
is loaded, not every time you load it.

Note that the case of doing (compile nil (lambda ...)) [without the single
quote mark, as Rob wrote it] is rare because it supposes you already 
had reason to call EVAL on the function, which means you either could 
have compiled it then or else you already knew at programming time what
the text was, begging the question: why did you not just say 
 (defun my-function (x y) ...)
and later call it as
 (my-function ...)
?  Only if you've actually consed it up newly at runtime does it make
a huge lot of sense for you to be still calling COMPILE on it, since 
generally only in that case is the definition a surprise at runtime.
(At least as far as I could imagine in a couple minutes thought just now.
Maybe I'm not thinking of some case.)
From: Rob Warnock
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <D7CdnWKpg5uw3xrbnZ2dnUVZ_vmqnZ2d@speakeasy.net>
Kent M Pitman  <······@nhplace.com> wrote:
+---------------
| ····@rpw3.org (Rob Warnock) writes:
| > Tamas Papp <······@gmail.com> wrote:
| > +---------------
| > | I am not writing any program, just going through On Lisp and playing
| > | around in the REPL.
| > +---------------
| > 
| > O.k., in that case others have answered you -- FUNCALL is your friend:
| >     > (funcall (compile nil (lambda (x y) (1- (expt x y)))) 2 100)
| >     ; Compiling LAMBDA (X Y): 
| >     ; Compiling Top-Level Form: 
| >     1267650600228229401496703205375
| 
| With the caveat that compiling just to call something once is horribly 
| expensive.  At least TRY to do
|  (defvar *my-function* (compile nil '(lambda (x y) ...)))
+---------------

Yes, yes, but he *did* say he was just playing around in the REPL.  ;-}  ;-}

I confess that even I do that sometimes, although usually for rather
different reasons. E.g., the other day I found myself typing this
several times in a row:

    > (disassemble (compile nil (lambda () (dotimes (i 2000000000)))))
    ...[unimportant stuff]...
	 BF1: L0:   INC     EAX                  ; [:BLOCK-START]
	 BF2: L1:   MOV     ECX, EAX             ; [:BLOCK-START]
	 BF4:       CMP     ECX, 2000000000
	 BFA:       JB      L0
    ...[unimportant stuff]...
    > 

to try to figure out why (TIME (DOTIMES (I 2000000000))) *sometimes*
takes 2 CPU clock cycles per iteration on my Athlon and other times
takes 3 clocks. [Tentative conclusion: It takes 3 cycles/count when
the inner loop happens to span a cache block boundary.]


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Kent M Pitman
Subject: Re: calling compiled lambda forms
Date: 
Message-ID: <uwsxkuqq3.fsf@nhplace.com>
····@rpw3.org (Rob Warnock) writes:

> I confess that even I do that sometimes, although usually for rather
> different reasons. E.g., the other day I found myself typing this
> several times in a row:
> 
>     > (disassemble (compile nil (lambda () (dotimes (i 2000000000)))))

Oh, clearly it's useful for interactive debugging.  I just worried newbies
would miss the subtle context from upthread.

And, incidentally, for better or worse, I think DISASSEMBLE does the
compile for you, so you don't have to do that.  From CLHS's entry on
DISASSEMBLE:

  If fn is a lambda expression or interpreted function, 
  it is compiled first and the result is disassembled. 

  If the fn designator is a function name, the function 
  that it names is disassembled. (If that function is an
  interpreted function, it is first compiled but the 
  result of this implicit compilation is not installed.)

But lest you think I'm saying there's no reasonable situation where you'd
do this interactively, I'll point out that I often do:

 (progn (compile (defun foo () ...)) (time (foo)))

when timing things interactively (to avoid having TIME capture the time it
takes to do macroexpansion).  I wouldn't, of course, do this in a delivered
application.

So again I think we're in agreement.  I'm probably just more paranoid
about making explicit disclaimers.  Sorry about that.