From: Simon Leinen
Subject: Why Lisp is "less efficient" than C in practice
Date: 
Message-ID: <SIMON.92Jul6120515@liasg1.epfl.ch>
In article <··················@informatik.uni-hannover.de>
··@informatik.uni-hannover.de (Michael Sperber) writes:

   Ken Dickey writes:
   >Huh?  To which reality are you referring?

   I was referring to the sad reality that most of the EXISTING Lisp
   systems I know are in fact written in C, and that even compiled
   code usually doesn't match up to C performance.

I know a couple of Lisp systems, and very few of them are "written in
C".  Some have an OS interface layer written in C, but the operations
on Lisp data structures are usually implemented in terms of primitive
pointer operations (primops) which the Lisp compiler compiles directly
into target machine code, so that nearly all of the system can be
written in "Lisp" (syntactically Lisp, but with low-level operations).

Here are some of the reasons why Lisp is generally perceived "less
efficient" than C.  One reason is of course that Lisp offers some
functionality not available in the average C environment, which is
convenient to use but not very efficient.  This is especially true for
Common Lisp, but Scheme also offers many "performance traps" such as
implementing data structures in terms of lists.

Dynamic typing, in particular generic arithmetic.  Type declarations
can help a lot (in Lisps that have them), but many people don't like
to clutter their code with these, especially because most Lisp
compilers need type declarations all over the place, most of which are
in fact redundant.  Also, it is somewhat counter-intuitive that a
harmless expression such as (1+ FOO) is compiled into something
horrible in the absence of type information for FOO.

Function call is usually also less efficient in Lisp than in C,
because of "varadic" functions; even if you never use them, there is a
runtime penalty for them.  Again, Common Lisp offers the possibility
of freezing a function's signature by means of (DECLAIM (FTYPE ...)),
so that the compiler could generate C-like function entry/exit.
Again, hardly anybody uses this in practice.  A problem is that the
declamation has to precede the function definition and calls to make
life easier for the compiler, which is not nice from a Software
Engineering point of view (you'd rather not want to distribute related
definitions over files etc.).

"Named" Lisp function calls (the common case) are slowed down even
more by the fact that functions can always be redefined; where a C
named function call is implemented by a jump subroutine instruction to
a "fixed" (at link time) address, the Lisp function call has to fetch
the function cell of the symbol that was used as the function name.
This is evil not only because a couple more instructions are involved,
but especially since the additional indirection results in additional
memory being accessed, and thus contributes to caching and virtual
memory overhead.  Some Lisps have "block compilation" modes which
"nail down" function definitions.  Unfortunately this is not
necessarily sufficient to resolve target addresses statically, since
the code objects may be moved by the GC.

Closures and garbage collection add further to general funcall
inefficiency, because a "function" is usually implemented as a pointer
to an object containing (1) a *pointer* to a chunk of machine code and
(2) the constants and [pointers to cells for] closed-over variables of
the function.  This means another indirection for every funcall
(thrash, thrash), even if you don't close over anything in your
program.

   Well, abstraction support is a weak point in C.  However, that's
   another point where much can be done in C.  (Also, much has to be
   done in R4RS Scheme - no 1st class closures or environments.)

No 1st class closures in R4RS Scheme? I don't think anything without
first class closures could be called Scheme.

   Icon - high level symbol & text manipulation
   Scheme - dynamic program systems
   C - device drivers, numerics, system libraries
   Mathematica - symbolic maths (well, what else)
   Pascal - homework assignments if must be

   What's your language which scores on all of the above points?

Common Lisp (as you might have guessed by now :-).

Ideally, one should be able to develop and debug programs under a full
Common Lisp or Scheme environment, with all the benefits of being able
to look at the state of your program under the debugger, change a
function/method/class definition, restart from some stack frame etc.
(This is what many CL environments provide today, and what makes CL my
language of choice for many applications that are neither time- nor
space-critical.)

But after you have everything running, you should be able(*) to
compile your program to a small standalone chunk of code that really
*flies*: If your program doesn't redefine functions, then the binary
should not look at the symbols for function calls.  If your program
doesn't intern symbols, the resulting binary should not even have
packages.  If your program does not use Plists, then the symbols in
the binary won't have plist cells, if it doesn't print symbols, then
the symbols won't have pname cells etc.  If you don't need symbols
(and many applications can quite well be written without them), then
there won't be any symbols in the resulting process.  I'm not aware of
any Lisp "delivery" systems that go that far, but they could be
written (Just A Simple Matter Of Programming ;-).  This would be my
ideal programming environment for nearly everything, including device
drivers.

Regards,
-- 
Simon.
*) By adding a reasonable amount of type decorations, declaring the
   appropriate metaclasses for your data structures (for CLOS), and
   selecting "production mode".  The system must permit to revert to
   "development mode" without having to change your "production" code.