From: ··········@tfeb.org
Subject: Re: Lisp compilation
Date: 
Message-ID: <cg2u1b$drn@odah37.prod.google.com>
Tim Lavoie wrote:

> Does it have some parts of Motif compiled statically? The program
> appears to try and read parts of itself once it is started. Would you
> have any idea what it is doing?

I'm fairly sure it's loading libraries with dlopen, so you won't see
them with ldd.

From: Tim Lavoie
Subject: Re: Lisp compilation
Date: 
Message-ID: <87k6vuq320.fsf@theasylum.dyndns.org>
>>>>> "tfb" == tfb  <··········@tfeb.org> writes:

    tfb> Tim Lavoie wrote:
    >> Does it have some parts of Motif compiled statically? The
    >> program appears to try and read parts of itself once it is
    >> started. Would you have any idea what it is doing?

    tfb> I'm fairly sure it's loading libraries with dlopen, so you
    tfb> won't see them with ldd.

Ah, good catch. I did a bit of poking around, and does use dlopen, but
it's a bit more interesting yet.

It:

  - finds its own path, and reads in a chunk of itself.

  - does a dlopen(NULL), treating itself as a libary, and mapping
  functions to there.

  - looks for libXm in a couple places, doesn't find them on my system.

  - creates a new temporary file, and writes to it, then opens that as
  a library.

  - does a dlopen() on the new temp file, and reads all the Motif bits
  out of there.


That's pretty nifty altogether, and a handy way of packaging the
minimal pieces required for the app into one binary.
From: Tim Lavoie
Subject: Re: Lisp compilation
Date: 
Message-ID: <874qmyq0rc.fsf@theasylum.dyndns.org>
Minor self correction... a couple of Motif libraries are found, and
read from directly, not dlopen().

Note to self: keep aware of both ltrace and strace...


-- 
The trouble with computers is that they do what you tell them, not what
you want.
                -- D. Cohen
From: Jim Newton
Subject: Re: Lisp compilation
Date: 
Message-ID: <2okkooFbnt48U1@uni-berlin.de>
can someone tell me what the difference in byte code generation
and compilation is?  a friend of mine claims that cmucl is not
really compiled because it does not produce an ELF file
in UNIX.  anyone know whether that is the case... is there
any way to look at a fasl file and tell if it is really
compiled?

-jim


Tim Lavoie wrote:
> Minor self correction... a couple of Motif libraries are found, and
> read from directly, not dlopen().
> 
> Note to self: keep aware of both ltrace and strace...
> 
> 
From: Zach Beane
Subject: Re: Lisp compilation
Date: 
Message-ID: <m3y8ka945s.fsf@unnamed.xach.com>
Jim Newton <·····@rdrop.com> writes:

> can someone tell me what the difference in byte code generation
> and compilation is?  a friend of mine claims that cmucl is not
> really compiled because it does not produce an ELF file
> in UNIX.  anyone know whether that is the case... is there
> any way to look at a fasl file and tell if it is really
> compiled?

If your friend defines compilation as "producing an ELF file in UNIX",
you may have trouble convincing him of much of anything.

Compilation is often defined as translating from one language to
another. Producing byte codes from Lisp is compilation, and producing
native code from Lisp is compilation. Producing C or Java from Lisp is
also compilation.

Native code compilation is pretty common in Lisp. I found it to be
really eye-opening to use DISASSEMBLE to look at the x86 assembly code
of various functions, especially functions I defined myself.

Here's a really trivial example:

   > (defun double-and-1+ (n)
       (declare (type (unsigned-byte 16) n))
       (logand #xFFFF (1+ (* n 2))))

   DOUBLE-AND-1+
   
   > (compile 'double-and-1+)
   DOUBLE-AND-1+
   NIL
   NIL

   > (disassemble 'double-and-1+)
   ; 09432548:       MOV EDX, EBX                ; no-arg-parsing entry point
   ;       4A:       SAR EDX, 2
   ;       4D:       SHL EDX, 1
   ;       4F:       INC EDX
   ;       50:       AND EDX, 65535
   ;       56:       SHL EDX, 2
   ;       59:       MOV ECX, [EBP-8]
   ;       5C:       MOV EAX, [EBP-4]
   ;       5F:       ADD ECX, 2
   ;       62:       MOV ESP, EBP
   ;       64:       MOV EBP, EAX
   ;       66:       JMP ECX
   ;       68:       BREAK 10                    ; error trap
   ;       6A:       BYTE #X02
   ;       6B:       BYTE #X16                   ; INVALID-ARG-COUNT-ERROR
   ;       6C:       BYTE #X4D                   ; ECX
   ; 
   NIL

You don't have to know a whole lot of x86 assembly to see it's
performing the lisp code almost verbatim with x86 operations, so the
above stuff is basically what DOUBLE-AND-1+ looks like when compiled
to machine code. My system runs the function about ten million times
in a second. 

Zach
From: Pascal Bourguignon
Subject: Re: Lisp compilation
Date: 
Message-ID: <877jru6amb.fsf@thalassa.informatimago.com>
Jim Newton <·····@rdrop.com> writes:

> can someone tell me what the difference in byte code generation
> and compilation is? 

Absolutely none.


> a friend of mine claims that cmucl is not
> really compiled because it does not produce an ELF file
> in UNIX.  anyone know whether that is the case... is there
> any way to look at a fasl file and tell if it is really
> compiled?

So older unix compiler that produced COFF files instead of ELF where
not really compilers...

Compilation is merely the translation of a program text in language L1
to a program text in language L2.  Usually, L1 is higher level
language than L2, but it's not needed and "higher level" is a fuzzy
notion anyway.  Some think that such a translation can be named
"compilation" only when there is a hardware implementation of an
interpreter for the language L2 (so called "native code"), but the
existance of micro-code, emulators, and virtual machines, and the fact
that there is no difference in architecture between what they'd accept
as a "compiler" and what they'd not accept as a "compiler", prove that
they're wrong.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we.
From: Edi Weitz
Subject: Re: Lisp compilation
Date: 
Message-ID: <87isbeu5ug.fsf@bird.agharta.de>
On Thu, 19 Aug 2004 22:37:09 +0200, Jim Newton <·····@rdrop.com> wrote:

> can someone tell me what the difference in byte code generation and
> compilation is?

The question should rather be what the difference between compilation
to bytecode and compilation to native machine code is. In both cases
the (Lisp) source code is compiled (translated) into another
representation.

In the first case this representation is called "bytecode" and what is
meant is an internal representation that needs to be interpreted by
another program in order to be executed. That's the way Perl, Java[1],
or CLISP work.

In the second case the code is compiled to machine code - a sequence
of bits which can be directly understood and executed by your
processor. This is what CMUCL, SBCL, AllegroCL, LispWorks, OpenMCL,
and various other Lisp implementations do. Zach already talked about
the DISASSEMBLE function which can be used to look at the code that is
created by your Lisp if you compile a function.

> a friend of mine claims that cmucl is not really compiled because it
> does not produce an ELF file in UNIX.

If he has said that he obviously doesn't know what he's talking
about. ELF is a file format which can be used to create "executable"
files meaning that the operating system happens to know how to create
a process image from this file. It happens to be the default binary
file format on Linux and most Unix systems but that's about all. You
can easily build a Linux kernel which can also execute a.out binaries,
and you can also make CMUCL FASL files "executable:"

  <http://www.cons.org/cmucl/doc/executable.html>

Edi.

[1] except for JIT compilation