From: vishnuvyas
Subject: newbie question on making executables in Lisp.
Date: 
Message-ID: <1130702087.297886.40570@g47g2000cwa.googlegroups.com>
I have used most of the free distros of lisps i can find including
SBCL,CMUCL, CLISP, etc.

Even though both SBCL and CMUCL are written primarily in lisp and can
compile themselves, and they both only produce .fas/.x86f type files on
compiling. However I always find that there is an elf type executable I
am compiling SBCL/CMUCL. How is that possible? Is there anyway of
making sbcl/cmucl generate executables directly?

Is there a standard procedure for doing this in other Common Lisps.
(SBCL site says it can even build from CLISP, which doesn't even
produce intel x86 code.)

Cheers
Vishnu

From: Pascal Bourguignon
Subject: Re: newbie question on making executables in Lisp.
Date: 
Message-ID: <878xwar48c.fsf@thalassa.informatimago.com>
"vishnuvyas" <··········@gmail.com> writes:

> I have used most of the free distros of lisps i can find including
> SBCL,CMUCL, CLISP, etc.
>
> Even though both SBCL and CMUCL are written primarily in lisp and can
> compile themselves, and they both only produce .fas/.x86f type files on
> compiling. However I always find that there is an elf type executable I
> am compiling SBCL/CMUCL. How is that possible? Is there anyway of
> making sbcl/cmucl generate executables directly?
>
> Is there a standard procedure for doing this in other Common Lisps.
> (SBCL site says it can even build from CLISP, which doesn't even
> produce intel x86 code.)

This is the classical compiler bootstrapping.

A machine (processor, virtual machine, interpreter) is symbolized as a
triangle:


                     +------+
                      \ix86/
                       \  /
                        \/

                     +------+
                      \ CL / (clisp interpreter for example)
                       \  /
                        \/


A compiler is symbolized by a T:

            +-----------------+
            | CL         ix86 |
            +----+ SBCL +-----+   SBCL compiler sources written in CL
                 |      |         takes a program written in CL and
                 |  CL  |         produces a programme written in ix86
                 +------+      

You want to obtain this:

            +-----------------+
            | CL         ix86 |
            +----+ SBCL +-----+   SBCL compiler compiled for ix86
                 |      |
                 | ix86 |
                 +------+


To compile SBCL with SBCL, you just need to have a CL interpreter:


                                      
            +-----------------+      +-----------------+
            | CL    #1   ix86 |      | CL    #2   ix86 |
            +----+ SBCL +-----+      +----+ SBCL +-----+
                 |      |                 |      |      
                 |  CL  |     ----->      | ix86 |      
                 +------+                 +------+      
                 +-------------------------------+      
                 | CL            #1         ix86 |      
                 +------------+ SBCL +-----------+
                              |      |            
                              |  CL  |            
                              +------+            
                              +------+            
                               \ CL / 
                                \  /  
                                 \/   

The T on the left and the T below are the same (#1): SBCL compiles
itself into a binary executable #2 with the help of a CL interpreter.

If you don't have a CL interpreter, you can use a different compiler.


                 +-----------------+ +-----------------+  
                 | CL    #1   ix86 | | CL   #3    ix86 | 
                 +----+ SBCL +-----+ +----+ SBCL +-----+  
                      |      |            |      |        
                      |  CL  |  ---->     | ix86 |        
                      +------+            +------+        
+-----------------+   +--------------------------+        
| CL   #1    ix86 |   |  CL      #2         ix86 |        
+----+ SBCL +-----+   +-------+ SBCL +-----------+        
     |      |                 |      |                    
     |  CL  |     ---->       | ix86 |                    
     +------+                 +------+                    
     +-------------------------------+                    
     | CL            #0         ix86 |                    
     +------------+ CMUCL+-----------+                    
                  |      |                                
                  | ix86 |                                
                  +------+                                
                  +------+                                
                   \ix86/                                 
                    \  /                                    
                     \/                                   

Here, we compile SBCL sources (#1) first with the CMUCL compiler (#0),
obtaining a binary SBCL (#2). Since this binary has not been compiled
by SBCL itself, we use it to compile the SBCL sources (#1) once more,
to obtain the final SBCL (#3) which has been compiled by itself.


If you ever compile gcc, you'll see too that it compiles itself
several times, with a bootstrap stage, and two or three other stages.
In particular, having the compiler compile itself once more than what
I've shown in the above diagrams allow us to compare the output of the
last two stage: they should be identical.

--                    
__Pascal Bourguignon__                     http://www.informatimago.com/

In a World without Walls and Fences, 
who needs Windows and Gates?
From: vishnuvyas
Subject: Re: newbie question on making executables in Lisp.
Date: 
Message-ID: <1130716533.361794.34850@g47g2000cwa.googlegroups.com>
Pascal Bourguignon wrote:
 [snip]

thanks for the detailed reply, I am get all of that, but that still
doesn't answer
my basic question.

1. Any of CLISP/CMUCL/SBCL do-not generate ELF type executables on
linux systems directly with the (compile-file) function. but how is
that when they are built, they are able to make themselves ELF
executables?

>
> --
> __Pascal Bourguignon__                     http://www.informatimago.com/
>
> In a World without Walls and Fences, 
> who needs Windows and Gates?
From: Pascal Bourguignon
Subject: Re: newbie question on making executables in Lisp.
Date: 
Message-ID: <874q6yr02r.fsf@thalassa.informatimago.com>
"vishnuvyas" <··········@gmail.com> writes:

> Pascal Bourguignon wrote:
>  [snip]
>
> thanks for the detailed reply, I am get all of that, but that still
> doesn't answer
> my basic question.
>
> 1. Any of CLISP/CMUCL/SBCL do-not generate ELF type executables on
> linux systems directly with the (compile-file) function. but how is
> that when they are built, they are able to make themselves ELF
> executables?

Well CLISP is a C program, so it has no difficulty to be linked into a
ELF executable.

SBCL is mostly a Common Lisp program, and indeed, while it compiles to
ix86 (or other processors), Common Lisp programs are stored into lisp
images or .fasl files, not into a ELF file.  In SBCL there's a small C
"loader" program that is linked into an ELF executable and that loads
the lisp image.

CMUCL is similar to SBCL, only its build process is more complex.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

This is a signature virus.  Add me to your signature and help me to live
From: Jamie Border
Subject: Re: newbie question on making executables in Lisp.
Date: 
Message-ID: <dk3qss$7cl$1@nwrdmz02.dmz.ncs.ea.ibs-infra.bt.com>
"Pascal Bourguignon" <····@mouse-potato.com> wrote:

> "vishnuvyas" <··········@gmail.com> writes:
>
>> Pascal Bourguignon wrote:
>>  [snip]
>>
>> thanks for the detailed reply, I am get all of that, but that still
>> doesn't answer
>> my basic question.
>>
>> 1. Any of CLISP/CMUCL/SBCL do-not generate ELF type executables on
>> linux systems directly with the (compile-file) function. but how is
>> that when they are built, they are able to make themselves ELF
>> executables?
>
> Well CLISP is a C program, so it has no difficulty to be linked into a
> ELF executable.
>
> SBCL is mostly a Common Lisp program, and indeed, while it compiles to
> ix86 (or other processors), Common Lisp programs are stored into lisp
> images or .fasl files, not into a ELF file.  In SBCL there's a small C
> "loader" program that is linked into an ELF executable and that loads
> the lisp image.

Before I got to the point where I didn't care, I was thinking about the 
following:

a) Hacking the build process to produce an ELF "custom-loader", which would 
load an image from a .data segment.
b) Writing a drop-in replacement for the above, thereby not f*cking with the 
build at all.

Before I got round to doing it, I stopped caring.about this.

However, it's still kicking around in the back of my head, as I'm going to 
have to produce some genuine stand-alone executables at some point for an 
obscure reason (limited to one ELF binary, read-only filesystem, so no temp 
files, yada yada yada).

I guess I (or the OP) should ask the SBCL/CMUCL guys about the best way to 
go about it...

>
> CMUCL is similar to SBCL, only its build process is more complex.

Hell yes.


Jamie
--

>
>
> -- 
> __Pascal Bourguignon__                     http://www.informatimago.com/
>
> This is a signature virus.  Add me to your signature and help me to live