From: Pierre THIERRY
Subject: Implementation of CL data structures
Date: 
Message-ID: <pan.2006.07.18.10.25.59.538015@levallois.eu.org>
As it seems to me that any data structure I know can be implemented if
you have scalars, structures and efficient indice-accessed containers
(typically contiguous allocated space), I'm wondering if CL implementors
tend to implement the various data structures of CL in CL itself.

That is, does the typical CL implementor that implements it in C also
write everything that is mandated by the CL standard in C, or the bare
minimum, or something in-between?

Curiously,
Nowhere man
-- 
···········@levallois.eu.org
OpenPGP 0xD9D50D8A

From: Lars
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <e9idsi$q85$1@info.service.rug.nl>
Pierre THIERRY wrote:
> That is, does the typical CL implementor that implements it in C also
> write everything that is mandated by the CL standard in C, or the bare
> minimum, or something in-between?

Somewhere in-between, with the C/Lisp ratio probably higher for 
interpreters, which need the extra bit of speed.

Here are some very crude figures, the number of Lisp and C source files 
in three OS CL implementations:

        CLISP  CMUCL  SBCL
.[ch]    429    130   139
.lisp    189    855   786

(Note: I've included the .d files in CLISP.)
From: Pascal Bourguignon
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <87mzb7p23t.fsf@thalassa.informatimago.com>
Lars <··@spam.please> writes:

> Pierre THIERRY wrote:
>> That is, does the typical CL implementor that implements it in C also
>> write everything that is mandated by the CL standard in C, or the bare
>> minimum, or something in-between?
>
> Somewhere in-between, with the C/Lisp ratio probably higher for
> interpreters, which need the extra bit of speed.
>
> Here are some very crude figures, the number of Lisp and C source
> files in three OS CL implementations:
>
>        CLISP  CMUCL  SBCL
> .[ch]    429    130   139
> .lisp    189    855   786
>
> (Note: I've included the .d files in CLISP.)

LoC would still be more significant: CLISP source files are BIG! (and
compact, and use a lot of macrology, so the .c generated from the .d
are even bigger).

And I've got the impression that it's the exception to implement Lisp
in C.  Of course, there's always a few C files, for the interface with
the POSIX or UNIX VM, but most implementations of lisp are implemented
in lisp.  

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -- Georges W. Bush
From: Lars
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <e9ikri$s5m$1@info.service.rug.nl>
Pascal Bourguignon wrote:
> And I've got the impression that it's the exception to implement Lisp
> in C.  Of course, there's always a few C files, for the interface with
> the POSIX or UNIX VM, but most implementations of lisp are implemented
> in lisp.  

But to interface well to C code, shouldn't things like records, cons 
cells, vectors, objects etc. all have a C representation so C code can 
mess with them?

(Sorry if I use non-CL names, I'm a Schemer :))
From: Alexander
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <1153237687.799464.112140@s13g2000cwa.googlegroups.com>
Lars wrote:
> But to interface well to C code, shouldn't things like records, cons
> cells, vectors, objects etc. all have a C representation so C code can
> mess with them?
>
Yes,And to infterface well to C++ code, we should have vtbl-pointers ,
function name mangling and so on
http://www.codesourcery.com/cxx-abi/abi.html... :)))
From: Pierre THIERRY
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <pan.2006.07.18.16.15.05.898901@levallois.eu.org>
Le Tue, 18 Jul 2006 15:06:41 +0200, Lars a écrit :
> But to interface well to C code, shouldn't things like records, cons
> cells, vectors, objects etc. all have a C representation so C code can
> mess with them?

That's the role of a FFI, I think. Some Lisp interface with FORTRAN, I
bet they don't double-implement anything in C and in FORTRAN! ;-)

That is, I don't think any FFI could just pass to a C function the
in-memory representation of a Lisp entity, but instead has to do a bunch
of adaptations (at least when it comes to pointers and memory
management). So I'm not sure it really gives a great advantage to
implement it in Lisp.

But I'm anything but a FFI guru, so I may be plain wrong here (if I am,
I'd be glad to know, in fact).

Doubtfully,
Nowhere man
-- 
···········@levallois.eu.org
OpenPGP 0xD9D50D8A
From: ···············@gmail.com
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <1153473989.887717.50670@i42g2000cwa.googlegroups.com>
Lars wrote:
> But to interface well to C code, shouldn't things like records, cons
> cells, vectors, objects etc. all have a C representation so C code can
> mess with them?

Yes, but that can be generated automatically from the Lisp description.
 In SBCL, for example, the layout of "builtin" objects is set by
DEFINE-PRIMITIVE-OBJECT macros in src/compiler/generic/objdef.lisp, and
part of the build process is to create equivalent C representations in
src/runtime/genesis/*.h

Incidentally, the immediate goal here is not to make life easier for
third-party C code - these header files are not installed in any public
directory -  it's so that the runtime system and GC (both written in C)
know what/where the objects in memory are.

(Disclaimer: was true as of 0.9.11.30; details may have changed since)

-dan
From: funkyj
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <1153521584.319047.179970@i42g2000cwa.googlegroups.com>
Pascal Bourguignon wrote:
> Lars <··@spam.please> writes:
>
> > Pierre THIERRY wrote:

> > Here are some very crude figures, the number of Lisp and C source
> > files in three OS CL implementations:
> >
> >        CLISP  CMUCL  SBCL
> > .[ch]    429    130   139
> > .lisp    189    855   786


> And I've got the impression that it's the exception to implement Lisp
> in C.  Of course, there's always a few C files, for the interface with
> the POSIX or UNIX VM, but most implementations of lisp are implemented
> in lisp.

Is that the case with CLisp, CMUCL and/or SBCL?  Are the C files in
those CLs only for interfacing with external C libraries?

  --jfc
From: Rob Warnock
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <3LSdnQW_V-UfCFzZnZ2dnUVZ_t2dnZ2d@speakeasy.net>
funkyj <······@gmail.com> wrote:
+---------------
| Pascal Bourguignon wrote:
| > Lars <··@spam.please> writes:
| > > Here are some very crude figures, the number of Lisp and C source
| > > files in three OS CL implementations:
| > >        CLISP  CMUCL  SBCL
| > > .[ch]    429    130   139
| > > .lisp    189    855   786
| 
| > And I've got the impression that it's the exception to implement Lisp
| > in C.  Of course, there's always a few C files, for the interface with
| > the POSIX or UNIX VM, but most implementations of lisp are implemented
| > in lisp.
| 
| Is that the case with CLisp, CMUCL and/or SBCL?  Are the C files in
| those CLs only for interfacing with external C libraries?
+---------------

Since it only provides a byte-compiler, CLISP uses much more C than
the other two -- most of the control & data "primitives". Still, quite
a lot of CLISP is written in CL [e.g., the compiler, FORMAT, etc.].

But almost *all* of CMUCL & SBCL -- *including* control & data
"primitives" -- is written in Common Lisp, which is hard-compiled
directly to machine code which is then included in the startup
heap image -- ~24 MB, for CMUCL-19c. The C files are used only to
build the *much* smaller -- ~260 KB, for CMUCL-19c -- executable
file which "loads" (mmap's, really) the heap image and then jumps
to it. The C code also contains a relatively small amount of machine
and/or OS architecture-dependent routines[2] (e.g., how to enter
and return from an interrupt and the related "setsigcontext" stuff,
how to install/remove a machine-language breakpoint, or do a system
call), some and -- last but not least! -- the garbage collector.
All in all, barely 1% of the bits in a freshly loaded copy of CMUCL
were coded in C. [SBCL should be similar.]


-Rob

[1] Including the compiler itself, which is why bootstrapping
    or rebuilding one of these is "interesting".  ;-}  Though
    at least SBCL can now use some *other* CL implementation
    besides itself for such bootstrapping. [CMUCL badly needs
    that feature too, IMHO.]

[2] This is why "number of C files" is a very bad metric for
    CMUCL & SBCL, since there are many copies of the "same"
    files in the distribution, only one of each particular kind
    needed for a given OS and/or machine architecture, e.g.:

       $ cd cmucl-19c/src/lisp
       $ ls *arch*
       alpha-arch.c  arch.h       mips-arch.c  search.c  sparc-arch.c
       amd64-arch.c  hppa-arch.c  ppc-arch.c   search.h  x86-arch.c
       $ ls *os*
       Config.alpha_osf1  Linux-os.h    hpux-os.h    os.h
       Darwin-os.c        NetBSD-os.c   irix-os.c    osf1-os.c
       Darwin-os.h        NetBSD-os.h   irix-os.h    osf1-os.h
       FreeBSD-os.c       OpenBSD-os.c  mach-os.c    solaris-os.c
       FreeBSD-os.h       OpenBSD-os.h  mach-os.h    sunos-os.c
       Linux-os.c         hpux-os.c     os-common.c  sunos-os.h
       $ ls *lispregs*
       alpha-lispregs.h  hppa-lispregs.h  mips-lispregs.h  sparc-lispregs.h
       amd64-lispregs.h  lispregs.h       ppc-lispregs.h   x86-lispregs.h
       $ 

    Therefore, to get a reasonable "LOC" comparison between CL & C
    source would require counting only the files needed for *one*
    complete machine/OS pair (e.g., x86/Linux).

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Rob Thorpe
Subject: Re: Implementation of CL data structures
Date: 
Message-ID: <1153523796.062464.273490@75g2000cwc.googlegroups.com>
Lars wrote:
> Pierre THIERRY wrote:
> > That is, does the typical CL implementor that implements it in C also
> > write everything that is mandated by the CL standard in C, or the bare
> > minimum, or something in-between?
>
> Somewhere in-between, with the C/Lisp ratio probably higher for
> interpreters, which need the extra bit of speed.
>
> Here are some very crude figures, the number of Lisp and C source files
> in three OS CL implementations:
>
>         CLISP  CMUCL  SBCL
> .[ch]    429    130   139
> .lisp    189    855   786
>
> (Note: I've included the .d files in CLISP.)

Lines of code tell a somewhat similar story:-

SBCL
 28198 .[ch] in Core
245299 .lisp in Core
 20060 .lisp in PCL

So SBCL is overwhelmingly lisp, and its a large program, 293557 lines.

ECL v0.9i
 44786 .[cdh] in Core
129755 .[ch] in the GMP library
 42143 .[ch] in Boehm GC
 40148 .lsp in Core
 30348 .lisp in CLX

So, fairly equal amounts of C and lisp in the core itself.  But aid
from large C libraries and a large lisp library.  I expect much of the
C library code isn't used.
The amount of lines of code actually distributed is actually similar to
SBCL, 287180 but I don't think this means much.

GCL is similar to ECL in this way, it uses lots of C and lots of lisp.
I looked at CLISP some time ago in this way.  The vast majority of
CLISP is the "d" variant of C.