From: His Holiness the Reverend Doktor Xenophon Fenderson, the Carbon(d)ated
Subject: System programming in Lisp?
Date: 
Message-ID: <w4ou2no44q8.fsf@nemesis.irtnog.org>
OK, so this is a newbie question of the basest kind.  I suppose this
kind of question exposes me as a wannabe-C-hax0r, but then again, I
suspect there are a lot of those lurking in c.l.l (hence questions
like the following, or about the nature of COMPILE-FILE/LOAD, etc.).
Now where did I put that asbestos underwear...

Are there any examples out there of low-level code (e.g. a device
driver or a GC) written in Lisp?  I mean, Lisp machine hackers had to
write their O/S in *something*, and it seems to me that lots of
assembly code is a tad unwieldy.  Much of the UNIX kernel (and lots of
the really to-the-bare-metal code, even) is written in C, if Linux and
BSD are any indication.

So how did the Lisp machine hackers do it?  (I think this is what
everyone means when they ask "what are the primitives of the Lisp
language".)  To be honest, I'm mystified.  No wonder "LispOS" is so
much vapour.  I kind-of know where to start if I were using a mix of C
and assembly, but I don't even know where I'd begin (or even what
compiler to use) if I were to try writing an operating system in Lisp.

-- 
Rev. Dr. Xenophon Fenderson, the Carbon(d)ated, KSC, DEATH, SubGenius, mhm21x16
Pope, Patron Saint of All Things Plastic fnord, and Salted Litter of r.g.s.b
"I try to take life one day at a time, but lately several days have attacked me
at once." -- Carrie Fish

From: Barry Margolin
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <xcRO3.60$UJ6.1908@burlma1-snr2>
In article <···············@nemesis.irtnog.org>,
His Holiness the Reverend Doktor Xenophon Fenderson, the Carbon(d)ated <········@irtnog.org> wrote:
>Are there any examples out there of low-level code (e.g. a device
>driver or a GC) written in Lisp?  I mean, Lisp machine hackers had to
>write their O/S in *something*, and it seems to me that lots of
>assembly code is a tad unwieldy.  Much of the UNIX kernel (and lots of
>the really to-the-bare-metal code, even) is written in C, if Linux and
>BSD are any indication.
>
>So how did the Lisp machine hackers do it?  (I think this is what
>everyone means when they ask "what are the primitives of the Lisp
>language".)  To be honest, I'm mystified.  No wonder "LispOS" is so
>much vapour.  I kind-of know where to start if I were using a mix of C
>and assembly, but I don't even know where I'd begin (or even what
>compiler to use) if I were to try writing an operating system in Lisp.

Lisp Machines have a number of low-level operations that are not found in
standard Common Lisp, such as the ability to address specific memory
addresses, like those that are mapped to I/O channels, and commands for
accessing special machine registers, etc.  They also have custom microcode
that implements these special operations, as well as interfacing to the
hardware assist features for GC.

Other than these very low-level operations, the design of an OS in Lisp is
not really that much different from any other high-level language.  Some of
the above features correspond to things that are done in assembly in more
common operating systems, or things that make use of non-portable features
in the high-level language (e.g. casting an integer to a pointer in C).

-- 
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Joe Marshall
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <iw%O3.29579$DV.38982@dfw-read.news.verio.net>
> Rev. Dr. Xenophon Fenderson, the Carbon(d)ated, KSC, DEATH, SubGenius,
mhm21x16
> Pope, Patron Saint of All Things Plastic fnord, and Salted Litter of
r.g.s.b
> <········@irtnog.org> wrote in message
····················@nemesis.irtnog.org...

I'm not sure I'm qualified to answer to someone with a title like that,
(does that even *fit* in a
CrackerJack box?), but here is a bit of an answer:

The CADR and its descendents implemented Lisp in microcode.
(See the papers at http://www.vsf.cape.com/~emergent/lispos.php3 )
The LMI K-Machine used Lisp all the way down to the metal via use
of `subprimitives'.  Subprimitives are primitive lisp expressions that have
certain requirements not usually needed for regular lisp expressions.  For
instance, in the K-machine, we had a subprimitive called
`%%alu-add' which took 4 arguments:  a width specifier, a type check code,
and a left and right input.  The width specifier was used to tell the AMD
29332
to use 1 2 3 or 4 bytes in the addition, the type check code told the type
check
hardware what kind of type codes to allow, and the left and right inputs
were
used as the data to add.  There were restrictions on the use of this
primitive:
the width specifier and the type-check code had to be compile time
constants,
and the left and right inputs had to be in registers (and thus could only be
variables,
not computed expressions).  If you obeyed these restrictions, you could
pretty
much put an ADD instruction anywhere you wanted to in the instruction
stream.
By adding similar subprimitives, you can write your assembly code in lisp.

This wasn't unique to LMI.  For example, Lucid used some similar techniques
to implement
low level functionality in lisp.

> Are there any examples out there of low-level code (e.g. a device
> driver or a GC) written in Lisp?

Yes, there are.  I've seen a lot of GC's written in Lisp (the two tricks
there is that the
GC has to cons slower than it reclaims, and it has to get permission to
bypass the
GC barriers.)  Device drivers in Lisp are a bit rarer.  Since device vendors
usually
provide a driver (in C, and for the most popular OS's), it is just easier to
try to
use their code than to rewrite it in Lisp.

>No wonder "LispOS" is so much vapour.

I don't think that trickiness is the problem.  I think there are a lot of
reasons, and
one of them is the fact that there doesn't seem to be a good, free, fairly
optimizing,
production quality, portable lisp compiler that runs on Linux and Windows.

> I kind-of know where to start if I were using a mix of C
> and assembly, but I don't even know where I'd begin (or even what
> compiler to use) if I were to try writing an operating system in Lisp.

Assuming you have a compiler, I would start with the virtual memory system.
That seems to be the natural dividing line between the world of
hard real-time response (device drivers and interrupts), and the higher
order
OS abstractions (processes, file systems).

~jrm
From: Fernando D. Mato Mira
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <380C9293.95D84E5@iname.com>
Joe Marshall wrote:

> > Rev. Dr. Xenophon Fenderson, the Carbon(d)ated, KSC, DEATH, SubGenius,
> mhm21x16
> > Pope, Patron Saint of All Things Plastic fnord, and Salted Litter of
> r.g.s.b
> > <········@irtnog.org> wrote in message
> ····················@nemesis.irtnog.org...
>
> >No wonder "LispOS" is so much vapour.
>
> I don't think that trickiness is the problem.  I think there are a lot of
> reasons, and
> one of them is the fact that there doesn't seem to be a good, free, fairly
> optimizing,
> production quality, portable lisp compiler that runs on Linux and Windows.

Why would you need more than one development platform if your goal is writing
an OS?

--
Fernando D. Mato Mira
Real-Time SW Eng & Networking
Advanced Systems Engineering Division
CSEM
Jaquet-Droz 1                   email: matomira AT acm DOT org
CH-2007 Neuchatel                 tel:       +41 (32) 720-5157
Switzerland                       FAX:       +41 (32) 720-5720

www.csem.ch      www.vrai.com     ligwww.epfl.ch/matomira.html
From: Joe Marshall
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <LpjP3.30045$DV.44344@dfw-read.news.verio.net>
Fernando D. Mato Mira <········@iname.com> wrote in message
·····················@iname.com...
>
> Why would you need more than one development platform if your goal is
writing
> an OS?
>

Selecting just Linux or just Windows would alienate too many people who, for
one
reason or another, are unwilling or unable to switch to the other platform.
From: Fernando Mato Mira
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <380DF741.5BB3A874@iname.com>
Joe Marshall wrote:
> 
> Fernando D. Mato Mira <········@iname.com> wrote in message
> ·····················@iname.com...
> >
> > Why would you need more than one development platform if your goal is
> writing
> > an OS?
> >
> 
> Selecting just Linux or just Windows would alienate too many people who, for
> one
> reason or another, are unwilling or unable to switch to the other platform.

Hmm.. If you can't download it for free, $1.95 + shipping takes care of
Linux.
You'll need some disk space of course.
Unwilling to boot anything but Windows and willing to write a Lisp OS? I
don't think so..

-- 
Fernando D. Mato Mira                    
Real-Time SW Eng & Networking            
Advanced Systems Engineering Division
CSEM                             
Jaquet-Droz 1                   email: matomira AT acm DOT org
CH-2007 Neuchatel                 tel:       +41 (32) 720-5157
Switzerland                       FAX:       +41 (32) 720-5720

www.csem.ch      www.vrai.com     ligwww.epfl.ch/matomira.html
From: Matthew X. Economou
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <w4ok8oj3uk6.fsf@nemesis.irtnog.org>
>>>>> "JM" == Joe Marshall <·········@alum.mit.edu> writes:

    JM> I'm not sure I'm qualified to answer to someone with a title
    JM> like that, (does that even *fit* in a CrackerJack box?), but
    JM> here is a bit of an answer:

:)  I usually post in alt.*, and I forgot to strip the pseudo.  I
actually got flamed somewhere else in comp.* (or maybe it was in
sci.*) for posting with a pseudo.

    JM> http://www.vsf.cape.com/~emergent/lispos.php3

Thanks for the URL.  I've been looking for the Eval-Apply papers and
such for a while.  What ever happened to the site?  It's still in some
of the search engines (like Google).

    JM> The LMI K-Machine used Lisp all the way down to the metal via
    JM> use of `subprimitives'.

I'm not certain yet (I don't understand enough of the architecture),
but the Alpha microprocessor's PALcode seems like it could provide
this.  In fact, the PALcode reference manual suggests (to me, anyway)
that this library is the place to put memory management functions,
etc.---really low-level OS and language support code.

Does Genera only run under OSF/1, or will it boot by itself on the
Alpha?  I'd really like to know if/how they use the PAL.

    JM> By adding similar subprimitives, you can write your assembly
    JM> code in lisp.

Are there any current Lisp implementations that provide these kinds of
sub-primitives?  Or even just an "escape to assembly" macro like GCC's
__asm directive?  I didn't notice anything like that when I browsed
CMU CL's online documentation (on www.cons.org), but I could have
missed it (and I haven't had time to go through the EncyCMUCLopedia).

    JM> I don't think that trickiness is the problem [in writing a
    JM> LispOS].  I think there are a lot of reasons, and one of them
    JM> is the fact that there doesn't seem to be a good, free, fairly
    JM> optimizing, production quality, portable lisp compiler that
    JM> runs on Linux and Windows.

What do you think are other reasons a free LispOS remains vaporware?

    JM> Assuming you have a compiler, I would start with the virtual
    JM> memory system.

Especially since Dybvig, et al, recommend tying GC, the run-time type
system, and the memory manager all together in "Don't stop the BIBOP".
And since the VM system doesn't necessarily need to talk to the I/O
bus at boot.

Thanks for the food for thought.  I appreciate it.

-- 
"I try to take life one day at a time, but lately several days have attacked me
at once." -- Carrie Fish
From: Chris Double
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <wk3dv64zy3.fsf@double.co.nz>
"Matthew X. Economou" <········@irtnog.org> writes:

> Are there any current Lisp implementations that provide these kinds
> of sub-primitives?  Or even just an "escape to assembly" macro like
> GCC's __asm directive?

Corman Lisp for Windows (http://www.corman.net) allows assembly
language to be intermixed with Lisp.

Chris.
-- 
http://www.double.co.nz/dylan
From: Christopher R. Barry
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <87emeqvejz.fsf@2xtreme.net>
"Matthew X. Economou" <········@irtnog.org> writes:

> Are there any current Lisp implementations that provide these kinds of
> sub-primitives?  Or even just an "escape to assembly" macro like GCC's
> __asm directive?

Most (all?) Lisps that compile to native code include something called 
a LAP (Lisp Assembler Program).

Christopher
From: Joe Marshall
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <GOjP3.30124$DV.44536@dfw-read.news.verio.net>
Christopher R. Barry <······@2xtreme.net> wrote in message
···················@2xtreme.net...
> "Matthew X. Economou" <········@irtnog.org> writes:
>
> > Are there any current Lisp implementations that provide these kinds of
> > sub-primitives?  Or even just an "escape to assembly" macro like GCC's
> > __asm directive?
>
> Most (all?) Lisps that compile to native code include something called
> a LAP (Lisp Assembler Program).

But not all support mixing Lap and Lisp, or expose the Lap interface to
the client.  I don't think Franz has much in the way of a Lap interface.

~jrm
From: Christopher R. Barry
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <87bt9tvqgm.fsf@2xtreme.net>
"Joe Marshall" <·········@alum.mit.edu> writes:

> Christopher R. Barry <······@2xtreme.net> wrote in message
> ···················@2xtreme.net...
> > "Matthew X. Economou" <········@irtnog.org> writes:
> >
> > > Are there any current Lisp implementations that provide these kinds of
> > > sub-primitives?  Or even just an "escape to assembly" macro like GCC's
> > > __asm directive?
> >
> > Most (all?) Lisps that compile to native code include something called
> > a LAP (Lisp Assembler Program).
> 
> But not all support mixing Lap and Lisp, or expose the Lap interface to
> the client.  I don't think Franz has much in the way of a Lap interface.

I can't find any documentation about it in the Allegro CL
distribution. In:

http://x40.deja.com/getdoc.xp?AN=461555549&CONTEXT=940442891.290128013&hitnum=0

Erik mentions "hackable LAP code" in an article about optimization
(where he got a 623 microsecond function down to 4 microseconds). So
I'm guessing they have some kind of interface.

Christopher
From: Jochen Schneider
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <uzoxd11n9.fsf@yod.cs.uni-magdeburg.de>
You might have to get a source code license to see how Franz used LAP
in their own code. (Just a guess, unfortunately, I don't have such a
license myself. Would be glad to hear that I'm wrong.) It seems to be
a mere problem of documentation. Is there some Allegro LAP reference
floating around on the 'net?

	Jochen
From: Duane Rettig
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <4yacwiwsq.fsf@beta.franz.com>
······@2xtreme.net (Christopher R. Barry) writes:

> "Joe Marshall" <·········@alum.mit.edu> writes:
> 
> > Christopher R. Barry <······@2xtreme.net> wrote in message
> > ···················@2xtreme.net...
> > > "Matthew X. Economou" <········@irtnog.org> writes:
> > >
> > > > Are there any current Lisp implementations that provide these kinds of
> > > > sub-primitives?  Or even just an "escape to assembly" macro like GCC's
> > > > __asm directive?
> > >
> > > Most (all?) Lisps that compile to native code include something called
> > > a LAP (Lisp Assembler Program).
> > 
> > But not all support mixing Lap and Lisp, or expose the Lap interface to
> > the client.  I don't think Franz has much in the way of a Lap interface.
> 
> I can't find any documentation about it in the Allegro CL
> distribution. In:

It's because it is not documented.  Lap code is extremely machine-dependent
and possibly volatile (I add or remove instructions from some architectures
once in a while, to implement new optimizations or to take advantage of
new instructions).

However, I never program in LAP myself; it's much too low-level and
architecture-dependent.  For system programming tasks, I use a
low-level language which is very similar to CL, but not quite; so
to emit, for example, a LAP instruction like
  (asr.l (im 2) (:reg 0 :eax :ax :al)),
which turns into
  22: c1 f8 02    sarl	eax,$2 
according to the x86 disassembler, I will use (in the compiler package)
the form (ll :asr x (ll :fixnum-to-mi 2)) in open lisp code, or if
I'm compiling to assembler source to assemble/link with the rest of the
core lisp shared-library, I can drop the explicit force to
machine-integer, because it is assumed: (ll :asr x 2).  Also when
compiling in this low-level-lisp, other CL constructs work: defmacro
(with an implied eval-when (compile)), loops, if, cond, etc., as well as
many operations that would normally be though of as emitting a single
instruction.  Thus, (+ a b) turns into (ll :+ a b), which emits a single
add instruction, with no overflow detection.

The problem with documenting or exporting any of this is that it is
intended only for building core lisp exectuables/shared-libraries.
In fact, half of the "ll" functions used to build the lisps are not
included in the released lisp (it is a waste of space).

> Erik mentions "hackable LAP code" in an article about optimization
> (where he got a 623 microsecond function down to 4 microseconds). So
> I'm guessing they have some kind of interface.

I have posted this a couple of times previously.  Try defining function
foo, then (setq comp::*hack-compiler-output* '(foo)) and then compile
foo.  The instructions should be self-explanatory.  You're on your own
with the laps, though.

-- 
Duane Rettig          Franz Inc.            http://www.franz.com/ (www)
1995 University Ave Suite 275  Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253   ·····@Franz.COM (internet)
From: Joe Marshall
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <lKjP3.30083$DV.44371@dfw-read.news.verio.net>
Matthew X. Economou <········@irtnog.org> wrote in message
····················@nemesis.irtnog.org...
>
>     JM> http://www.vsf.cape.com/~emergent/lispos.php3
>
> Thanks for the URL.  I've been looking for the Eval-Apply papers and
> such for a while.  What ever happened to the site?  It's still in some
> of the search engines (like Google).

I didn't get a bill from Internic and thus forgot to renew the domain name.
Some company that buys out expired domain names bought it and I just
haven't felt like beating on them to get it back.  And I'm so sick of
NSI in general.

>     JM> The LMI K-Machine used Lisp all the way down to the metal via
>     JM> use of `subprimitives'.
>
> I'm not certain yet (I don't understand enough of the architecture),
> but the Alpha microprocessor's PALcode seems like it could provide
> this.  In fact, the PALcode reference manual suggests (to me, anyway)
> that this library is the place to put memory management functions,
> etc.---really low-level OS and language support code.
>
> Does Genera only run under OSF/1, or will it boot by itself on the
> Alpha?  I'd really like to know if/how they use the PAL.

I'm sorry but I know close to nil about the Alpha.

>     JM> By adding similar subprimitives, you can write your assembly
>     JM> code in lisp.
>
> Are there any current Lisp implementations that provide these kinds of
> sub-primitives?  Or even just an "escape to assembly" macro like GCC's
> __asm directive?  I didn't notice anything like that when I browsed
> CMU CL's online documentation (on www.cons.org), but I could have
> missed it (and I haven't had time to go through the EncyCMUCLopedia).

Corman Lisp does.  I don't think Franz does.


> What do you think are other reasons a free LispOS remains vaporware?

Among the reasons:

Most people interested in LispOS have their own agenda that diverges
significantly
from OS support.  (Too many cooks, maybe.)

Most hackers have a day job.

I haven't gotten off my ass and dug up some old code I promised to
John Morrison.  He has a byte code engine ready to go.

A LispOS is on the order of 10 man-years of effort.  That's either one or
two
*really* dedicated individuals or a *lot* of time.  I think we're seeing the
latter.  Also, look at how long the GNU HURD project has been running.
Stallman is nothing if not prolific, and I seem to remember the idea of a
Unix-like clone circa 1985.  I'm not surprised by the lack of a LispOS,
just disappointed.  If I were independently wealthy, I'd put a lot more
effort
into it.

We need our own personal Torvalds.


>     JM> Assuming you have a compiler, I would start with the virtual
>     JM> memory system.
>
> Especially since Dybvig, et al, recommend tying GC, the run-time type
> system, and the memory manager all together in "Don't stop the BIBOP".
> And since the VM system doesn't necessarily need to talk to the I/O
> bus at boot.

I'm not familiar with this.  Is this a paper they wrote?  I'd like to read
it.
In my experience, I prefer to more loosly couple the VM and GC.  I was
thinking about this last night, actually.  The GC is really easy to write if
you can assume you can get as much address space as you want
whenever you want.

> Thanks for the food for thought.  I appreciate it.

Anytime.

~jrm
From: Matthew Economou
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <w4oyacx3fsy.fsf@nemesis.irtnog.org>
>>>>> "JM" == Joe Marshall <·········@alum.mit.edu> writes:

    JM> Assuming you have a compiler, I would start with the virtual
    JM> memory system.

That's what I was thinking.

>>>>> "MXE" == Matthew Economou <········@irtnog.org> writes:

    MXE> Especially since Dybvig, et al, recommend tying GC, the
    MXE> run-time type system, and the memory manager all together in
    MXE> "Don't stop the BIBOP". And since the VM system doesn't
    MXE> necessarily need to talk to the I/O bus at boot.

    JM> I'm not familiar with this.  Is this a paper they wrote?  I'd
    JM> like to read it. In my experience, I prefer to more loosly
    JM> couple the VM and GC.  I was thinking about this last night,
    JM> actually.  The GC is really easy to write if you can assume
    JM> you can get as much address space as you want whenever you
    JM> want.

Hm.  I reread the paper today, and I must have confused the
tightly-coupled-GC-and-VM idea with their work.  "Don't Stop the
BIBOP" describes a hybrid system for storage and type management,
combining both tag fields (the least-significant 3 bits of a pointer)
with a BIBOP.  The BIBOP segments the heap, and they describe a
generational garbage collector that walks the segments.  I think I
read a paper (or maybe a FAQ entry?) in Harlequin's memory management
reference (http://www.harlequin.com/mm/reference/) that mentions
combining a BIBOP with a virtual memory manager.  It says that some
RISC machines support "inverted page tables", and that these are a
really good way of implementing a BIBOP.

(I don't think I have a clear understanding of what all anyone is
talking about.  I never really studied GC algorithms in school, and
Dybvig uses terms (like "weak pair"---what's that?) that I've never
heard before.  I'm going to troll through the citations and see if I
can find a good summary paper.  There's a GC book that the gclist FAQ
recommends, so I think I'll order that this weekend, as well.)

P.S. Dybvig, Eby, and Bruggeman wrote the paper.  It's IU technical
report #400, "Don't Stop the BIBOP: Flexible and Efficient Storage
Management for Dynamically-Typed Languages".  You can download the
PostScript at http://www.cs.indiana.edu/pub/techreports/TR400.html.
I've actually met Bruggeman---he's a sharp fellow, really interesting
to talk to.  Too bad that my knowledge of Lisp at the time consisted
entirely of Emacs customizations.  :)

-- 
"I try to take life one day at a time, but lately several days have attacked me
at once." -- Carrie Fish
From: Frank A. Adrian
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <vKwP3.648$r8.21010@news.uswest.net>
Good references for GC are the surveys by Paul R. Wilson at UT Austin
(http://www.cs.utexas.edu/users/oops/papers.html) and Jones' and Lins' book
on Garbage collection (http://www.amazon.com/exec/obidos/ASIN/0471941484).
Both are rather exhaustive of the state of the art as of about 4-5 years
ago.  For more recent stuff (most of which has been about interfacing with
common cache hardware and distributed stuff), see the Symposia on Functional
Languages and Programming and OOPSLA Proceedings for the last few years.  If
nothing else, you'll get a few references.

faa

Matthew Economou <········@irtnog.org> wrote in message
····················@nemesis.irtnog.org...
> >>>>> "JM" == Joe Marshall <·········@alum.mit.edu> writes:
>
>     JM> Assuming you have a compiler, I would start with the virtual
>     JM> memory system.
>
> That's what I was thinking.
>
> >>>>> "MXE" == Matthew Economou <········@irtnog.org> writes:
>
>     MXE> Especially since Dybvig, et al, recommend tying GC,
thehttp://www.amazon.com/exec/obidos/ASIN/0471941484
>     MXE> run-time type system, and the memory manager all together in
>     MXE> "Don't stop the BIBOP". And since the VM system doesn't
>     MXE> necessarily need to talk to the I/O bus at boot.
>
>     JM> I'm not familiar with this.  Is this a paper they wrote?  I'd
>     JM> like to read it. In my experience, I prefer to more loosly
>     JM> couple the VM and GC.  I was thinking about this last night,
>     JM> actually.  The GC is really easy to write if you can assume
>     JM> you can get as much address space as you want whenever you
>     JM> want.
>
> Hm.  I reread the paper today, and I must have confused the
> tightly-coupled-GC-and-VM idea with their work.  "Don't Stop the
> BIBOP" describes a hybrid system for storage and type management,
> combining both tag fields (the least-significant 3 bits of a pointer)
> with a BIBOP.  The BIBOP segments the heap, and they describe a
> generational garbage collector that walks the segments.  I think I
> read a paper (or maybe a FAQ entry?) in Harlequin's memory management
> reference (http://www.harlequin.com/mm/reference/) that mentions
> combining a BIBOP with a virtual memory manager.  It says that some
> RISC machines support "inverted page tables", and that these are a
> really good way of implementing a BIBOP.
>
> (I don't think I have a clear understanding of what all anyone is
> talking about.  I never really studied GC algorithms in school, and
> Dybvig uses terms (like "weak pair"---what's that?) that I've never
> heard before.  I'm going to troll through the citations and see if I
> can find a good summary paper.  There's a GC book that the gclist FAQ
> recommends, so I think I'll order that this weekend, as well.)
>
> P.S. Dybvig, Eby, and Bruggeman wrote the paper.  It's IU technical
> report #400, "Don't Stop the BIBOP: Flexible and Efficient Storage
> Management for Dynamically-Typed Languages".  You can download the
> PostScript at http://www.cs.indiana.edu/pub/techreports/TR400.html.
> I've actually met Bruggeman---he's a sharp fellow, really interesting
> to talk to.  Too bad that my knowledge of Lisp at the time consisted
> entirely of Emacs customizations.  :)
>
> --
> "I try to take life one day at a time, but lately several days have
attacked me
> at once." -- Carrie Fish
From: Gareth McCaughan
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <866701sjnj.fsf@g.local>
Matthew Economou wrote:

> (I don't think I have a clear understanding of what all anyone is
> talking about.  I never really studied GC algorithms in school, and
> Dybvig uses terms (like "weak pair"---what's that?) that I've never
> heard before.  I'm going to troll through the citations and see if I
> can find a good summary paper.  There's a GC book that the gclist FAQ
> recommends, so I think I'll order that this weekend, as well.)

A weak X is an X that doesn't prevent (all of) the things
it references from being GCed if it's the only reference
to them.

So, if you had a function WEAK-CONS for making weak pairs,
and if "weak" here means "weak with respect to both halves",
then you could in principle observe effects like this:

  --> (setf foo (weak-cons (list 1 2 3) (list 1 2 3)))
  ((1 2 3) 1 2 3)
  --> foo
  ((1 2 3) 1 2 3)
  --> (gc)
  NIL
  --> foo
  (NIL)

The "(all of)" above is there because e.g. I've seen
something or other that defines a "weak hash table"
as "a hash table which doesn't stop its keys getting
GCed when there are no other references to them left".

-- 
Gareth McCaughan  ················@pobox.com
sig under construction
From: Paolo Amoroso
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <380dd34a.75350@news.mclink.it>
On 19 Oct 1999 16:32:57 -0500, "Matthew X. Economou" <········@irtnog.org>
wrote:

> Are there any current Lisp implementations that provide these kinds of
> sub-primitives?  Or even just an "escape to assembly" macro like GCC's
> __asm directive?  I didn't notice anything like that when I browsed
> CMU CL's online documentation (on www.cons.org), but I could have

CMU CL might provide something similar. Have a look at the files in the
src/assembler and src/compiler directories (particularly
src/compiler/new-assem.lisp) of the source distribution. The relevant
symbols should be in the NEW-ASSEM and C packages.


> missed it (and I haven't had time to go through the EncyCMUCLopedia).

It doesn't provide much relevant documentation beyond what is included in
the Implementor Documentation/System Internals section of the
EncyCMUCLopedia index. The most useful resource listed there is probably
the "Design of CMU Common Lisp" manual (see Chapter 23).


Paolo
-- 
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
From: Barry Margolin
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <kplP3.112$UJ6.4315@burlma1-snr2>
In article <···············@nemesis.irtnog.org>,
Matthew X. Economou <········@irtnog.org> wrote:
>>>>>> "JM" == Joe Marshall <·········@alum.mit.edu> writes:
>    JM> By adding similar subprimitives, you can write your assembly
>    JM> code in lisp.
>
>Are there any current Lisp implementations that provide these kinds of
>sub-primitives?

IIRC, Lucid CL has sub-primitives.  When I was at Thinking Machines, we
used them to implement *Lisp's interface to the Connection Machine, which
used memory-mapped registers.

-- 
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
From: Matthew Economou
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <w4o4sfl51xh.fsf@nemesis.irtnog.org>
>>>>> "BM" == Barry Margolin <······@bbnplanet.com> writes:

    BM> IIRC, Lucid CL has sub-primitives.

Do you (or anyone else) have code examples I could look at?

-- 
"I try to take life one day at a time, but lately several days have attacked me
at once." -- Carrie Fish
From: R. Matthew Emerson
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <87ln8xdd74.fsf@nightfly.apk.net>
Matthew Economou <········@irtnog.org> writes:

> >>>>> "BM" == Barry Margolin <······@bbnplanet.com> writes:
> 
>     BM> IIRC, Lucid CL has sub-primitives.
> 
> Do you (or anyone else) have code examples I could look at?

Here's a sample LAP function from MCL 4.2 that might give you a
sense of how these sorts of things look.

; Copy an IEEE-single (aka "short") float from a macptr to a lisp short float
#+ppc-target
(defppclapfunction %ref-ieee-single-float ((macptr arg_y) (dest arg_z))
  (check-nargs 2)
  (lwz imm0 ppc::macptr.address macptr)
  (lfs fp1 0 imm0)
  (put-single-float fp1 dest)
  (blr))

(In MCL, a short-float is an IEEE single precision value.)

MCL comes with a fair amount of source code for the implementation;
you might see about taking a look at it.

-- 
R. Matthew Emerson
http://www.thoughtstuff.com/rme/
From: ·····@onshore.com
Subject: Re: System programming in Lisp?
Date: 
Message-ID: <lriu409x0i.fsf@playip3.onshore.com>
"Matthew X. Economou" <········@irtnog.org> writes:

> Are there any current Lisp implementations that provide these kinds of
> sub-primitives?  Or even just an "escape to assembly" macro like GCC's
> __asm directive?  I didn't notice anything like that when I browsed
> CMU CL's online documentation (on www.cons.org), but I could have
> missed it (and I haven't had time to go through the EncyCMUCLopedia).

There doesn't seem to be a whole lot of documentation about how this
works in CMUCL; here's my naive take, based mostly on source reading:
there's no 'escape to assembly' per say, but one of the more
interesting aspects of CMUCL is its native compiler, by which lisp
code is converted by stages into native machine code. At one point
(IR2?) code is generated for a 'virtual machine', which is defined in
terms of a set of lisp registers, for environment pointer, allocation
pointer, program counter, eg., and a set of virtual operations, or
VOPs. Many of these VOPs are wrappers around snippets of native
assembly code. The sequence of VOPs is assembled into the final
output.

So the compiler is, in a sense, marshalling what you are calling
'subprimitives'.

Try, in CMUCL, doing:

(compile-file "foo.lisp" :trace-file t)

where "foo.lisp" contains perhaps a simple function or two, then look
at "foo.trace", which gives a verbose narrative of the compilation to
native code.

I don't think there are any real barriers stopping you from finding a
place to mix in your own assembly code, other than a comprehensive
knowledge of how the CMUCL VM works, ... uh, hm, maybe that is a bit
of a barrier.

Jesse Bouwman