From: Tom Lord
Subject: building a new run-time system
Date: 
Message-ID: <v26p4mbordea17@corp.supernews.com>
        > ... I was advocating for some work to build an original free
        > software run-time system for high-level languages as a
        > practical alternative to dragging the free software world
        > into a swamp of .NET reimplementation.

        Erann Gat:

        I believe this is a very worthy goal.  I am actually in active
        need of such a system (for reasons that I won't go into here).
        I've found that Hans's GC, Bruno's CLN system, and C++ get me
        a very long way towards where I want to be.  If I wanted to
        use a precise GC instead of Hans's system, what would you
        recommend?  (Note: I need a > thread-safe GC.)

I'm not aware of any off-the-shelf precise GC that is packaged as
conveniently as Hans' conservative GC.  That's an aspect of the
project I was trying to drum up support for: to build such a GC, in C.
(Actually, "not aware of" is a slight exaggeration -- I _think_ I
recall some recent (six months) comp.lang.scheme traffic about a
reusable precise GC -- you might want to google.)

So, I'd say: roll your own (but check c.l.s. first).

That you are committed to C++ might help automate the bookkeeping
tasks.  If you were working in C, I'd suggest at least planning for
`gclint(1)'.



        Henrik Motakef:

        Isn't [such a run-time system] one of the goals of the Parrot
        project? AFAIK it's being developed as the default backend of
        Perl 6, but with cross-language-ability in mind, and some Ruby
        and Python people used to be quite excited about it. 



Looking at their web page...

   "Parrot is a virtual machine used to efficiently execute bytecode
    for interpreted languages"

The run-time system I want to build is not a virtual machine.  It is a
library that one might want to use directly from C/C++ or as the
non-interpreting component of an interpreted language implementation
or as a set of primitive function support for a compiled language
implementation.

For example, it will be the job of this library to provide various
forms of strings, arrays, pairs, hash tables, numbers, I/O, and so
forth -- but interpreting code will be a (planned for) layer on top of
this library.

Parrot has some run-time system elements as well.  For example, being
Perl oriented, it has a string library.  The string library I plan to
build goes a bit beyond what they are offering.  I plan to to have low
levels which are close to libc string functions (with some obvious
gaffs cleaned up) but with added support for the good Unicode
encodings.  Intermediate levels that are closer to what they are
offering.  And at the highest levels, I plan to have attributed,
editable strings with support for markers.

I didn't see any support in Parrot for numbers or for the basic lisp
types, but perhaps I missed it.



        Would be a CL-to-Parrot-compiler be a good idea?

Politically?  or "Might it be fun in the same way that building a
working high-precision drill press from items commonly found in the
kitchen is fun?"  or "Is this a good way to approach implementing CL?"

(yes, yes, no - not from what I've seen so far)

From: Paul F. Dietz
Subject: Re: building a new run-time system
Date: 
Message-ID: <KfycncAJjJ7V-r6jXTWcqQ@dls.net>
Tom Lord wrote:

> I'm not aware of any off-the-shelf precise GC that is packaged as
> conveniently as Hans' conservative GC.  That's an aspect of the
> project I was trying to drum up support for: to build such a GC, in C.
> (Actually, "not aware of" is a slight exaggeration -- I _think_ I
> recall some recent (six months) comp.lang.scheme traffic about a
> reusable precise GC -- you might want to google.)


You might want to get involved with the development of gcc, so it
can be extended with precise gc.  IIRC their java compiler generates
code using the BDW conservative collector, and gcc itself also uses
BDW now (with some distressing effects on performance).

Considering the complexity of gcc, I've sometimes wondered if they
should rewrite it in Lisp. :)

	Paul
From: Tom Lord
Subject: Re: building a new run-time system
Date: 
Message-ID: <v270en7rdtefe6@corp.supernews.com>
        You might want to get involved with the development of gcc, so
        it can be extended with precise gc.  

I'm interested in a precise GC which is portable C, not
compiler-specific.


        IIRC their java compiler generates code using the BDW
        conservative collector, 

I recall that their Java uses conservative GC:  that's something I
wanted to help fix.

	and gcc itself also uses BDW now (with some distressing
        effects on performance).

There are recent threads on performance and GC-internal-to-GCC if you
want to drill down into the details.

I believe that GCC internally uses _precise_ GC -- the switch to
conservative was just part of an experiment in performance
measurement, not something on the mainline.


	Considering the complexity of gcc, I've sometimes wondered if
	they should rewrite it in Lisp. :)

Clearly (if you've looked at the code) they need some higher level
notation than what they've got.   Little experiments like `lcc' (from
the same author whose code morphed into GCC, long ago) make me wonder
if starting from scratch wouldn't be a good idea.  Initially, it could
start out as a project to build a companion compiler optimized for
compilation speed....

-t
From: Nikodemus Siivola
Subject: Re: building a new run-time system
Date: 
Message-ID: <b0098n$2ult8$1@midnight.cs.hut.fi>
Tom Lord <····@emf.emf.net> wrote:

> I'm interested in a precise GC which is portable C, not
> compiler-specific.

This is good news! Stupid question: are you just interested, or are you
*working* on it, or are you sort of rolling up your sleeves? 

> Clearly (if you've looked at the code) they need some higher level

Oh yeah... but then again I can see why they would be reluctant to let go: it
would sort of amount to throwing away your god-hood. (I mean, any schmuck can
write a compiler, but few are those that can navigate the internals of
gcc.) ;)

> notation than what they've got.   Little experiments like `lcc' (from
> the same author whose code morphed into GCC, long ago) make me wonder
> if starting from scratch wouldn't be a good idea.  Initially, it could
> start out as a project to build a companion compiler optimized for
> compilation speed....

Sort of makes me think of tcc. Very nice.

-- 
  -- Nikodemus
From: Erann Gat
Subject: Re: building a new run-time system
Date: 
Message-ID: <gat-1301032327450001@192.168.1.51>
In article <··············@corp.supernews.com>, ····@emf.emf.net (Tom
Lord) wrote:

> I'm not aware of any off-the-shelf precise GC that is packaged as
> conveniently as Hans' conservative GC.  That's an aspect of the
> project I was trying to drum up support for: to build such a GC, in C.
> (Actually, "not aware of" is a slight exaggeration -- I _think_ I
> recall some recent (six months) comp.lang.scheme traffic about a
> reusable precise GC -- you might want to google.)
> 
> So, I'd say: roll your own (but check c.l.s. first).

Given the rate that my C++ coding skills are progressing I'll be ready to
tackle my own GC in about fifteen years.  :-(

> That you are committed to C++ might help automate the bookkeeping
> tasks.  If you were working in C, I'd suggest at least planning for
> `gclint(1)'.

Yes, that's one reason I'm using C++.  C++ is a godawful mess, but at
least it's *possible* to write a few decent abstractions.  (It is also
possible to write some serious hairballs.)

> The run-time system I want to build is not a virtual machine.  It is a
> library that one might want to use directly from C/C++ or as the
> non-interpreting component of an interpreted language implementation
> or as a set of primitive function support for a compiled language
> implementation.
> 
> For example, it will be the job of this library to provide various
> forms of strings, arrays, pairs, hash tables, numbers, I/O, and so
> forth -- but interpreting code will be a (planned for) layer on top of
> this library.

I've found that the STL + CLN is not a bad start.  Add Hans's GC, and a
surprisingly capable interpreter for a Lisp-like language can be had for a
few hundred LOC.

E.
From: Nikodemus Siivola
Subject: Re: building a new run-time system
Date: 
Message-ID: <b00hje$2sv2p$1@midnight.cs.hut.fi>
Erann Gat <···@jpl.nasa.gov> wrote:

> I've found that the STL + CLN is not a bad start.  Add Hans's GC, and a
> surprisingly capable interpreter for a Lisp-like language can be had for a
> few hundred LOC.

Surprisingly capable interpreter for a Lisp-like language can be had for a
few hundred LOC of *anything*. That as a measure is more of a testament to
the power of the unified syntax paradigm and simple core language, than to
C++. ;)

-- 
  -- Nikodemus
------------------------------------------------------------
 I refuse to have a battle of wits with an unarmed person.
From: Christopher Browne
Subject: Re: building a new run-time system
Date: 
Message-ID: <b0110p$knv74$1@ID-125932.news.dfncis.de>
Martha Stewart called it a Good Thing whenNikodemus Siivola <········@conflagration.cs.hut.fi>wrote:
> Erann Gat <···@jpl.nasa.gov> wrote:
>
>> I've found that the STL + CLN is not a bad start.  Add Hans's GC,
>> and a surprisingly capable interpreter for a Lisp-like language can
>> be had for a few hundred LOC.
>
> Surprisingly capable interpreter for a Lisp-like language can be had
> for a few hundred LOC of *anything*. That as a measure is more of a
> testament to the power of the unified syntax paradigm and simple
> core language, than to C++. ;)

Ah, but do you normally get a pretty decent GC in a few hundred LOC in
other languages?
-- 
output = ("cbbrowne" ·@acm.org")
http://cbbrowne.com/info/languages.html
"I'd much rather punch my timecards with a punch awl and blood
spattered fist than use the Solomon IV "Timekeeper" module."
-- Peter da Silva
From: Erann Gat
Subject: Re: building a new run-time system
Date: 
Message-ID: <gat-1401030749320001@192.168.1.51>
In article <··············@ID-125932.news.dfncis.de>, Christopher Browne
<········@acm.org> wrote:

> Martha Stewart called it a Good Thing whenNikodemus Siivola
<········@conflagration.cs.hut.fi>wrote:
> > Erann Gat <···@jpl.nasa.gov> wrote:
> >
> >> I've found that the STL + CLN is not a bad start.  Add Hans's GC,
> >> and a surprisingly capable interpreter for a Lisp-like language can
> >> be had for a few hundred LOC.
> >
> > Surprisingly capable interpreter for a Lisp-like language can be had
> > for a few hundred LOC of *anything*. That as a measure is more of a
> > testament to the power of the unified syntax paradigm and simple
> > core language, than to C++. ;)
> 
> Ah, but do you normally get a pretty decent GC in a few hundred LOC in
> other languages?

And a full numeric tower (including unlimited-precision floats)? 
catch/throw/unwind-protect?  An FFI?  Native code compilation?

E.
From: Nikodemus Siivola
Subject: Re: building a new run-time system
Date: 
Message-ID: <b01p24$2t9q0$2@midnight.cs.hut.fi>
Erann Gat <···@jpl.nasa.gov> wrote:

> And a full numeric tower (including unlimited-precision floats)? 
> catch/throw/unwind-protect?  An FFI?  Native code compilation?

Let's get this straight: are you saying that you have an lisp-like language
implemented in C++ "in a few hundred LOC" that does all that? Wow! 

Can I see? 'cos otherwise I don't believe. (No offense meant, just
incredulous.)

-- 
  -- Nikodemus
From: Erann Gat
Subject: Re: building a new run-time system
Date: 
Message-ID: <gat-1401031309500001@k-137-79-50-102.jpl.nasa.gov>
In article <··············@midnight.cs.hut.fi>, Nikodemus Siivola
<········@kekkonen.cs.hut.fi> wrote:

> Erann Gat <···@jpl.nasa.gov> wrote:
> 
> > And a full numeric tower (including unlimited-precision floats)? 
> > catch/throw/unwind-protect?  An FFI?  Native code compilation?
> 
> Let's get this straight: are you saying that you have an lisp-like language
> implemented in C++ "in a few hundred LOC" that does all that? Wow!

Well, it depends on how you count.  It uses CLN, the Boehm GC, and the STL
to provide most of its functionality, so if you coun't all those systems
it's a lot more than a few hundred LOC.  My few-hundred LOC are little
more than a thin layer of glue on top of those facilities.  The point is
not so much that I'm a wizardly language implementor (I'm not) but that
even a non-wizard like me can get a surprising amount of mileage out of
C++ and off-the-shelf open-source code nowadays.

Some more truth-in-advertising:

1.  The compiler isn't written.  But the FFI is (it's just dlopen) so you
can do native-code compilation by translating Ciel (that's the name of my
language) code into C++ and invoking gcc.  If you don't care too much
about efficiency this translator could be very simple.  Note that I didn't
say *good* native code compilation ;-) though I suspect that's possible as
well without getting too hairy.  My gut feeling is that something like the
twobit compiler, or some simple type inferencing, will help a lot.  But
the jury is still very much out on that.

2.  The library is extremely sparse at the moment.  It's got only the bare
minimum needed to write some rudimentary benchmarks.  Adding library
functions is pretty easy, I just haven't gotten around to it.  So in its
current state it's not usable for much.

> Can I see? 'cos otherwise I don't believe. (No offense meant, just
> incredulous.)

Well, it's not ready for prime time yet.  But Kent Pitman has seen it. 
Will an affidavit from Kent satisfy you for now?  :-)

Here's a transcript of the sytem actually running.

[···@alvin ciel]$ wc cl.cc
   1000    3067   23886 cl.cc
[···@alvin ciel]$ ./ciel
Ciel: (set fib (fn (x) (if (<= x 1) 1 (+ (fib (- x 1)) (fib (- x 2))
((((... ))))
--> fn(x)((if (<= x 1) 1 (+ (fib (- x 1)) (fib (- x 2)))))
Ciel: (time (fib 20))
(fib 20) took 420 msec.  ; On a 1 GHZ Pentium running Linux.  It's SLOOOOOWWWW.
--> 10946
Ciel: (sqrt -2.000000000000000000000000000000000)
--> 1.41421356237309504880168872420969807857L0i
Ciel: x
*** Uncaught exception: ("Unbound symbol" . x)
Ciel: (try x (print "foo"))
"foo"
--> "foo"
Ciel: (set x ((fn (x) (fn (y) (set x (+ x y)) x)) 5))
--> fn(y)((set x (+ x y)) x)
Ciel: (x 1)
--> 6
Ciel: (x 1)
--> 7
Ciel: (set wenn if)
--> <Special form if>
Ciel: (wenn (<= 1 2) (print 3) (print 4))
3
--> 3
--> <Special form fn>
Ciel: (set lambda fn)
--> <Special form fn>
Ciel: ((lambda x x) 1 2 3)
--> (1 2 3)
Ciel: (dynload "foo__FR8consCell" "/home/gat/ciel/fftest")
--> Nil
Ciel: (foo__FR8consCell)
Hello world
--> Nil

The last few lines loaded and executed the following "foreign" code:

clObject& foo(consCell& args) {
  cout << "Hello world\n";
  return nil;
}

E.
From: Nikodemus Siivola
Subject: Re: building a new run-time system
Date: 
Message-ID: <b02s9j$2vcfb$1@midnight.cs.hut.fi>
Erann Gat <···@jpl.nasa.gov> wrote:

> Well, it depends on how you count.  It uses CLN, the Boehm GC, and the STL
> to provide most of its functionality, so if you coun't all those systems

> 1.  The compiler isn't written.  But the FFI is (it's just dlopen) so you

Ok, fair enough. The way you said it just sounded like you had *implemented* 
all that in a few hundred LOC, not *interfaced* to it...

Cheers,

  -- Nikodemus
From: Jules Grosse
Subject: Re: building a new run-time system
Date: 
Message-ID: <8d844ffa.0301151029.78fe5095@posting.google.com>
> Ciel: (dynload "foo__FR8consCell" "/home/gat/ciel/fftest")
> --> Nil
> Ciel: (foo__FR8consCell)

Aah, the wonderful realm of name mangling. ;)

jules
From: Nikodemus Siivola
Subject: Re: building a new run-time system
Date: 
Message-ID: <b01ol9$2t9q0$1@midnight.cs.hut.fi>
Christopher Browne <········@acm.org> wrote:

> Ah, but do you normally get a pretty decent GC in a few hundred LOC in
> other languages?

You cheat and let the task fall to the implementation languages GC by
default. ;)

-- 
  -- Nikodemus
From: Joe Marshall
Subject: Re: building a new run-time system
Date: 
Message-ID: <fzrvhg7o.fsf@ccs.neu.edu>
···@jpl.nasa.gov (Erann Gat) writes:

> C++ is a godawful mess, but at least it's *possible* to write a few
> decent abstractions.

It's possible to write 90% of a decent abstraction.
From: Erann Gat
Subject: Re: building a new run-time system
Date: 
Message-ID: <gat-1401030907260001@k-137-79-50-101.jpl.nasa.gov>
In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu> wrote:

> ···@jpl.nasa.gov (Erann Gat) writes:
> 
> > C++ is a godawful mess, but at least it's *possible* to write a few
> > decent abstractions.
> 
> It's possible to write 90% of a decent abstraction.

Yeah, but I work for the government.  90% is good enough :-)

E.
From: Marco Antoniotti
Subject: Re: building a new run-time system
Date: 
Message-ID: <3E24556E.9000806@cs.nyu.edu>
Erann Gat wrote:
> In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu> wrote:
> 
> 
>>···@jpl.nasa.gov (Erann Gat) writes:
>>
>>
>>>C++ is a godawful mess, but at least it's *possible* to write a few
>>>decent abstractions.
>>
>>It's possible to write 90% of a decent abstraction.
> 
> 
> Yeah, but I work for the government.  90% is good enough :-)

That's why you are so expensive.  In industry it is usually 42%.

Cheers

-- 
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group        tel. +1 - 212 - 998 3488
715 Broadway 10th Floor                 fax  +1 - 212 - 998 3484
New York, NY 10003, USA                 http://bioinformatics.cat.nyu.edu
                     "Hello New York! We'll do what we can!"
                            Bill Murray in `Ghostbusters'.
From: Duane Rettig
Subject: Re: building a new run-time system
Date: 
Message-ID: <4bs2jd23n.fsf@beta.franz.com>
···@jpl.nasa.gov (Erann Gat) writes:

> In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu> wrote:
> 
> > ···@jpl.nasa.gov (Erann Gat) writes:
> > 
> > > C++ is a godawful mess, but at least it's *possible* to write a few
> > > decent abstractions.
> > 
> > It's possible to write 90% of a decent abstraction.
> 
> Yeah, but I work for the government.  90% is good enough :-)

Ouch.  No wonder you have such trouble selling Lisp to your management.

:-)

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Kenny Tilton
Subject: Re: building a new run-time system
Date: 
Message-ID: <3E257F52.2020209@nyc.rr.com>
Duane Rettig wrote:
> ···@jpl.nasa.gov (Erann Gat) writes:
> 
> 
>>In article <············@ccs.neu.edu>, Joe Marshall <···@ccs.neu.edu> wrote:
>>
>>
>>>···@jpl.nasa.gov (Erann Gat) writes:
>>>
>>>
>>>>C++ is a godawful mess, but at least it's *possible* to write a few
>>>>decent abstractions.
>>>
>>>It's possible to write 90% of a decent abstraction.
>>
>>Yeah, but I work for the government.  90% is good enough :-)
> 
> 
> Ouch.  No wonder you have such trouble selling Lisp to your management.
> 
> :-)
> 

Nah, "close enough for government work" is a well-established (ironic) 
QC encomium. I use it all the time, and I do not even work for 
government (well, looking at my taxes...)

:)



-- 

  kenny tilton
  clinisys, inc
  http://www.tilton-technology.com/
  ---------------------------------------------------------------
"Cells let us walk, talk, think, make love and realize
  the bath water is cold." -- Lorraine Lee Cudmore
From: Basile STARYNKEVITCH
Subject: Re: building a new run-time system
Date: 
Message-ID: <q5rptr08ei3.fsf@hector.lesours>
>>>>> "Tom" == Tom Lord <····@emf.emf.net> writes:


    Tom>         I believe this is a very worthy goal.  I am actually
    Tom> in active need of such a system (for reasons that I won't go
    Tom> into here).  I've found that Hans's GC, Bruno's CLN system,
    Tom> and C++ get me a very long way towards where I want to be.
    Tom> If I wanted to use a precise GC instead of Hans's system,
    Tom> what would you recommend?  (Note: I need a > thread-safe GC.)

    Tom> I'm not aware of any off-the-shelf precise GC that is
    Tom> packaged as conveniently as Hans' conservative GC.  That's an
    Tom> aspect of the project I was trying to drum up support for: to
    Tom> build such a GC, in C.  (Actually, "not aware of" is a slight
    Tom> exaggeration -- I _think_ I recall some recent (six months)
    Tom> comp.lang.scheme traffic about a reusable precise GC -- you
    Tom> might want to google.)

My Qish runtime is a reusable precise GC for C but it is not yet
thread-safe of multi-thread capable. I might consider going
multithread some day....

See http://starynkevitch.net/Basile/qishintro.html and
http://freshmeat.net/projects/qish/ (where you can subscribe to new
release announcements).


-- 

Basile STARYNKEVITCH         http://starynkevitch.net/Basile/ 
email: basile<at>starynkevitch<dot>net 
alias: basile<at>tunes<dot>org 
8, rue de la Fa�encerie, 92340 Bourg La Reine, France
From: Erann Gat
Subject: Re: building a new run-time system
Date: 
Message-ID: <gat-1301032317370001@192.168.1.51>
In article <···············@hector.lesours>, Basile STARYNKEVITCH
<·········@SPAM+starynkevitch.net.invalid> wrote:

> >>>>> "Tom" == Tom Lord <····@emf.emf.net> writes:
> 
> 
>     Tom>         I believe this is a very worthy goal.  I am actually
>     Tom> in active need of such a system (for reasons that I won't go
>     Tom> into here).  I've found that Hans's GC, Bruno's CLN system,
>     Tom> and C++ get me a very long way towards where I want to be.
>     Tom> If I wanted to use a precise GC instead of Hans's system,
>     Tom> what would you recommend?  (Note: I need a > thread-safe GC.)
> 
>     Tom> I'm not aware of any off-the-shelf precise GC that is
>     Tom> packaged as conveniently as Hans' conservative GC.  That's an
>     Tom> aspect of the project I was trying to drum up support for: to
>     Tom> build such a GC, in C.  (Actually, "not aware of" is a slight
>     Tom> exaggeration -- I _think_ I recall some recent (six months)
>     Tom> comp.lang.scheme traffic about a reusable precise GC -- you
>     Tom> might want to google.)
> 
> My Qish runtime is a reusable precise GC for C but it is not yet
> thread-safe of multi-thread capable. I might consider going
> multithread some day....

I looked ay Qish, but single-threaded is a show-stopper.  Also, I found
the programmer constraints rather onerous.

E.