From: Jeff Dalton
Subject: Re: C is faster than lisp (lisp vs c++ / Rick Graham...)
Date:
Message-ID: <Cuy0ro.ByK@cogsci.ed.ac.uk>
In article <··········@disuns2.epfl.ch> ········@di.epfl.ch (Fernando Mato Mira) writes:
>In article <··········@triple-i.com>, ····@triple-i.com (Kirk Rader) writes:
>
>> entirely in a commercial Common Lisp, using lisp to implement all of
> ^^^^^^^^^^^^^^^^^^^^^^^
>> the bells-and-whistles like multi-threading that a large-scale
> ^^^^^^^^^^^^^^^
>> application is likely to require.
>
>That's what I hate the most about current implementations.
>Take advantage of the OS. Do not rewrite it.
>No wonder if performance and COMPATIBILITY stink..
What exactly is the problem with providing (multiple) threads for Lisp
programs? If the OS-/C-library-provided lw process mechanism isn't
sufficient, what are we supposed to do? Give up?
From: Fernando Mato Mira
Subject: Re: C is faster than lisp (lisp vs c++ / Rick Graham...)
Date:
Message-ID: <33aq17$9u0@disuns2.epfl.ch>
In article <··········@cogsci.ed.ac.uk>, ····@aiai.ed.ac.uk (Jeff Dalton) writes:
> What exactly is the problem with providing (multiple) threads for Lisp
> programs? If the OS-/C-library-provided lw process mechanism isn't
> sufficient, what are we supposed to do? Give up?
FFI syndrome. It seems that vendors assumed for some time that providing
a way to call C (or FORTRAN, or whatever), and `be called back' was enough.
The problem is, that `be called back' assumes a single C thread running,
which means you're in trouble when you want to program a callback function
in lisp to be passed to some C library (like a MIDI library, or the
Performer visual simulation toolkit for SGI). The parallel C thread cannot
just call lisp because this might happen in the middle of a GC (even worse,
what's the lisp context for that call?). Also, it is not possible to write
low-level threads in lisp only with a reasonable degree of performance.
I do not know. I guess that 95% of the time, the OS thread mechanism is
enough. For that 5% you can have a more complex mechanism, if you want
(after all, they are already programmed).
And if you want to push lisp into the `mainstream', what other mechanisms
do they know, after all?
PS: By the way, upto now Performer FORKS its processes, which could be a problem
even in an sproc-based lisp implementation. However, I've seen the developers
at SIGGRAPH, and they promised (they deemed it very easy) a switch so you'll
be able to set up Performer to SPROC its processes. I've done my part, now
it's up to the lisp implementors to fix the other side so as to show that
lisp can kick butt for rapid prototyping/deployment of Visual Sim/VR apps.
By the way, you know what sells Performer? Just that, _rapid_prototyping_
and graphics tuning. Oh, I forgot, I when you do a two-week
gotta-show-something-to-get-real-nonresearch-project-money consulting,
every core dump counts.
Hey, you can't say an Onyx with 8 processors is not fast enough to do
your VR in lisp! (watch for GC bites, though).
--
F.D. Mato Mira
Computer Graphics Lab ········@epfl.ch
EPFL FAX: +41 (21) 693-5328
In article <··········@disuns2.epfl.ch> ········@di.epfl.ch (Fernando Mato Mira) writes:
The problem is, that `be called back' assumes a single C thread running,
which means you're in trouble when you want to program a callback function
in lisp to be passed to some C library (like a MIDI library, or the
Performer visual simulation toolkit for SGI). The parallel C thread cannot
just call lisp because this might happen in the middle of a GC (even worse,
what's the lisp context for that call?).
This is only a problem if you let the assembly language (C) be in control.
If the Lisp is in control, the only reason you are spending time in C space
is because some Lisp thread called some C function. Presumably then the
callback from C into Lisp should be executed in that same thread.
Bill
--
Bill Janssen <·······@parc.xerox.com> (415) 812-4763 FAX: (415) 812-4777
Xerox Palo Alto Research Center, 3333 Coyote Hill Rd, Palo Alto, CA 94304
URL: ftp://parcftp.parc.xerox.com/pub/ilu/misc/janssen.html
From: Fernando Mato Mira
Subject: Re: C is faster than lisp (lisp vs c++ / Rick Graham...)
Date:
Message-ID: <33fk1q$fil@disuns2.epfl.ch>
In article <·····················@holmes.PARC.Xerox.Com>, ·······@parc.xerox.com (Bill Janssen) writes:
> In article <··········@disuns2.epfl.ch> ········@di.epfl.ch (Fernando Mato Mira) writes:
>
> The problem is, that `be called back' assumes a single C thread running,
> which means you're in trouble when you want to program a callback function
> in lisp to be passed to some C library (like a MIDI library, or the
> Performer visual simulation toolkit for SGI). The parallel C thread cannot
> just call lisp because this might happen in the middle of a GC (even worse,
> what's the lisp context for that call?).
>
> This is only a problem if you let the assembly language (C) be in control.
> If the Lisp is in control, the only reason you are spending time in C space
> is because some Lisp thread called some C function. Presumably then the
> callback from C into Lisp should be executed in that same thread.
I thought we were talking here about what's lacking in Lisp to
be more competitive in `the real world'. In this case `the real world'
means being able to interface with existing tools, in this case, a
graphics toolkit developped by SGI, which supposedly optimizes the use
of the graphics hardware in their very diverse configurations.
Now, how am I going to be `in control' of some already existing tool?
You can only be `in control' to the point it allows to do so.
Performer is more like an `engine' than a library. It is designed to
do constant-frame-rate rendering (if you want, and normally you do in
Visual Simulation apps). An application will tipically:
read database / setup sim
while (!quit) {
do non latency-critical stuff
/* wait until next frame boundary */
pfSync();
update viewpoint
/* initiate traversal using current state */
pfFrame();
}
This is the application loop. pfFrame() resumes the pipeline
processes to render the next frame. There can typically be a delay
of two frames between the time you define it in the application, and
the time it gets rendered, as you can set up the toolkit to spawn many
processes.
For example, let's say you have a flight simulator application, and that
you have a big projection display consisting of 3 screens in a configuration
like this:
/---\
/ o \
^
|
pilot
What's more, the guy is wearing stereo glasses, so that the screens
are actually refreshed at 120Hz, with views interleaved for each eye.
That makes then for 6 visual channels in total.
You've got the money, so you buy an Onyx rack with 3 RealityEngine2
pipelines, and 6-channel multichannel option, and 10 processors.
Then, you can set up your machine, for example, like:
Proc 0: Unix/other stuff (I do not want other things to mess up timing)
Proc 1: app main
Proc 2: plane dynamics (this eats up a lot of CPU, it deserves one)
Proc 3: input processing (for real-time guarantees)
Proc 4: Culling process 1
Proc 5: Culling process 2
Proc 6: Culling process 3
Proc 7: Draw process 1
Proc 8: Draw process 2
Proc 9: Draw process 3
Each cull/draw pair is taking care of rendering 2 channels
(left/right view).
OK. This post is pretty long by now. Let me just ask:
If you want to be `in control', who is going to implement a Performer
in lisp? _Not_ SGI. And then, who else has got the know-how of the
machine/OS/graphic libs internals to do it?
I would have liked Symbolics Graphics Division to make it, but
they didn't, so that's the world right now, and that's why I
have to worry about how to hook these things up.
--
F.D. Mato Mira
Computer Graphics Lab ········@epfl.ch
EPFL FAX: +41 (21) 693-5328
From: Fernando Mato Mira
Subject: Threads (was: C is faster than lisp (lisp vs c++ / Rick Graham...))
Date:
Message-ID: <33kr5l$bkf@disuns2.epfl.ch>
In article <··········@cogsci.ed.ac.uk>, ····@aiai.ed.ac.uk (Jeff Dalton) writes:
> Why is that a problem? Let's take a fairly ordinary Lisp
> implementation in which GC is a subroutine that "cons" and some
> other operations might call. Now, if that subroutine is running,
> then the parallel C thread isn't, because lw processes don't
> provide real parallelism.
First, there can be "real parallelism" very easily, given the
current "symmetric multiprocessing" workstation trend (not the
lisp lw processes, the Unix threads).
Even if you have only one processor, let's say you call a C routine
which sprocs some Unix threads and returns (for example: a MIDI
library whose initialization routine spawns a couple of processes
to read/write from the MIDI ports. I developped stuff like that
to port a C application from Mach (NeXT), and used it in the porting
I did of the MIDI support of Common Music for SGI. You can get this
from CCRMA, midi/sgi subdir if you want to look at the code (could
be cleaner)).
So, the C routine you called from lisp returns, but there's now
another Unix threads running in the same VM space.
If these routines stay in "C space", everything is OK. However,
they can't call lisp. Why? Because as the lisp is a separate thread,
it gets multitasked and it my happen that just before you tried to
call it, it started a GC and then gets the processor preempted. So,
when you try to call something now, the lisp space might be in an
inconsistent state for lisp code to execute.
> Also what is the "lisp context"?
Lisp process, for one. The implementation should provide a facility
so that the C thread can create a lisp lightweight process corresponding
to itself.
Somebody said that OS compatibility should not imply leaving
a `more powerful multiprocessing mechanism' (I would like an example,
to get an idea). I said "OK. the multiprocessing implementation in
lisp is there (if you have the right compiler). Do one using the
OS threads, and you can still lisp-multitask in one of these".
Now, the question is, could these two different types of threads
be made compatible at the lisp level (if an OS thread has no children),
are these OS threads stack groups, really, or we need another
name/abstraction for them?
He also said that "In the same way, you wouldn't want to leave ADA tasks".
Now, how do ADA tasks relate to Unix processes (is there a POSIX
equivalent for sproc?).
--
F.D. Mato Mira
Computer Graphics Lab ········@epfl.ch
EPFL FAX: +41 (21) 693-5328
In article <··········@disuns2.epfl.ch> ········@di.epfl.ch (Fernando Mato Mira) writes:
In article <··········@cogsci.ed.ac.uk>, ····@aiai.ed.ac.uk (Jeff Dalton) writes:
> Why is that a problem? Let's take a fairly ordinary Lisp
> implementation in which GC is a subroutine that "cons" and some
> other operations might call. Now, if that subroutine is running,
> then the parallel C thread isn't, because lw processes don't
> provide real parallelism.
First, there can be "real parallelism" very easily, given the
current "symmetric multiprocessing" workstation trend (not the
lisp lw processes, the Unix threads).
The posting is stimulating, but is is still somewhat far away from
current practice. First of all, you are mentioning 'sproc' which I
infer is a SGI thing. The POSIX threads are not widely available. They
will be, but once again they are OS features that will have to be
dealt with within the OS dependent layer of the language.
...
So, the C routine you called from lisp returns, but there's now
another Unix threads running in the same VM space.
If these routines stay in "C space", everything is OK. However,
they can't call lisp. Why? Because as the lisp is a separate thread,
it gets multitasked and it my happen that just before you tried to
call it, it started a GC and then gets the processor preempted. So,
when you try to call something now, the lisp space might be in an
inconsistent state for lisp code to execute.
Well, you could always spawn a C routine that would clear your files
from your directory behind your back. :)
Seriously, this argument (which you somewhat reinforce when speaking
about ADA) speaks for the "one language per OS". Hence, since
virtually all OS' are implemented with C today, this would leave us
with a single useable language.
...
I said "OK. the multiprocessing implementation in
lisp is there (if you have the right compiler). Do one using the
OS threads, and you can still lisp-multitask in one of these".
Now, the question is, could these two different types of threads
be made compatible at the lisp level (if an OS thread has no children),
are these OS threads stack groups, really, or we need another
name/abstraction for them?
I do not see wy it shouldn't be possible. Note that the lack of proper
multithread system calls in UN*X is what prevented ADA (and CL) to
have true thread primitives in the language. Remember POSIX threads
are not available everywhere yet. My machine still uses SunOS 4.1.1:
there is a lightweight library, but if I had been in the business to
develop an ADA or CL implentation a few years ago, I would have not
counted on it as a "standard" LWP library. The same for 'sproc' and
friends.
He also said that "In the same way, you wouldn't want to leave ADA tasks".
Now, how do ADA tasks relate to Unix processes (is there a POSIX
equivalent for sproc?).
From corridor chat with the ADA men here at NYU, the current
experimental GNAT (ADA) runtime uses an implementation of the POSIX
threads proposal on the Suns (which is defined to be still a "moving
target"). I remember reading somewhere that an ADA implementation (not
NYU) on the VxWorks RT OS, was able to map ADA tasks with VxWorks
tasks (called through a C interface to the OS). I do not see why a
"new implementation" of a language (any language) shouldn't take
advantage of the "available" facilities of the hosting OS.
Happy Lisping
--
Marco Antoniotti - Resistente Umano
-------------------------------------------------------------------------------
Robotics Lab | room: 1220 - tel. #: (212) 998 3370
Courant Institute NYU | e-mail: ·······@cs.nyu.edu
...e` la semplicita` che e` difficile a farsi.
...it is simplicity that is difficult to make.
Bertholdt Brecht
In article <··········@disuns2.epfl.ch> ········@di.epfl.ch (Fernando Mato Mira) writes:
From: ········@di.epfl.ch (Fernando Mato Mira)
Newsgroups: comp.lang.lisp
Date: 30 Aug 1994 14:24:53 GMT
Organization: Ecole Polytechnique Federale de Lausanne
Lines: 166
Distribution: world
Here's an answer that appeared in comp.lang.ada .
Seems Common Lisp implementations only are being autistic, not
Ada ones.
LOTS DELETED >>>>>>>>>>>>>>>>>>>>>>>>
--
F.D. Mato Mira
Computer Graphics Lab ········@epfl.ch
EPFL FAX: +41 (21) 693-5328
Thanks for the informative response. But your comment on Common Lisp
is not con-divisible.
Of course ADA has plenty or True Multitasking implementations. But:
(1) Ada has concurrency built in the language (Common Lisp and C/C++ do not)
(2) UN*X threading kernels have not being available until recently
(the SGI 'sproc' is a SysCall which calls for totally unportable
code). If you plan to run your code exclusively on SGI machines,
fine. But this is not generally the case for tons of other applications
(3) The lack of the above (not talking about RT OS' where CL is not at
home) made ADA a total headache to implement on your regular Sun
or SGI.
This is the problem. Now that we are going to have a reasonable
standard for Threads (POSIX) the ground will be firmer for new
implementations. What is needed is a proposal for CL Threads within
the language, so that we will not have to juggles among N different
implementations (note that Allegro, Lucid and Harlequin do offer some
Multiprocessing extension).
Happy Lisping
--
Marco Antoniotti - Resistente Umano
-------------------------------------------------------------------------------
Robotics Lab | room: 1220 - tel. #: (212) 998 3370
Courant Institute NYU | e-mail: ·······@cs.nyu.edu
...e` la semplicita` che e` difficile a farsi.
...it is simplicity that is difficult to make.
Bertholdt Brecht
In article <·····················@graphics.cs.nyu.edu>,
·······@graphics.cs.nyu.edu (Marco Antoniotti) writes:
[ comments about POSIX and multiproceesing not being part of CL deleted]
I know and I agree. It's probably a good thing that multiprocessing
has not been standardized in CL so that it can now be approached with
POSIX 1003.4 (real time) and POSIX 1003.4a (threads) compatibility in
mind.
--
F.D. Mato Mira
Computer Graphics Lab ········@epfl.ch
EPFL FAX: +41 (21) 693-5328
From: Marcus Daniels
Subject: Re: Threads (was: C is faster than lisp (lisp vs c++ / Rick Graham...))
Date:
Message-ID: <MARCUS.94Aug27065700@tdb.ee.pdx.edu>
In-reply-to: ········@di.epfl.ch's message of 26 Aug 1994 10:03:58 -0400
>>>>> "matomira" == Fernando Mato Mira <········@di.epfl.ch> writes:
In article <··········@disuns2.epfl.ch> ········@di.epfl.ch (Fernando Mato Mira) writes:
matomira> So, the C routine you called from lisp returns, but there's
matomira> now another Unix threads running in the same VM space. If
matomira> these routines stay in "C space", everything is OK. However,
matomira> they can't call lisp. Why? Because as the lisp is a separate
matomira> thread, it gets multitasked and it my happen that just
matomira> before you tried to call it, it started a GC and then gets
matomira> the processor preempted. So, when you try to call something
matomira> now, the lisp space might be in an inconsistent state for
matomira> lisp code to execute.
In other words, garbage-collected (type) tagged allocation that is
performed at the system call level -- complete with shared memory, &
VM/file mapping? The remaining issues can be solved with locking &
serialization -- can they not?
Does anyone know if there any practical microkernel implementations of
distributed garbage collection? By `practical', I mean an O.S. where
there is a fully integrated development environment (e.g. Ada, C++,
LISP, Scheme, SML). Or how about prototypes that run as Unix processes?
(I've read a couple papers from INRIA's SOR project, and some other
papers on distributed garbage collection -- but I didn't find many
implementation-specifics).
In article <··········@cogsci.ed.ac.uk> ····@aiai.ed.ac.uk (Jeff Dalton) writes:
Newsgroups: comp.lang.lisp
Lines: 17
In article <··········@disuns2.epfl.ch> ········@di.epfl.ch (Fernando Mato Mira) writes:
>In article <··········@triple-i.com>, ····@triple-i.com (Kirk Rader) writes:
>
>> entirely in a commercial Common Lisp, using lisp to implement all of
> ^^^^^^^^^^^^^^^^^^^^^^^
>> the bells-and-whistles like multi-threading that a large-scale
> ^^^^^^^^^^^^^^^
>> application is likely to require.
>
>That's what I hate the most about current implementations.
>Take advantage of the OS. Do not rewrite it.
>No wonder if performance and COMPATIBILITY stink..
What exactly is the problem with providing (multiple) threads for Lisp
programs? If the OS-/C-library-provided lw process mechanism isn't
sufficient, what are we supposed to do? Give up?
I second that. If we were writing an ADA compiler for a UNIX (or were
we talking about MSDOS :-) ) system, should we give up the tasking
facilities of the language?
--
Marco Antoniotti - Resistente Umano
-------------------------------------------------------------------------------
Robotics Lab | room: 1220 - tel. #: (212) 998 3370
Courant Institute NYU | e-mail: ·······@cs.nyu.edu
...e` la semplicita` che e` difficile a farsi.
...it is simplicity that is difficult to make.
Bertholdt Brecht