From: RPG
Subject: Help with SBCL multiprocessing
Date: 
Message-ID: <1136822400.689029.95040@f14g2000cwb.googlegroups.com>
I am maintaining the Garnet Common Lisp UI prototyping toolkit (see the
sourceforge project page for more information:
http://sourceforge.net/projects/garnetlisp).  I have it running
successfully on Allegro CL and CMUCL, but only partially on SBCL.  The
key stumbling block is managing the multiprocessing that handles the
event loop; without such multiprocessing, one cannot interact with a
Garnet UI and work with the REPL simultaneously, which makes Garnet
unacceptable  (IMHO) for development.

Unfortunately, I don't have the time available to dive into the details
of SBCL's multiprocessing.  I hope that some kind SBCL expert will
provide me guidance on how to replace the following code with something
SBCL-friendly.  This is the key function, and I provide the CMUCL
version, hoping that will be fairly familiar to SBCL experts.

[Please, before you get angry, if you are inclined that way --- I am
not asking anyone to do my work for me!  I don't use SBCL myself; I am
just trying to provide a piece of software to people in that community.
 I hope someone in that community will see this as worthwhile; if you
don't, thanks for your time.]



(defun launch-main-event-loop-process ()
  "Spawn a process which is doing Garnet interaction all of the time.
   RETURN the process."
   (when (mp:processp *main-event-loop-process*)
      (mp:destroy-process *main-event-loop-process*))
   (setf *main-event-loop-process*
	 (mp:make-process
            #'(lambda ()
		;; first, throw away any pending events
		(discard-all-pending-events)
		(let ((root-window (gv device-info :current-root)))
		  (loop
		   (restart-case
		    (loop
		     (inter::default-event-handler root-window))
		    (abort () :report "Discard pending X events, restart loop"
			   (discard-all-pending-events))))))
	    :name "Garnet event loop"))
   (setf mp::*idle-process* mp::*initial-process*)
   ;; Lower the timeout for serve-event to give good process response.
   (setf lisp::*max-event-to-usec* 50000)
   (setf lisp::*max-event-to-sec* 0)
   *main-event-loop-process*)

Many thanks!

From: Thomas F. Burdick
Subject: Re: Help with SBCL multiprocessing
Date: 
Message-ID: <xcvlkxojr90.fsf@conquest.OCF.Berkeley.EDU>
"RPG" <·········@gmail.com> writes:

> I am maintaining the Garnet Common Lisp UI prototyping toolkit (see the
> sourceforge project page for more information:
> http://sourceforge.net/projects/garnetlisp).  I have it running
> successfully on Allegro CL and CMUCL, but only partially on SBCL.  The
> key stumbling block is managing the multiprocessing that handles the
> event loop; without such multiprocessing, one cannot interact with a
> Garnet UI and work with the REPL simultaneously, which makes Garnet
> unacceptable  (IMHO) for development.

I don't have access to any machines on which SBCL supports MP, so I
can't help with that bit.  However, IIRC Garnet's main loop calls into
SERVE-EVENT, which would still allow a SLIME repl (using :fd-handler
communication) to function.

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | Free Mumia Abu-Jamal! |
     ,--'    _,'   | Abolish the racist    |
    /       /      | death penalty!        |
   (   -.  |       `-----------------------'
   |     ) |                               
  (`-.  '--.)                              
   `. )----'                               
From: RPG
Subject: Re: Help with SBCL multiprocessing
Date: 
Message-ID: <1136950582.351182.139790@g14g2000cwa.googlegroups.com>
Thank you for the advice.  I have discussed this a little with
Christophe Rhodes.  I think using :fd-handler communication would
require more restructuring of the existing code than I have time or
energy for, although perhaps for a true SBCL expert, this would be the
right way to go.

Unfortunately, I have only the most limited time to devote to SBCL
support for Garnet (and it's a matter of unsupported conjecture that
this will find any users if I manage it), so I must try to do something
that will salvage as much of the existing code (which is built to use
ACL, CMUCL, or LispWorks multiprocessing) as possible.

I freely admit that this may be the wrong thing, and would be happy to
adopt better solutions offered by others.

Best,
R
From: RPG
Subject: Re: Help with SBCL multiprocessing [conditionalization]
Date: 
Message-ID: <1136950935.777469.142070@g44g2000cwa.googlegroups.com>
On a slightly varying topic, I note that SBCL multiprocessing (really
sb-threads) is only available on linux x86 (or x86-64?).

Question: would it make sense to provide a :sb-thread feature on
platforms where this is available?

I ask because I was writing some sbcl threading code (probably a mess)
for Garnet, and contemplating something like #+(and sbcl linux x86),
but that is really the wrong thing to do, if there is any prospect of
sbcl threading support spreading to other platforms.  #+(and sbcl
sbcl-thread) intuitively seems like a better thing to have.

Admittedly, I have little understanding of SBCL, so this might be a
dumb idea for some reason of which I am (pace Xach!) ignorant.

Thanks,
R
From: Juho Snellman
Subject: Re: Help with SBCL multiprocessing [conditionalization]
Date: 
Message-ID: <slrnds9mpi.cns.jsnell@sbz-30.cs.Helsinki.FI>
RPG <·········@gmail.com> wrote:
> On a slightly varying topic, I note that SBCL multiprocessing (really
> sb-threads) is only available on linux x86 (or x86-64?).
> 
> Question: would it make sense to provide a :sb-thread feature on
> platforms where this is available?

That's already the case :-) The :SB-THREAD feature will be present if
and only if that SBCL installation supports threads.

> I ask because I was writing some sbcl threading code (probably a mess)
> for Garnet, and contemplating something like #+(and sbcl linux x86),
> but that is really the wrong thing to do, if there is any prospect of
> sbcl threading support spreading to other platforms.

Right. It's also incorrect the other way around, since it's possible
(and fairly common) to build a SBCL without thread support on x86.

-- 
Juho Snellman
From: RPG
Subject: Re: Help with SBCL multiprocessing [conditionalization]
Date: 
Message-ID: <1136994643.197107.122200@g14g2000cwa.googlegroups.com>
Thank you, Juho.  I hadn't realized that I was getting a version of
SBCL w/o thread support, when I built it.

BTW, would it be useful to modify the introduction to the "Threading"
chapter of the manual to state that the default build is made without
threading support?  As it stands, the introduction reads:

"SBCL supports a fairly low-level threading interface that maps onto
the
host operating system's concept of threads or lightweight processes.
This means that threads may take advantage of hardware multiprocessing
on machines that have more than one CPU, but it does not allow Lisp
control of the scheduler.  This is found in the SB-THREAD package.

"This requires x86/x86-64 and Linux kernel 2.6 or systems with NPTL
backports."

It's not obvious to the casual reader that (a) the default install will
not have this package or (b) checking for :sb-thread in the *features*
is a Good Thing to do.

Perhaps this is documented elsewhere (it is explained in the
installation notes, for example), but it might be handy to have it
here, as well.
From: Juho Snellman
Subject: Re: Help with SBCL multiprocessing [conditionalization]
Date: 
Message-ID: <slrndsb61n.fs6.jsnell@sbz-30.cs.Helsinki.FI>
RPG <·········@gmail.com> wrote:
> Thank you, Juho.  I hadn't realized that I was getting a version of
> SBCL w/o thread support, when I built it.
> 
> BTW, would it be useful to modify the introduction to the "Threading"
> chapter of the manual to state that the default build is made without
> threading support?  As it stands, the introduction reads:

Not a bad idea.

-- 
Juho Snellman
From: Thomas F. Burdick
Subject: Re: Help with SBCL multiprocessing
Date: 
Message-ID: <xcvd5iy4xn3.fsf@conquest.OCF.Berkeley.EDU>
"RPG" <·········@gmail.com> writes:

> Thank you for the advice.  I have discussed this a little with
> Christophe Rhodes.  I think using :fd-handler communication would
> require more restructuring of the existing code than I have time or
> energy for, although perhaps for a true SBCL expert, this would be the
> right way to go.
> 
> Unfortunately, I have only the most limited time to devote to SBCL
> support for Garnet (and it's a matter of unsupported conjecture that
> this will find any users if I manage it), so I must try to do something
> that will salvage as much of the existing code (which is built to use
> ACL, CMUCL, or LispWorks multiprocessing) as possible.

I didn't mean that you should re-write Garnet to act as a proper
SERVE-EVENT application.  Blocking i/o already calls into SERVE-EVENT
if you're using buffered streams.  This means that a SLIME user can
call into the Garnet main loop from the inferior lisp buffer, and the
SLIME repl will continue to function.

> I freely admit that this may be the wrong thing, and would be happy to
> adopt better solutions offered by others.

It's probably a good idea to add support for SBCL's threads, but it's
also probably more work than just trying to get it working along side
serve-event.  And of course, sb-threads only works on x86 and x86-64
machines running Linux >= 2.6.

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | Free Mumia Abu-Jamal! |
     ,--'    _,'   | Abolish the racist    |
    /       /      | death penalty!        |
   (   -.  |       `-----------------------'
   |     ) |                               
  (`-.  '--.)                              
   `. )----'                               
From: ····@hotpop.com
Subject: Re: Help with SBCL multiprocessing
Date: 
Message-ID: <1136978585.622219.148460@g49g2000cwa.googlegroups.com>
RPG wrote:
> (defun launch-main-event-loop-process ()
>   "Spawn a process which is doing Garnet interaction all of the time.
>    RETURN the process."
>    (when (mp:processp *main-event-loop-process*)
>       (mp:destroy-process *main-event-loop-process*))

  (when (typep *m-e-l-p* 'sb-thread:thread)
    (sb-thread:terminate-thread *m-e-l-p*))

>    (setf *main-event-loop-process*
> 	 (mp:make-process

  (sb-thread:make-thread

>             #'(lambda ()
> 		;; first, throw away any pending events
> 		(discard-all-pending-events)
> 		(let ((root-window (gv device-info :current-root)))
> 		  (loop
> 		   (restart-case
> 		    (loop
> 		     (inter::default-event-handler root-window))
> 		    (abort () :report "Discard pending X events, restart loop"
> 			   (discard-all-pending-events))))))
> 	    :name "Garnet event loop"))

  These is not needed:

>    (setf mp::*idle-process* mp::*initial-process*)

  Neither are these:
>    ;; Lower the timeout for serve-event to give good process response.
>    (setf lisp::*max-event-to-usec* 50000)
>    (setf lisp::*max-event-to-sec* 0)

>    *main-event-loop-process*)
> 
> Many thanks!

Cheers, Gábor Melis