From: David Bakhash
Subject: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m33djke2yk.fsf@cadet.dsl.speakeasy.net>
Hi,

Many Lisp users have happily used the MP package for multiprocessing,
whether under LW, ACL or whatever else (Genera, MCL?)

I have used it both under ACL and LW, and have had success.  But in
some recent work, I've encountered some problems that were pretty
straightforward under ACL, and I'm trying to figure out the best
approach for LW.

The multiprocessing philosophy for Common Lisp seems to be upwards
compatibility for ANSI CL, but augmenting it to have threading.  Every 
MP implementation lets these threads (I prefer to call them threads)
share global variables.  Of course, this is the behavior I like.
However, ACL also lets you very simply define any thread-specific
special variables while LW doesn't as far as I can tell.  So, say you
had a global variable that you knew a thread would mess with, but you
didnt' want other threads to be affected by any setq of that variable
in the thread that was using it.  Then, upon creation of that thread,
you could make it special inside that thread like this:

(defvar *done-message* "Done!")

(defun f ()
  (sleep 2)
  (pprint *done-message*))

(defun g ()
  (setq *done-message* "new done!")
  (f))

Now, if you do:

(mp:process-run-function
 `(:name "proc 1"
   :initial-bindings ((*done-message* . *done-message*)	 
		      ,@excl:*cl-default-special-bindings*))
 #'g)

then you happily get:

==>

"new done!"

then, if you examine the global value of *done-message*, you get:

==>

"Done!"

That's pretty nice.  You get to easily create thread-level global
variables safely, and you can even choose if you want their initial
values to inherit from the thread which created it or from the global
environment.  In addition, the dynamic value of the special variable
only affects the thread in which it was created, so other threads are
protected from sudden changes.  I don't know how to do this with LW.
It might be possible, but reading the docs has not led me to the
solution.

Of course, I can store these values in my own alist, or hashtable, and 
initialize one with the creation of each new thread.  But I'd rather
not.  I'm looking for a way which is a bit more transparent.

thanks,
dave

From: Sunil Mishra
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <39AFE149.9030406@everest.com>
So far as I can tell, both lispworks and ACL handle the concept of a
dynamic binding correctly. In other words, say you have two threads 
running, which may loosely be represented by the parallel execution of 
the following functions:

(lambda (...) (let ((*binding* 'foo)) ...)
(lambda (...) (let ((*binding* 'bar)) ...)

Then, in both lispworks and ACL, the value of *binding* will depend on 
the thread in which you query it. Now, if you do a setq on *binding* 
after creating a let-binding for *binding* I'd imagine you would get the 
desired effect.

Sunil

David Bakhash wrote:

> Hi,
> 
> Many Lisp users have happily used the MP package for multiprocessing,
> whether under LW, ACL or whatever else (Genera, MCL?)
> 
> I have used it both under ACL and LW, and have had success.  But in
> some recent work, I've encountered some problems that were pretty
> straightforward under ACL, and I'm trying to figure out the best
> approach for LW.
> 
> The multiprocessing philosophy for Common Lisp seems to be upwards
> compatibility for ANSI CL, but augmenting it to have threading.  Every 
> MP implementation lets these threads (I prefer to call them threads)
> share global variables.  Of course, this is the behavior I like.
> However, ACL also lets you very simply define any thread-specific
> special variables while LW doesn't as far as I can tell.  So, say you
> had a global variable that you knew a thread would mess with, but you
> didnt' want other threads to be affected by any setq of that variable
> in the thread that was using it.  Then, upon creation of that thread,
> you could make it special inside that thread like this:
> 
> (defvar *done-message* "Done!")
> 
> (defun f ()
>   (sleep 2)
>   (pprint *done-message*))
> 
> (defun g ()
>   (setq *done-message* "new done!")
>   (f))
> 
> Now, if you do:
> 
> (mp:process-run-function
>  `(:name "proc 1"
>    :initial-bindings ((*done-message* . *done-message*)	 
> 		      ,@excl:*cl-default-special-bindings*))
>  #'g)
> 
> then you happily get:
> 
> ==>
> 
> "new done!"
> 
> then, if you examine the global value of *done-message*, you get:
> 
> ==>
> 
> "Done!"
> 
> That's pretty nice.  You get to easily create thread-level global
> variables safely, and you can even choose if you want their initial
> values to inherit from the thread which created it or from the global
> environment.  In addition, the dynamic value of the special variable
> only affects the thread in which it was created, so other threads are
> protected from sudden changes.  I don't know how to do this with LW.
> It might be possible, but reading the docs has not led me to the
> solution.
> 
> Of course, I can store these values in my own alist, or hashtable, and 
> initialize one with the creation of each new thread.  But I'd rather
> not.  I'm looking for a way which is a bit more transparent.
> 
> thanks,
> dave
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3ya1cce1o.fsf@cadet.dsl.speakeasy.net>
Sunil Mishra <············@everest.com> writes:

> So far as I can tell, both lispworks and ACL handle the concept of a
> dynamic binding correctly. In other words, say you have two threads 
> running, which may loosely be represented by the parallel execution of 
> the following functions:
> 
> (lambda (...) (let ((*binding* 'foo)) ...)
> (lambda (...) (let ((*binding* 'bar)) ...)
> 
> Then, in both lispworks and ACL, the value of *binding* will depend on 
> the thread in which you query it. Now, if you do a setq on *binding* 
> after creating a let-binding for *binding* I'd imagine you would get the 
> desired effect.

This is essentially what I was looking for.  I'm not sure it's
"thread-safe", but I'll try it out, of course.  It basically amounts
to wrapping the actual function body around a let form.  My concern is
in the following case.  I'll start with the implementation I'd use if
the above worked.

Imagine a function which takes as arguments a function and then a list
of bindings as follows (forgive the sloppyness; I'm just trying to
make a point here):

(defmacro wrapper (fn bindings)
  `(compile nil
	    (function
	     (lambda (&rest args)
	       (let ,(eval bindings)
		 (apply ,fn args))))))

You'd call it like this:

(wrapper #'f
	 `((*binding-1* ,*binding-1*)
	   (*binding-2* 35)
           ...))

So in LW, for example, one would try something like:

(mp:process-run-function "proc 1" nil
                         (wrapper #'f
				  `((*print-case* ,*print-case*))))

hmm...I'll do a very simple check.  Sounds pretty straightforward to
me.

<doing a cursory check...>

done.  As I suspected might happen...It did not work, even in the
simplest case.  The global value of the bindings was changed when
setq'd.  You can try it yourself.  Here's a very basic test that will
show you what I mean:

---------

USER(99): (defvar *done-message* "Done!")
*DONE-MESSAGE*
USER(100): 
(defun f ()
  (sleep 2)
  (pprint *done-message*))

USER(100): F
USER(101): 
(defun g ()
  (setq *done-message* "new done!")
  (f))

USER(101): G
USER(104): *done-message*
"Done!"
USER(105): 
(mp:process-run-function "test"
			 (wrapper #'g
				  `((*done-message ,*done-message*))))
USER(105): ; While compiling (:ANONYMOUS-LAMBDA 10):
Warning: Variable *DONE-MESSAGE is never used.
#<PROCESS test @ #x204e2c4a>
USER(106): 
"new done!"
USER(106): *done-message*
"new done!"

---------

Now, if I did something wrong, or if I'm missing something very basic,
please let me know what it is.  I also tested it out with the addition
of SPECIAL declarations, and that didn't work either.  And, even if
these did work, I still couldn't be sure that everything I'm trying to
accomplish will work, which is more than the above (i.e. thread-safe;
setting value in one thread doesn't affect other threads; global
variables not affected...)

dave
From: Sunil Mishra
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <39B0266B.3050205@everest.com>
David,

I looked at your code, and decided to write my own tests that perhaps 
better demonstrate my intention. Here's an (edited) transcript. (There 
were minor modifications to the function f in the course of clarifying 
its flow, but nothing that should change the outcome.)

CL-USER 11 > (defvar *binding*)
*BINDING*

CL-USER 13 > (setq *binding* 1)
1

CL-USER 24 > (defun f (x1 x2 sleep1 sleep2)
(let ((*binding* x1))
(format t "~&~A: Value before: ~D"
(mp:process-name mp:*current-process*) *binding*)
(sleep sleep1)
(format t "~&~A: Value after first sleep: ~D; Setting to ~D."
(mp:process-name mp:*current-process*)
*binding* (setq *binding* x2))
(sleep sleep2)
(format t "~&~A: Value after second sleep: ~D."
(mp:process-name mp:*current-process*) *binding*)))
F

CL-USER 25 > (compile 'f)
F
NIL
NIL

CL-USER 26 > (progn (mp:process-run-function "Test thread one" nil #'f 6 
8 2 2)
(mp:process-run-function "Test thread two" nil #'f 20 30 1 5))
#<MP:PROCESS Name "Test thread two" Priority 850000 State "Running">

CL-USER 27 >
Test thread one: Value before: 6
Test thread two: Value before: 20
Test thread two: Value after first sleep: 20; Setting to 30.
Test thread one: Value after first sleep: 6; Setting to 8.
Test thread one: Value after second sleep: 8.
Test thread two: Value after second sleep: 30.

CL-USER 28 > *binding*
1

I am thus able to set the value of *binding* independently in the two 
threads. Is this not what you were looking for?

Sunil

David Bakhash wrote:

> Sunil Mishra <············@everest.com> writes:
> 
> > So far as I can tell, both lispworks and ACL handle the concept of a
> > dynamic binding correctly. In other words, say you have two threads 
> > running, which may loosely be represented by the parallel execution of 
> > the following functions:
> > 
> > (lambda (...) (let ((*binding* 'foo)) ...)
> > (lambda (...) (let ((*binding* 'bar)) ...)
> > 
> > Then, in both lispworks and ACL, the value of *binding* will depend on 
> > the thread in which you query it. Now, if you do a setq on *binding* 
> > after creating a let-binding for *binding* I'd imagine you would get the 
> > desired effect.
> 
> This is essentially what I was looking for.  I'm not sure it's
> "thread-safe", but I'll try it out, of course.  It basically amounts
> to wrapping the actual function body around a let form.  My concern is
> in the following case.  I'll start with the implementation I'd use if
> the above worked.
> 
> Imagine a function which takes as arguments a function and then a list
> of bindings as follows (forgive the sloppyness; I'm just trying to
> make a point here):
> 
> (defmacro wrapper (fn bindings)
>   `(compile nil
> 	    (function
> 	     (lambda (&rest args)
> 	       (let ,(eval bindings)
> 		 (apply ,fn args))))))
> 
> You'd call it like this:
> 
> (wrapper #'f
> 	 `((*binding-1* ,*binding-1*)
> 	   (*binding-2* 35)
>            ...))
> 
> So in LW, for example, one would try something like:
> 
> (mp:process-run-function "proc 1" nil
>                          (wrapper #'f
> 				  `((*print-case* ,*print-case*))))
> 
> hmm...I'll do a very simple check.  Sounds pretty straightforward to
> me.
> 
> <doing a cursory check...>
> 
> done.  As I suspected might happen...It did not work, even in the
> simplest case.  The global value of the bindings was changed when
> setq'd.  You can try it yourself.  Here's a very basic test that will
> show you what I mean:
> 
> ---------
> 
> USER(99): (defvar *done-message* "Done!")
> *DONE-MESSAGE*
> USER(100): 
> (defun f ()
>   (sleep 2)
>   (pprint *done-message*))
> 
> USER(100): F
> USER(101): 
> (defun g ()
>   (setq *done-message* "new done!")
>   (f))
> 
> USER(101): G
> USER(104): *done-message*
> "Done!"
> USER(105): 
> (mp:process-run-function "test"
> 			 (wrapper #'g
> 				  `((*done-message ,*done-message*))))
> USER(105): ; While compiling (:ANONYMOUS-LAMBDA 10):
> Warning: Variable *DONE-MESSAGE is never used.
> #<PROCESS test @ #x204e2c4a>
> USER(106): 
> "new done!"
> USER(106): *done-message*
> "new done!"
> 
> ---------
> 
> Now, if I did something wrong, or if I'm missing something very basic,
> please let me know what it is.  I also tested it out with the addition
> of SPECIAL declarations, and that didn't work either.  And, even if
> these did work, I still couldn't be sure that everything I'm trying to
> accomplish will work, which is more than the above (i.e. thread-safe;
> setting value in one thread doesn't affect other threads; global
> variables not affected...)
> 
> dave
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3r973d9iw.fsf@cadet.dsl.speakeasy.net>
David Bakhash <·····@alum.mit.edu> writes:

> (mp:process-run-function "test"
> 			 (wrapper #'g
> 				  `((*done-message ,*done-message*))))
> USER(105): ; While compiling (:ANONYMOUS-LAMBDA 10):
> Warning: Variable *DONE-MESSAGE is never used.
> #<PROCESS test @ #x204e2c4a>
> USER(106): 
> "new done!"
> USER(106): *done-message*
> "new done!"

How'd you guys let me get this one by you?  I mis-typed *done-message* 
as *done-message.  (without the trailing `*').

Anyway, adding it didn't do the right thing, at least for ACL.  I
guess the thing is that in ACL, when you start a new thread, it does
not inherit *anything* from the current environment unless you
specifically ask it to.  So I still have to test this out with LW.

dave
From: Tim Bradshaw
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <ey3itsfsovz.fsf@tfeb.org>
* David Bakhash wrote:
> (defmacro wrapper (fn bindings)
>   `(compile nil
> 	    (function
> 	     (lambda (&rest args)
> 	       (let ,(eval bindings)
> 		 (apply ,fn args))))))

> You'd call it like this:

> (wrapper #'f
> 	 `((*binding-1* ,*binding-1*)
> 	   (*binding-2* 35)
>            ...))

I'm not sure if I understand this code, but you can have nasty cases
where you evaluate things in the wrong context, which I suspect may be
the case here.  Something like this works for me.

 (defmacro rebinding ((&rest variables) &body body)
   `(let ,(mapcar #'(lambda (v)
		      (list v v))
	   variables)
      ,@body))

 (defvar *message* "Foo")

 (defun test-it (x)
   (mp:process-run-function ""
			    #'(lambda ()
				(rebinding (*message*)
					   (setf *message* x)))))


you can obviously get a whole bunch more elaborate...

--tim
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3u2bzda0z.fsf@cadet.dsl.speakeasy.net>
Tim Bradshaw <···@tfeb.org> writes:

> * David Bakhash wrote:
> > (defmacro wrapper (fn bindings)
> >   `(compile nil
> > 	    (function
> > 	     (lambda (&rest args)
> > 	       (let ,(eval bindings)
> > 		 (apply ,fn args))))))
> 
> > You'd call it like this:
> 
> > (wrapper #'f
> > 	 `((*binding-1* ,*binding-1*)
> > 	   (*binding-2* 35)
> >            ...))
> 
> I'm not sure if I understand this code, but you can have nasty cases
> where you evaluate things in the wrong context, which I suspect may be
> the case here.  Something like this works for me.

well, I don't understand what's not to understand about it.  I wanted
a wrapper that takes a function and returns the same function, but
somehow with the thread-local variables protected.

If you'd macroexpanded my code, you'd see something like this:

USER(107): 
(pprint
 (macroexpand-1
  '(wrapper #'f
	    `((*done-message ,*done-message*)))))
USER(107): 
(COMPILE NIL
         #'(LAMBDA (&REST ARGS)
             (LET ((*DONE-MESSAGE "Done!"))
               (APPLY #'F ARGS))))

Which basically does exactly what the post from Sunil suggested, as
far as I know.  Basically, the suggestion was to wrap the function
around a LET, binding the variables that were intended to be
protected.  I just threw in the compile because I'm used to compiling
functions at all times, since ultimately that's what I try to do
anyway.  I don't really see the ambiguity, yet I like this kind of
interface, since it lets me easily and expressively transform an
existing function and add the new local bindings as I want to add
them.


>  (defmacro rebinding ((&rest variables) &body body)
>    `(let ,(mapcar #'(lambda (v)
> 		      (list v v))
> 	   variables)
>       ,@body))

Yes.  I think I see where the problem lies.  I can't just make a
closure where I create a new function which is the old function, but
inside a LET, and expect everything to work.  Why?  I don't know.  But
I can tell you that this REBINDING macro does not do the right thing
as I would use it.

Instead of going back-and-fourth with code snippets, I'd like to ask
for the following:

provide a macro (like the one I wrote) which takes:

 1) a function object
 2) a list of bindings that will be thread-local, with initial values
    (also as mine does)

and returns a new function that I can equivalently pass to
#'MP:PROCESS-RUN-FUNCTION that will protect these variables, and do
#the right thing as it was described.

I specifically do *not* want to re-write the function.  I would like
to have a work-around that does not require me to have the code inside 
the function itself.  I want to somehow encapsulate a compiled
function object, and have a nice, simple way to pass the new function
to #'MP:PROCESS-RUN-FUNCTION.  I'll keep trying random stuff, and if I 
figure it out I'll post my solution.  I just hope it works with LW,
ACL, CMUCL, etc.

dave
From: Tim Bradshaw
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <ey3g0njrry9.fsf@tfeb.org>
* David Bakhash wrote:

> provide a macro (like the one I wrote) which takes:

>  1) a function object
>  2) a list of bindings that will be thread-local, with initial values
>     (also as mine does)

> and returns a new function that I can equivalently pass to
> #'MP:PROCESS-RUN-FUNCTION that will protect these variables, and do
> #the right thing as it was described.

This does that:

 (defmacro wrap/rebinding (f bindings)
   `#'(lambda (&rest args)
	(declare (dynamic-extent args))	;?
	(let ,(mapcar #'(lambda (vd)
			  (etypecase vd
			    ;; note different semantics to LET --
			    ;; naked symbol means to rebind to current
			    ;; value
			    (cons vd)
			    (symbol (list vd vd))))
	       bindings)
	  (apply ,f args))))

I was worried about yours because I think the macro calls compile at
runtime, which is probably not what you want -- better to rely on the
compiler to compile the expansion, and also I just didn't understand
it because of the eval...

But it turns out that it's just a typo.  You wrote:

    (mp:process-run-function "test"
			     (wrapper #'g
				      `((*done-message ,*done-message*))))
    USER(105): ; While compiling (:ANONYMOUS-LAMBDA 10):
    Warning: Variable *DONE-MESSAGE is never used.
    #<PROCESS test @ #x204e2c4a>
    USER(106): 
    "new done!"
    USER(106): *done-message*
    "new done!"

But you're rebinding *DONE-MESSAGE not *DONE-MESSAGE*!

--tim
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3itsfchci.fsf@cadet.dsl.speakeasy.net>
Tim Bradshaw <···@tfeb.org> writes:

> This does that:
> 
>  (defmacro wrap/rebinding (f bindings)
>    `#'(lambda (&rest args)
> 	(declare (dynamic-extent args))	;?
> 	(let ,(mapcar #'(lambda (vd)
> 			  (etypecase vd
> 			    ;; note different semantics to LET --
> 			    ;; naked symbol means to rebind to current
> 			    ;; value
> 			    (cons vd)
> 			    (symbol (list vd vd))))
> 	       bindings)
               ^^^^^^^^ (should be: ,bindings ???)
> 	  (apply ,f args))))

I think I see a typo again, though I havn't tested this:

`bindings' above should be comma'd out.  Which probably means that you 
havn't actually tested this out either.  So I'll test it.  No problem.

> I was worried about yours because I think the macro calls compile at
> runtime, which is probably not what you want -- better to rely on the
> compiler to compile the expansion, and also I just didn't understand
> it because of the eval...
> 
> But it turns out that it's just a typo.  You wrote:
> 
>     (mp:process-run-function "test"
> 			     (wrapper #'g
> 				      `((*done-message ,*done-message*))))

> But you're rebinding *DONE-MESSAGE not *DONE-MESSAGE*!

Yeah.  See my follow-up post on that.  I noticed it shortly
thereafter, but it actually didn't make a difference (i.e. it didn't
fix the problem; it only changed to another behavior which was also
not what I wanted).

dave
From: Tim Bradshaw
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <ey34s3ysvwi.fsf@tfeb.org>
* David Bakhash wrote:

> I think I see a typo again, though I havn't tested this:

> `bindings' above should be comma'd out.  Which probably means that you 
> havn't actually tested this out either.  So I'll test it.  No problem.

no, as the whole (mapcar ...) form was within a comma.  Here's what
happens on acl with this:

 (defvar *x* 1)

 (defun test-it-1 (x)
   (mp:process-run-function "foo"
			    (wrap/rebinding #'(lambda ()
						(setf *x* x))
					    (*x*))))

 CL-USER(9): *x*
 1
 CL-USER(10): (test-it-1 2)
 #<multiprocessing:process foo @ #x4582df2>
 CL-USER(11): *x*
 1

Here's a more elaborate version of basically the same thing.

 (defvar *variables-to-rebind* '())

 (defmacro define-rebinding-variable (v &optional (value nil valuep))
   `(progn
      (let ((found (assoc ',v *variables-to-rebind*)))
	(if found
	    (setf (cadr found) (if ,valuep ',value ',v))
	    (push (list ',v (if ,valuep ',value ',v)) *variables-to-rebind*)))
      (setf (symbol-function 'process-run-function)
	(compile nil 
		 `(lambda (name f &rest args)
		    (block process-run-function
		      (mp:process-run-function 
		       name
		       #'(lambda ()
			   (let ,(mapcar #'(lambda (vd)
					     (etypecase vd
					       (cons vd)
					       (symbol (list vd vd))))
				  *variables-to-rebind*)
			     (apply f args))))))))
      ',v))

 (defvar *x* 1)
 (define-rebinding-variable *x*)

 (defun test-it-2 (new)
   (process-run-function "foo" 
			 #'(lambda ()
			     (setf *x* new))))

(Note this assumes that PROCESS-RUN-FUNCTION and
MP:PROCESS-RUN-FUNCTION are different!)

--tim
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3wvguc3ux.fsf@cadet.dsl.speakeasy.net>
Tim Bradshaw <···@tfeb.org> writes:

> * David Bakhash wrote:
> 
> > I think I see a typo again, though I havn't tested this:
> 
> > `bindings' above should be comma'd out.  Which probably means that you 
> > havn't actually tested this out either.  So I'll test it.  No problem.
> 
> no, as the whole (mapcar ...) form was within a comma.  Here's what
> happens on acl with this:

damn this newsserver!  I canceled that article about 10 seconds after
I left, when I saw the ,(mapcar ...)

anyway, sorry about that.

dave
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3bsy7cgvi.fsf@cadet.dsl.speakeasy.net>
Tim Bradshaw <···@tfeb.org> writes:

> I was worried about yours because I think the macro calls compile at
> runtime, which is probably not what you want -- better to rely on the
> compiler to compile the expansion, and also I just didn't understand
> it because of the eval...
> 
> But it turns out that it's just a typo.  You wrote:
> 
>     (mp:process-run-function "test"
> 			     (wrapper #'g
> 				      `((*done-message ,*done-message*))))

> But you're rebinding *DONE-MESSAGE not *DONE-MESSAGE*!

Yeah.  See my follow-up post on that.  I noticed it shortly
thereafter, but it actually didn't make a difference (i.e. it didn't
fix the problem; it only changed to another behavior which was also
not what I wanted).

dave
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m33djjc56a.fsf@cadet.dsl.speakeasy.net>
Tim Bradshaw <···@tfeb.org> writes:

> I was worried about yours because I think the macro calls compile at
> runtime, which is probably not what you want -- better to rely on
> the compiler to compile the expansion, and also I just didn't
> understand it because of the eval...

As far as I know, the EVAL in mine was just there to make the
macroexpansion output look the way I wanted it to.  I posted it really 
quickly, and didn't put too much thought into it.  However, for the
purposes of compile-time macroexpansion it should be fine, especially
since I tested the macroexpansion.

I wanted to be able to pass in BINDINGS such that it could be a
(back-)quoted list, allowing the user to easily choose what objects
were quoted and what weren't, without having to use #'LIST.  I used
the EVAL to undo the quote, and since BINDINGS was already inside the
BACKQUOTE in the macro it already was comma'd out.

In other words, since evaluation undoes quoting, I used it.

Hope that explains the eval.  I still don't think it makes a
difference, since the macroexpansion was correct, i.e.:

> USER(107): 
> (pprint
>  (macroexpand-1
>   '(wrapper #'f
> 	    `((*done-message ,*done-message*)))))
> USER(107): 
> (COMPILE NIL
>          #'(LAMBDA (&REST ARGS)
>              (LET ((*DONE-MESSAGE "Done!"))
>                (APPLY #'F ARGS))))

Incidentally, I tried this without the #'COMPILE call, and it did the
same thing.

Someone posted a solution for LW that will probably work just fine.
It uses a special variable that I wasn't aware of.  Since I was gonna
write a macro that would make the #'MP:PROCESS-RUN-FUNCTION in LW
resemble the one in ACL anyway, this variable will come in handy to
deal with the alist that ACL lets you pass in.

dave
From: Tim Bradshaw
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <ey3aedrrq1x.fsf@tfeb.org>
* I wrote:
> I was worried about yours because I think the macro calls compile at
> runtime, which is probably not what you want -- better to rely on the
> compiler to compile the expansion [...]

What I *meant* was `better to rely on the expansion being processed by
the compiler in the normal way' or something.

Incidentally, for my one, in ACL, I get the answer I (and you I think)
expect -- the top-level value of a special does not get clobbered.

--tim
From: Kent M Pitman
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <sfwhf80cb6c.fsf@world.std.com>
David Bakhash <·····@alum.mit.edu> writes:

> Many Lisp users have happily used the MP package for multiprocessing,
> whether under LW, ACL or whatever else (Genera, MCL?)

Heh.  For varying values of happy.  The argument conventions are not the same
and it drives me nuts.  I can't believe we as a community can't come to some
consensus.
From: Lieven Marchand
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3n1hryf9x.fsf@localhost.localdomain>
David Bakhash <·····@alum.mit.edu> writes:

> Hi,
> 
> Many Lisp users have happily used the MP package for multiprocessing,
> whether under LW, ACL or whatever else (Genera, MCL?)
> 
> I have used it both under ACL and LW, and have had success.  But in
> some recent work, I've encountered some problems that were pretty
> straightforward under ACL, and I'm trying to figure out the best
> approach for LW.
> 

This works for me:

<Other definitions identical to yours elided>

CL-USER 7 > (let ((mp:*process-initial-bindings* (list (cons '*done-message* *done-message*))))
              (mp:process-run-function "proc 1" nil #'g))

#<MP:PROCESS Name "proc 1" Priority 850000 State "Running">
CL-USER 8 > *done-message*
"Done!"

You get the output "new done!" in the tty you started Lispworks in,
not in the listener or the output browser.

-- 
Lieven Marchand <···@bewoner.dma.be>
Lambda calculus - Call us a mad club
From: David Bakhash
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m38ztbcfto.fsf@cadet.dsl.speakeasy.net>
Lieven Marchand <···@bewoner.dma.be> writes:

> This works for me:
> 
> <Other definitions identical to yours elided>
> 
> CL-USER 7 > (let ((mp:*process-initial-bindings* (list (cons '*done-message* *done-message*))))
>               (mp:process-run-function "proc 1" nil #'g))
> 
> #<MP:PROCESS Name "proc 1" Priority 850000 State "Running">
> CL-USER 8 > *done-message*
> "Done!"
> 
> You get the output "new done!" in the tty you started Lispworks in,
> not in the listener or the output browser.

So far, this is great.  Now would have liked to bind
MP:*PROCESS-INITIAL-BINDINGS* a bit later (i.e. around the function
call itself), but it's not a big deal.  It might be nice to write a
macro around #'MP:PROCESS-RUN-FUNCTION.

One concern I have is that of simultaneous threads all having the same 
initial bindings variables.  I just wonder if they are kept separate.
If they are, then this is exactly the fix I was looking for.

thanks so much,
dave
From: Lieven Marchand
Subject: Re: MP stuff (issues, concerns, work-arounds)
Date: 
Message-ID: <m3r972511w.fsf@localhost.localdomain>
David Bakhash <·····@alum.mit.edu> writes:

> So far, this is great.  Now would have liked to bind
> MP:*PROCESS-INITIAL-BINDINGS* a bit later (i.e. around the function
> call itself), but it's not a big deal.  It might be nice to write a
> macro around #'MP:PROCESS-RUN-FUNCTION.
> 
> One concern I have is that of simultaneous threads all having the same 
> initial bindings variables.  I just wonder if they are kept separate.
> If they are, then this is exactly the fix I was looking for.

They seem to be:

CL-USER 2 > (defvar *done-message* "Done!")
*DONE-MESSAGE*

CL-USER 3 > (defun f (n)
              (sleep n)
              (pprint *done-message*))
F

CL-USER 11 > (defun g (n)
              (setq *done-message* (format nil "new done: ~A!" n))
              (f n))
G

CL-USER 12 > (let ((mp:*process-initial-bindings* (cons (cons '*done-message* *done-message*) mp:*process-initial-bindings*)))
              (mp:process-run-function "proc 1" nil #'(lambda () (g 2)))
              (mp:process-run-function "proc 2" nil #'(lambda () (g 10))))

#<MP:PROCESS Name "proc 2" Priority 850000 State "Running">

CL-USER 13 > *done-message*
"Done!"

And as output:

"new done: 2!"
"new done: 10!"

I seem to recall someone wrote a portable process library and
announced it here some time ago. That implementation together with a
write up of the semantics could perhaps be entered as a proposal to
X3J13.

-- 
Lieven Marchand <···@bewoner.dma.be>
Lambda calculus - Call us a mad club
From: David Bakhash
Subject: Re: MP stuff (and a trailing CMUCL question)
Date: 
Message-ID: <m3zolqc43r.fsf_-_@cadet.dsl.speakeasy.net>
Lieven Marchand <···@bewoner.dma.be> writes:

> CL-USER 12 > (let ((mp:*process-initial-bindings* (cons (cons '*done-message* *done-message*) mp:*process-initial-bindings*)))
>               (mp:process-run-function "proc 1" nil #'(lambda () (g 2)))
>               (mp:process-run-function "proc 2" nil #'(lambda () (g 10))))

Just a note, though it's more stylistic:

#'MP:PROCESS-RUN-FUNCTION has it's last argument as an &rest, so you
could have said:

(mp:process-run-function "proc 1" nil #'g 2)

Anyway, your test is valid, and I am convinced and very happy that my
question was answered.  I'm much more comfortable with this than using 
LET, since I feel like I'm depending on the treading implementation to 
somehow know to do the "right" thing, when the right thing is really
unspecified.  Obviously, this variable exists for a reason, and the LW 
people thought it out very well.  

> I seem to recall someone wrote a portable process library and
> announced it here some time ago. That implementation together with a
> write up of the semantics could perhaps be entered as a proposal to
> X3J13.

There's "proc.lisp" at http://clocc.sourceforge.net but I wouldn't
bother with it since it just doesn't do the right thing.  It ignores
details like what I'm doing completely.  For example, MCL might also
have a way to handle these thread-local special variables, and same
with even CormanLisp and Lucid/LCL, but these things get ignored.  And 
who the heck would have the time to first figure out if each and every 
implementation could do it, and then fix that one function to handle
it?

In most applications, there's not that much thread-calling code to
have to worry about porting problems if you suddenly switch
implementations.  But if I were suddenly changing to something like
CormanLisp, which has totally different names and conventions for
multiprocessing (i.e. its #'TH:CREATE-THREAD function), then having
used something like proc.lisp would be useful.

My last question is:

Could what I asked for be done relatively easily in CMUCL?

thanks,
dave