Hello,
I'd like to have Erlang style micro-threads in Lisp and I'm
investigating if that's a good idea or not and how much effort it
would be to implement. My goal is to is to be able to take advantage
of multi-core cpus while eliminating the pain of locking,
synchronization, etc. After some reading, I'm thinking that
implementing the micro-threads using continuations is a possible way
to go and I'd like to hear some opinions on it.
I've got some ideas on how to implement continuations in Common Lisp,
but it's shaping up to be a lot of work and before I go to far down
that trail, I'd like to see if anyone else has gone there before me.
Jim
jim wrote:
> Hello,
>
> I'd like to have Erlang style micro-threads in Lisp and I'm
> investigating if that's a good idea or not and how much effort it
> would be to implement. My goal is to is to be able to take advantage
> of multi-core cpus while eliminating the pain of locking,
> synchronization, etc. After some reading, I'm thinking that
> implementing the micro-threads using continuations is a possible way
> to go and I'd like to hear some opinions on it.
>
> I've got some ideas on how to implement continuations in Common Lisp,
> but it's shaping up to be a lot of work and before I go to far down
> that trail, I'd like to see if anyone else has gone there before me.
>
> Jim
Although not based on micro-threads, you may find cl-muproc (http://common-lisp.net/project/cl-muproc) interesting - it
implements many significant Erlang concepts including linking, monitoring, supervisors, and generic servers -- all in
Common Lisp.
-Klaus.
On Jan 11, 3:47 pm, Klaus Harbo <·····@harbo.net> wrote:
> jim wrote:
> > Hello,
>
> > I'd like to have Erlang style micro-threads in Lisp and I'm
> > investigating if that's a good idea or not and how much effort it
> > would be to implement. My goal is to is to be able to take advantage
> > of multi-core cpus while eliminating the pain of locking,
> > synchronization, etc. After some reading, I'm thinking that
> > implementing the micro-threads using continuations is a possible way
> > to go and I'd like to hear some opinions on it.
>
> > I've got some ideas on how to implement continuations in Common Lisp,
> > but it's shaping up to be a lot of work and before I go to far down
> > that trail, I'd like to see if anyone else has gone there before me.
>
> > Jim
>
> Although not based on micro-threads, you may find cl-muproc (http://common-lisp.net/project/cl-muproc) interesting - it
> implements many significant Erlang concepts including linking, monitoring, supervisors, and generic servers -- all in
> Common Lisp.
>
> -Klaus.
Whoah awesome! I didn't know about this. I'm going to try it out
tonight.
On Jan 10, 4:08 am, jim <········@gmail.com> wrote:
> I'd like to haveErlangstylemicro-threads in Lisp and I'm
> investigating if that's a good idea or not and how much effort it
> would be to implement. My goal is to is to be able to take advantage
> of multi-core cpus while eliminating the pain of locking,
> synchronization, etc. After some reading, I'm thinking that
> implementing the micro-threads using continuations is a possible way
> to go and I'd like to hear some opinions on it.
>
> I've got some ideas on how to implement continuations in Common Lisp,
> but it's shaping up to be a lot of work and before I go to far down
> that trail, I'd like to see if anyone else has gone there before me.
It's pretty easy to implement a cross-platform erlang concurrency
model in common lisp. (See cl-muproc.) And it results to be quite
handy and useful too. But, IMHO, the real problem is the
implementation of process migration. For instance, Termite Scheme guys
needed to fiddle with Gambit internals to be able to serialize thunks.
If we'd forget platform independency for a moment, it's even a complex
task to implement function serialization for a single CL
implementation too.
Regards.
On Feb 7, 2:56 pm, Volkan YAZICI <·············@gmail.com> wrote:
> On Jan 10, 4:08 am, jim <········@gmail.com> wrote:
>
> > I'd like to haveErlangstylemicro-threads in Lisp and I'm
> > investigating if that's a good idea or not and how much effort it
> > would be to implement. My goal is to is to be able to take advantage
> > of multi-core cpus while eliminating the pain of locking,
> > synchronization, etc. After some reading, I'm thinking that
> > implementing the micro-threads using continuations is a possible way
> > to go and I'd like to hear some opinions on it.
>
> > I've got some ideas on how to implement continuations in Common Lisp,
> > but it's shaping up to be a lot of work and before I go to far down
> > that trail, I'd like to see if anyone else has gone there before me.
>
> It's pretty easy to implement a cross-platform erlang concurrency
> model in common lisp. (See cl-muproc.) And it results to be quite
> handy and useful too. But, IMHO, the real problem is the
> implementation of process migration. For instance, Termite Scheme guys
> needed to fiddle with Gambit internals to be able to serialize thunks.
> If we'd forget platform independency for a moment, it's even a complex
> task to implement function serialization for a single CL
> implementation too.
>
> Regards.
Hmmm. It seems cl-muproc doesn't actually do any networking yet, so
this still needs to happen. Even in a platform independent way, how
would you transfer functions? Do you also need to transfer state?
Could you use a special defun macro that saved the s-expressions
alongside functions so they could be sent?
-jeff
P� Fri, 08 Feb 2008 13:36:55 +0100, skrev Jeff <······@gmail.com>:
>
> Hmmm. It seems cl-muproc doesn't actually do any networking yet, so
> this still needs to happen. Even in a platform independent way, how
> would you transfer functions? Do you also need to transfer state?
> Could you use a special defun macro that saved the s-expressions
> alongside functions so they could be sent?
>
> -jeff
Most proffesional applications use CORBA for this kind of thing.
--------------
John Thingstad
"John Thingstad" <·······@online.no> writes:
> P� Fri, 08 Feb 2008 13:36:55 +0100, skrev Jeff <······@gmail.com>:
>
>>
>> Hmmm. It seems cl-muproc doesn't actually do any networking yet, so
>> this still needs to happen. Even in a platform independent way, how
>> would you transfer functions? Do you also need to transfer state?
>> Could you use a special defun macro that saved the s-expressions
>> alongside functions so they could be sent?
>>
>> -jeff
>
> Most proffesional applications use CORBA for this kind of thing.
That's a highly dubious statement... CORBA had its day, but
that was several years ago, and it never reached anything like
ubiquity. Good thing, too, because it is severely flawed, and a bitch
to use from most programming languages.
Den Fri, 08 Feb 2008 16:44:04 +0100 skrev John Thingstad:
>> Hmmm. It seems cl-muproc doesn't actually do any networking yet, so
>> this still needs to happen. Even in a platform independent way, how
>> would you transfer functions? Do you also need to transfer state?
>> Could you use a special defun macro that saved the s-expressions
>> alongside functions so they could be sent?
>
> Most proffesional applications use CORBA for this kind of thing.
CORBA is used only by people who've invested too much already to admit
that it's complete and utter crap. It's a horribly flawed implementation
on top of a horribly flawed idea, coming from people who gave us beauties
such as UML.
Cheers,
Maciej
On Feb 8, 2:36 pm, Jeff <······@gmail.com> wrote:
> Even in a platform independent way, how would you transfer functions?
> Do you also need to transfer state? Could you use a special defun macro
> that saved the s-expressions alongside functions so they could be sent?
For instance, see David Lichteblau's reply[1] to "Closure
Serialization" question at SBCL developers mailing list.
[1] http://sourceforge.net/mailarchive/message.php?msg_name=20080111231119.GC8921%40radon
Regards.
On Jan 10, 3:08 am, jim <········@gmail.com> wrote:
> Hello,
>
> I'd like to have Erlang style micro-threads in Lisp
So do I.
> and I'm investigating if that's a good idea or not
It is
> and how much effort it would be to implement.
A lot of effort.
> My goal is to is to be able to take advantage
> of multi-core cpus while eliminating the pain of locking,
> synchronization, etc.
Currently only lisp that could do SMP is scieneer, as per my knwoledge
never actually tried it,
there was a talk about smp in sbcl and allegro google this group for
more details
> After some reading, I'm thinking that
> implementing the micro-threads using continuations is a possible way
> to go and I'd like to hear some opinions on it.
It's one way. CMUCL uses his own green threads.
The problem is in the big lisp runtime, everything could change in
runtime in lisp .
Probably making a purely functional subset would be a feasible
project.
>
> I've got some ideas on how to implement continuations in Common Lisp,
> but it's shaping up to be a lot of work and before I go to far down
> that trail, I'd like to see if anyone else has gone there before me.
If you want only delimited continuations they're alrady here
http://common-lisp.net/project/cl-cont/ for the full blown call/cc you
need to poke with lisp implementations.
cheers
Slobodan
On 2008-02-08 10:43:55 -0500, Slobodan Blazeski
<·················@gmail.com> said:
> Currently only lisp that could do SMP is scieneer, as per my knwoledge
> never actually tried it,
> there was a talk about smp in sbcl and allegro google this group for
> more details
openmcl/Clozure Common Lisp also does SMP:
from:
<http://openmcl.clozure.com/Doc/index.html#Programming-with-Threads>
"For a variety of reasons - better utilization of CPU resources on
single and multiprocessor systems and better integration with the OS in
general - threads in OpenMCL 0.14 and later are preemptively scheduled.
In this model, lisp threads are native threads and all scheduling
decisions involving them are made by the OS kernel. (Those decisions
might involve scheduling multiple lisp threads simultaneously on
multiple processors on SMP systems.) This change has a number of subtle
effects:
- it is possible for two (or more) lisp threads to be executing
simultaneously, possibly trying to access and/or modify the
same data structures."
cl-muporoc runs under openmcl/CCL as well.
Den Fri, 08 Feb 2008 11:40:20 -0500 skrev Raffael Cavallaro:
> On 2008-02-08 10:43:55 -0500, Slobodan Blazeski
> <·················@gmail.com> said:
>
>> Currently only lisp that could do SMP is scieneer, as per my knwoledge
>> never actually tried it,
>> there was a talk about smp in sbcl and allegro google this group for
>> more details
>
> openmcl/Clozure Common Lisp also does SMP:
As does SBCL. As I'm writing this post, it's maxing out both cores on my
Core 2 Duo at 197% CPU usage.
Cheers,
Maciej
On Fri, 8 Feb 2008 20:49:25 +0000 (UTC), Maciej Katafiasz <········@gmail.com> wrote:
> As does SBCL.
Only on certain combinations of OS and CPU, though, AFAIK. Luckily,
Linux on Intel is amongst them and this is probably what most people
use it for.
--
European Common Lisp Meeting, Amsterdam, April 19/20, 2008
http://weitz.de/eclm2008/
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
On Feb 8, 4:43 pm, Slobodan Blazeski <·················@gmail.com>
wrote:
> On Jan 10, 3:08 am, jim <········@gmail.com> wrote:> Hello,
>
> > I'd like to have Erlang style micro-threads in Lisp
> So do I.
> > and I'm investigating if that's a good idea or not
> It is
> > and how much effort it would be to implement.
> A lot of effort.
> > My goal is to is to be able to take advantage
> > of multi-core cpus while eliminating the pain of locking,
> > synchronization, etc.
>
> Currently only lisp that could do SMP is scieneer, as per my knwoledge
SBCL has support for native threads.
> It's one way. CMUCL uses his own green threads.
> The problem is in the big lisp runtime, everything could change in
> runtime in lisp .
Cool. If you use both the native and system threads you could
probably get something pretty close to Erlang style multi-processing
that can still max out a multicore system. I don't understand this
stuff about changing and sharing variables though. Just because lisp
lets you share variables doesn't mean you have to. Is there something
important here I'm missing, or could you just limit your communication
to mailboxes and it would basically be the same?
-Jeff
On Feb 8, 11:03 pm, Jeff <······@gmail.com> wrote:
> On Feb 8, 4:43 pm, Slobodan Blazeski <·················@gmail.com>
> wrote:
>
> > On Jan 10, 3:08 am, jim <········@gmail.com> wrote:> Hello,
>
> > > I'd like to have Erlang style micro-threads in Lisp
> > So do I.
> > > and I'm investigating if that's a good idea or not
> > It is
> > > and how much effort it would be to implement.
> > A lot of effort.
> > > My goal is to is to be able to take advantage
> > > of multi-core cpus while eliminating the pain of locking,
> > > synchronization, etc.
>
> > Currently only lisp that could do SMP is scieneer, as per my knwoledge
>
> SBCL has support for native threads.
>
> > It's one way. CMUCL uses his own green threads.
> > The problem is in the big lisp runtime, everything could change in
> > runtime in lisp .
>
> Cool. If you use both the native and system threads you could
> probably get something pretty close to Erlang style multi-processing
> that can still max out a multicore system. I don't understand this
> stuff about changing and sharing variables though. Just because lisp
> lets you share variables doesn't mean you have to. Is there something
> important here I'm missing, or could you just limit your communication
> to mailboxes and it would basically be the same?
Erlang is functional language , variable could be declared once and it
stays that way for good.
Elang for example doesn't have even mutable arrays.It's like using cl
defconstant for every variable there is. Look at Haskell and Prolog
where they get the idea. In lisp we have setf, push pop & co. We have
classes changed in runtime we have mutable arrays etc .
So if you want to make something like Erlang concurrency you will have
to limit the cl to a functional subset or something like that.
cheers
Slobodan
>
> -Jeff
P� Sat, 09 Feb 2008 22:00:36 +0100, skrev Slobodan Blazeski
<·················@gmail.com>:
> So if you want to make something like Erlang concurrency you will have
> to limit the cl to a functional subset or something like that.
>
You might want to check out ACL2. http://www.cs.utexas.edu/~moore/acl2/
http://www.cs.utexas.edu/users/moore/acl2/v3-3/PROGRAMMING.html
Gives a list of a functional subset of Common Lisp with some extraes used
by ACL2.
--------------
John Thingstad
On Feb 9, 11:04 pm, "John Thingstad" <·······@online.no> wrote:
> På Sat, 09 Feb 2008 22:00:36 +0100, skrev Slobodan Blazeski
> <·················@gmail.com>:
>
> > So if you want to make something like Erlang concurrency you will have
> > to limit the cl to a functional subset or something like that.
>
> You might want to check out ACL2.http://www.cs.utexas.edu/~moore/acl2/http://www.cs.utexas.edu/users/moore/acl2/v3-3/PROGRAMMING.html
> Gives a list of a functional subset of Common Lisp with some extraes used
> by ACL2.
>
> --------------
> John Thingstad
Thanks for the link but I already know it. However I want look at it
because it's GPL'ed.
Currently I have in my mind lispy syntaxed crosbreed of prolog and
haskell embedded language.
And If got something clean that I would like to program in I would
implement it.
cheers
Slobodan