From: Fractal
Subject: Clisp and System Threads
Date: 
Message-ID: <1126017401.174929.182790@g43g2000cwa.googlegroups.com>
Hi,

I am trying to call a C function that will create a system thread and
callback a lisp routine...

The callback is going ok ... but immediately afterwards I get
I division by zero error and Clisp crash. (it happens after the lisp
callback function returns)

Is it even possible to do this?
(i wrote a C program to check the creat_thread routine and it's working
just fine)

Please your advice.

Thanks,
Gal

From: Pascal Bourguignon
Subject: Re: Clisp and System Threads
Date: 
Message-ID: <87u0gynlbm.fsf@thalassa.informatimago.com>
"Fractal" <···········@gmail.com> writes:
> I am trying to call a C function that will create a system thread and
> callback a lisp routine...
>
> The callback is going ok ... but immediately afterwards I get
> I division by zero error and Clisp crash. (it happens after the lisp
> callback function returns)
>
> Is it even possible to do this?

In general, no, it's not possible.

The problem is that clisp is not re-entrant (not "thread-safe").

So you can run independant threads, but you cannot call back randomly
like you do.

To be able to call a call-back from another thread, you would have to
modify clisp to add a rendez-vous point, something like what is done
to handle signals.

Or you could just write in lisp an event loop that would poll for
other requests from other threads and dispatch them, always from the
clisp thread.  I assume some synchronizations would be needed so you'd
have to use some IPC (eg. semaphores).  Have a look at:

cvs -z3 -d ··················@cvs.informatimago.com:/usr/local/cvs/public/chrooted-cvs/cvs co common/susv3

(depends on the following too:

cvs -z3 -d ··················@cvs.informatimago.com:/usr/local/cvs/public/chrooted-cvs/cvs co common/makedir
cvs -z3 -d ··················@cvs.informatimago.com:/usr/local/cvs/public/chrooted-cvs/cvs co common/common-lisp
cvs -z3 -d ··················@cvs.informatimago.com:/usr/local/cvs/public/chrooted-cvs/cvs co common/clisp

)



Of course, there's always the possibility to add full fleshed threads
to clisp ;-)


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

In a World without Walls and Fences, 
who needs Windows and Gates?
From: Fractal
Subject: Re: Clisp and System Threads
Date: 
Message-ID: <1126075559.625692.197970@g47g2000cwa.googlegroups.com>
i don't feel ready yet to dive into clisp impl.

what is that:
--------------------
Have a look at:
cvs -z3 -d
··················@cvs.informatimago.com:/usr/lo­cal/cvs/public/chrooted-cvs/cv­s
co common/susv3
(depends on the following too:
cvs -z3 -d
··················@cvs.informatimago.com:/usr/lo­cal/cvs/public/chrooted-cvs/cv­s
co common/makedir
cvs -z3 -d
··················@cvs.informatimago.com:/usr/lo­cal/cvs/public/chrooted-cvs/cv­s
co common/common-lisp
cvs -z3 -d
··················@cvs.informatimago.com:/usr/lo­cal/cvs/public/chrooted-cvs/cv­s
co common/clisp
----------------------
and how do i access this ?

i wouldn't mind using the select facility to create user threads...
But with a single OS thread the I/O operation is bound to block.
(relinquishing it's cpu time slot - this will hurt performance)

and correct me if i am wrong ... but modern computers I/O device can
operate concurrently to the CPU and themselves.

it would be nice if there was some way to write somthing like a (or "MS
I/O completion port" "LINUX async I/O")
with existing Clisp impl ... is there a way ?
From: Pascal Bourguignon
Subject: Re: Clisp and System Threads
Date: 
Message-ID: <87vf1cg4yr.fsf@thalassa.informatimago.com>
"Fractal" <···········@gmail.com> writes:

> i don't feel ready yet to dive into clisp impl.
>
> what is that:
> --------------------
> Have a look at:
> [...some newsreader damanged shell commands...]
> ----------------------

Please change your newsreader!


> and how do i access this ?

These are the CVS commands you'd use to fetch the sources. See http://cvshome.org
cvs has not url scheme :-(  Perhaps it's time to switch to arch...


> i wouldn't mind using the select facility to create user threads...

Not to create user threads, to synchronize!


> But with a single OS thread the I/O operation is bound to block.
> (relinquishing it's cpu time slot - this will hurt performance)

Who said you'd have only one thread?


> and correct me if i am wrong ... but modern computers I/O device can
> operate concurrently to the CPU and themselves.

Yeah.


> it would be nice if there was some way to write somthing like a (or "MS
> I/O completion port" "LINUX async I/O")
> with existing Clisp impl ... is there a way ?

-- 
"Remember, Information is not knowledge; Knowledge is not Wisdom;
Wisdom is not truth; Truth is not beauty; Beauty is not love;
Love is not music; Music is the best." -- Frank Zappa
From: Pascal Bourguignon
Subject: Re: Clisp and System Threads
Date: 
Message-ID: <87ll28g43r.fsf@thalassa.informatimago.com>
Pascal Bourguignon <····@mouse-potato.com> writes:
> "Fractal" <···········@gmail.com> writes:
>> I am trying to call a C function that will create a system thread and
>> callback a lisp routine...
>>
>> The callback is going ok ... but immediately afterwards I get
>> I division by zero error and Clisp crash. (it happens after the lisp
>> callback function returns)
>>
>> Is it even possible to do this?
>
> In general, no, it's not possible.
>
> The problem is that clisp is not re-entrant (not "thread-safe").
>
> So you can run independant threads, but you cannot call back randomly
> like you do.
>
> To be able to call a call-back from another thread, you would have to
> modify clisp to add a rendez-vous point, something like what is done
> to handle signals.
>
> Or you could just write in lisp an event loop that would poll for
> other requests from other threads and dispatch them, always from the
> clisp thread.  I assume some synchronizations would be needed so you'd
> have to use some IPC (eg. semaphores).

Or using pipes or sockets, for example something like this:


#+clisp
#-linux
(defun make-pipe (&key (ELEMENT-TYPE 'character)
                  (EXTERNAL-FORMAT custom:*default-file-encoding*)
                  (BUFFERED t))
  "RETURN: two interconnected IPC streams.
This would be a couple of pipes, but since we don't have internal pipes
in clisp, we do it with a socket.
These are bi-directionnal!"
  (let ((lsock (socket:socket-server))
        (asock nil)
        (bsock nil))
    (unwind-protect
         (progn
           (setf asock (socket:socket-connect
                        (socket:socket-server-port lsock)
                        (socket:socket-server-host lsock)
                        :ELEMENT-TYPE ELEMENT-TYPE
                        :EXTERNAL-FORMAT external-format
                        :BUFFERED BUFFERED
                        :TIMEOUT 5))
           (if (socket:socket-wait lsock 5)
               (progn
                 (setf bsock (socket:socket-accept
                              lsock 
                              :ELEMENT-TYPE ELEMENT-TYPE
                              :EXTERNAL-FORMAT EXTERNAL-FORMAT
                              :BUFFERED BUFFERED
                              :TIMEOUT 5))
                 (if bsock
                     (values asock bsock)
                     (values nil nil)))
               (values nil nil)))
      (socket:socket-server-close lsock)
      (unless bsock (close asock)))))
                                       
   
#+clisp
#+linux
(defun make-pipe (&key (ELEMENT-TYPE 'character)
                  (EXTERNAL-FORMAT custom:*default-file-encoding*)
                  (BUFFERED t))
  "RETURN: two interconnected IPC streams.
These are pipes."
  (multiple-value-bind (res handles) (linux:|pipe|)
    (when (zerop res)
      (values (ext:make-stream (aref handles 0)
                               :direction :input
                               :element-type element-type
                               :external-format external-format
                               :buffered buffered)
              (ext:make-stream (aref handles 1)
                               :direction :output
                               :element-type element-type
                               :external-format external-format
                               :buffered buffered)))))



(let (from-thread to-lisp from-lisp   to-thread)
  (multiple-value-setq (from-thread to-lisp)   (make-pipe))
  (multiple-value-setq (from-lisp   to-thread) (make-pipe))
  
  (create-thread
   (lambda ()                        ; this would be written in C actually!
     (some-thread-function     
      (lambda (request)              ; this would be written in C actually!
        (let ((*print-readably*))
          (format to-lisp "~S~%" request)
          (read from-lisp))))))

  (loop
     (case (socket:socket-status from-thread 0)
       ((:io :input)
        ;; process thread call back:
        (let ((*print-readably*))
           (format to-thread "~S~%" (ignore-errors (eval (read from-thread))))))
       (otherwise
        (do-main-lisp-stuff)))))

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

This is a signature virus.  Add me to your signature and help me to live
From: Fractal
Subject: Re: Clisp and System Threads
Date: 
Message-ID: <1126130099.076984.226130@g47g2000cwa.googlegroups.com>
thanks alot, this will work out great :-)