From: skibud2
Subject: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <1168561564.069728.131940@51g2000cwl.googlegroups.com>
Hey Guys -- I have the following function:

(def-call-out PQescapeString (:return-type uint)
  (:language :stdc)
  (:name "PQescapeString")
  (:library "libpq.so.4")
  (:arguments (to (c-ptr (c-array-max char 256)) :out :alloca)
              (from c-string)
              (length uint)) )

I would like to make the 256 dynamic. Is there a way to do this? Also,
is the "to" array allocated everything I make the call to
PQescapeString?

Thanks,

Mike

From: Ken Tilton
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <7oDph.148$AJ5.145@newsfe10.lga>
skibud2 wrote:
> Hey Guys -- I have the following function:
> 
> (def-call-out PQescapeString (:return-type uint)
>   (:language :stdc)
>   (:name "PQescapeString")
>   (:library "libpq.so.4")
>   (:arguments (to (c-ptr (c-array-max char 256)) :out :alloca)
>               (from c-string)
>               (length uint)) )
> 
> I would like to make the 256 dynamic. Is there a way to do this? Also,
> is the "to" array allocated everything I make the call to
> PQescapeString?

Just guessing, but I would expect that to be no more than a type 
declaration. You still have to allocate the array and pass that as an 
argument in the function /call/. And then you can reuse the same array 
as many times as you like.

but I am just guessing from general principles. purely fwiw.

kt

-- 
The Dalai Lama gets the same crap all the time.
   -- Kenny Tilton on c.l.l when accused of immodesty
From: skibud2
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <1168576096.918602.138620@i15g2000cwa.googlegroups.com>
OK, so this is what I came up with. It is not pretty, but it works. I
have still yet to optimize it.


(def-call-out PQescapeString (:return-type uint)
  (:language :stdc)
  (:name "PQescapeString")
  (:library "libpq.so.4")
  (:arguments (to c-pointer)
              (from c-string)
              (length uint)) )


(defun escape-string (str)
  (let* ((result (foreign-allocate 'char :count 100))
         (len (PQescapeString result str (length str)))
         (c-type (parse-c-type `(c-array char ,len)))
         (result-array (memory-as result c-type)) )
    (prog1
      (map 'string #'code-char result-array)
      (foreign-free result) ) ) )
From: skibud2
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <1168576235.193107.106450@s34g2000cwa.googlegroups.com>
On Jan 11, 11:28 pm, "skibud2" <·············@gmail.com> wrote:
> OK, so this is what I came up with. It is not pretty, but it works. I
> have still yet to optimize it.
>
> (def-call-out PQescapeString (:return-type uint)
>   (:language :stdc)
>   (:name "PQescapeString")
>   (:library "libpq.so.4")
>   (:arguments (to c-pointer)
>               (from c-string)
>               (length uint)) )
>
> (defun escape-string (str)
>   (let* ((result (foreign-allocate 'char :count 100))
>          (len (PQescapeString result str (length str)))
>          (c-type (parse-c-type `(c-array char ,len)))
>          (result-array (memory-as result c-type)) )
>     (prog1
>       (map 'string #'code-char result-array)
>       (foreign-free result) ) ) )


Actually, it was supposed to be:
(foreign-allocate 'char :count (+ (length str) 100))
From: Ken Tilton
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <HcFph.120$ch2.45@newsfe08.lga>
skibud2 wrote:
> 
> On Jan 11, 11:28 pm, "skibud2" <·············@gmail.com> wrote:
> 
>>OK, so this is what I came up with. It is not pretty, but it works. I
>>have still yet to optimize it.
>>
>>(def-call-out PQescapeString (:return-type uint)
>>  (:language :stdc)
>>  (:name "PQescapeString")
>>  (:library "libpq.so.4")
>>  (:arguments (to c-pointer)
>>              (from c-string)
>>              (length uint)) )
>>
>>(defun escape-string (str)
>>  (let* ((result (foreign-allocate 'char :count 100))
>>         (len (PQescapeString result str (length str)))
>>         (c-type (parse-c-type `(c-array char ,len)))
>>         (result-array (memory-as result c-type)) )
>>    (prog1
>>      (map 'string #'code-char result-array)
>>      (foreign-free result) ) ) )
> 
> 
> 
> Actually, it was supposed to be:
> (foreign-allocate 'char :count (+ (length str) 100))
> 

Hang on. I belatedly smelled a rat. Considering this:

    size_t PQescapeString (char *to, const char *from, size_t length);

...(which you should have posted!) and the fact that CLisp's FFI has 
very nice support for C strings, I think the answer is simply for you to 
look at the CLisp FFI doc a little closer. The "to" string I am sure 
requires just something simple syntactically, such as 'with-cstring' or 
similar. If no CLisp guru steps up, your mission is to dive into the 
Cells-Gtk CVs repo and look for the original pure-clisp (not CFFI) 
version, which had all sorts of cool FFI stuff to steal.

kt


-- 
The Dalai Lama gets the same crap all the time.
   -- Kenny Tilton on c.l.l when accused of immodesty
From: skibud2
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <1168614209.942488.165630@q2g2000cwa.googlegroups.com>
I am all for another way. I tried working with with-foreigns-string and
with c-var, but had no luck. I am sure it is going to come down to a
simple two line result.




On Jan 12, 12:44 am, Ken Tilton <·········@gmail.com> wrote:
> skibud2 wrote:
>
> > On Jan 11, 11:28 pm, "skibud2" <·············@gmail.com> wrote:
>
> >>OK, so this is what I came up with. It is not pretty, but it works. I
> >>have still yet to optimize it.
>
> >>(def-call-out PQescapeString (:return-type uint)
> >>  (:language :stdc)
> >>  (:name "PQescapeString")
> >>  (:library "libpq.so.4")
> >>  (:arguments (to c-pointer)
> >>              (from c-string)
> >>              (length uint)) )
>
> >>(defun escape-string (str)
> >>  (let* ((result (foreign-allocate 'char :count 100))
> >>         (len (PQescapeString result str (length str)))
> >>         (c-type (parse-c-type `(c-array char ,len)))
> >>         (result-array (memory-as result c-type)) )
> >>    (prog1
> >>      (map 'string #'code-char result-array)
> >>      (foreign-free result) ) ) )
>
> > Actually, it was supposed to be:
> > (foreign-allocate 'char :count (+ (length str) 100))Hang on. I belatedly smelled a rat. Considering this:
>
>     size_t PQescapeString (char *to, const char *from, size_t length);
>
> ...(which you should have posted!) and the fact that CLisp's FFI has
> very nice support for C strings, I think the answer is simply for you to
> look at the CLisp FFI doc a little closer. The "to" string I am sure
> requires just something simple syntactically, such as 'with-cstring' or
> similar. If no CLisp guru steps up, your mission is to dive into the
> Cells-Gtk CVs repo and look for the original pure-clisp (not CFFI)
> version, which had all sorts of cool FFI stuff to steal.
>
> kt
>
> --
> The Dalai Lama gets the same crap all the time.
>    -- Kenny Tilton on c.l.l when accused of immodesty
From: Ken Tilton
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <zGPph.299$3w7.108@newsfe12.lga>
skibud2 wrote:
> I am all for another way. I tried working with with-foreigns-string and
> with c-var, but had no luck.

The "no luck" is what you should have asked us about. :)

Go back, do it the right way (as best you can work out, with that 
parameter declared as a c-string, etc), and then ask for help with 
/that/ error.

IIRC, there were two cases for foreign string, one in which the (Lisp) 
caller provided (allocated) the storage, maybe you got the wrong one. 
But I might be thinking of a diff FFI altogether.

> I am sure it is going to come down to a
> simple two line result.

Here is the original cells-gtk, done with the CLisp FFI:

    http://common-lisp.net/cgi-bin/viewcvs.cgi/clisp-cgtk/?root=cells-gtk

Might be a pertinent example in there.

kt
From: Ken Tilton
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <9kTph.258$yZ1.28@newsfe10.lga>
skibud2 wrote:
> Hey Guys -- I have the following function:
> 
> (def-call-out PQescapeString (:return-type uint)
>   (:language :stdc)
>   (:name "PQescapeString")
>   (:library "libpq.so.4")
>   (:arguments (to (c-ptr (c-array-max char 256)) :out :alloca)
>               (from c-string)
>               (length uint)) )

I got curious and went digging. Looks like the above is The Right Way 
for the declaration (but the call should be diff, as per below), based 
on example 31.8 from here:

      http://clisp.cons.org/impnotes/dffi.html#dffi-examples

(FFI:DEF-CALL-OUT gethostname
   (:arguments (name (FFI:C-PTR (FFI:C-ARRAY-MAX ffi:char 256))
                     :OUT :ALLOCA)
               (len ffi:int))
   (:language :stdc)
   (:return-type ffi:int))

(defun myhostname ()
   (multiple-value-bind (success name)
       ;; :OUT and :IN-OUT parameters are returned as multiple values
       (gethostname 256)
     (if (zerop success) name
         (error ...)))) ;; strerror(errno)

(defvar hostname (myhostname))

> 
> I would like to make the 256 dynamic. Is there a way to do this? Also,
> is the "to" array allocated everything I make the call to
> PQescapeString?

The doc says :alloca allocates on the stack, so you are not incurring 
the cost of malloc/free. In that case, do you still need the 256 to be 
dynamic? There is a :NONE option for allocation in which you are 
expected to have provided the space, but that is described as "dangerous 
and deprecated", not too enticing.

kt


-- 
The Dalai Lama gets the same crap all the time.
   -- Kenny Tilton on c.l.l when accused of immodesty
From: skibud2
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <1168650419.176581.300750@38g2000cwa.googlegroups.com>
Unfortunetly, I need the 256 to be dynamic. The function will execute
on small strings and large blocks of text. I figured I could always
malloc the memory once and leave it around as a scratch buffer for this
function. Since clisp is not multithreaded yet ;), the above should do
the job.


On Jan 12, 4:44 pm, Ken Tilton <·········@gmail.com> wrote:
> skibud2 wrote:
> > Hey Guys -- I have the following function:
>
> > (def-call-out PQescapeString (:return-type uint)
> >   (:language :stdc)
> >   (:name "PQescapeString")
> >   (:library "libpq.so.4")
> >   (:arguments (to (c-ptr (c-array-max char 256)) :out :alloca)
> >               (from c-string)
> >               (length uint)) )I got curious and went digging. Looks like the above is The Right Way
> for the declaration (but the call should be diff, as per below), based
> on example 31.8 from here:
>
>      http://clisp.cons.org/impnotes/dffi.html#dffi-examples
>
> (FFI:DEF-CALL-OUT gethostname
>    (:arguments (name (FFI:C-PTR (FFI:C-ARRAY-MAX ffi:char 256))
>                      :OUT :ALLOCA)
>                (len ffi:int))
>    (:language :stdc)
>    (:return-type ffi:int))
>
> (defun myhostname ()
>    (multiple-value-bind (success name)
>        ;; :OUT and :IN-OUT parameters are returned as multiple values
>        (gethostname 256)
>      (if (zerop success) name
>          (error ...)))) ;; strerror(errno)
>
> (defvar hostname (myhostname))
>
>
>
> > I would like to make the 256 dynamic. Is there a way to do this? Also,
> > is the "to" array allocated everything I make the call to
> > PQescapeString?The doc says :alloca allocates on the stack, so you are not incurring
> the cost of malloc/free. In that case, do you still need the 256 to be
> dynamic? There is a :NONE option for allocation in which you are
> expected to have provided the space, but that is described as "dangerous
> and deprecated", not too enticing.
>
> kt
>
> --
> The Dalai Lama gets the same crap all the time.
>    -- Kenny Tilton on c.l.l when accused of immodesty
From: ·················@yahoo.com
Subject: Re: Clisp FFI, strings and dynamic allocation
Date: 
Message-ID: <1168983028.504791.303640@51g2000cwl.googlegroups.com>
I found this to be the biggest problem with the CLisp FFI for my
application.

I have a similar function where sometimes I need the output buffer to
be 128 characters,
and occasionally I need it to be 4 MB. I just can't bring myself to
leave 4 MB lying around
as a scratch buffer, so I jump through hoops to grab the 4 MB data
block in 1 kB chunks.

Just adding my $0.02 in case the CLisp gods are taking notes on
requested features.

-Matt

skibud2 wrote:
> Unfortunetly, I need the 256 to be dynamic. The function will execute
> on small strings and large blocks of text. I figured I could always
> malloc the memory once and leave it around as a scratch buffer for this
> function. Since clisp is not multithreaded yet ;), the above should do
> the job.
>
>
> On Jan 12, 4:44 pm, Ken Tilton <·········@gmail.com> wrote:
> > skibud2 wrote:
> > > Hey Guys -- I have the following function:
> >
> > > (def-call-out PQescapeString (:return-type uint)
> > >   (:language :stdc)
> > >   (:name "PQescapeString")
> > >   (:library "libpq.so.4")
> > >   (:arguments (to (c-ptr (c-array-max char 256)) :out :alloca)
> > >               (from c-string)
> > >               (length uint)) )I got curious and went digging. Looks like the above is The Right Way
> > for the declaration (but the call should be diff, as per below), based
> > on example 31.8 from here:
> >
> >      http://clisp.cons.org/impnotes/dffi.html#dffi-examples
> >
> > (FFI:DEF-CALL-OUT gethostname
> >    (:arguments (name (FFI:C-PTR (FFI:C-ARRAY-MAX ffi:char 256))
> >                      :OUT :ALLOCA)
> >                (len ffi:int))
> >    (:language :stdc)
> >    (:return-type ffi:int))
> >
> > (defun myhostname ()
> >    (multiple-value-bind (success name)
> >        ;; :OUT and :IN-OUT parameters are returned as multiple values
> >        (gethostname 256)
> >      (if (zerop success) name
> >          (error ...)))) ;; strerror(errno)
> >
> > (defvar hostname (myhostname))
> >
> >
> >
> > > I would like to make the 256 dynamic. Is there a way to do this? Also,
> > > is the "to" array allocated everything I make the call to
> > > PQescapeString?The doc says :alloca allocates on the stack, so you are not incurring
> > the cost of malloc/free. In that case, do you still need the 256 to be
> > dynamic? There is a :NONE option for allocation in which you are
> > expected to have provided the space, but that is described as "dangerous
> > and deprecated", not too enticing.
> >
> > kt
> >
> > --
> > The Dalai Lama gets the same crap all the time.
> >    -- Kenny Tilton on c.l.l when accused of immodesty