From: Kevin L
Subject: Very simple UFFI:  what is wrong with this code?
Date: 
Message-ID: <%KOzg.4746$zV6.3786@trnddc03>
Hi all,

I'm trying to incorporate two C functions:  getgrnam(3) so I can easily
render a Unix group name, and statfvs(2) so that I can have a
get-available-disk-space function.  This is my first real try with UFFI.

(get-group-name gid) works fine in both CMUCL and SBCL.

However, (get-available-disk-space "/") crashes in both CMUCL and SBCL.

Here is the source for two UFFI functions.  I *think* I'm doing it
right, but obviously I'm not.

-----snip-----snip-----

(eval-when (:compile-toplevel :load-toplevel :execute)

  ;; getgrgid(3) -----

  ;; Debian: /usr/include/asm/posix_types.h
  (uffi:def-foreign-type gid-t :unsigned-short)

  (uffi:def-struct
   group
   (gr-name (* :unsigned-char))
   (gr-passwd (* :unsigned-char))
   (gid gid-t)
   (gr-mem (* (* :unsigned-char))))

  (uffi:def-function
   ("getgrgid" c-getgrgid)
   ((gid gid-t))
   :returning (* group))

  (defun get-group-name (gid)
    "Returns the name of a group given its GID from the operating
system.  See getgrgid(3)."
    (let ((x (c-getgrgid gid)))
      (if (uffi:null-pointer-p x)
	  (error "Invalid group ID")
	(uffi:convert-from-foreign-string (uffi:get-slot-value x 'group
'gr-name)))))

  ;; statvfs(2) -----

  ;; Debian:  /usr/include/bits/typesizes.h
  (uffi:def-foreign-type fsblkcnt-t :unsigned-long)

  (uffi:def-struct
   statvfs
   (f-bsize :unsigned-long)
   (f-frsize :unsigned-long)
   (f-blocks fsblkcnt-t)
   (f-bfree fsblkcnt-t)
   (f-bavail fsblkcnt-t)
   (f-files fsblkcnt-t)
   (f-ffree fsblkcnt-t)
   (f-favail fsblkcnt-t)
   (f-fsid :unsigned-long)
   (f-flag :unsigned-long)
   (f-namemax :unsigned-long))

  (uffi:def-function
   ("statvfs" c-statvfs)
   ((path :cstring)
    (buf (* statvfs)))
   :returning :int)

  (defun get-available-disk-space (path)
    "Returns the number of bytes available on a filesystem.  See
statvfs(2)."
    (uffi:with-cstring
     (cpath path)
     (uffi:with-foreign-object
      (fs 'statvfs)
      (print fs)
      (terpri)
      (print cpath)
      (terpri)

      (if (not (zerop (c-statvfs cpath fs)))
	  (error "error in statvfs()"))
        (* (uffi:get-slot-value fs 'statvfs 'f-bavail)
           (uffi:get-slot-value fs 'statvfs 'f-bsize)))))

  )

From: Kevin L
Subject: Re: Very simple UFFI:  what is wrong with this code?
Date: 
Message-ID: <1%1Ag.4487$ee1.3430@trnddc06>
For future reference, I found the solution.  struct statvfs is larger
than indicated in the man page.  Adding some extra space to the UFFI
declaration makes it work fine:

   (uffi:def-struct
    statvfs
    (f-bsize :unsigned-long)
    (f-frsize :unsigned-long)
    (f-blocks fsblkcnt-t)
    (f-bfree fsblkcnt-t)
    (f-bavail fsblkcnt-t)
    (f-files fsblkcnt-t)
    (f-ffree fsblkcnt-t)
    (f-favail fsblkcnt-t)
    (f-fsid :unsigned-long)
    (f-flag :unsigned-long)
    (f-namemax :unsigned-long)
    ;; There are additional fields in the struct not discussed
    ;; in the manpage.  This should leave plenty of space.
    (reserved (:array :char 128))
   )
From: Lars Rune Nøstdal
Subject: Re: Very simple UFFI: what is wrong with this code?
Date: 
Message-ID: <1154532643.167060.230460@p79g2000cwp.googlegroups.com>
Kevin L wrote:
> For future reference, I found the solution.  struct statvfs is larger
> than indicated in the man page.  Adding some extra space to the UFFI
> declaration makes it work fine:
>
>    (uffi:def-struct
>     statvfs
>     (f-bsize :unsigned-long)
>     (f-frsize :unsigned-long)
>     (f-blocks fsblkcnt-t)
>     (f-bfree fsblkcnt-t)
>     (f-bavail fsblkcnt-t)
>     (f-files fsblkcnt-t)
>     (f-ffree fsblkcnt-t)
>     (f-favail fsblkcnt-t)
>     (f-fsid :unsigned-long)
>     (f-flag :unsigned-long)
>     (f-namemax :unsigned-long)
>     ;; There are additional fields in the struct not discussed
>     ;; in the manpage.  This should leave plenty of space.
>     (reserved (:array :char 128))
>    )

By the way; have you looked at the "new UFFI" called CFFI...?

-- 
mvh, Lars Rune Nøstdal
http://lars.nostdal.org/
From: Kevin L
Subject: Re: Very simple UFFI: what is wrong with this code?
Date: 
Message-ID: <0etAg.9488$jt.9200@trnddc04>
Lars Rune N�stdal wrote:
> By the way; have you looked at the "new UFFI" called CFFI...?

No, but it looks like a much better (more supported) way to go.  Thanks
for the mention.
From: Ken Tilton
Subject: Re: Very simple UFFI: what is wrong with this code?
Date: 
Message-ID: <mzQAg.34$qy1.7@fe11.lga>
Kevin L wrote:
> Lars Rune N�stdal wrote:
> 
>>By the way; have you looked at the "new UFFI" called CFFI...?
> 
> 
> No, but it looks like a much better...

yes, but...

... (more supported)

well, it is more that the maintainer wants to keep UFFI stable since it 
is in a lot of production code and brittle (his word), so changes are a 
pain. But it is definitely well-supported in that iff anyone finds a bug 
I am sure it would get fixed. /Enhancements/ I believe are being 
reserved for UFFI-2, the need for which perhaps is being obviated by 
CFFI. UFFI-2 was not going to be backwards compatible with UFFI, so no 
continuity win would be had with UFFI-2 over CFFI.

kt


-- 
Cells: http://common-lisp.net/project/cells/

"I'll say I'm losing my grip, and it feels terrific."
    -- Smiling husband to scowling wife, New Yorker cartoon
From: Kevin Rosenberg
Subject: Re: Very simple UFFI: what is wrong with this code?
Date: 
Message-ID: <slrned8kkf.lmh.kevin@tiger.med-info.com>
On 2006-08-04, Ken Tilton <·········@gmail.com> wrote:
>
>
> Kevin L wrote:
>> Lars Rune N�stdal wrote:
>> 
>>>By the way; have you looked at the "new UFFI" called CFFI...?
>> 
>> 
>> No, but it looks like a much better...
>
> yes, but...
>
> ... (more supported)
>
> well, it is more that the maintainer wants to keep UFFI stable since it 
> is in a lot of production code and brittle (his word), so changes are a 
> pain. But it is definitely well-supported in that iff anyone finds a bug 
> I am sure it would get fixed. /Enhancements/ I believe are being 
> reserved for UFFI-2, the need for which perhaps is being obviated by 
> CFFI. UFFI-2 was not going to be backwards compatible with UFFI, so no 
> continuity win would be had with UFFI-2 over CFFI.

Hi Kenny, you're right on the mark here.

-- 
Kevin Rosenberg
·····@hypershots.com
From: Ken Tilton
Subject: Re: Very simple UFFI: what is wrong with this code?
Date: 
Message-ID: <ZjZAg.250$Fs3.65@fe09.lga>
Kevin Rosenberg wrote:
> On 2006-08-04, Ken Tilton <·········@gmail.com> wrote:
> 
>>
>>Kevin L wrote:
>>
>>>Lars Rune N�stdal wrote:
>>>
>>>
>>>>By the way; have you looked at the "new UFFI" called CFFI...?
>>>
>>>
>>>No, but it looks like a much better...
>>
>>yes, but...
>>
>>... (more supported)
>>
>>well, it is more that the maintainer wants to keep UFFI stable since it 
>>is in a lot of production code and brittle (his word), so changes are a 
>>pain. But it is definitely well-supported in that iff anyone finds a bug 
>>I am sure it would get fixed. /Enhancements/ I believe are being 
>>reserved for UFFI-2, the need for which perhaps is being obviated by 
>>CFFI. UFFI-2 was not going to be backwards compatible with UFFI, so no 
>>continuity win would be had with UFFI-2 over CFFI.
> 
> 
> Hi Kenny, you're right on the mark here.
> 

Whew! I felt quite far out on a limb there. :)

Thanks again for UFFI.

kenny

-- 
Cells: http://common-lisp.net/project/cells/

"I'll say I'm losing my grip, and it feels terrific."
    -- Smiling husband to scowling wife, New Yorker cartoon
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: Very simple UFFI:  what is wrong with this code?
Date: 
Message-ID: <87slkd5x5b.fsf@qrnik.zagroda>
Kevin L <······@nowhere.com> writes:

>    (uffi:def-struct
>     statvfs
>     (f-bsize :unsigned-long)
>     (f-frsize :unsigned-long)
>     (f-blocks fsblkcnt-t)
>     (f-bfree fsblkcnt-t)
>     (f-bavail fsblkcnt-t)
>     (f-files fsblkcnt-t)
>     (f-ffree fsblkcnt-t)
>     (f-favail fsblkcnt-t)
>     (f-fsid :unsigned-long)
>     (f-flag :unsigned-long)
>     (f-namemax :unsigned-long)
>     ;; There are additional fields in the struct not discussed
>     ;; in the manpage.  This should leave plenty of space.
>     (reserved (:array :char 128))
>    )

I don't like that many FFIs require the programmer to specify
the layout of structures. Most C/C++ libraries, including ISO/ANSI
standards and POSIX / Single Unix Specification, don't specify
the order of fields, and allow for private fields not listed
in the specification. They also leave expansion of many typedefs
implementation-defined, only hinting about some constraints,
and often rely on compile-time features like #define.

In other words, they have a well defined source interface, but not
an official binary interface.

This leaves three basic ways of inferring the missing information
for interoperability:

1. Using the C compiler to produce wrappers with a known binary
   interface which access the fields. (Easiest but might be
   inefficient; efficient if wrappers can be inlined into code
   generated from the other language.)

2. Using the C compiler to produce a program which reads this
   information, and embedding the resulting offsets and sizes
   in the source code in the other language. (Problematic when
   cross-compiling.)

3. Parsing C headers with custom tools to extract the information,
   duplicating the work of the C compiler. (Brittle and problematic
   in the face of compiler extensions.)

Many FFIs though, and I'm afraid that this includes Lisp FFIs, follow
the fourth way which invites unportable and brittle programs (errors
in inferring the binary interface generally cause undefined behavior):

4. Leave the binary interface for the programmer to discover.

-- 
   __("<         Marcin Kowalczyk
   \__/       ······@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/
From: James Bielman
Subject: Re: Very simple UFFI:  what is wrong with this code?
Date: 
Message-ID: <87d5bhtrqj.fsf@jamesjb.com>
Marcin 'Qrczak' Kowalczyk <······@knm.org.pl> writes:

> I don't like that many FFIs require the programmer to specify
> the layout of structures. Most C/C++ libraries, including ISO/ANSI
> standards and POSIX / Single Unix Specification, don't specify
> the order of fields, and allow for private fields not listed
> in the specification. They also leave expansion of many typedefs
> implementation-defined, only hinting about some constraints,
> and often rely on compile-time features like #define.
>
> In other words, they have a well defined source interface, but not
> an official binary interface.
>
> This leaves three basic ways of inferring the missing information
> for interoperability:
>
> 1. Using the C compiler to produce wrappers with a known binary
>    interface which access the fields. (Easiest but might be
>    inefficient; efficient if wrappers can be inlined into code
>    generated from the other language.)

If only C programmers would always provide accessors instead of making
structure layouts part of the public interface... :(

> 2. Using the C compiler to produce a program which reads this
>    information, and embedding the resulting offsets and sizes
>    in the source code in the other language. (Problematic when
>    cross-compiling.)

FWIW, the cffi-grovel utility can be used to accomplish this (and I
would still like to see the groveller become part of CFFI proper
someday as well):

   http://www.cliki.net/cffi-grovel

James