From: Frank Goenninger DG1SBG
Subject: UFFI-izing varargs
Date: 
Message-ID: <m2zmuc8t6b.fsf@pcsde001.de.goenninger.net>
While trying to write a (working) interface to the syslog 
family of C functions I came across the varargs problem:

The C function is defined as:

void syslog( int priority, const char *messae, ...)

Hm - how to transform this into a suitable UFFI definition
and how to call it then ?

I have so far:

###

;; FUNCTION DEFINITION

(uffi:def-function ("syslog"   _syslog)
		   ((priority  :int)
		    (message   :cstring))
		   :returning  :void)

;; SYSLOG: CALLING THE C FUNCTION

(defun syslog (priority message &rest args)

  (declare (ignorable args))

  ;; write message to syslog

  (with-cstring (c-message message)
    (_syslog priority c-message))
)

###

That's obviously only half of the thing I want. So,
my questions are: 

1. How to deal with the &rest args ?
   Is this the right approach ?

2. How to call `format' to transform the args into one
   string which can then be passed to syslog ?

I see that using approach 2) properly there's no need
for a varargs definition any more. So, 2) means how to
loop ober the args ... Oh, wait, that should be easy ;-)
It once again shows that I am still a newbie.

Thanks for any input and feedback.

Frank

--
   Frank, the Ham DG1SBG

From: Kenny Tilton
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <M2Qme.1210$jU5.1160427@twister.nyc.rr.com>
Frank Goenninger DG1SBG wrote:

> While trying to write a (working) interface to the syslog 
> family of C functions I came across the varargs problem:
> 
> The C function is defined as:
> 
> void syslog( int priority, const char *messae, ...)
> 
> Hm - how to transform this into a suitable UFFI definition
> and how to call it then ?
> 
> I have so far:
> 
> ###
> 
> ;; FUNCTION DEFINITION
> 
> (uffi:def-function ("syslog"   _syslog)
> 		   ((priority  :int)
> 		    (message   :cstring))
> 		   :returning  :void)
> 
> ;; SYSLOG: CALLING THE C FUNCTION
> 
> (defun syslog (priority message &rest args)
> 
>   (declare (ignorable args))
> 
>   ;; write message to syslog
> 
>   (with-cstring (c-message message)
>     (_syslog priority c-message))
> )
> 
> ###
> 
> That's obviously only half of the thing I want. So,
> my questions are: 
> 
> 1. How to deal with the &rest args ?

Don't you want an array of pointers where you have (message :cstring) in 
the UFFI definition? Then you just populate the Lisp array (suitably 
defined using UFFI) and pass that.

>    Is this the right approach ?
> 
> 2. How to call `format' to transform the args into one
>    string which can then be passed to syslog ?
> 
> I see that using approach 2) properly there's no need
> for a varargs definition any more. So, 2) means how to
> loop ober the args ... Oh, wait, that should be easy ;-)
> It once again shows that I am still a newbie.

For fun you can look at format args ~{/~} to explore handily formatting 
a list.

kenzo

-- 
Cells? : http://www.common-lisp.net/project/cells/
Cello? : http://www.common-lisp.net/project/cello/
Cells-Gtk? : http://www.common-lisp.net/project/cells-gtk/
Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film

"Doctor, I wrestled with reality for forty years, and I am happy to 
state that I finally won out over it." -- Elwood P. Dowd
From: Frank Goenninger DG1SBG
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <m2fyw0fqf8.fsf@pcsde001.de.goenninger.net>
Kenny Tilton <·······@nyc.rr.com> writes:

> Frank Goenninger DG1SBG wrote:
>
>> While trying to write a (working) interface to the syslog family of
>> C functions I came across the varargs problem:
>> The C function is defined as:
>> void syslog( int priority, const char *messae, ...)
>> Hm - how to transform this into a suitable UFFI definition
>> and how to call it then ?
>> I have so far:
>> ###
>> ;; FUNCTION DEFINITION
>> (uffi:def-function ("syslog"   _syslog)
>> 		   ((priority  :int)
>> 		    (message   :cstring))
>> 		   :returning  :void)
>> ;; SYSLOG: CALLING THE C FUNCTION
>> (defun syslog (priority message &rest args)
>>   (declare (ignorable args))
>>   ;; write message to syslog
>>   (with-cstring (c-message message)
>>     (_syslog priority c-message))
>> )
>> ###
>> That's obviously only half of the thing I want. So,
>> my questions are: 1. How to deal with the &rest args ?
>
> Don't you want an array of pointers where you have (message :cstring)
> in the UFFI definition? Then you just populate the Lisp array
> (suitably defined using UFFI) and pass that.

Hm? Kenny, you lost me here. Where would I want to put that array
of pointer in? The C function syslog does not accept an array of
pointers.

>
>>    Is this the right approach ?
>> 2. How to call `format' to transform the args into one
>>    string which can then be passed to syslog ?
>> I see that using approach 2) properly there's no need
>> for a varargs definition any more. So, 2) means how to
>> loop ober the args ... Oh, wait, that should be easy ;-)
>> It once again shows that I am still a newbie.
>
> For fun you can look at format args ~{/~} to explore handily
> formatting a list.
>

Hehe - here we are together again. Yes, that's what I'll do.
It is part of a solution others pointed out by private email:

1. Write directly to syslog daemon via sockets
2. Use something like

   (apply #'format nil args)

   to achieve the desired results of getting one string to send
   to the syslog daemon.

Thanks for all the feedback !!

Cheers,
   Frank
   
From: Kenny Tilton
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <429FA0E7.3030405@nyc.rr.com>
[sent to group also]

Frank Goenninger DG1SBG wrote:
> Kenny Tilton <·······@nyc.rr.com> writes:
> 
> 
>>Frank Goenninger DG1SBG wrote:
>>
>>
>>>While trying to write a (working) interface to the syslog family of
>>>C functions I came across the varargs problem:
>>>The C function is defined as:
>>>void syslog( int priority, const char *messae, ...)
>>>Hm - how to transform this into a suitable UFFI definition
>>>and how to call it then ?
>>>I have so far:
>>>###
>>>;; FUNCTION DEFINITION
>>>(uffi:def-function ("syslog"   _syslog)
>>>		   ((priority  :int)
>>>		    (message   :cstring))
>>>		   :returning  :void)
>>>;; SYSLOG: CALLING THE C FUNCTION
>>>(defun syslog (priority message &rest args)
>>>  (declare (ignorable args))
>>>  ;; write message to syslog
>>>  (with-cstring (c-message message)
>>>    (_syslog priority c-message))
>>>)
>>>###
>>>That's obviously only half of the thing I want. So,
>>>my questions are: 1. How to deal with the &rest args ?
>>
>>Don't you want an array of pointers where you have (message :cstring)
>>in the UFFI definition? Then you just populate the Lisp array
>>(suitably defined using UFFI) and pass that.
> 
> 
> Hm? Kenny, you lost me here. Where would I want to put that array
> of pointer in? The C function syslog does not accept an array of
> pointers.

My C is rusty. I thought I remembered stepping thru the varargs as if it 
were an array of pointers. Checking my K&R 2nd edition what I see is 
that the implementation of varargs is, um, implementation dependent and 
so you have to use the va_* API. So... I believe you have some glue to 
write unless you are content to hardcode for one implementation (and 
then you need to find out how that implementation expands the va* macros.

Or duck the issue and put the string together in Lisp, as you have 
already worked out. :)

kenny

ps. I could have sworn the signature for main was (argc, void **argv)?

-- 
Cells? : http://www.common-lisp.net/project/cells/
Cello? : http://www.common-lisp.net/project/cello/
Cells-Gtk? : http://www.common-lisp.net/project/cells-gtk/
Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film

"Doctor, I wrestled with reality for forty years, and I am happy to 
state that I finally won out over it." -- Elwood P. Dowd
From: M Jared Finder
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <epGdnUss4OdObQLfRVn-ug@speakeasy.net>
Kenny Tilton wrote:

>> Hm? Kenny, you lost me here. Where would I want to put that array
>> of pointer in? The C function syslog does not accept an array of
>> pointers.
> 
> My C is rusty. I thought I remembered stepping thru the varargs as if it 
> were an array of pointers. Checking my K&R 2nd edition what I see is 
> that the implementation of varargs is, um, implementation dependent and 
> so you have to use the va_* API. So... I believe you have some glue to 
> write unless you are content to hardcode for one implementation (and 
> then you need to find out how that implementation expands the va* macros.

Usually, varargs are implemented by packing the extra parameters one 
right after another on the stack.  The va_* API just abstracts the low 
level pointer manipulation.  All implementations I've seen do that.

> Or duck the issue and put the string together in Lisp, as you have 
> already worked out. :)
> 
> kenny
> 
> ps. I could have sworn the signature for main was (argc, void **argv)?

It's actually int main( int argc, char** argv ).  Main only accepts a 
varying array of strings, so it gets an easier to use signature.

   -- MJF
From: Duane Rettig
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <4sm00exud.fsf@franz.com>
M Jared Finder <·····@hpalace.com> writes:

> Kenny Tilton wrote:
> 
> >> Hm? Kenny, you lost me here. Where would I want to put that array
> >> of pointer in? The C function syslog does not accept an array of
> >> pointers.
> > My C is rusty. I thought I remembered stepping thru the varargs as
> > if it were an array of pointers. Checking my K&R 2nd edition what I
> > see is that the implementation of varargs is, um, implementation
> > dependent and so you have to use the va_* API. So... I believe you
> > have some glue to write unless you are content to hardcode for one
> > implementation (and then you need to find out how that
> > implementation expands the va* macros.
> 
> 
> Usually, varargs are implemented by packing the extra parameters one
> right after another on the stack.  The va_* API just abstracts the low
> level pointer manipulation.  All implementations I've seen do that.

You've not seen them all.  At least one calling sequence: LinuxPPC /
Yellow Dog Linux) has a singularity after the 8th argument (8 is the
number of arguments passed in registers).  As I recall, the called
function has the option where to place the first 8 arguments, but
there is no "shadow" section, and the standard calling sequence calls
for the 9th argument to show up right after the frame link and the
return address, so it is impossible to place the first 8 arguments
contiguous with the higher numbered ones.  Of course, this is far
from an ideal situation, and we use a different calling sequence
for lisp-to-lisp calls (one which does indeed reserve shadow locations).
It doesn't help varargs compatibility with C, though - its varargs
macros have to deal with the singlarity.

Also, be carefull on the HP/UX systems; the arguments travel _downward_
rather than upward in address - i.e. arg0 is a higher address than
arg1, unlike other systems, where arg0 is lower address than arg1.

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Marcin 'Qrczak' Kowalczyk
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <87fyvzmz5m.fsf@qrnik.zagroda>
No serious C library provides an essential functionality which is
accessible only through varargs. C varargs have serious limitations,
the primary being that all call sites must have the layout of
arguments statically known. You can't forward all parameters from
another vararg function, you can't pass all elements of an array.
In Lisp terms, there is only funcall, no apply.

Also, varargs are not type safe. They shouldn't be exposed directly
to other languages.

The two most common uses are passing a list of homogeneous elements
(execl) or passing values to format (printf). In the first case every
serious library provides an alternative function which takes a pointer
to the first element of an array (execv). In the second case formatting
should be done in a way native to the calling language, such that C
gets only a simple string to emit; exposing generic printf to other
languages is impractical.

-- 
   __("<         Marcin Kowalczyk
   \__/       ······@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/
From: M Jared Finder
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <gd-dnR1F56dUszzfRVn-sA@speakeasy.net>
Duane Rettig wrote:
> M Jared Finder <·····@hpalace.com> writes:
> 
>>Usually, varargs are implemented by packing the extra parameters one
>>right after another on the stack.  The va_* API just abstracts the low
>>level pointer manipulation.  All implementations I've seen do that.
> 
> You've not seen them all.  At least one calling sequence: LinuxPPC /
> Yellow Dog Linux) has a singularity after the 8th argument (8 is the
> number of arguments passed in registers).  As I recall, the called
> function has the option where to place the first 8 arguments, but
> there is no "shadow" section, and the standard calling sequence calls
> for the 9th argument to show up right after the frame link and the
> return address, so it is impossible to place the first 8 arguments
> contiguous with the higher numbered ones.  Of course, this is far
> from an ideal situation, and we use a different calling sequence
> for lisp-to-lisp calls (one which does indeed reserve shadow locations).
> It doesn't help varargs compatibility with C, though - its varargs
> macros have to deal with the singlarity.

So if I call say, printf( "%d+%d=%d", 1, 2, 3 ), then all the arguments 
are passed in registers, but if I call
printf( "%d,%d,%d,%d,%d,%d,%d,%d", 1, 2, 3, 4, 5, 6, 7, 8 ), then the 8 
will be on the stack but the other parameters will all be in registers?

The varargs macros must be forcing the poor C compiler to work overtime 
to support that.

   -- MJF
From: Duane Rettig
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <48y1qr38w.fsf@franz.com>
M Jared Finder <·····@hpalace.com> writes:

> Duane Rettig wrote:
> > M Jared Finder <·····@hpalace.com> writes:
> >
> 
> >>Usually, varargs are implemented by packing the extra parameters one
> >>right after another on the stack.  The va_* API just abstracts the low
> >>level pointer manipulation.  All implementations I've seen do that.
> > You've not seen them all.  At least one calling sequence: LinuxPPC /
> 
> > Yellow Dog Linux) has a singularity after the 8th argument (8 is the
> > number of arguments passed in registers).  As I recall, the called
> > function has the option where to place the first 8 arguments, but
> > there is no "shadow" section, and the standard calling sequence calls
> > for the 9th argument to show up right after the frame link and the
> > return address, so it is impossible to place the first 8 arguments
> > contiguous with the higher numbered ones.  Of course, this is far
> > from an ideal situation, and we use a different calling sequence
> > for lisp-to-lisp calls (one which does indeed reserve shadow locations).
> > It doesn't help varargs compatibility with C, though - its varargs
> > macros have to deal with the singlarity.
> 
> So if I call say, printf( "%d+%d=%d", 1, 2, 3 ), then all the
> arguments are passed in registers, but if I call
> 
> printf( "%d,%d,%d,%d,%d,%d,%d,%d", 1, 2, 3, 4, 5, 6, 7, 8 ), then the
> 8 will be on the stack but the other parameters will all be in
> registers?

Yup.  The calling convention specifies whether the register arguments
are supposed to have "shadow" locations where the caller or callee can
save them optionally, and usually they do, and they are usually
specified to be in a location which places them contiguous on the
stack with the non-register arguments, but not always...

> The varargs macros must be forcing the poor C compiler to work
> overtime to support that.

That's why we always define our own Lisp calling convention, because
some C calling conventions don't allow for the large-scale use of
such common idioms as passing large and/or unspecified-length lists
of parameters.  Hey, they don't even _know_ how many parameters have
actually been passed!

-- 
Duane Rettig    ·····@franz.com    Franz Inc.  http://www.franz.com/
555 12th St., Suite 1450               http://www.555citycenter.com/
Oakland, Ca. 94607        Phone: (510) 452-2000; Fax: (510) 452-0182   
From: Frank Goenninger DG1SBG
Subject: Re: UFFI-izing varargs
Date: 
Message-ID: <m2slzytbdo.fsf@pcsde001.de.goenninger.net>
Hi all who answered my little question:

Thanks for all the feedback. It's again one of those cases
where a small question produced quite a list of follow-ups.

And, you know what: I learned a lot more than I asked for.

Keep this up!

Thanks again!

   Frank