From: skibud2
Subject: clisp, fcgi and stdin example
Date: 
Message-ID: <1169180437.884463.178600@m58g2000cwm.googlegroups.com>
About a week ago, I ran into a problem with using clisp and fastcgi.
The slurp-stdin function does support reading one character at a time.
Therefore, some posts may get truncated at Null characters. As a
result, I wrote this small ffi wrapper to create this function. Below
is the c code for the library and ffi call. Hopefully this will be
usefull for other people.

Mike


(use-package :ffi)

(ffi:default-foreign-language :stdc)

(def-call-out fcgi_read_char
  (:name "fcgi_read_char")
  (:arguments (c (c-ptr (c-array-max char 1)) :out :alloca))
  (:library "fcgi_read.so")
  (:return-type int) )


(defun fcgi-read-char ()
  (multiple-value-bind (success result)
      (fcgi_read_char)
    (if (> success 0)
      (character (aref result 0))
      nil )))


////////////// C Code //////////////////

#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif

#include <string.h>

#include "fcgi_stdio.h"


int fcgi_read_char(char* c) {
  return fread(c, 1, 1, stdin);
}

From: Kaz Kylheku
Subject: Re: clisp, fcgi and stdin example
Date: 
Message-ID: <1169184915.773181.273850@v45g2000cwv.googlegroups.com>
skibud2 wrote:
> About a week ago, I ran into a problem with using clisp and fastcgi.
> The slurp-stdin function does support reading one character at a time.

I don't see how a silly wrapping of a one-character of fread can
possibly solve any conceivable problem that anyone might think he is
having with CLISP. Common Lisp already has streams, including
pre-instantiated ones for standard input and standard output. And it
has a READ-CHAR function.

One thing you have to watch out for when using text streams under CLISP
is encodings.  CLISP rigorously imposes encodings on streams, and both
*standard-input* and *standard-output* are text streams. The encoding
is determined from the environment. If the encoding falls back on
ASCII, then if any character is read that is outside of the ASCII range
0-127, it will signal an error condition.

Your fread wrapper does not bypass this concern, because an encoding is
also applied to string conversions in the FFI. Actually it's buggy
because your method can't be used to read, for instance, multi-byte
encodings such as UTF-8. If two or three bytes must be read from the
stream to reconstruct a character, your function will instead convert
them to two or three incorrect characters.

> (use-package :ffi)
>
> (ffi:default-foreign-language :stdc)
>
> (def-call-out fcgi_read_char
>   (:name "fcgi_read_char")
>   (:arguments (c (c-ptr (c-array-max char 1)) :out :alloca))
>   (:library "fcgi_read.so")
>   (:return-type int) )
>
>
> (defun fcgi-read-char ()
>   (multiple-value-bind (success result)
>       (fcgi_read_char)
>     (if (> success 0)
>       (character (aref result 0))
>       nil )))

All this could be done by targetting the getchar() function instead,
and then converting the returned integer to a character (if it is
non-negative). But of course, that still wouldn't represent a solution
to any real problem.

(def-call-out getchar
  (:arguments)
  (:return-type int)
  (:library "name-of-your-libc.so"))

> ////////////// C Code //////////////////
>
> #ifdef HAVE_UNISTD_H
> #include <unistd.h>
> #endif

Useless header; stdin and fread are in <stdio.h>

> #include <string.h>

Ditto.

> #include "fcgi_stdio.h"
> 
> int fcgi_read_char(char* c) {
>   return fread(c, 1, 1, stdin);
> }
From: skibud2
Subject: Re: clisp, fcgi and stdin example
Date: 
Message-ID: <1169215527.059581.296240@v45g2000cwv.googlegroups.com>
You have made some valid points in your post, but there is one thing
that you are missing. The stdin mentioned in my example is the fastcgi
stdin, not the standard stdin. The function slurp-stdin returns a
string, so null characters will terminate the string early, and you
will miss data.



Kaz Kylheku wrote:
> skibud2 wrote:
> > About a week ago, I ran into a problem with using clisp and fastcgi.
> > The slurp-stdin function does support reading one character at a time.
>
> I don't see how a silly wrapping of a one-character of fread can
> possibly solve any conceivable problem that anyone might think he is
> having with CLISP. Common Lisp already has streams, including
> pre-instantiated ones for standard input and standard output. And it
> has a READ-CHAR function.
>
> One thing you have to watch out for when using text streams under CLISP
> is encodings.  CLISP rigorously imposes encodings on streams, and both
> *standard-input* and *standard-output* are text streams. The encoding
> is determined from the environment. If the encoding falls back on
> ASCII, then if any character is read that is outside of the ASCII range
> 0-127, it will signal an error condition.
>
> Your fread wrapper does not bypass this concern, because an encoding is
> also applied to string conversions in the FFI. Actually it's buggy
> because your method can't be used to read, for instance, multi-byte
> encodings such as UTF-8. If two or three bytes must be read from the
> stream to reconstruct a character, your function will instead convert
> them to two or three incorrect characters.
>
> > (use-package :ffi)
> >
> > (ffi:default-foreign-language :stdc)
> >
> > (def-call-out fcgi_read_char
> >   (:name "fcgi_read_char")
> >   (:arguments (c (c-ptr (c-array-max char 1)) :out :alloca))
> >   (:library "fcgi_read.so")
> >   (:return-type int) )
> >
> >
> > (defun fcgi-read-char ()
> >   (multiple-value-bind (success result)
> >       (fcgi_read_char)
> >     (if (> success 0)
> >       (character (aref result 0))
> >       nil )))
>
> All this could be done by targetting the getchar() function instead,
> and then converting the returned integer to a character (if it is
> non-negative). But of course, that still wouldn't represent a solution
> to any real problem.
>
> (def-call-out getchar
>   (:arguments)
>   (:return-type int)
>   (:library "name-of-your-libc.so"))
>
> > ////////////// C Code //////////////////
> >
> > #ifdef HAVE_UNISTD_H
> > #include <unistd.h>
> > #endif
>
> Useless header; stdin and fread are in <stdio.h>
>
> > #include <string.h>
>
> Ditto.
>
> > #include "fcgi_stdio.h"
> >
> > int fcgi_read_char(char* c) {
> >   return fread(c, 1, 1, stdin);
> > }
From: skibud2
Subject: Re: clisp, fcgi and stdin example
Date: 
Message-ID: <1169215656.441914.99500@l53g2000cwa.googlegroups.com>
Take a loop at slurp-stdin here:

http://clisp.cons.org/impnotes/fastcgi.html

By the way, I was not complaining about clisp, I was just posting my
work around for fastcgi.


Kaz Kylheku wrote:
> skibud2 wrote:
> > About a week ago, I ran into a problem with using clisp and fastcgi.
> > The slurp-stdin function does support reading one character at a time.
>
> I don't see how a silly wrapping of a one-character of fread can
> possibly solve any conceivable problem that anyone might think he is
> having with CLISP. Common Lisp already has streams, including
> pre-instantiated ones for standard input and standard output. And it
> has a READ-CHAR function.
>
> One thing you have to watch out for when using text streams under CLISP
> is encodings.  CLISP rigorously imposes encodings on streams, and both
> *standard-input* and *standard-output* are text streams. The encoding
> is determined from the environment. If the encoding falls back on
> ASCII, then if any character is read that is outside of the ASCII range
> 0-127, it will signal an error condition.
>
> Your fread wrapper does not bypass this concern, because an encoding is
> also applied to string conversions in the FFI. Actually it's buggy
> because your method can't be used to read, for instance, multi-byte
> encodings such as UTF-8. If two or three bytes must be read from the
> stream to reconstruct a character, your function will instead convert
> them to two or three incorrect characters.
>
> > (use-package :ffi)
> >
> > (ffi:default-foreign-language :stdc)
> >
> > (def-call-out fcgi_read_char
> >   (:name "fcgi_read_char")
> >   (:arguments (c (c-ptr (c-array-max char 1)) :out :alloca))
> >   (:library "fcgi_read.so")
> >   (:return-type int) )
> >
> >
> > (defun fcgi-read-char ()
> >   (multiple-value-bind (success result)
> >       (fcgi_read_char)
> >     (if (> success 0)
> >       (character (aref result 0))
> >       nil )))
>
> All this could be done by targetting the getchar() function instead,
> and then converting the returned integer to a character (if it is
> non-negative). But of course, that still wouldn't represent a solution
> to any real problem.
>
> (def-call-out getchar
>   (:arguments)
>   (:return-type int)
>   (:library "name-of-your-libc.so"))
>
> > ////////////// C Code //////////////////
> >
> > #ifdef HAVE_UNISTD_H
> > #include <unistd.h>
> > #endif
>
> Useless header; stdin and fread are in <stdio.h>
>
> > #include <string.h>
>
> Ditto.
>
> > #include "fcgi_stdio.h"
> >
> > int fcgi_read_char(char* c) {
> >   return fread(c, 1, 1, stdin);
> > }
From: skibud2
Subject: Re: clisp, fcgi and stdin example
Date: 
Message-ID: <1169216196.607313.17690@v45g2000cwv.googlegroups.com>
Kaz Kylheku,

The more I read your post, the more it irritates me. Here I am posting
an example to help my fellow lisp programmers and you you write this
pompous response. It may not be the perfect example, but it is better
than not having one.

Also, please note, the function will be reading data from attachment
posts, so multi-character support is not what I was looking for.






skibud2 wrote:
> Take a loop at slurp-stdin here:
>
> http://clisp.cons.org/impnotes/fastcgi.html
>
> By the way, I was not complaining about clisp, I was just posting my
> work around for fastcgi.
>
>
> Kaz Kylheku wrote:
> > skibud2 wrote:
> > > About a week ago, I ran into a problem with using clisp and fastcgi.
> > > The slurp-stdin function does support reading one character at a time.
> >
> > I don't see how a silly wrapping of a one-character of fread can
> > possibly solve any conceivable problem that anyone might think he is
> > having with CLISP. Common Lisp already has streams, including
> > pre-instantiated ones for standard input and standard output. And it
> > has a READ-CHAR function.
> >
> > One thing you have to watch out for when using text streams under CLISP
> > is encodings.  CLISP rigorously imposes encodings on streams, and both
> > *standard-input* and *standard-output* are text streams. The encoding
> > is determined from the environment. If the encoding falls back on
> > ASCII, then if any character is read that is outside of the ASCII range
> > 0-127, it will signal an error condition.
> >
> > Your fread wrapper does not bypass this concern, because an encoding is
> > also applied to string conversions in the FFI. Actually it's buggy
> > because your method can't be used to read, for instance, multi-byte
> > encodings such as UTF-8. If two or three bytes must be read from the
> > stream to reconstruct a character, your function will instead convert
> > them to two or three incorrect characters.
> >
> > > (use-package :ffi)
> > >
> > > (ffi:default-foreign-language :stdc)
> > >
> > > (def-call-out fcgi_read_char
> > >   (:name "fcgi_read_char")
> > >   (:arguments (c (c-ptr (c-array-max char 1)) :out :alloca))
> > >   (:library "fcgi_read.so")
> > >   (:return-type int) )
> > >
> > >
> > > (defun fcgi-read-char ()
> > >   (multiple-value-bind (success result)
> > >       (fcgi_read_char)
> > >     (if (> success 0)
> > >       (character (aref result 0))
> > >       nil )))
> >
> > All this could be done by targetting the getchar() function instead,
> > and then converting the returned integer to a character (if it is
> > non-negative). But of course, that still wouldn't represent a solution
> > to any real problem.
> >
> > (def-call-out getchar
> >   (:arguments)
> >   (:return-type int)
> >   (:library "name-of-your-libc.so"))
> >
> > > ////////////// C Code //////////////////
> > >
> > > #ifdef HAVE_UNISTD_H
> > > #include <unistd.h>
> > > #endif
> >
> > Useless header; stdin and fread are in <stdio.h>
> >
> > > #include <string.h>
> >
> > Ditto.
> >
> > > #include "fcgi_stdio.h"
> > >
> > > int fcgi_read_char(char* c) {
> > >   return fread(c, 1, 1, stdin);
> > > }
From: skibud2
Subject: Re: clisp, fcgi and stdin example
Date: 
Message-ID: <1169216254.614566.138440@l53g2000cwa.googlegroups.com>
Kaz Kylheku,

The more I read your post, the more it irritates me. Here I am posting
an example to help my fellow lisp programmers and you you write this
pompous response. It may not be the perfect example, but it is better
than not having one.

Also, please note, the function will be reading data from attachment
posts, so multi-byte encoding support is not what I was looking for.






skibud2 wrote:
> Take a loop at slurp-stdin here:
>
> http://clisp.cons.org/impnotes/fastcgi.html
>
> By the way, I was not complaining about clisp, I was just posting my
> work around for fastcgi.
>
>
> Kaz Kylheku wrote:
> > skibud2 wrote:
> > > About a week ago, I ran into a problem with using clisp and fastcgi.
> > > The slurp-stdin function does support reading one character at a time.
> >
> > I don't see how a silly wrapping of a one-character of fread can
> > possibly solve any conceivable problem that anyone might think he is
> > having with CLISP. Common Lisp already has streams, including
> > pre-instantiated ones for standard input and standard output. And it
> > has a READ-CHAR function.
> >
> > One thing you have to watch out for when using text streams under CLISP
> > is encodings.  CLISP rigorously imposes encodings on streams, and both
> > *standard-input* and *standard-output* are text streams. The encoding
> > is determined from the environment. If the encoding falls back on
> > ASCII, then if any character is read that is outside of the ASCII range
> > 0-127, it will signal an error condition.
> >
> > Your fread wrapper does not bypass this concern, because an encoding is
> > also applied to string conversions in the FFI. Actually it's buggy
> > because your method can't be used to read, for instance, multi-byte
> > encodings such as UTF-8. If two or three bytes must be read from the
> > stream to reconstruct a character, your function will instead convert
> > them to two or three incorrect characters.
> >
> > > (use-package :ffi)
> > >
> > > (ffi:default-foreign-language :stdc)
> > >
> > > (def-call-out fcgi_read_char
> > >   (:name "fcgi_read_char")
> > >   (:arguments (c (c-ptr (c-array-max char 1)) :out :alloca))
> > >   (:library "fcgi_read.so")
> > >   (:return-type int) )
> > >
> > >
> > > (defun fcgi-read-char ()
> > >   (multiple-value-bind (success result)
> > >       (fcgi_read_char)
> > >     (if (> success 0)
> > >       (character (aref result 0))
> > >       nil )))
> >
> > All this could be done by targetting the getchar() function instead,
> > and then converting the returned integer to a character (if it is
> > non-negative). But of course, that still wouldn't represent a solution
> > to any real problem.
> >
> > (def-call-out getchar
> >   (:arguments)
> >   (:return-type int)
> >   (:library "name-of-your-libc.so"))
> >
> > > ////////////// C Code //////////////////
> > >
> > > #ifdef HAVE_UNISTD_H
> > > #include <unistd.h>
> > > #endif
> >
> > Useless header; stdin and fread are in <stdio.h>
> >
> > > #include <string.h>
> >
> > Ditto.
> >
> > > #include "fcgi_stdio.h"
> > >
> > > int fcgi_read_char(char* c) {
> > >   return fread(c, 1, 1, stdin);
> > > }