From: jmckitrick
Subject: Araneida and file uploads?
Date: 
Message-ID: <1158312923.732225.172410@b28g2000cwb.googlegroups.com>
I've seen archived emails that araneida doesn't, but then I've seen
posts that ucw (which can use araneida) does do file uploads.

Does anyone know for sure if araneida supports file uploads?

From: vanekl
Subject: Re: Araneida and file uploads?
Date: 
Message-ID: <1158330453.325808.154600@h48g2000cwc.googlegroups.com>
jmckitrick wrote:
> I've seen archived emails that araneida doesn't, but then I've seen
> posts that ucw (which can use araneida) does do file uploads.
>
> Does anyone know for sure if araneida supports file uploads?

Araneida opens and reads its sockets as character
streams, which means that if you want to upload binary
files it probably wont work unless you encode the files
a priori in something like base64, and use an rfc2388
parser that is more robust when parsing multipart mime
headers than normal, especially line endings. (E.g., Clisp
will reinterpret line endings if the stream format is
character.)

Araneida doesn't parse rfc2388 multipart mime headers--it's
just a simple httpd server. UCW calls a rfc2388 parser, but
it doesn't handle character streams, just binary. This is
technically the correct way to handle http data since it's
(mostly) a binary protocol, not character, but it leaves
first-gen httpd servers out in the cold.

UCW does come with several modules with support for other
httpd backends. I guess the recommended backend for
development work is the in-house 'httpd' server. This
backend works great on toy problems, but gave me random EOF
socket closings when I tried uploading files greater than
120K. Or sometimes would fall into a mode that would return
an unending stream of ascii control-code 2s. Mmmmkay. So I
ditched :httpd for :araneida.

The :araneida backend that comes with ucw required a lot
of love. It doesn't seem to have been maintained. It also
requires a dual rfc2388 parser that is capable of handling
character streams. If you look on the UCW mailing list you
can find a rfc2388 parser that can read character streams.
The stock parser wont work with a stock araneida, though.

Anyway, it's POSSIBLE to get this combination to work. I've
done it with both clisp and a current version of sbcl. And
this is what I currently use, but it takes a lot of work
and only handles character files (which suits my purposes). 

Lou Vanek
From: jmckitrick
Subject: Re: Araneida and file uploads?
Date: 
Message-ID: <1158344262.632963.324460@e3g2000cwe.googlegroups.com>
vanekl wrote:
> Anyway, it's POSSIBLE to get this combination to work. I've
> done it with both clisp and a current version of sbcl. And
> this is what I currently use, but it takes a lot of work
> and only handles character files (which suits my purposes).

Right now I'm looking at cl-mime to get the file contents.  The files
are probably going to be powerpoint and .doc files, and that means
binary.

I guess another option (and a rather heavyweight one at that) is to run
apache with araneida.  Ugh.
From: vanekl
Subject: Re: Araneida and file uploads?
Date: 
Message-ID: <1158346654.782455.192630@m73g2000cwd.googlegroups.com>
jmckitrick wrote:
> vanekl wrote:
> > Anyway, it's POSSIBLE to get this combination to work. I've
> > done it with both clisp and a current version of sbcl. And
> > this is what I currently use, but it takes a lot of work
> > and only handles character files (which suits my purposes).
>
> Right now I'm looking at cl-mime to get the file contents.  The files
> are probably going to be powerpoint and .doc files, and that means
> binary.
>
> I guess another option (and a rather heavyweight one at that) is to run
> apache with araneida.  Ugh.

If you meant apache/mod_lisp and ucw, yeah, that should work. But
araneida wont handle powerpoint nor doc files as is. Haven't used
cl-mime.
I use a heavily modified version of rfc2388. UCW comes with a bunch
of example programs. One of them is a file uploader. If you're able to
get UCW to run it's worth taking a look at. UCW is probably too heavy-
weight if all you're going to be doing is uploading files. But you may
be
able to extract the small amount of code that you need, or update
araneida to use binary streams. Or maybe find another http server. I
don't know, but maybe something like TBNL would work. Anybody know
whether it processes binary or character streams?
From: jmckitrick
Subject: Re: Araneida and file uploads?
Date: 
Message-ID: <1158375146.858381.97070@k70g2000cwa.googlegroups.com>
vanekl wrote:
> I use a heavily modified version of rfc2388. UCW comes with a bunch

I got a suggestion and some seed code from Richard Newman.  He seems to
have done a lot of work on araneida, while the official repo has,
sadly, stagnated.  Believe it or not, bringing in rfc2388, along with a
few of Richard's bits, has allowed me to test uploading text and a jpg.

I'm hoping this will solve the problem, though I'd love to see araneida
incorporate these changes officially.
From: vanekl
Subject: Re: Araneida and file uploads?
Date: 
Message-ID: <1158400189.478863.33970@h48g2000cwc.googlegroups.com>
jmckitrick wrote:
> I got a suggestion and some seed code from Richard Newman.  He seems to
> have done a lot of work on araneida, while the official repo has,
> sadly, stagnated.  Believe it or not, bringing in rfc2388, along with a
> few of Richard's bits, has allowed me to test uploading text and a jpg.
>
> I'm hoping this will solve the problem, though I'd love to see araneida
> incorporate these changes officially.

What did you do to get a jpg to upload? Are your streams bivalent?
If so, which lisp allows it? Or did you mod araneida? Good to hear
you found a solution.
From: jmckitrick
Subject: Re: Araneida and file uploads?
Date: 
Message-ID: <1158426150.630383.120700@d34g2000cwd.googlegroups.com>
vanekl wrote:
> What did you do to get a jpg to upload? Are your streams bivalent?
> If so, which lisp allows it? Or did you mod araneida? Good to hear
> you found a solution.

I'll answer the best I can, but Richard deserves all the credit here.

There are a few patches he's added to araneida for parsing the body of
urlencoded and multipart form data, using rfc2388.  Then, in the
handler, I do this:

    (multiple-value-bind (data data-properties)
	(araneida::form-data-param "upload" body)
      (declare (ignore data-properties))
      ;(format t "~A~%" fname)
      (with-open-file (s (concatenate 'string +path-app+ +path-home+
fname)
			 :element-type '(unsigned-byte 8)
			 :direction :output
			 :if-does-not-exist :create
			 :if-exists :supersede)
	(write-sequence (loop for char across data collecting (char-code
char)) s))))

Lo! and Behold!  It works.  There has to be a better way of writing the
data, but this works for now.  What I'd like to try next is inserting
that binary data as a BLOB into a db, then writing it back to the
stream when requested.