I'm trying to automate some requests for daily weather data with
aserve. The site uses html forms which return cgi-created pages of
preformatted ASCII text representing info from specified days. Each
char is always prefixed with a title + column headers, and postfixed
with a little disclaimer (whether there is any info in between or not).
After groking the request forms to a call to a client request all I can
get back is the title and headers--no data and no disclaimer. I'm
moderately positive it's not a mistake in translating the html form,
and I thought maybe aserve closed do-http-request before the cgi
finished generating a response, but multiple
client-request-read-sequence calls on the client-request object made by
make-http-client-request return an eof after the colmn headers so I
think I'm wrong.
Anyone have any ideas? I'm just wondering if this is a simple diagnosis
to someone with a little more web-dev experience before I go through
the effort of poking around with another webserver (ok, so I'm lazy ;-)
Nick
ps
the site is
http://www.wrcc.dri.edu/cgi-bin/rawMAIN.pl?nvsage
a sample aserve request is at
http://homepage.mac.com/nick_a/sage-req.lisp
········@gmail.com writes:
> http://homepage.mac.com/nick_a/sage-req.lisp
I got the same thing(1). Have you succeeded doing this with wget or
similar tools? Do you know what the javascript returned is doing?
Petter
(((:|| :XML "xml" :VERSION "1.0" :ENCODING "iso-8859-1")
(:!DOCTYPE " html
PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\"
\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\"")
((:HTML :XMLNS "http://www.w3.org/1999/xhtml" :LANG "en-US"
:|XML:LANG| "en-US")
(:HEAD
(:TITLE
"RAWS Daily data listing (Simple version) - Stead Nevada - DRI SAGE Building, 19200511 to 192005131")
((:SCRIPT :LANGUAGE "JavaScript" :TYPE "text/javascript") "
<!-- Hide script
//<![CDATA[
//]]> End script hiding -->
"))
(:BODY)
((:BODY :BGCOLOR "FFFFFF")
(:PRE " Stead Nevada - DRI SAGE Building
Daily Data run on 3/7/2006 1:17
Day Day Solar Wind Air Relative
of of Rad. Speed Dir. Speed Temperature Humidity Precip.
Date Year Year Run Total Ave. Vector Gust Ave. Max. Min. Ave. Max. Min. Total
ly. mph deg. mph Deg. Fahrenheit percent in.
")))))
--
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Peter wrote:
>I got the same thing(1). Have you succeeded doing this with wget or
>similar tools? Do you know what the javascript returned is doing?
hrm... good thinking, I get the same thing with wget
(http://homepage.mac.com/nick_a/wget-sage.txt) so I guess it's not
something aserve's doing ;-)
I have no idea what the javascript is doing (I actually didn't think
there was any since the <![CDATA[ .. ]]> looks blank), I'll have to do
some research.
thanks for the suggestion
Nick
On Tue, 07 Mar 2006 08:59:35 -0800, nallen05 wrote:
> Peter wrote:
>>I got the same thing(1). Have you succeeded doing this with wget or
>>similar tools? Do you know what the javascript returned is doing?
>
> hrm... good thinking, I get the same thing with wget
> (http://homepage.mac.com/nick_a/wget-sage.txt) so I guess it's not
> something aserve's doing ;-)
>
> I have no idea what the javascript is doing (I actually didn't think
> there was any since the <![CDATA[ .. ]]> looks blank), I'll have to do
> some research.
I don't think it's the (indeed empty) JavaScript. I ran you code forcing
hte protocol to HTTP/1.0 (therefore blocking chunked encoding) and got
the same results. Same with GET instead of POST ... looks pretty much like
a broken server.
HTH Ralf Mattes
> thanks for the suggestion
>
> Nick
R. Mattes wrote:
> I don't think it's the (indeed empty) JavaScript. I ran you code forcing
> hte protocol to HTTP/1.0 (therefore blocking chunked encoding) and got
> the same results. Same with GET instead of POST ... looks pretty much like
> a broken server.
Thanks for taking a look at it. I think I found the source of the
problem:
http://www.wrcc.dri.edu/cgi-bin/wea_listex.pl?subacc
turns out there's a "raw data access policy". Hopefully my need falls
within their requirements of an acceptable "use and quantity" of
information about the weather ;-(
thanks again
Nick
resolution:
like most others, this problem has swiftly been revealed to be mostly
my fault! The form was *not* properly Y2K compliment *cough* *cough* so
after replacing instances of "2005" with "05" a friend discovered
within minutes that the site will, in fact, return tables of data in
HTML format (though still not in nice, column-delineated ASCII, but
that doesn't mean the parameter list for a grok-the-weather function
has to be any longer then it would before ;-) so... there you go:, if
you ever need hourly wind speed and direction highs, lows, and standard
deviations from stead nevada in for the last five years you now know
what to do.
signing off
thanks again for the help
Nick