From: Kuo-Chieh Ting
Subject: Answer: How to get a html content from net in lisp
Date: Wed, 20 Nov 1996 00:00:00 +0000
Message-ID: <56vlej$nn2@george.rutgers.edu>

Hi folks:

	 Thanks for everybody, I got lots of response about how to use lisp to grap a HTML file from the net.   This is a organized list.  if someone has better idea, please post it to us. 


Kuo-Chieh Ting
······@paul.rutgers.edu
	

-------------------------------------------------------------------------
generic system shell 
 
From Leif Nixon (·····@softlab.se):
I'm using the Lynx browser for such things. You should have
no trouble finding it on e.g. Yahoo. It can be used in batch
mode to download webpages either in HTML or text
form.

Provided your lisp implementation permits you to call
an external program and return its result in a string, you
should have no problem to implement what you want.

I'm using scsh, a Scheme implementation intended for Unix
shell scripting and systems programming. In scsh you can
write get-page-from-url like this, provided you have Lynx
installed:

(define (get-page-from-url url)
  (run/string lynx -source ,url))

---------------------------------------------------------------------------
UNIX & PC:

From Lawrence Hunter  (······@nlm.nih.gov):
My advice would be to start with http://www.ics.uci.edu/WebSoft/libwww-perl,
and then call the perl script from within lisp.


From Judy Winn     (···@Franz.COM):
We have a socket package for Solaris 2, Sun OS and HP that makes this easy.
The sockets are available via ftp from ftp/franz/pub/socket, read me file is
socket.n.  There is also some source code example which you can find under
clients.

----------------------------------------------------------------------------
Machintosh:

from Shannon Spires	(·······@telespin.com):
This is pretty close to what you want, and it works in MCL:

(in-package :ccl)

(require :mactcp)

(defun get-http-string (server command)
  (let ((stream (open-tcp-stream server 80)))
    ;send the command
    (format stream  command)
    (write-char #\Newline stream)
    ;(write-char #\Newline stream) ; WebStar expects 2 Newlines
    (write-char #\Linefeed stream)
    (force-output stream)
    ;read the answer
    (let ((result nil))
      (push (read-char stream nil :end) result)
      (loop
        (cond ((ccl::stream-eofp stream)
               (return))
              (T
               (let ((char (read-char stream nil :end)))
                 (if (eq char :end)
                   (return)
                   (push char result))))
              ))
      (prog1
        (coerce (nreverse result) 'string)
        (close stream))
      )
    )
  )

(get-http-string "www.adobe.com" "GET /")