From: Frank Buss
Subject: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <tujply4mjczy.sk2o1f0t26c4$.dlg@40tude.net>
I have installed Portable AllegroServe (for http://www.luaplayer.org ) and
I want to give a detailed step-by-step howto in this posting how someone
can do this, starting with a new Debian Sarge installation. I don't know,
if I'm doing it "the right way", or if there are some security holes in
doing it like I've done it, so please correct me.

adduser --disabled-password --disabled-login web
aptitude cmucl
aptitude cmucl-source
aptitude install cl-aserve 

add /usr/share/common-lisp/source/aserve/ to *central-registry* in
/usr/share/common-lisp/source/asdf/asdf.lisp:

(defvar *central-registry*                                  
  '(*default-pathname-defaults*                             
    "/usr/share/common-lisp/source/aserve/"                 
    #+nil "/home/dan/src/sourceforge/cclan/asdf/systems/"   
    #+nil "telent:asdf;systems;"))                          

add a file /etc/init.d/packetfilter (make it executable) for firewall and
redirecting 80 to 8080 and 443 to 8443 (change the "local_ip" entry to your
IP address) :

----------------------------------------------------------

#!/bin/sh
#
# Startup script for packetfilter
#
# chkconfig: 23456 86 14
# description: packetfilter\
# processname: n/a

depmod -a
modprobe ip_conntrack
modprobe ip_conntrack_ftp


# A basic stateful local firewall


# The "uplink" interface (connection to the internet)
UPLINK="eth0"

# All interfaces, including lo
INTERFACES="lo eth0"

# The list of services that we allow incoming connection to
SERVICES="ssh https 80 8080 8443"

if [ "$1" = "start" ]
then

	echo "Starting firewall..."

	# Flush all rules on INPUT chain
	iptables -F INPUT

	# Set default policy to drop packets...
	iptables -P INPUT DROP

	# ...but allow incoming any packets on the non-uplink interfaces
	iptables -A INPUT -i ! ${UPLINK} -j ACCEPT

	# Allow incoming packets on tracked connections
	iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

	# enable public access to services
	for x in ${SERVICES}
	do
		iptables -A INPUT -i ${UPLINK} -p tcp --dport ${x} -m state --state NEW
-j ACCEPT
	done

	iptables -A INPUT -p tcp --dport 60000:60255 -j LOG --log-prefix
"portknock "
	# allow ICMP (ping)
	iptables -A INPUT -p ICMP -j ACCEPT

	# do not allow any ICMP (ping)
	# /bin/echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all

	# no explicit congestion notification
	# ("ECN freaks out a number of Internet routers, if ECN is enabled,
	#  your Linux machine won't be able to carry on network communications
	#  with 8% of the Internet.")
	if [ -e /proc/sys/net/ipv4/tcp_ecn ]
	then
		echo 0 > /proc/sys/net/ipv4/tcp_ecn
	fi

	# Linux kernel antispoofing filter	
	for x in ${INTERFACES}
	do
		echo 1 > /proc/sys/net/ipv4/conf/${x}/rp_filter
	done

	# better rejects for a brighter tomorrow
	iptables -A INPUT -p tcp -i ${UPLINK} -j REJECT --reject-with tcp-reset
	iptables -A INPUT -p udp -i ${UPLINK} -j REJECT --reject-with
icmp-port-unreachable

	# redirect 443 to 8443
	local_ip=1.2.3.4
 	iptables -t nat -A OUTPUT     --destination localhost   -p tcp --dport
443 -j REDIRECT --to-ports 8443
 	iptables -t nat -A OUTPUT     --destination ${local_ip} -p tcp --dport
443 -j REDIRECT --to-ports 8443
 	iptables -t nat -A PREROUTING --destination ${local_ip} -p tcp --dport
443 -j REDIRECT --to-ports 8443

	# redirect 80 to 8080
        iptables -t nat -A OUTPUT     --destination localhost   -p tcp
--dport 80 -j REDIRECT --to-ports 8080
        iptables -t nat -A OUTPUT     --destination ${local_ip} -p tcp
--dport 80 -j REDIRECT --to-ports 8080
        iptables -t nat -A PREROUTING --destination ${local_ip} -p tcp
--dport 80 -j REDIRECT --to-ports 8080
			

elif [ "$1" = "stop" ]
then

	echo "Stopping firewall..."
	iptables -F INPUT
	iptables -P INPUT ACCEPT
	
fi

----------------------------------------------------------

update-rc.d packetfilter defaults 

create a file /home/web/start.cl (assuming you want two virtual hosts,
substitue luaplyer and it4systems with your names) :

----------------------------------------------------------
;; init base
(setf ext:*gc-verbose* NIL)
(load "/usr/share/common-lisp/source/asdf/asdf.lisp")

;; load webserver
(asdf:operate 'asdf:load-op :aserve)
(mp::startup-idle-and-top-level-loops)

;; define package and global settings
(defpackage #:it4systems.web
  (:use #:common-lisp #:cl-user #:acl-compat.excl #:net.html.generator
#:net.aserve))

(in-package #:it4systems.web)
(unpublish :all t)

(defun open-log-file (path)
  (open (concatenate 'string "/home/web/" path)
        :direction :output
        :if-exists :append
        :if-does-not-exist :create))

(defparameter *wserver* (make-instance 'wserver))
(defparameter *vhost-table* (wserver-vhosts *wserver*))

;; it4systems web space
(defparameter *it4systems-names* '("www.it4-systems.com"
"it4-systems.com"))
(defparameter *it4systems-vhost* (wserver-default-vhost *wserver*))
(setf (vhost-names *it4systems-vhost*) *it4systems-names*)
(dolist (name *it4systems-names*)
  (setf (gethash name *vhost-table*) *it4systems-vhost*))
(setf (vhost-log-stream *it4systems-vhost*) (open-log-file
"it4systems.log"))
(setf (vhost-error-stream *it4systems-vhost*) (open-log-file
"it4systems.error.log"))
(publish-directory :host *it4systems-vhost*
                   :prefix "/"
                   :destination "/home/web/it4systems/")

;; luaplayer web space
(defparameter *luaplayer-names* '("www.luaplayer.org" "luaplayer.org"))
(defparameter *luaplayer-vhost* (make-instance 'vhost :names
*luaplayer-names*))
(dolist (name *luaplayer-names*)
  (setf (gethash name *vhost-table*) *luaplayer-vhost*))
(setf (vhost-log-stream *luaplayer-vhost*) (open-log-file "luaplayer.log"))
(setf (vhost-error-stream *luaplayer-vhost*) (open-log-file
"luaplayer.error.log"))
(publish-directory :host *luaplayer-vhost*
                   :prefix "/"
                   :destination "/home/web/luaplayer/")

;; start server
(net.aserve:start :port 8080 :listeners 20)
----------------------------------------------------------

create the directories /home/web/luaplayer and /home/web/it4system and
create index.html in both. Make sure, the owner of the directories and
files is "web.web".

add the file /home/web/start.sh (with web.web as owner and executable flag
set) :

----------------------------------------------------------
#!/bin/sh

cd /home/web
cat start.cl | cmucl &
----------------------------------------------------------

add the file /etc/init.d/web (make it executable) :

----------------------------------------------------------
#!/bin/sh
#
# Startup script for webserver
#
# chkconfig: 23456 86 14
# description: webserver\
# processname: n/a

# See how we were called.
case "$1" in
  start)
 echo -n "Starting webserver: "
 su -c /home/web/start.sh web
 echo
 ;;
  stop)
 echo -n "Shutting down webserver: "
 echo
 ;;
  restart)
 $0 stop
 $0 start
 ;;
  *)
 echo "Usage: $0 {start|stop|restart}"
 exit 1
esac

exit 0
----------------------------------------------------------

update-rc.d web defaults

restart your computer to make sure that after reboot it is started without
the need to start it manual

try http://yourserver/ and you'll see the apache-like access-logs in
/home/web/*.log.

-- 
Frank Bu�, ··@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de

From: Frank Buss
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <71p4no3atg85.1byzmk2cjrpud.dlg@40tude.net>
Frank Buss wrote:

> ----------------------------------------------------------
> #!/bin/sh
> 
> cd /home/web
> cat start.cl | cmucl &
> ----------------------------------------------------------

How can I improve this? I think start.cl does a lot of things, which could
be saved in a custom Lisp image to speed up the start time. How can I do
this?

And I'm login to the server with ssh. How can I start the process on boot
in such a way that I can open a REPL to the running Lisp process? Is it
possible to open it remote from Emacs, over a ssh connection?

-- 
Frank Bu�, ··@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de
From: Alex Mizrahi
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <42fc830c$0$18644$14726298@news.sunsite.dk>
(message (Hello 'Frank)
(you :wrote  :on '(Fri, 12 Aug 2005 12:46:11 +0200))
(

 FB> How can I improve this? I think start.cl does a lot of things, which
 FB> could be saved in a custom Lisp image to speed up the start time. How
 FB> can I do this?

find a point where it is loaded and is about to start server, and save
image.
you can find an example in ucw distribution.

 FB> And I'm login to the server with ssh. How can I start the process on
 FB> boot in such a way that I can open a REPL to the running Lisp process?

detachtty

 FB> Is it possible to open it remote from Emacs, over a ssh connection?

possible, with ssh port forwarding.
you can also use SLIME (but it will require some tuning -- disable separate
connection for output, or fix it to work on non-local machine).

)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"People who lust for the Feel of keys on their fingertips (c) Inity")
From: Tim Lavoie
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <6ntus2-oeg.ln1@theasylum.dyndns.org>
On 2005-08-12, Alex Mizrahi <········@users.sourceforge.net> wrote:
>  FB> How can I improve this? I think start.cl does a lot of things, which
>  FB> could be saved in a custom Lisp image to speed up the start time. How
>  FB> can I do this?
>
> find a point where it is loaded and is about to start server, and save
> image.
> you can find an example in ucw distribution.
>
>  FB> And I'm login to the server with ssh. How can I start the process on
>  FB> boot in such a way that I can open a REPL to the running Lisp process?
>
> detachtty
>
>  FB> Is it possible to open it remote from Emacs, over a ssh connection?
>
> possible, with ssh port forwarding.
> you can also use SLIME (but it will require some tuning -- disable separate
> connection for output, or fix it to work on non-local machine).

You can have both detachtty and SLIME, if that helps. I use a somewhat
modified form of the startup code from UCW for an app I'm working on,
which does use both. The detachtty part isn't something I regularly
connect to for editing or REPL use, but is a handy way to launch it
all. I start up XEmacs, do a M-x slime-connect, and it's all good.

You may want to just use detachtty for connectivity, and not launch
the SLIME server at all, since the latter will accept connections from
anybody on localhost. Since you already log in to the server with SSH,
you can still use SLIME if you want, by getting it to call attachtty
to connect.

I haven't used this in a while, but my .xemacs/init file used this:

(defun slime-sbclremote ()
  "connect to sbcl already running with detachtty"
  (interactive)
  (let ((inferior-lisp-program "sbcl-attach")
        (coding-system-for-read 'us-ascii-dos))
    (slime)))

There wasn't much to the sbcl-attach program, but I'd split it off so
I could fiddle with command-line arguments if I wanted:

   attachtty /home/tim/procs/run/sbcl.socket

This means that there is no socket listening that just anybody could
connect to, as they instead need access to that unix-domain socket on
the filesystem.

	Cheers,
	Tim

-- 
If you can't be a good example, then you'll just have to be a horrible
warning.
    -- Catherine Aird
From: Marco Baringer
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <m2mznnf5v0.fsf@soma.local>
"Alex Mizrahi" <········@users.sourceforge.net> writes:

> possible, with ssh port forwarding.
> you can also use SLIME (but it will require some tuning -- disable separate
> connection for output, or fix it to work on non-local machine).

both these slime issue are fixable. the variable
*dedicated-output-stream-port* and the elisp variables
slime-translate-from-lisp-filename-function and
slime-translate-to-lisp-filename-function.

(*dedicated-output-stream-port* is relativly new, but
slime-translate-(to|from)-lisp-filename-function has been around for
quite a while)

-- 
-Marco
Ring the bells that still can ring.
Forget the perfect offering.
There is a crack in everything.
That's how the light gets in.
	-Leonard Cohen
From: Frank Buss
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <1ura4gs4xvi29.1r419ici3awfo$.dlg@40tude.net>
Alex Mizrahi wrote:

>  FB> And I'm login to the server with ssh. How can I start the process on
>  FB> boot in such a way that I can open a REPL to the running Lisp process?
> 
> detachtty

thanks, this works. There was a problem with
"mp::startup-idle-and-top-level-loops" calling ends the load process, but I
have written it like (eval "mp::startup-idle-and-top-level-loops") and now
it works.

But there is another problem: When the server is running some time (about a
day), it stops serving webpages. The port is still open, I can telnet to
the webserver port, but it doesn't reponds. It is not much traffic on the
server, perhaps every some minutes a request. I've tried it with more
listeners and ":keep-alive nil", but this morning the server was not
responding again. I've enabled debugging now, hoping to see something more
and with attachtty I can check the running server. Any ideas, what I can
check on the REPL, when the freeze occurs again?

-- 
Frank Bu�, ··@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de
From: Frank Buss
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <vbhkydpvlebn$.18ld0x1gx2w0k$.dlg@40tude.net>
I don't know where the problem is, but it freezes again. I logged into the
server with attachtty, but it doesn't responds (I tried "(+ 1 2)", but no
result). Then I tried Ctrl-C, which stopped something, but didn't help. I'm
sorry if this sounds silly, but I have no clue how to debug a running
aserver. The last lines from the dribble file:

-----------------------------------------------------------------
d> (3-aserve-worker): file strategy is (:USE-SOCKET-STREAM)
d> (3-aserve-worker): 206 "Partial Content" - 976742 bytes
d> (2-aserve-worker): file strategy is (:USE-SOCKET-STREAM)
d> (2-aserve-worker): 206 "Partial Content" - 652134 bytes
d> (1-aserve-worker): file strategy is (:USE-SOCKET-STREAM)
d> (1-aserve-worker): 206 "Partial Content" - 327526 bytes

(+ 1 2)
^C
Interrupted at #xB7F5495E.
   [Condition of type SIMPLE-CONDITION]

Restarts:
  0: [CONTINUE] Return from BREAK.
  1: [ABANDON ] Abandon this request and wait for the next one
  2: [DESTROY ] Destroy the process

Debug  (type H for help)

(UNIX::SIGINT-HANDLER #<unused-arg> #<unused-arg> #.(SYSTEM:INT-SAP
#x3FFFC320))
Source:
; File: target:code/signal.lisp
:
-----------------------------------------------------------------


The error logfile shows some errors, perhaps the SIGPIPE killed the
processes? After some time the message "all threads busy, pause" occured
for every new request and the server freezed:


-----------------------------------------------------------------
9-aserve-worker: 08/14/05 - 17:38:16 - while processing command "GET
/gallery/sn
ake.png HTTP/1.1"
got error
Error in function UNIX::SIGPIPE-HANDLER:  SIGPIPE at #xB7F5495E.

7-aserve-worker: 08/14/05 - 17:41:44 - while processing command "GET
/design.jpg
 HTTP/1.1"
got error
Error in function GET-PEER-HOST-AND-PORT:
   Error "Transport endpoint is not connected" getting peer host and port
on FD
7.

4-aserve-worker: 08/14/05 - 18:13:02 - while processing command "GET
/luaplayerw
indows-alpha-release.zip HTTP/1.1"
got error
Error in function GET-PEER-HOST-AND-PORT:
   Error "Transport endpoint is not connected" getting peer host and port
on FD
7.

2-aserve-worker: 08/14/05 - 18:13:12 - while processing command "GET
/luaplayerw
indows-alpha-release.zip HTTP/1.1"
got error

aserve-accept-11: 08/14/05 - 20:14:38 - all threads busy, pause

-----------------------------------------------------------------


I'm sure I'm doing something wrong with my starting script, but I can't use
it with this buggy behaviour, so I have installed Apache as an intermediate
solution. Any ideas how I can fix the problem or do you know of a more
stable pure Lisp webserver? Is it possible at all to run a stable portable
aserve with CMUCL on Debian?

-- 
Frank Bu�, ··@frank-buss.de
http://www.frank-buss.de, http://www.it4-systems.de
From: Petter Gustad
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <87hddr60m8.fsf@filestore.home.gustad.com>
Frank Buss <··@frank-buss.de> writes:

> I'm sure I'm doing something wrong with my starting script, but I can't use
> it with this buggy behaviour, so I have installed Apache as an intermediate
> solution. Any ideas how I can fix the problem or do you know of a more
> stable pure Lisp webserver? Is it possible at all to run a stable portable
> aserve with CMUCL on Debian?


I've been running an intranet server based on CMUCL and Portable
Allegroserve for a couple years without any problems. Not very heavy
load though, only ca. 20 people entering and query data from a mysql
database.


$cat /etc/slackware-version 
Slackware 8.1
$screen -r
* *aserve-version*

(1 2 24)
*  *aserve-version*

(1 2 24)
* (lisp-implementation-type)

"CMU Common Lisp"
* (lisp-implementation-version)

"18d"


This server is typically started started by starting screen manually,
starting lisp, loading paserve and:

(mp::startup-idle-and-top-level-loops)
(net.aserve:start :port 8888 :chunking nil)


I recently started a new project where I'm using CMUCL�, Portable
Allegroserve, Webactions, CLSQL/Postgresql under Gentoo Linux. This
system hasn't been through anything other than alpha-testing, but
still witout any problems. Here I'm using crontab to start the
webserver:

crontab -u lisp -l
@reboot $HOME/src/web/rc.d/startpaserve >> $HOME/src/web/rc.d/startpaserve.log 2>&1

My startpaserve script looks like:

screen -D -m -S aserve /usr/bin/lisp -eval "(asdf:operate 'asdf:load-op :web)" -eval "(mp:make-process #'web:start-web-server)" -eval '(mp::startup-idle-and-top-level-loops)' 

This will start the server under screen after a reboot. I can then
connect to the server by typing screen -r and start the swank server

(asdf:operate 'asdf:load-op :swank)
(swank:create-swank-server 4005)

What I haven't figured out is how to make swank listen on a specific
interface (e.g. eth1) only.

Petter


�Common Lisp CVS release-19a 19a-release-20040728 + minimal debian patches

-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
From: Petter Gustad
Subject: Re: using Portable AllegroServe on CMUCL on Debian Sarge
Date: 
Message-ID: <87vf2731ya.fsf@filestore.home.gustad.com>
Petter Gustad <·············@gustad.com> writes:

> This server is typically started started by starting screen manually,
> starting lisp, loading paserve and:

I'll try to actually read the message before posting next time :-)

Petter
-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?