Quick boring version:
Anyone got UFFI declarations for Internet sockets in Linux?
Long less boring version:
I'm poking at my "writing a network security scanner in Lisp" project
again. So far, I have a TCP portscanner which will ably plod through
any number of ports, using SBCL'S SB-BSD-SOCKETS to attempt connecting
to each port on a host.
This is remarkably slow: scanning 65535 ports on the loopback interface
takes 37 seconds -- ten times as slow as the conventional (C-based)
portscanner "nmap". Now, I know full well, from reading this august
forum among others, that Lisp is not slow -- so something had to be the
matter with my assumptions. :)
* (time (tcp-port-scan #(127 0 0 1) 1 65535))
Evaluation took:
37.238 seconds of real time
30.715332 seconds of user run time
6.496013 seconds of system run time
59 page faults and
78102744 bytes consed.
(22 25 80 111 113 139 445 515 631 709 714 5060 5222 5223 5269 7101 8005
8008 8118 8180 8990 32768)
78102744 bytes consed. That strikes me as a lot. SB-BSD-SOCKETS uses a
CLOS object to represent each socket, and I'm making 64k of those. CLOS
objects for sockets would be great if I were writing an ordinary TCP
application which doesn't use a lot of sockets, but a portscanner is not
that.
So now I have wandered into the terrain of FFI, and specifically UFFI
since it seems to be a Good Thing for portability. I expect that
talking directly to the operating system's sockets API will be a good
deal faster than consing over a kilobyte per socket. Plus, there's a
nice UFFI example in the Debian UFFI package for _file_ sockets.
However, the Internet sockets API itself is pretty complicated, and I am
a little over my head in AF_INETs and sockaddr_in's and socklen_t's.
Since I don't count myself a C library guru, I wonder if perhaps there
is (to bastardize Newton) a handy giant somewhere around here who could
lend a shoulder for standing upon. So: Anyone got UFFI declarations
for Internet sockets in Linux? :)
Just for fun, here's the portscanner as it is today:
;
; SBCL portscanner.
(require :sb-bsd-sockets)
(use-package :sb-bsd-sockets)
(deftype port-number () '(integer 1 65535))
(defun make-tcp-socket ()
"Create a brand new TCP socket, good for all your TCP-ing needs."
(make-instance 'inet-socket
:type :stream
:protocol (get-protocol-by-name "tcp")))
(defmacro with-tcp-socket (sock &body body)
"Guaranteed close on socket."
`(let ((,sock (make-tcp-socket)))
(unwind-protect
(progn ,@body)
(socket-close ,sock))))
(defun probe-tcp-port (inet-addr port &optional (timeout 10))
(declare (port-number port))
"Returns T on open socket or NIL with a condition on failure."
(handler-case
(with-timeout timeout
(with-tcp-socket sock
(socket-connect sock inet-addr port)
t))
(sb-bsd-sockets::socket-error (condition)
(values nil condition))
(timeout (condition)
(values nil condition))))
(defun tcp-port-scan (inet-addr start-port end-port &optional (timeout 5))
(declare (port-number start-port end-port))
"Scan INET-ADDR for TCP ports from START-PORT to END-PORT."
(assert (< start-port end-port) (start-port end-port)
"Start port ~S is greater than end port ~S." start-port end-port)
(loop for port from start-port to end-port
if (probe-tcp-port inet-addr port timeout)
collect port))
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
From: Joe Marshall
Subject: Re: Seeking UFFI for sockets on Linux
Date:
Message-ID: <znebmeal.fsf@ccs.neu.edu>
"Karl A. Krueger" <········@example.edu> writes:
> I'm poking at my "writing a network security scanner in Lisp" project
> again. So far, I have a TCP portscanner which will ably plod through
> any number of ports, using SBCL'S SB-BSD-SOCKETS to attempt connecting
> to each port on a host.
>
> This is remarkably slow: scanning 65535 ports on the loopback interface
> takes 37 seconds -- ten times as slow as the conventional (C-based)
> portscanner "nmap". Now, I know full well, from reading this august
> forum among others, that Lisp is not slow -- so something had to be the
> matter with my assumptions. :)
What exactly is nmap doing? I can think of a couple of reasons it
*might* be faster:
Multiple threads so work is done during the network pauses.
Lower level protocols so that TCP connections don't have to be
attempted for ports that are not in use. (I don't know if it might
be quicker to ping the port, probably not, but setting up and
initializing the TCP state and buffers just to be told that the
connection failed might be a problem.)
Caching of certain results. I notice that a call is made to
(get-protocol-by-name "tcp") each time that a TCP socket is
created. If this is searching the etc/services file each and every
time, it could be a problem.
Joe Marshall <···@ccs.neu.edu> writes:
> Caching of certain results. I notice that a call is made to
> (get-protocol-by-name "tcp") each time that a TCP socket is
> created. If this is searching the etc/services file each and every
> time, it could be a problem.
It calls getprotobyname(), which might be /etc/protocols or might be
something involving nsswitch (dns, nis, ldap etc) dependingon the site.
CL-USER> (time (dotimes (i 65535) (sb-bsd-sockets:get-protocol-by-name "tcp")))
Evaluation took:
11.281 seconds of real time
9.24 seconds of user run time
1.86 seconds of system run time
1 page fault and
11952600 bytes consed.
But it's very unlikely to change during the course of a run, so, yes,
cache it
-dan
--
http://web.metacircles.com/ - Open Source software development and support
Joe Marshall <···@ccs.neu.edu> wrote:
> "Karl A. Krueger" <········@example.edu> writes:
>> This is remarkably slow: scanning 65535 ports on the loopback interface
>> takes 37 seconds -- ten times as slow as the conventional (C-based)
>> portscanner "nmap". Now, I know full well, from reading this august
>> forum among others, that Lisp is not slow -- so something had to be the
>> matter with my assumptions. :)
>
> What exactly is nmap doing? I can think of a couple of reasons it
> *might* be faster:
nmap doesn't use multiple threads, and in its default mode it does use
socket connect() calls, not lower-level network mangling. (Otherwise it
would only be able to run as root on Unix systems.)
However, caching GET-PROTOCOL-BY-NAME does speed the process up by 30%.
Good call!
(let ((tcp (get-protocol-by-name "tcp")))
(defun make-tcp-socket ()
"Create a brand new TCP socket, good for all your TCP-ing needs."
(make-instance 'inet-socket
:type :stream
:protocol tcp)))
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
From: Joe Marshall
Subject: Re: Seeking UFFI for sockets on Linux
Date:
Message-ID: <r7znjecv.fsf@ccs.neu.edu>
"Karl A. Krueger" <········@example.edu> writes:
> Joe Marshall <···@ccs.neu.edu> wrote:
>> "Karl A. Krueger" <········@example.edu> writes:
>>> This is remarkably slow: scanning 65535 ports on the loopback interface
>>> takes 37 seconds -- ten times as slow as the conventional (C-based)
>>> portscanner "nmap". Now, I know full well, from reading this august
>>> forum among others, that Lisp is not slow -- so something had to be the
>>> matter with my assumptions. :)
>>
>> What exactly is nmap doing? I can think of a couple of reasons it
>> *might* be faster:
>
> nmap doesn't use multiple threads, and in its default mode it does use
> socket connect() calls, not lower-level network mangling. (Otherwise it
> would only be able to run as root on Unix systems.)
>
> However, caching GET-PROTOCOL-BY-NAME does speed the process up by 30%.
> Good call!
Can you re-use the socket if the socket-connect fails?
Joe Marshall <···@ccs.neu.edu> wrote:
> "Karl A. Krueger" <········@example.edu> writes:
>> However, caching GET-PROTOCOL-BY-NAME does speed the process up by 30%.
>> Good call!
>
> Can you re-use the socket if the socket-connect fails?
It looks like you can, and it is an improvement:
(let ((socket (make-tcp-socket)))
(defun probe-tcp-port-2 (inet-addr port &optional (timeout 10))
(declare (port-number port))
"Documentation goes here."
(when (null socket) (setq socket (make-tcp-socket)))
(handler-case
(with-timeout timeout
(progn
(socket-connect socket inet-addr port)
(setq socket nil)
t))
(sb-bsd-sockets::socket-error (condition)
(values nil condition))
(timeout (condition)
(values nil condition)))))
This improves the run time for 65535 localhost ports from 22 sec to 14
sec. It also reduces consing from ~65 MiB to ~58 MiB. However, I am a
little suspicious of it, since it does not call SOCKET-CLOSE and I don't
know if I should trust the system not to leak fds.
This variant which seems like it should work, is broken -- catching
BAD-FILE-DESCRIPTOR-ERRORs from invalid socket reuse when used
repeatedly:
(let ((socket (make-tcp-socket)))
(defun probe-tcp-port-3 (inet-addr port &optional (timeout 10))
(declare (port-number port))
"Warning -- Broken!"
(handler-case
(prog1
(with-timeout timeout
(progn
(socket-connect socket inet-addr port)
t))
(socket-close socket)
(setq socket (make-tcp-socket)))
(sb-bsd-sockets::socket-error (condition)
(values nil condition))
(timeout (condition)
(values nil condition)))))
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
From: Joe Marshall
Subject: Re: Seeking UFFI for sockets on Linux
Date:
Message-ID: <he0idgug.fsf@ccs.neu.edu>
"Karl A. Krueger" <········@example.edu> writes:
> Joe Marshall <···@ccs.neu.edu> wrote:
>>
>> Can you re-use the socket if the socket-connect fails?
>
> It looks like you can, and it is an improvement:
>
> This improves the run time for 65535 localhost ports from 22 sec to 14
> sec. It also reduces consing from ~65 MiB to ~58 MiB. However, I am a
> little suspicious of it, since it does not call SOCKET-CLOSE and I don't
> know if I should trust the system not to leak fds.
I'm pretty sure you won't leak if the connect fails.
In poking around the sources, it also looks like a SOCKADDR is created
and `pinned' each time around the loop. It'd probably be better if a
single SOCKADDR were created (and created in non-gc'd memory) and then
you could simply bash the port each time around.
I imagine that pinning the SOCKADDR has to do some work like entering
the array in some table that the GC can consult, and that might be an
issue, especially if GC interlocking is involved.
Joe Marshall <···@ccs.neu.edu> wrote:
> "Karl A. Krueger" <········@example.edu> writes:
>> This improves the run time for 65535 localhost ports from 22 sec to 14
>> sec. It also reduces consing from ~65 MiB to ~58 MiB. However, I am a
>> little suspicious of it, since it does not call SOCKET-CLOSE and I don't
>> know if I should trust the system not to leak fds.
>
> I'm pretty sure you won't leak if the connect fails.
That's the trouble. If the connect fails, I keep the socket object
around for the next pass -- but if the connect succeeds, I throw it to
the garbage collector by doing (SETQ SOCKET NIL), because a used socket
is no good anymore.
Here's the dilemma:
1. In order to avoid (the impression of) an fd leak, the
underlying system must be sure that the fd is reclaimed when
the socket object is gc'd. However, I know from reading
Lisp-vs-Java flamewars that one cannot rely on destructors.
It seems to me to be the Right Thing to call SOCKET-CLOSE
before throwing the socket away, just as one would call
close() in C to reclaim the fd.
2. However, adding a SOCKET-CLOSE call before (SETQ SOCKET NIL)
causes the program to break intermittently (possibly due to
the timeout?) -- for some reason as yet unknown to me, it
starts to throw invalid-fd errors. This is where I'm digging
now -- "where are these errors coming from?"
> In poking around the sources, it also looks like a SOCKADDR is created
> and `pinned' each time around the loop. It'd probably be better if a
> single SOCKADDR were created (and created in non-gc'd memory) and then
> you could simply bash the port each time around.
So perhaps the way to go would be to fiddle with SB-BSD-SOCKETS, add a
way to keep track of your own SOCKADDR rather than having one consed up
for you on each connect, and add a SETF form for modifying the port ...
... This is getting complicated. :)
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
"Karl A. Krueger" <········@example.edu> writes:
>
> 2. However, adding a SOCKET-CLOSE call before (SETQ SOCKET NIL)
> causes the program to break intermittently (possibly due to
> the timeout?) -- for some reason as yet unknown to me, it
> starts to throw invalid-fd errors. This is where I'm digging
> now -- "where are these errors coming from?"
The problem might spring from the fact that closing a TCP socket does
not only release the file descriptor, but also attempts to terminate
the TCP connection. When you call close on a socket, the client (you)
will send a TCP segment with the FIN flag set and change state to
FIN_WAIT_1 (should be visible using netstat). The server should ACK
the FIN and change state to CLOSE_WAIT, then close the connection
and send a FIN. Upon receiving the ACK, the client will change state
to FIN_WAIT_2 and then TIME_WAIT when the FIN arrives. The FIN will
be acknowledged by the client, and the client will usually wait for
30 seconds or so before actually closing the connection.
The conclusion is that it will usually take at least 30 seconds from
you close a socket until the file descriptor can be reclaimed. I'm
not sure whether this is related to your problem or not, but it should
be easy to verify using netstat and/or a packet analyser.
--
We tend to meet any new situation by reorganising; and a wonderful method
it can be for creating the illusion of progress while producing confusion,
inefficiency and demoralisation. -- Gaius Petronius, 60 AD
From: Joe Marshall
Subject: Re: Seeking UFFI for sockets on Linux
Date:
Message-ID: <oeupc1k2.fsf@ccs.neu.edu>
"Karl A. Krueger" <········@example.edu> writes:
> That's the trouble. If the connect fails, I keep the socket object
> around for the next pass -- but if the connect succeeds, I throw it to
> the garbage collector by doing (SETQ SOCKET NIL), because a used socket
> is no good anymore.
>
> Here's the dilemma:
>
> 1. In order to avoid (the impression of) an fd leak, the
> underlying system must be sure that the fd is reclaimed when
> the socket object is gc'd. However, I know from reading
> Lisp-vs-Java flamewars that one cannot rely on destructors.
> It seems to me to be the Right Thing to call SOCKET-CLOSE
> before throwing the socket away, just as one would call
> close() in C to reclaim the fd.
>
> 2. However, adding a SOCKET-CLOSE call before (SETQ SOCKET NIL)
> causes the program to break intermittently (possibly due to
> the timeout?) -- for some reason as yet unknown to me, it
> starts to throw invalid-fd errors. This is where I'm digging
> now -- "where are these errors coming from?"
Ok, now I see what you are saying.
Here's my wild theory (based on nothing but my imagination): Closing a
socket takes a fair amount of time, but the process need not stall
while the socket is being closed. The call to SOCKET-CLOSE no doubt
initiates the close, but doesn't wait until the socket is `truly
closed'. The (SETQ SOCKET NIL) call causes the GC to discard the
socket and makes the memory associated with the socket a candidate for
re-use. My completely wild guess is that the socket code tries to use
the SOCKADDR or something during closing and therefore uses the memory
after it has been free'd by the GC. Your intermittent error would
occur if the GC happens between the call to SOCKET-CLOSE and the
actual closing of the socket *and* you manage to re-use the freed
storage.
So try putting something like (PUSH SOCKET *CLOSING-SOCKETS*) just
before the (SETQ SOCKET NIL) to keep the GC from reclaiming the
storage. If it works, I think I'm on the right track. If not, well,
I've been wrong before.
Joe Marshall <···@ccs.neu.edu> wrote:
> So try putting something like (PUSH SOCKET *CLOSING-SOCKETS*) just
> before the (SETQ SOCKET NIL) to keep the GC from reclaiming the
> storage. If it works, I think I'm on the right track. If not, well,
> I've been wrong before.
It works. *CLOSING-SOCKETS* accumulates one socket for each open port
on the target system, so those end up needing to be reaped later on to
reclaim space, but that's not a show-stopper.
More importantly, the results are correct and it doesn't appear to leak
fds, as all of those socket objects on *CLOSING-SOCKETS* refer to the
same fd number.
Thanks for your help -- I think I'm starting to Get It. :)
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
From: Joe Marshall
Subject: Re: Seeking UFFI for sockets on Linux
Date:
Message-ID: <ad69adsc.fsf@ccs.neu.edu>
"Karl A. Krueger" <········@example.edu> writes:
> It works. *CLOSING-SOCKETS* accumulates one socket for each open port
> on the target system, so those end up needing to be reaped later on to
> reclaim space, but that's not a show-stopper.
>
> More importantly, the results are correct and it doesn't appear to leak
> fds, as all of those socket objects on *CLOSING-SOCKETS* refer to the
> same fd number.
>
> Thanks for your help -- I think I'm starting to Get It. :)
You're welcome. How's the performance?
Joe Marshall <···@ccs.neu.edu> wrote:
> "Karl A. Krueger" <········@example.edu> writes:
>> Thanks for your help -- I think I'm starting to Get It. :)
>
> You're welcome. How's the performance?
The current version is twice as fast as what I started with.
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
Joe Marshall <···@ccs.neu.edu> writes:
> Here's my wild theory (based on nothing but my imagination): Closing
> a socket takes a fair amount of time, but the process need not stall
> while the socket is being closed. The call to SOCKET-CLOSE no doubt
> initiates the close, but doesn't wait until the socket is `truly
> closed'. The (SETQ SOCKET NIL) call causes the GC to discard the
> socket and makes the memory associated with the socket a candidate
> for re-use. My completely wild guess is that the socket code tries
> to use the SOCKADDR or something during closing and therefore uses
> the memory after it has been free'd by the GC.
Just checking, but shouldn't this--if, as later posts seem to suggest,
it is confirmed to be what's going on--be construed as a bug in the
socket code. Presumably this is happening because the code that reuses
the SOCKADDR or whatever (assuming your theory is correct) is C code
which isn't playing nice with the GC. I.e. if the object is still
referenced, how could it be GC'd. Not that Karl shouldn't go with your
workaround for the time being but this should be reported to the Lisp
vendor, right? (I forget which Lisp he's using.)
-Peter
--
Peter Seibel ·····@javamonkey.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
Peter Seibel <·····@javamonkey.com> writes:
> socket code. Presumably this is happening because the code that reuses
> the SOCKADDR or whatever (assuming your theory is correct) is C code
> which isn't playing nice with the GC. I.e. if the object is still
> referenced, how could it be GC'd. Not that Karl shouldn't go with your
> workaround for the time being but this should be reported to the Lisp
> vendor, right? (I forget which Lisp he's using.)
It's SBCL, and I'm watching carefully - unfortunately I've been unable
to replicate it thus far. Karl, what version are you using?
-dan
--
http://web.metacircles.com/ - Open Source software development and support
Daniel Barlow <···@telent.net> wrote:
> Peter Seibel <·····@javamonkey.com> writes:
>> socket code. Presumably this is happening because the code that reuses
>> the SOCKADDR or whatever (assuming your theory is correct) is C code
>> which isn't playing nice with the GC. I.e. if the object is still
>> referenced, how could it be GC'd. Not that Karl shouldn't go with your
>> workaround for the time being but this should be reported to the Lisp
>> vendor, right? (I forget which Lisp he's using.)
>
> It's SBCL, and I'm watching carefully - unfortunately I've been unable
> to replicate it thus far. Karl, what version are you using?
SBCL 0.8.4, which is what's in Debian "testing" at the moment.
Linux 2.6.0-test9 kernel; x86 SMP (Intel Xeon) architecture.
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
Peter Seibel <·····@javamonkey.com> writes:
> Joe Marshall <···@ccs.neu.edu> writes:
>
> Just checking, but shouldn't this--if, as later posts seem to suggest,
> it is confirmed to be what's going on--be construed as a bug in the
> socket code. Presumably this is happening because the code that reuses
> the SOCKADDR or whatever (assuming your theory is correct) is C code
> which isn't playing nice with the GC. I.e. if the object is still
> referenced, how could it be GC'd. Not that Karl shouldn't go with your
> workaround for the time being but this should be reported to the Lisp
> vendor, right? (I forget which Lisp he's using.)
There was a bug in SB-BSD-SOCKETS' use of finalizers, causing
occasional closure of currently-in-use file descriptors which has been
fixed in SBCL's CVS. It's quite possible that this might have been
the cause of strange behaviour if the OP was indeed using SBCL.
Christophe
--
http://www-jcsu.jesus.cam.ac.uk/~csr21/ +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%") (pprint #36rJesusCollegeCambridge)
"Karl A. Krueger" <········@example.edu> writes:
> ... This is getting complicated. :)
I reckon it'll get more complicated when you have to talk to remote
hosts too. The current scheme looks good for when latency is low and
most ports are closed. Some more things to think about are:
Network latency can make you spend a lot of time waiting. The worst
case is if a firewall is discarding your packets, so that your
connects will have to timeout. Making requests in parallel will
probably be important.
If the remote end accepts a lot of your connections, this causes a
lot of work for the kernel. If they didn't want to be scanned, they
could possibly crash your computer just by acknowledging your
connection requests and ignoring everything else, forcing your
kernel to open and hold a lot of sockets. Maybe you'll want to be
careful to avoid some bad cases.
So good optimizations for real usage may be much different to what
works well over the loopback interface.
On the other hand, I'm speaking without knowing what you actually
plan to use the program for :-)
Cheers,
Luke
Luke Gorrie <····@bluetail.com> wrote:
> Network latency can make you spend a lot of time waiting. The worst
> case is if a firewall is discarding your packets, so that your
> connects will have to timeout. Making requests in parallel will
> probably be important.
It's true. I have a couple of approaches in mind for this ...
My eventual goal is not a clone of nmap, but rather a port of an
existing program I have written in Python. This is a multi-threaded
program that doesn't scan 65535 ports on a host; rather, it scans a few
ports on each of many hosts. It then invokes external programs that do
the actual vulnerability testing, and it records its results in a
PostgreSQL database.
This program is kind of tailored to networks that look like my
workplace's: thousands of machines of several different kinds, scattered
sparsely over many subnets. So it spends a lot of effort eliminating
addresses and ports it doesn't need to scan before spawning threads for
those it does -- it doesn't beat on Windows vulnerabilities on Unix
hosts or nonexistent ones. It isn't meant to be good for "bad guys"
scanning other people's networks through firewalls.
What I hope out of porting it to Lisp is really twofold: first, to
learn a lot more Lisp; second, to bring more of the external code inside
and make it faster.
> If the remote end accepts a lot of your connections, this causes a
> lot of work for the kernel. If they didn't want to be scanned, they
> could possibly crash your computer just by acknowledging your
> connection requests and ignoring everything else, forcing your
> kernel to open and hold a lot of sockets. Maybe you'll want to be
> careful to avoid some bad cases.
Sure, if people are running things like LaBrea, a scanner has to eat a
timeout for every port touched. That would suck. :)
--
Karl A. Krueger <········@example.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews
"Karl A. Krueger" <········@example.edu> writes:
> This is a multi-threaded program that doesn't scan 65535 ports on a
> host; rather, it scans a few ports on each of many hosts. It then
> invokes external programs that do the actual vulnerability testing,
> and it records its results in a PostgreSQL database.
>
> This program is kind of tailored to networks that look like my
> workplace's: thousands of machines of several different kinds, scattered
> sparsely over many subnets.
Sounds really cool :-). I like to write programs like this too. I'd
like to offer one (slightly verbose) idea that might be a nice way to
have a crack at it.
You start with a search space of some set of TCP ports on some set of
IP subnets. The expected output is the subset of IP/port pairs that
are actually accepting connections.
The program could be seen as two concurrent activities (coroutines):
A "sender" that iterates through the search space, sending a TCP SYN
packet for each element. It would send the packets at some fixed
rate in packets per second.
A "receiver" that looks at each SYN+ACK packet arriving on your
machine and collects the IP/port pair for each one that belongs in
the search space.
The program terminates some seconds after the "sender" has sent the
SYN to the last destination, and the result is the set of IP/port
pairs collected by the receiver.
One nice property of this program is that it's guaranteed to terminate
in (/ size-of-search-space packets-per-second) seconds, even if you
have awkward firewalls, honeypots, broken computers, and so on. With
thousands of machines in total, that might be important.
If that doesn't sound too crazy, here's an idea for implementing it:
Linux has a great feature called a 'tun' device, which is described in
/usr/src/linux/Documentation/networking/tuntap.txt. This is a very
simple way of writing a virtual network interface as a user-space
program.
The upshot is that if you open a file called '/dev/net/tun', then a
new virtual network interface called 'tun0' appears on your
computer. You can assign tun0 an IP address to with 'ifconfig' as
usual. The file descriptor you got from opening the tun device can be
used to write() IP packets directly to the kernel, and to read() any
packets that the kernel decided to route into the tun0 interface.
Putting this together, if you opened a tun device from Lisp, you could
use it to write() TCP SYN packets to the appropriate destinations and
read() the SYN+ACKs that arrive in reply. Taking this approach means
you have to parse and unparse TCP/IP headers, but all of the other
problems - complex OS interfaces, file descriptor leaks, scalability,
threading and concurrency - should all totally disappear. You don't
even need to run as root, provided you have read/write permissions on
/dev/net/tun.
That's one way to do it, anyhow. :-)
Cheers,
Luke
From: Bulent Murtezaoglu
Subject: Re: Seeking UFFI for sockets on Linux
Date:
Message-ID: <87r7zlpzqr.fsf@cubx.internal>
>>>>> "KK" == Karl A Krueger <········@example.edu> writes:
KK> [...] This is a
KK> multi-threaded program that doesn't scan 65535 ports on a
KK> host; rather, it scans a few ports on each of many hosts. It
KK> then invokes external programs that do the actual
KK> vulnerability testing, and it records its results in a
KK> PostgreSQL database. [...]
Hmm, I assume, then, you don't need to have sockets functionality
in this code. Maybe you could look into something like
http://www.cliki.net/Slitch and deal with raw stuff and maintain
state for tcp manually in your code.
cheers,
BM
Joe Marshall <···@ccs.neu.edu> writes:
> I'm pretty sure you won't leak if the connect fails.
>
> In poking around the sources, it also looks like a SOCKADDR is created
> and `pinned' each time around the loop. It'd probably be better if a
> single SOCKADDR were created (and created in non-gc'd memory) and then
> you could simply bash the port each time around.
Yeah. The interface is really not optimized for creating a zillion
sockets, but I'm observing this thread with interest.
> I imagine that pinning the SOCKADDR has to do some work like entering
> the array in some table that the GC can consult, and that might be an
> issue, especially if GC interlocking is involved.
Assuming we're using x86 here, pinning the sockaddr means pushing a
tagged pointer to it onto the C stack. The GC is conservative with
respect to potential pointers found on the C stack (as in, it doesn't
move the objects they might be pointing at). Allocating the sockaddr
is probably a bigger time sink.
If we're not using x86 we have to put up with the low-tech semispace
GC, so the with-pinned-objects call just disables GC for the duration.
This is probably not a problem given that these ports have no thread
support anyway, so there's not a lot else that might be consing during
the extent of the C call.
For anyone about to ask about the overhead involved in consing: each
thread gets an 8k local region to allocate from. Each allocation
within the region involves setting a pseudo-atomic flag, doing the
alloc, initializing the memory (at least the header word, plus whatever
else is necessary to make the space safe for GC), then checking
the pseudo-atomic-interrupted flag to see if a signal was received
during the section. When the region's full, creating a new one
involves getting a lock from the GC, which is obviously slower.
It would be interesting to experiement with the region size and see
what different sizes so, but there are other things to do first...
-dan
--
http://web.metacircles.com/ - Open Source software development and support
"Karl A. Krueger" <········@example.edu> writes:
> I'm poking at my "writing a network security scanner in Lisp" project
> again. So far, I have a TCP portscanner which will ably plod through
> any number of ports, using SBCL'S SB-BSD-SOCKETS to attempt connecting
> to each port on a host.
If you want to do heavy traffic, I'd recommend using a low-level raw
IP or Ethernet interface. The kernel is likely to be a major
bottleneck - it keeps a lot of state for TCP sockets, and sometimes
for a very long time (stuck in TIME_WAIT state and so on). Linuxen at
least tend to have a lot of bad cases for performance around the place
for these Denial-of-Service style workloads.
Most operating systems should have low-level interfaces. On Linux you
can use PF_PACKET sockets (see PACKET(7) manpage).
I sometimes use Lisp for stress-testing IP networking stuff at
work. With Linux and PF_PACKET sockets I can generate about 20K small
packets per second on my 1.7Ghz laptop ("cheating" by pregenerating
all the frames and looping over them).
The code I use is pretty half-baked, but it's in the sourceforge
project 'slitch' if you want to have a look. It's based on CMUCL and
Linux, and at least complete enough to let you write a working
Ethernet switch in about a page of Lisp [1] :-)
Also, 80MB of consed data from short-lived objects probably takes a
decent copying generational collector like CMUCL's about 50ms to
collect and be totally insignificant in a program like this. (Consing
is good for you!!)
P.S., a totally different approach would be to use Linux's
kernel-space packet generator ('pktgen' module), which your Lisp
program could script via the /proc filesystem. I didn't have any luck
getting it working myself.
Cheers and best of luck,
Luke
1: A small ethernet switch.
http://www.bluetail.com/~luke/misc/lisp/switch.lisp