Network programming is something that can be tricky in every programming language. Fortunately all network programming makes use of BSD sockets, a paradigm that almost universal in all modern operating systems and programming languages.
In this article, I'll show you how you can program sockets in Lisp using the SBCL implementation. You see, the Common Lisp specification has no specification for socket programming. So every Lisp compiler is free to specify it's own implementation.
Note: This is a first version of this article and only focusses on writing a Lisp server. I'll update this article later to include a Lisp client.
The completed examples we build below can be found here:
General flow for sockets
All systems making use of network sockets follow a similar flow. The flow does differ between client and server implementations.
The first step is to create a socket and set the desired socket options. Next you have to bind the socket to a specific address. An address in socket programming is a combination of an IP address and a port number. After the socket is bound to an address, the server can listen to incoming connections of clients.
Writing a Lisp server
Let's start by writing a simple server in Lisp. Note that this code won't work just yet, follow along.
(defparameter *address* #(0 0 0 0))
(defparameter *port* 7127)
(defun start-server ()
(let ((server (make-instance 'sb-bsd-sockets:inet-socket :type :stream :protocol :tcp)))
; Set socket options
(setf (sb-bsd-sockets:sockopt-reuse-address server) t)
(setf (sb-bsd-sockets:non-blocking-mode server) t)
; Bind to an address
(sb-bsd-sockets:socket-bind server *address* *port*)
(sb-bsd-sockets:socket-listen server 1)))
First we define two parameters, address and port. The address parameter is a vector respresenting the IP address 0.0.0.0. The port parameter is the integer 8080. Binding our socket to this address will let it listen on all IP addresses of the computer we run this program on.
In the start-server function we first create a socket by creating an instance of inet-socket. Next we set the options and then bind it to the address. Finally we start listening on the socket.
Next we have to accept any client connections and handle those connections. For our example we will create a server that will echo everything a client sends and then close the connection.
Let's update our start-server function to accept connections and pass the connection to a handler function. To make reading from the socket a bit easier, I've added a helper function read-from-connection.
(defun start-server ()
(let ((server (make-instance 'sb-bsd-sockets:inet-socket :type :stream :protocol :tcp)))
; Set socket options
(setf (sb-bsd-sockets:sockopt-reuse-address server) t)
(setf (sb-bsd-sockets:non-blocking-mode server) t)
; Bind to an address
(sb-bsd-sockets:socket-bind server *address* *port*)
(sb-bsd-sockets:socket-listen server 1)
(loop
(let ((connection (sb-bsd-sockets:socket-accept server)))
(when connection
(handle-connection connection))))))
(defun read-from-connection (connection)
"Read data from a connection."
(multiple-value-bind (buffer length) (sb-bsd-sockets:socket-receive connection nil 1024)
(let (data)
(if (> length 0)
(subseq buffer 0 length)
data))))
(defun handle-connection (connection)
"Handle an incoming connection."
(let ((data (read-from-connection connection)))
(sb-bsd-sockets:socket-send connection data nil)
(sb-bsd-sockets:socket-close connection)))
As you can see we had to do some processing of the data received from the socket. Our function read-from-connection reads up to 1024 bytes from the connection. Lisp has to allocate an entire buffer of that length, but only a certain number of bytes are received. So we simple return the part of the buffer equal to the length of the data written.
In the handle-connection function, we read data from the connection using our helper function. Then we simply send that data back to the connection by using the socket-send function, and finally close our socket by making use of the socket-close function.
To connect to our server you can use the telnet command on your computer:
michael@winston:~|⇒ telnet 127.0.0.1 9000
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Lisp is amazing!
Lisp is amazing!
Connection closed by foreign host.
As you can see the telnet commands establishes a connection. We then enter something ("Lisp is amazing!" in my example) and press enter. The server immediately echos back what we sent it and closes the connection.
The complete example up untill now can be found in the simple echo server file.
Handling multiple client connections
Our design has one important flaw. Because of the way our server handles connections, only one connection can be handled by our server at a time.
You can test this out by connecting two telnet sessions to our server, just connect but don't send anything just yet. If you send a string to our server on the second telnet session, you won't get an immediate reply. The connection will stay open, without a response, untill the first session is finished. Send something through the first telnet session and you'll notice the second session will get a response and close after the first session does.
If you take a look at the flow of our server, you'll see that our loop that accepts connections is 'kept busy' by our handle-connection function. We can't accept new connections while our handle-connection function hasn't finished. In this particular case, the impact is negligible: since we simple read, send and then close the connection we will quickly return to the loop accepting connections. But image if our connection handler would keep the connection alive and read multiple times from a connection? Bad design!
Luckily, we can easily fix this by starting a new thread for each connection and handle the connection in the new thread.
(defun handle-connection (connection)
(sb-thread:make-thread
(lambda ()
; Handle our connection in this lambda, which is run in a new thread
(loop
(let ((data (read-from-connection connection)))
; Do something with the data received
))
(sb-bsd-sockets:socket-close connection))))We changed our handle-connection function to make a new thread. That thread will spin off from the main server thread and continue on its own. While the thread is running it's own loop in this case, the server is free to accept new connections that will be handled by threads of their own.
If you test our server with two telnet session like we did before, you'll notice our server now responds as expected. The fact that one client was connected first, but hasn't finished yet, makes no difference when handling a second client.
The full example can be found in the threaded echo server file.
More resources
If you're interested in more information, be sure to check out the following resources:
SBCL Manual: Networking All information about the sb-bsd-sockets module of SBCL.
USOCKET A Common Lisp library that wraps socket implementations of different Lisp compilers.
Beej's Guide to Network Programming A very good resource if you want to learn more about BSD sockets and networking internals.