Remotely Modifying a Running Program Using Swank

One of the strengths of Common Lisp is that it includes support for dynamically redefining classes and functions at run-time, while keeping data intact. Theoretically at least, this should enable support for keeping programs continually running in production while changing pieces of them – "Common Lisp Recipes" by Edi Weitz includes the following disclaimer about this functionality

If you'e ever talked to experienced Lispers, you've probably heard "war stories" of huge and complex systems to which substantial modifications were applied while they kept running and without interrupting the services they provided. Although this sometimes has to be taken with a grain of salt, it is in fact true that many COMMON LISP features were designed from the ground up to be dynamic in the sense that they can be changed at run time. This includes CLOS, where an object can change from one class to another, and where classes can be modified, although they already have objects "hanging off" of them.

– "Common Lisp Recipes" by Edi Weitz, section 13-8

My understanding is that some of this functionality at least is difficult/nonstandard to replicate in other languages such as Python and Java (please feel free let me know that I am wrong if that is the case!).

Anyway, I would like to remotely interact in lisp with remote instances of my current project I am working on – dumb-mq – so I figured it would be helpful to start with a small example of remote connection/redefinition.

This example uses sbcl and SLIME-mode in emacs, but it should work for other lisp implementations and other tools that support the swank protocol as well. The easiest way to get SBCL and emacs set up that I am aware currently of is to download the excellent Portacle IDE by Nicolas Hafner. Alternatively just figure out how to install them yourself.

Write the following lisp file: swankdemo.lisp

;; a little common lisp swank demo
;; while this program is running, you can connect to it from another terminal or machine
;; and change the definition of doprint to print something else out!
;; (ql:quickload :swank)
;; (ql:quickload :bordeaux-threads)

(require :swank)
(require :bordeaux-threads)

(defparameter *counter* 0)

(defun dostuff ()
  (format t "hello world ~a!~%" *counter*))

(defun runner ()
  (bt:make-thread (lambda ()
		    (swank:create-server :port 4006)))
  (format t "we are past go!~%")
  (loop while t do
       (sleep 5)
       (dostuff)
       (incf *counter*)
       ))

(runner)

You can run this program as follows:

sbcl --load swankdemo.lisp

The program will run indefinitely, printing out a message every five seconds and incrementing its counter. Imagine instead that this was program was accepting connections indefinitely and was providing an important service.

By default swank will accept connections only from localhost – if you would like to connect from a different computer you can use ssh tunneling to forward the port on the remote machine to a port on your local computer. For example

ssh -L4006:127.0.0.1:4006 username@example.com

will securely forward port 4006 on the server at example.com to your local computer's port 4006.

Let's connect to the program. Fire up emacs, type M-x slime-connect, at the prompts select 127.0.0.1 (the default) and port 4006 (type this in). If all went well, you are now connected to the remotely running lisp program! Just to check, see if you can retrieve the current value of the counter:

CL-USER> *counter*

Now let's say you want to change the definition of dostuff and reset the counter while you are at it. Type in the following either in an emacs scratch buffer, select it, and send it to the remote lisp program using M-x slime-eval-region (or an alternate method).

(defun dostuff ()
  (format t "goodbye world ~a!~%" *counter*))
(setf *counter* 0)

Observe swankdemo's output in the console – you will see the output change and the counter be reset. Success!

You can do more complicated redefinitions and changes – refer to the Common Lisp Standard (draft) section 7.2 and 7.3 for some information on modifying objects at run-time.