The Case Against Lisp - And When Would I Choose Lisp Today?

I wrote an article back in 2015 entitled The Case for Lisp .  It lays out a simple argument, founded on expressiveness, for choosing Lisp.

Now I would like to lay out some reasons why one might not choose Lisp for a project. Focusing on Common Lisp for this article.

  1. The very s-expression syntax which enables structural macros which gives lisp some extra expressiveness power vs most other programming languages, also serves to make the code harder to read for the vast majority of people. Perhaps the flexibility of s-expression syntax is overweighed by the increased cognitive overhead required to read the code.  Even after I spent quite a lot of time reading and writing lisp over a period of several years, I would still find it a bit more challenging to read lisp code than to read, for example, java code.
  2. I think in the age of autocomplete and ide-assisted coding, lisp's OOP model and function comes before object mentality is less ergonomic than the dominant object.method paradigm.  Consider `(bark dog)` vs dog.bark() to see what I mean.  The latter syntax is extremely amenable to autocomplete in an IDE and provides some "namespacing" to the bark() method where it won't pollute the global autocomplete context but instead only comes up in the context of the dog object.
  3. Another original benefit of lisp s-expression syntax is the ease with which you can write data using the same syntax as code.  Lists are integral -- with constructs like plists and alists maps and general nested structures are also well supported.  I think other languages have caught up to lisp in this regard for practical purposes -- JSON is arguably better than s-expressions for expressing data, and is well integrated into languages like Javascript and Python.
  4. Lisp structural macros are a two-edged sword.  They can enable some neat tricks but these tricks can completely disrupt the control flow of the code and make things quite hard to understand.  For example the book "Let over Lambda" by Doug Hoyte contains numerous macro tricks that mess with scope, bindings, evaluation, etc.  If you use a lot of these techniques in your code you can greatly obfuscate your code which is not generally considered a benefit.  If you really need a DSL to model a specific problem, you can always create one using non-lisp languages as well, but you will likely need to make use of a parser generator like ANTLR.  For 99.9% of projects I would say the common data, function, and object abstractions are sufficient and Lisp's "structural macro" chops aren't needed or provide only a marginal advantage.
  5. I personally think people sometimes go overboard with the Lisp "form" syntax function chaining programming style, when it would be easier to understand if values were saved to named variables in intermediate steps along the way, as is encouraged by many other programming languages.  Other languages technically support function chaining just the same, but it would be frowned on to have some of the massive effective one-liners that are considered fairly idiomatic in Lisp.
  6. Common Lisp has lots of arcane and somewhat archaic core functions and functionality.. especially around manipulating cons cells and the like.  (car), (cadr), (assoc), (elt), vs (nth), and so on and so forth.
  7. Lisp syntax tricks can only get you so far when solving real-world problems.  When you compare Lisp programs to solve specified programming tasks -- like for instance the tasks given here -- Debian Benchmarks Game (Lisp) -- do you actually see lisp clocking in consistently with less code and more readable code than other less-theoretically "expressive" languages?  To be honest, no you don't.  In the real world, there is a certain amount of domain-driven complexity that will be present regardless of what language you write in, and I think it is rare that choosing lisp for its supposed expressiveness advantages will actually allow you to greatly reduce the complexity of modelling a problem.
  8. Finally, for me the last but perhaps most important factor to evaluate a language/runtime and it's limitations is, to consider what systems have been created in each language, and the characteristics of those systems. I'll write another article to touch on what I observe as the pros and cons of other languages/runtimes from this point of view, but for Lisp I will say that most lisp systems that I have interacted with feel a tad bit unpolished and indeed perhaps unfinished, unless you are a hardcore lisper who wants to interact with the system primarily or entirely in Lisp itself.  Systems like Emacs, LispWorks IDE, and McClim are some that i've used that I've felt this way about. I just don't see the level of polish, performance, and system integration that is commonly seen in C++-based software, for instance.

Common Lisp is a good language and has several good runtimes -- including SBCL.  Lisp had many of the benefits of Java/managed runtimes before Java even existed, including to my knowledge memory safety (in theory at least), a runtime that allows live modifications (to a greater extent than the jvm!  but this can be a footgun in practice), lower memory usage than the JVM (in my experience), similar runtime performance to the JVM, and full support for object oriented programming with the CLOS (if with a bit unusual syntax/mental model).

Coming out of college, Lisp was my favorite language.  I tried to shoehorn it into my work at the first few places I worked at, and used it for personal systems for our family as well. Over time however, it's mostly been displaced from my personal workflow, in part because of network effect of others using other languages but also because I felt I could do the job better in other languages more often. Python has become my choice for most interactive or exploratory programming, Java remains a good choice for building a structured business system, for me C is the language for low-level programming if it is needed. C++ when necessary or called for. 

I still see Lisp as having certain strengths.  It's good for starting exploratory code in an interactive style similar to python that can then be transitioned into a real product with better performance than python. I think Lisp probably has saner packaging than python.  It has a good story around profiling and making real-time changes to the code if necessary.

In conclusion, would I choose Lisp for a project today? 

For me personally I think I would be a bit unlikely to choose Lisp for a new project unless I just had an excess of time and wanted to tinker -- or if I had specific requirements that matched Lisp's strengths as I view them.