American in Spain

Lisp Enlightenment and Emacs Frustration

July 6, 2007

I have recently had my interest piqued about Lisp. I learned a little Lisp back in college for some Artificial Intelligence classes that I took. It struck me as a rather awkward language. Several months ago, I found this article by Paul Graham about how he, in 1995, way before the dot-com boom, started a company, called Viaweb, to make customizable online stores, exactly five years before I, too, started a company to make customizable online stores. The only difference is that he sold his to Yahoo! for millions of dollars. In the article, he claims that the key factor that let his company beat out the competition at every turn was that they wrote their software in Lisp. He quotes Eric Raymond:

Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.

In my investigation, I've found that this moment of enlightenment is mentioned by many other Lisp users (aka. Lispers). As a non-Lisper, I was reminded of the feeling I had when those Magic Eye stereograms became popular. I felt like everyone around me was staring at something that appeared uninteresting and saying, "Whoa! Cooooool!!!" I just didn't get it.

Then, a few weeks ago, I found an article called The Nature of Lisp, by Slava Akhmechet. His stated goal of the article was to look back at his own path to Lisp enlightenment, and then reach out to the unenlightened and describe why Lisp is so cool in terms that non-Lispers understand, namely XML and Ant. Compressing the article into one sentence, the main point that non-Lispers don't get is that Lisp blurs the line between data and code much like Ant is code written in XML, a format used mostly for data, and this lets you manipulate your Lisp code with Lisp itself. See how hard that is to understand? There's a reason Slava's article is so long.

I was just beginning to see the light. I had seen through the keyhole to a beautiful world out there, if I could just pry the door open a bit more... The next step was reading Practical Common Lisp, a book by Peter Seibel, that he has released free on the web. I am the exact audience that the book was written for, a pretty good programmer, well versed in one language, and interested in understanding why Lisp is so great. In his introduction, he states,

If you're a hard-nosed pragmatist who wants to know what advantages Common Lisp has over languages such as Perl, Python, Java, C, or C#, this book should give you some ideas. Or maybe you don't even care about using Lisp--maybe you're already sure Lisp isn't really any better than other languages you know but are annoyed by some Lisper telling you that's because you just don't "get it." If so, this book will give you a straight-to-the-point introduction to Common Lisp. If, after reading this book, you still think Common Lisp is no better than your current favorite languages, you'll be in an excellent position to explain exactly why.

In the past few weeks, I've more or less read Practical Common Lisp from cover to cover. I skipped over some of the stuff at the end about parsing MP3 files. And the final conclusion is that: Lisp is far better than any programming language I've ever learned, including Java, which is more or less my "mother tongue" these days. Even if you tie macros (Lisp's true source of power) behind its back, its object inheritance, error handling, and variable scoping are far more robust and flexible than Java's.

My programming language path went from BASIC to C++ to Java, with slight dabbling in C and Pascal along the way. As such, I thought that concepts like "everything is an object", "every variable is really pointer to an object", garbage collection, and running on a virtual machine were all brand new concepts in Java. And just recently I've been amazed at the power of concepts like closures and first-class functions in Javascript. And now I learn that Lisp was doing all this and more a decade before I was born?? Mind. Blown.

So here I am. I've seen the light, and it's absolutely beautiful. I've pried open the door and am gazing, slack-jawed, at the blissful sunny meadows of efficient, elegant Lisp programming. As I'm preparing to leave behind my dank dark prison cell of Java, vowing never to return, a little voice in the back of my head says what I was trying all along not to think about.

If it's so nice and perfect out there, why is there no one out there? You think you're so smart that, in only two months, you've figured something out that none of the other poor schmucks in the neighboring cells haven't?

"Shut up," I murmured, and sprinted for the open door. And that was when I slammed, face first, into the invisible pane of glass called Emacs.

At work, I spend almost all of my time using an application called IntelliJ IDEA, an integrated development environment (IDE) for Java. IntelliJ makes Java programming as easy as it can possibly be. It more or less compiles your code as you type it, points out most bugs as they occur, intelligently suggests code completion to finish your words for you. I generates all the annoying repetitive code, like getters and setters and equals() methods for you. Without it, Java would be much more laborious to program in (but still not as bad as other languages in the C family tree). Java has many other IDE alternatives, but none truly stack up to IntelliJ.

A couple years ago, I began using a plugin for IntelliJ, called IdeaVIM, that allows me to use Vim key mappings in IntelliJ. Vim is a successor to Vi, which is an ancient UNIX text editor. The main value that it provides is allowing you to move your cursor around a text file without moving your fingers from the home row, thus greatly reducing RSI.

Lisp, on the other hand, has only one IDE. That IDE consists of Emacs, another ancient, extensible text editor, and SLIME. There are some commercial IDEs, but they all use Emacs key mappings and don't improve the interface that much, so what's the point? This 55 minute video (mov) (torrent) demonstration of Emacs and SLIME, by Marco Baringer (one of Lisp's leading developers, it seems), is pretty impressive and demonstrates an efficiency of development comparable to that of IntelliJ. You can even do some things that IntelliJ (and Java) can't do, such as compile code directly into a running remote server.

Vi and Emacs are long time rivals, and the internet is full of bitter arguments about which is superior. The fact is that they both provide ways of moving quickly around a text file and modifying entire blocks of text. And, most importantly, both of them have incredibly steep learning curves.

I just spent all this brain energy comprehending the infinite power of Lisp, and now I'm stuck at the bottom of a learning cliff that prevents me from going any further before learning an awkward set of key combinations to edit a text file. Coming from a Vim background, what I can't stand about Emacs is that you have to hold down the Ctrl or Alt key practically the entire time you're using it. This is enough to make a Vim user scream! Emacs seems like a sure way to RSI. This is a huge stumbling block for me at the moment, and it's particularly frustrating because of my desire to flex my newly formed Lisp muscles.

It seems 100% crystal clear that if you want to be a Lisp developer, you have to use Emacs. This is just a fact that I'm going to have to accept and either give up or learn Emacs. I suspect that this is the major reason that Lisp isn't more popular. There are other problems like how the lack of rigid structure makes managing large teams of programmers difficult, but I bet the main glass wall keeping people in their cells is the difficulty of learning a new and different language and a new and different text editor.

Hopefully it's just a matter of time. I clearly recall a time when I thought, "Vi is so hard to use! Why would anyone want to learn such a difficult editor?" And now I can't imagine life without it. I've mainly written this post as a diary entry, both to organize my thoughts on the matter, and hopefully to have something to look back down on after scaling the Emacs learning cliff.