100 Languages Speedrun: Episode 86: Emacs Lisp

The Editor Wars are long over. TextMate-style editors (Sublime Text, Atom, VSCode) won. Language-specific editors like Jupyter, Android Studio, and such significant use, and somehow even Notepad++ found its niche. Notably irrelevant are both main actors of the "Editor War" - Emacs and Vi. Emacs even more so, there are somehow still enough Vi diehards to keep Vi-style editors (Vim and NeoVim these days) alive. Emacs-style editors lack even that kind of following.

Anyway, what interests me here is not the editors - I've been early adopter of TextMate and never looked back - but the languages they used for their extensions.

Emacs Lisp was much more powerful, and whole programs that ran from within Emacs were written in it. Back then that was seen as strange, but now every editor works like that, so at least in this sense Emacs won. Emacs Lisp was basically a major Lisp dialect, and I guess it still is, as none of the Lisps are terribly popular. It seems that nobody liked it, and back when Emacs was relevant there was constant talk about switching to Common Lisp, Scheme, or anything else. Now it doesn't matter anymore, the whole ecosystem died.

Meanwhile Vimscript was never anywhere as big as Emacs Lisp, and what remains of the the Vim world (with NeoVim) switched to Lua anyway, only keeping Vimscript for backwards compatibility.

Hello, World!

With proper #! we can execute Emacs Lisp scripts from terminal, without ever seeing the editor:

#!/usr/bin/env emacs -Q --script

(princ "Hello, World!\n")

$ ./hello.el
Hello, World!

It's a Lisp of course, so parentheses everywhere. princ is a human-friendly print. A lot of Emacs Lisp functions have such cryptic names.

FizzBuzz

#!/usr/bin/env emacs -Q --script

; FizzBuzz in Emacs Lisp

(defun divisible (n m)
  (= 0 (% n m)))

(defun fizzbuzz (n)
  (cond
    ((divisible n 15) "FizzBuzz")
    ((divisible n 5) "Buzz")
    ((divisible n 3) "Fizz")
    (t (number-to-string n))))

(dotimes (i 100)
  (princ (fizzbuzz (+ i 1)))
  (princ "\n"))

The code isn't too bad, but we already run into some minor issues:

• (defun ...) defines a function
• (dotimes (i n)) only does iteration from 0 to n-1, there's no builtin a to b iteration
• (princ) only takes one argument, doesn't print newline, and there's no println equivalent function that would just work - I'm actually baffled why they won't let princ take multiple arguments, it's such an obvious thing to do, and most languages support it just fine
• (cond ...) is like if/elsif chain
• t means true
• nil means false, also empty list

Macros

OK, let's extend Emacs Lisp to be nicer. We'll add (dorange (i a b) ...) and many-arguments (prints ...).

#!/usr/bin/env emacs -Q --script

(defun prints (&rest args)
  (if (consp args)
    (progn
      (princ (car args))
      (apply 'prints (cdr args)))))

(defun fib (n)
  (if (<= n 2)
    1
    (+ (fib (- n 1)) (fib (- n 2)))))

(defmacro dorange (i a b &rest body)
  `(let ((,i ,a))
     (while (<= ,i ,b)
       ,@body
       (setq ,i (+ ,i 1)))))

(dorange n 1 30
  (prints "fib(" n ")=" (fib n) "\n"))

$ ./fib.el
fib(1)=1
fib(2)=1
fib(3)=2
fib(4)=3
fib(5)=5
fib(6)=8
fib(7)=13
fib(8)=21
fib(9)=34
fib(10)=55
fib(11)=89
fib(12)=144
fib(13)=233
fib(14)=377
fib(15)=610
fib(16)=987
fib(17)=1597
fib(18)=2584
fib(19)=4181
fib(20)=6765
fib(21)=10946
fib(22)=17711
fib(23)=28657
fib(24)=46368
fib(25)=75025
fib(26)=121393
fib(27)=196418
fib(28)=317811
fib(29)=514229
fib(30)=832040

Step by step:

• the naming is really awful, progn, car, cdr, setq etc. I know these are traditional Lisp names, they're all ass.
• consp means "is nonempty list"
• car means "first element of the list"
• cdr means "rest of the list"
• setq means "set variable"
• &rest means remaining arguments of a function
• why do we need to do this silliness like (apply 'prints (cdr args)) instead of (prints &rest (cdr args)) or (prints . (cdr args))?

Functional Programming

OK, let's try some super basic functional programming.

#!/usr/bin/env emacs -Q --script

(setq list '(1 2 3 4 5))

(setq add2 (lambda (n) (+ n 2)))
(print (mapcar add2 list))

(defun addn (n) (lambda (m) (+ n m)))
(setq add3 (addn 3))
(print (mapcar add3 list))

We create add2 as a lambda that adds 2 to its argument. Then we create add3 that adds 3. Surely that would work right?

$ ./functional.el

(3 4 5 6 7)
Symbol’s value as variable is void: n

Well add2 worked, but add3 didn't, wat? Well here we run into one of the major issues with Emacs Lisp - it does not use lexical scoping. For some insane reason EmacsLisp uses dynamic scoping for everything. This pretty much kills any idea of using functional programming.

Optional Don't Be Broken Mode

Weirdly at some point after everyone stopped using Emacs, Emacs Lisp added optional "don't be broken" mode, where you can request lexical scoping:

#!/usr/bin/env emacs -Q --script
;; -*- lexical-binding: t -*-

(setq list '(1 2 3 4 5))

(setq add2 (lambda (n) (+ n 2)))
(print (mapcar add2 list))

(defun addn (n) (lambda (m) (+ n m)))
(setq add3 (addn 3))
(print (mapcar add3 list))

$ ./functional2.el

(3 4 5 6 7)

(4 5 6 7 8)

Also what's up with those extra newlines with (print ...)? prin1 and princ don't print any newlines, while print prints one before and one after, WTF?

Step by step:

• (setq lest '(1 2 3 4 5)) - we need that quotation mark to distinguish list from a function call, without it Emacs Lisp would try to call function named 1
• (lambda (n) ...) is anonymous function taking argument n
• (mapcar f list) is map, another case of awful naming

Unicode

At least Unicode works. It would be an embarrassment if editor-specific language didn't support Unicode.

#!/usr/bin/env emacs -Q --script

(defun prints (&rest args)
  (if (consp args)
    (progn
      (princ (car args))
      (princ "\n")
      (apply 'prints (cdr args)))))

(prints
  (length "Hello")
  (length "Żółw")
  (length "💰")
  (downcase "Żółw")
  (upcase "Żółw"))

$ ./unicode.el
5
4
1
żółw
ŻÓŁW

Wordle

All right, let's do something slightly more complicated - a Wordle game.

#!/usr/bin/env emacs -Q --script

(defun read-file (path)
  (with-temp-buffer
    (insert-file-contents path)
    (buffer-string)))
(defun read-lines (path)
  (split-string (read-file path) "\n" t))
(defun random-element (list)
  (nth (random (length list)) list))

(defun report-wordle-blocks (guess word)
  (dotimes (i 5)
    (let ((gi (substring guess i (+ i 1)))
          (wi (substring word i (+ i 1))))
      (princ
        (cond
          ((equal gi wi) "🟩")
          ((string-match-p (regexp-quote gi) word) "🟨")
          (t "🟥")))))
  (princ "\n"))

(defun report-wordle (guess word)
  (if (/= (length guess) 5)
    (princ "Please enter a 5 letter word.\n")
    (report-wordle-blocks guess word)))

(setq word-list (read-lines "wordle-answers-alphabetical.txt"))
(setq word (random-element word-list))
(setq guess "")

(while (not (equal guess word))
  (setq guess (read-from-minibuffer "Guess: "))
  (report-wordle guess word))

And here's my first try, not amazing:

$ ./wordle.el
Guess: raise
🟥🟩🟥🟥🟩
Guess: maybe
🟥🟩🟥🟥🟩
Guess: dance
🟥🟩🟥🟥🟩
Guess: vague
🟥🟩🟥🟨🟩
Guess: haute
🟩🟩🟩🟩🟩

Step by step:

• Emacs Lisp lacks a lot of obvious functions like "read a file", "random element", or "string contains"
• to read a file we need to create "temporary buffer", insert file contents into that buffer, then read the buffer contents
• to readlines, we need to do that, and then split-string by "\n" - that extra t means to ignore empty strings (like the one at the end after final newline) - the whole thing is not quite right, but close enough
• random-element returns random element from a list
• report-wordle-blocks prints colored blocks for Wordle matches
• (string-match-p (regexp-quote gi) word) looks like the easiest way to check if a string contains another, which is baffling missing feature for a text editor
• overall so many small things about this code feel just a bit wrong

Should you use Emacs Lisp?

Obviously not. Emacs was a pioneer of making editors a application platform, and Emacs Lisp was good enough for that role, but both Emacs and Emacs Lisp are really obsolete. Maybe Emacs would have had a fighting chance with a better language, and less GUI-phobia, but history is what it is.

As for the language itself, Emacs Lisp the language is full of weird archaic quirks, misses so many basic features, and modern Lisps do it a bit better. Arguably none of the Lisps is all that great, but if you want to give Lisp a try, Racket and Clojure are much more reasonable.

And if you want to code some editor plugins, VSCode is all JavaScript, so you'll have to learn that.

Code

All code examples for the series will be in this repository.

Code for the Emacs Lisp episode is available here.

Comments

[Tomasz Wegrzanowski]

VSCode doesn't have builtin keyboard macros, but there are packages for it.

I suspect most things you'd do with keyboard macros are better done with multiple cursors.

[Beni Cherniavsky-Paskin]

to read a file we need to create "temporary buffer", insert file contents into that buffer, then read the buffer contents

Not so strange in a text editor. I know you're deliberately sticking to standalone CLI scripts in this series to avoid comparing languages' own environments, but for Elisp that really stretches it out of its niche of scripting Emacs itself :-)

Many weird choices Elisp made are actually very interesting as an exercise in "Language-Oriented Programming". A central goal was to maximize dual-use functions that one may invoke as editor commands OR call programmatically.

Consider string/regexp search function. In "normal" languages, it may take the string to search in, the pattern, some flags (e.g. case_sensitive) -> and return an index.
In Elisp, it operates on current buffer, it has side effect of moving the cursor(!), and case sensitivity is controlled by a config variable. Which is sensible for interactive search command, but how is a good API style?!
Well, that's gonna be a repeating pain with most dual-use functions, right?

Emacs mitigates that by adding constructs to combine such functions:

• what if you wanted to search within other string?
  => Several constructs like with-temp-buffer — compare to Unix standardizing on stdin/out and providing redirection facility in shell!
  (So see, a read-file function returning a string is not that useful. For any serious manipulation you'd upgrade it to a buffer anyway.)

• what if you didn't want side effects?
  => Several constructs like save-excursions restore previous state. (originally no Unix analogue, nowdays that's a bit like running in a container and deciding later whether to commit the modified FS.)

• Global case-fold-search config var is obviously not enough :-(
  => Unique scoping concept: buffer-local variables! E.g. set it sensitive in a Python buffer, but ignore case in english prose buffer.
  => Dynamic scoping?! It's weird and mathematically broken (and historically comes from old lisps) but it does make any function that references a "global" variable locally configurable: (let ((case-fold-search t)) (search-forward ...))

It's not all roses. In particular tons of elisp code doing searches and cursor movements for ad-hoc "parsing" of nearby context stinks... A code editor really should have decent DSLs for actual parsing, which for a looong time emacs really lacked :-(

So yes, pretty bad as general-purpose language, but an interesting point in language design.

[Beni Cherniavsky-Paskin]

It gets more interesting as a system. "GUI-phobia" accusation is true but it did provide, at its time, a very advanced TUI prototyping environment. It had a DOM, with redisplay taken care of for you, very flexible bindings from events (key sequences, mouse) to handler functions, and embedded interpreter for writing these handlers, and tight devel loop by modifying the system live as you're using it.
All this in many ways resembles modern browsers with DOM->JS handlers!

(Of course, it can't compete with modern browsers on layout/styling/graphics. Nor on flashy devtools. => You're right that nowdays VSCode is better choice for reusing web skills, and that on number of devs alone, elisp has no chance to compete in long run.)

Emacs had more on-ramps from "end-user" to programmer than modern browsers — it never wanted to split between page author / user. E.g. you can start from simple macro recording, later convert a macro to Elisp code, later edit it to add some conditionals/loops. You can override/"advise" almost any builtin function.
In comparison, even VSCode has much more of boundary between core and "extensions" :-( Though it has not-bad reasons to do that! It can provide more of "app-store" experience installing extensions. Running in external process with LSP protocol finally solved E*L problem to E+L (and both Emacs and Vi are now benefitting from LSP)

[vincentjgoh]

why do we need to do this silliness like (apply 'prints (cdr args)) instead of (prints &rest (cdr args)) or (prints . (cdr args))

You know why. You wrote this in a way where you want to pass in individual arguments, but then use cdr which returns a list, which is not the same thing? Like, you've very clearly made a programming decision here that would intentionally break. You clearly know how to use the map functions, so why wouldn't you use those instead, instead of a recursive solution? And map is a perfectly fine name for a class of functions; they map a function onto a list.

I'm not gonna argue that elisp is somehow the most accessible or best language to do things in--it carries a tonne of historical baggage with it like lots of other languages do--but at the very least, you can't misuse it and then say that's the fault of the language. Every language has its context. This is one that's inside an editor and is meant mainly to augment the functionality of the editor; other things are possible (I mean, that's a pretty decent wordle clone in very little space) but it's not really fair to compare its general purpose utility to other languages when it's not a general purpose language at all. (The extended functionality of emacs is definitely due to the skill of the plugin writers. Sometimes I'm amazed at something like emacs-hydra. Wrangling macros like that isn't easy.)

That said, this was fun. I'm looking forward to reading through the other languages. I hope you're at least this critical of C++. :D

[Tomasz Wegrzanowski]

map is a perfectly fine name, as would be maplist or listmap or so. mapcar is really weird.

The series is close to 100 episodes, and I have very few free slots left, so I don't think C++ will make it.

[vincentjgoh]

mapcar actually makes the most sense to me—it tells you how it's going to apply the function. For once, the documentation on the official manual page is very useful: if you use mapcar with cons and 2 lists, it cons the cars of each list together. In the context of a lisp, it's actually very clear. (mapconcat too, actually. It does what you'd expect.)

I find mapcan and mapc much less obvious names. mapc works just like mapcar but only for the side effects? Useful, but why is that what it means? mapcan? Why is it called that? It returns a single list of all the results which themselves must have been lists, so it's flattening the answer down. Again, useful, but that name is random as far as I'm concerned.

[Chris Ochsenreither]

I probably would have been into Emacs Lisp if I was programming when it was new-ish. Wikipedia says it was released in 1985. Back then, you were most likely siloed on your own computer and making various types of programs and games within emacs would have been fun (maybe with your friend who was actually in the room). I took a course in Racket but I find languages like F# much easier to read, if I'm looking for something functional.

[SREagle]

The article begins with "The Editor Wars are long over." - but this article also got by far the most comments, referencing each other.

This goes to show that there seems nothing more divisive than the choice of the most basic tool of a computer user: the editor.

Personal choice is king, and some tolerance for the choice of others goes a long way.
But man, this is fun to follow... 🍿

BTW: I like the 'hot take' approach of the articles, even while disagreeing on most languages I happen to know better. "No fanboying" indeed! 👍

[Tomasz Wegrzanowski]

Yeah, the posts with most views and reactions were Python (as E01), Crystal, Lua, 3 Raku posts, and Emacs Lisp.

Stack Overflow Survey shows that 71% of people use VSCode, and none on the top ones do any fighting. Like I've never seen any VSCode vs Notepad++, or PyCharm vs Jupyter posts.

[Andrey Orst]

I can highly recommend trying out Kakoune editor. It is built around multiple selections, and it's the best implementation of the feature I've seen so far. Kakoune combines macros, regex, and multiple selections into a very powerful editing language inspired by Vim

[Tomasz Wegrzanowski]

You can do next matching regex select for multicursor too.

I'm not really sure what's the use case for macros. Multicursor is a crazy powerful feature, so I really don't want any editor without it (unless I'm over ssh and all they have is nano or mcedit). Emacs doesn't have it, and I don't know how good third party packages that add it are.

If something is too complicated for multicursor, then you should probably write a proper extension with some interface for it.

I'm not really sure what kind of tasks would fall in the middle ground - too complex for multicursor, but not worth writing proper extension command for.

[Tomasz Wegrzanowski]

Emacs ecosystem is fairly small compared with VSCode ecosystem. Is there Github Copilot for Emacs? Jupyter for Emacs? Syntax highlighting for 90% of the languages I covered for Emacs?

If ecosystem is what you're after, you're using a wrong editor.

If you're already an Emacs user, with tons of customizations you've created for yourself, you might just as well keep using it, but I'd definitely not recommend it for anyone new.

[Andrey Orst]

Emacs ecosystem is fairly small compared with VSCode ecosystem.

That's a bold statement, that needs to be backed up at least by something. VSCode is fairly popular right now, so its package base grows rapidly, but new packages pop daily for Emacs as well. Emacs has packages for a lot of things VSCode probably would not even go for, like an NES emulator, or a tool to order salads from a specific restaurant.

Is there Github Copilot for Emacs?

Not yet, as Copilot is in closed beta, but there are already 3 other packages that implement what Copilot does, except those are FOSS.

Jupyter for Emacs?

Yes, yes. Actually, Org Mode can do everything Jupiter can, and with much more languages than Jupiter.

Syntax highlighting for 90% of the languages I covered for Emacs?

Oh boy. Because I have nothing more important to do this sunday morning, I've went and checked almost all languages you've written about:

• ✅ Python
• ✅ Emojicode
• ✅ CSS
• ✅ Lua
• ✅ Kotlin
• ✅ Tcl/Tk
• ✅ Sed and Regular Expression FizzBuzz
• ✅ Ada
• ✅ Befunge
• ✅ Tcsh
• ✅ D
• ✅ Arc
• ✅ Awk
• ✅ Octave
• ✅ Rake
• ✅ PLY Python Lex-Yacc
• ✅ Julia
• ✅ Forth
• ✅ Clojure
• ✅ XSLT
• ✅ Ruby
• ✅ Postscript
• ✅ JQ
• ✅ Raku (Perl 6)
• ❌ Whenever
• ✅ TeX
• ✅ Verilog
• ✅ SageMath
• ✅ Fortran
• ✅ Gherkin
• ✅ Logo
• ✅ Racket Scheme
• ✅ Groovy
• ✅ AppleScript
• ✅ OCaml
• ✅ BC
• ✅ SQLite
• ✅ Assembly
• ✅ Prolog
• ❌ Thue
• ✅ M4
• ✅ Elvish
• ✅ Crystal
• ✅ COBOL
• ✅ R
• ✅ Perl
• ✅ QBasic
• ✅ Haskell
• ✅ PHP
• ✅ Scala
• ✅ XQuery
• ✅ Smalltalk
• ❌ Asciidots
• ✅ POV-Ray
• ✅ LLVM Intermediate Representation
• ✅ CSVQ
• ✅ ChucK
• ✅ Xonsh
• ❌ Io
• ❌ Ioke
• ✅ Wren
• ❌ Factor
• ❌ Windows Batch Files
• ✅ Free Pascal
• ❌ Jasmin
• ❌ Designing New Esoteric Language Tuples
• ✅ Elixir
• ❌ Pyret
• ✅ PowerShell
• ❌ Lingua Romana Perligata
• ❌ Linguagem Potigol
• ✅ Emacs Lisp
• ❌ Sidef

So the stats are - 73 languages total, 60 covered, 13 uncovered, which is 82%.
Not quite 90%, but I bet still much higher than you've expected.
I didn't search too deep, mostly included everything that shows in top google results or directly in the package manager.

If you're already an Emacs user, with tons of customizations you've created for yourself, you might just as well keep using it, but I'd definitely not recommend it for anyone new.

Yes. Emacs is a journey you need to come to by yourself. Simply because Emacs is a tool that you can truly make your own thing.

[Tomasz Wegrzanowski]

Yeah, it's higher than I expected, but VSCode does all those plus 6 more (Io, Factor, Pyret, Linguagem Potigol, Asciidots, Windows Batch Files), getting it to 90%.

There's no sweetgreen, but there's NES emulators.

By simple count there are 33829 extensions in VSCode extension manager, 5241 on Sublime Text package manager, 5097 on Emacs MELPA.

This gap is only going to get widen, as there's a lot more people who can write JavaScript than Emacs Lisp. It's also quite easy to port stuff from the web to VSCode.

[Andrey Orst]

By simple count there are 33829 extensions in VSCode extension manager, 5241 on Sublime Text package manager, 5097 on Emacs MELPA.

I believe in quality, not quantity.

For instance, VSCode had three plugins for working with files over ftp protocol, and none of those worked as advertised, constantly getting de-synchronized and not being able to pull files. Emacs has one package that does it, and many more things for working with various remote file editing protocols, but the most important thing is that it works.

Emacs is already 36 years old, let's see how VSCode will do in 36 years :)

[Fadhil Radhian]

Amazing journey right there!

[bbrtj]

VS code will either be abandoned or set aside by the new hotness in let's say 5 years. Many editors come and go yet Emacs and vi stick around. It makes much more sense to spend time developing a plugin in elisp or vimscript if you know you will basically be able use it indefinitely.