Nixpkgs: A community-maintained package lock file for all of Unix

Disclaimer
I wrote this article a few years ago but never published it. A few things may be a little outdated (especially the lack of info on Nix Flakes), but it's still a nice little intro to Nixpkgs.

Meet Nix

Nix is the name of the programming language that the NixPkgs project is written in. NixPkgs can be thought of in a lot of ways; One of those ways is as a lazily evaluated, community-maintained, programmable package lock file for all of Unix. Let me explain:

A package lock file for all of Unix

Many programming languages come with their own package managers. For example, Node has npm, Python has pip, etc. When you write code that depends on some packages, you want to make sure that if someone else runs that code, they have the same packages at the same versions. This is where package lock files come in. They specify exactly which versions of which packages are needed to run some piece of code. For Node, that takes the form of package-lock.json, and for Python it's requirements.txt.

The NixPkgs repository is kind of like one of these package lock files, but huge: It specifies exactly which versions of which packages you need, for everything. Seriously; It's an effort to put pretty much everything in a single package lock file - kind of like the everything module for Node. But not just for Node, but for any software you'll find for any Unix system.

"But why would I ever install from this lock file?", you might ask, because you don't ever need "all the things". Well, as opposed to most lockfile formats, Nix lockfiles let you evaluate only part of it, making it so you only install one, or a few, of the packages it locks. It achieves this through lazy evaluation.

Lazily-evaluated

Some programming languages are evaluated eagerly (for example JavaScript), and some lazily (for example Haskell). Nix is of the latter sort. When I think about this difference, I usually think of it as the "order" in which the evaluator goes through the language. Take this JavaScript code, for example:

const one = 1;
const two = 2;

const added = one + two;
const multiplied = one * two;

const answer = multiplied + one;

Here, the evaluation of the statements goes from top to bottom. First, the values of one and two are evaluated, then added and multiplied, and finally answer. If I end up only using the value of answer, then added will still have been evaluated even though it was not used to compute answer*.

A lazily evaluated programming language kind of evaluates in backwards order. For example, here's some similar code in Nix:

rec {
  one = 1;
  two = 2;

  added = one + two;
  multiplied = one * two;

  answer = multiplied + one;
}

Here, if I ask for the value of answer, then the language will see that multiplied and one are needed, and when evaluating multiplied, it will see that one and two are needed, finally evaluating one and two, and completely skipping over added. The downside is that one may have been evaluated twice, because it was used in two distinct places*.

If we bring that back to our massive package lock file, we can see how this lazy evaluation strategy is helpful: Let's say we're just interested in the evaluation of a cowsay property, because we want a cow to tell us who we are:

$ cowsay $USER
 ______ 
< avaq >
 ------ 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Because of lazy evaluation, the thousands upon thousands of other properties in the giant lockfile don't need to be evaluated. Nix will start with the cowsay property, and traverse its dependency tree as needed.

* Let's pretend that neither programming language has any kind of optimizer in place, for the sake of illustration.

Programmable

Once you start maintaining a lockfile of this magnitude it becomes a little unwieldy. Luckily, Nix is a programming language, not just a data format. So it allows us to define functions, and create abstractions. These can then be used for things like customization of the lockfile, deriving development environments, deriving new lockfiles from existing ones, deriving entire operating systems from the lockfile, and much more.

Community Maintained

Instead of letting this package lock file be the result of a process that generates it from a smaller specification, the NixPkgs "lockfile" is manually created, maintained, and peer reviewed by a community of enthusiasts who each use their own slice of this lockfile to manage their package installations. Every change to it comes in the form of a pull-request, and every pull request is reviewed and exposed to a test suite.

As a result, you have more certainty (compared to a traditional lockfile which is updated automatically) that any given version of the lockfile will result in a valid installation of the software you want.

Using Nix

The NixPkgs project already defines some 80,000 Linux packages that can be installed through it. Nix can be easily installed using the Determinate Nix Installer on Linux and Mac. To learn more about Nix, check out the Learning Resources on nixos.com.