I use direnv to configure the environment in most projects I'm working on. In my day job a few projects use direnv to load Nix environments for developers on macOS and Linux, and I did some of the work to make that a good experience. It's in doing that that I've learned what can help, and what might also help you.
Be quick or be dead
There's one rule that should never, ever be broken, and it's almost all I'm going to talk about in this post, and it is: do not block.
By this I mean that .envrc
1 should run in a few hundred milliseconds, no more. 500ms is pushing it. That's not long. There's no way to instantiate a Nix shell2 in that time, for example.
So don't. Do not call nix-shell
in .envrc
.
The same rule holds if you're using NPM – maybe you're tempted to put npm install
into .envrc
for example – or another package management or build tool. Think twice before using curl
or wget
. The rule always applies: don't do anything in .envrc
that's not going to return quickly.
Why blocking is a problem
$ cd some_directory
direnv: loading .envrc
direnv: ([direnv export bash]) is taking a while to execute. Use CTRL-C to give up.
... a long time passes ...
Your interactive shell is blocked. You wanted to pop in to take a look at README.txt
but now you have to wait.
Bright idea #1: open another terminal and cd some_directory
Whereupon direnv starts a second long-running process. Facepalm: should've guessed that.
Bright idea #2: hit ctrl-c to give up
The background process was killed. Good. You type git pull
to make sure you're up to date.
You have just entered a command: git pull
. If direnv is configured correctly its hook will be run just before your shell displays the next prompt. direnv will find the .envrc
in the directory and run it, starting another long-running process.
You have been eaten by a grue.
A thing to note: hitting ctrl-c doesn't always work3. The long-running process sometimes keeps running in the background, writing to the terminal, using RAM, CPU cores, and your battery life. At this point you might use pkill
on the background process.
You have just entered a command: pkill some_thing
. If direnv is configured correctly ...
Long story short: you have been eaten by a grue.
Bright idea #3: look at README.txt
from your editor
You open README.txt
and your editor locks up. You wonder why your editor isn't responding. You force kill it and try again. It locks up. You reboot and try again. It locks ...
Facepalm #2: you installed the direnv plugin to your editor. Your editor has triggered direnv into starting yet another long-running process.
Bright idea #4: get angry
This actually works, but is ultimately unfulfilling and doesn't solve the underlying problem.
Anyway, rm some_directory/.envrc
or direnv deny some_directory
and you can freely come and go, but you're not getting the benefit of that direnv integration you signed up for. It might be all that you can do when working with someone else's code though.
How to do it instead
Have a separate process to build your environment. With Nix, for example, you could write a build script like:
#!/usr/bin/env bash
nix-shell --run 'direnv dump > .envrc.cache'
Your .envrc
could be as simple as:
source <(direnv apply_dump .envrc.cache)
This change, as basic as it looks, gives you back control:
- You can
cd
into any directory without worry that you'll trigger a build. - You can open any file in your editor without worrying that it'll freeze.
- You are unlikely to run the build script multiple times concurrently (and ctrl-c will probably work if you do).
- You choose when to build, e.g. when you're not running on battery power.
There is more you can do with this. For example, the build script above will show an error if you haven't created .envrc.cache
yet. You could make it instead prompt4 the user to run the build script.
A couple of other things
If you use ssh-agent
then the cache will contain a value for SSH_AUTH_SOCK
. This isn't sensitive but it will get out of date if you reboot, say. The symptom is that ssh
and commands that use ssh
, like git
, will always prompt you for your password.
The cache will also contain values for DIRENV_DIFF
, DIRENV_DIR
, and DIRENV_WATCHES
. These are direnv's bookkeeping records. Applying the dump without filtering these variables out can cause weird behaviour.
You can update your build script to address both of these problems like so:
#!/usr/bin/env bash
nix-shell --run \
'unset ${!SSH_@} ${!DIRENV_@} && direnv dump > .envrc.cache'
At NoRedInk we go further and cache only the difference in the environment so that loading the cache doesn't clobber environment variables unrelated to the project. We also detect if the cache is stale and prompt the user to recreate it.
There's more to talk about
Like: forking background processes from .envrc
(be prepared to be eaten by a grue many times before you get it right), tools like lorri, and perhaps some more about NoRedInk's tooling, but there's enough in this post already. I hope it's useful. Please like and subscribe :-)
-
direnv looks for a
.envrc
file in a directory youcd
into. If it finds one it interprets it as a Bash script, then copies all of the exported environment variables into your shell's environment (which doesn't have to be Bash). It does some bookkeeping too so that it knows how to restore the previous environment when youcd
out of that directory. ↩ -
The
nix-shell
command is how you start a Nix shell environment with the packages and configuration you've asked for in ashell.nix
file. How that works is way out of scope for this post. Suffice it to say thatnix-shell
almost never returns in less than ~5 seconds, and can take hours. ↩ -
Maybe this is a bug in direnv, something to do with process groups or some such. Thing is, it's a problem that exists and we have to account for it. ↩
-
Note that I wrote "prompt", not "run the build script" or "ask the user if they want to run the build script". Either of those would block. ↩
Top comments (6)
A lot of time has passed since this article was written, so here are a few updates:
and
I'm using
envrc.el
(instead ofdirenv.el
) for a few weeks now and it's a smoother, snappier experience overall, so I highly recommendenvrc.el
!Also see
melpa.org/#/direnv
direnv integration with emacs.
Useful when you want to binaries from nix tooling for code completion formatting etc.
#cat ./.envrc
DIRENV_LOG_FORMAT="" source <(direnv apply_dump .envrc.cache)
This silences the environment diff that would otherwise get printed after every shell command.
Holy shit, super useful -- finding easy to understand documentation on nix workflows seems impossible.
I never managed. How do you do it?