Livebook for Elixir: Just What the Docs Ordered

While initially conceived as a tool for data exploration (much like Jupyter for Python), Livebook has deservedly become a sensation in the Elixir community.

It has been fantastic to see all the wonderful ways teams are leveraging Livebook for a range of different use cases. We have seen Livebooks being used to:

• Create interactive documentation for libraries.
• Build onboarding material and guides.
• Audit and explore potential dependencies in your app.

Livebooks have also been used as the default REPL interface for project development.

In this post, we'll show how you can easily create interactive documentation with Livebook and outline some top tips for using Livebook. We will assume you have installed Livebook, following the guidance in their README.

But First: What is Livebook for Elixir?

Livebooks are supercharged markdown files where you can add sections of arbitrary executable Elixir code. They are inspired by similar notebooks for other languages (like Python's Jupyter), but Livebooks leverage LiveView and other BEAM goodies, so they are even better.

Livebook files get their own .livemd extension, and (somewhat confusingly) we create and run those Livebook markdown files using a Phoenix application also called Livebook.

That phoenix app runs in the browser and enables a whole host of interactive features like collaboration, as we will see.

The expectation is that you will install the Livebook repo locally and start a Livebook server from somewhere on your machine where there are Livebooks (the files).

The Livebook app will show you the working directory of where you started the Livebook server, so you can select any given Livebook to run from there.

Let's now look at a library to document.

Livebook Docs: A Single Source of Truth in Elixir

Our library is going to accept various inputs and rate them according to the following chonk chart:

It will output the chonk rating accordingly. First, let's create the library.

mix new chonk_o_meter && cd chonk_o_meter

Let's add ex_doc to our deps in our mix.exs:

defp deps do
  [
    {:ex_doc, ">= 0.0.0", runtime: false, only: [:docs, :dev]},
  ]
end

Now, download the chonk chart image above and put it into the root of the library in a directory called images so we can refer to it in our README. In the README.md, let's put a title and a short explanation of what the library aims to do:

# Chonk 'O' Meter

Chonk O Meter is a state-of-the-art size estimator. It will rate the size of anything according to the following chart:

![alt chart showing cats of various sizes](./images/chonk.jpg)

So far, so good. Now we will open the module and write our moduledoc. But here's the thing: what we really want to do is just copy what we already wrote for the README.

Having one source of truth for that information is super valuable as the library develops because having to update the documentation in multiple places is a recipe for errors!

It's good to repeat the same information in different formats because different people will come to the library via different paths.

Some may see the repo first (and therefore the README), whereas others may see the library on Hex first — and so only see the moduledoc.

You may think that it's easy enough to copy and paste, but once we add our Livebook into the mix, there will be three places we need to update docs when something changes!

Instead, let's be a bit smarter. We will section off a part of the README and read that section into the moduledoc at compile time. Sandwich the introduction to the library between two markdown comments in the README:

<!-- README START -->

.... Library introduction here.

<!-- README END -->

Now, we can write the introduction as if it were a moduledoc in between those two comments, like so:

<!-- README START -->

Chonk O Meter is a state-of-the-art size estimator. It will rate the size of anything according to the following chart:

![alt chart showing cats of various sizes](./images/chonk.jpg)

<!-- README END -->

In the main module ChonkOMeter, we can do this:

defmodule ChonkOMeter do
  @moduledoc File.read!(Path.expand("./README.md"))
             |> String.split("<!-- README START -->")
             |> Enum.at(1)
             |> String.split("<!-- README END -->")
             |> List.first()
end

This will extract the guide sandwiched between the markdown comments and set it as the moduledoc. Now we only need to change the README to update both!

We can generate the documentation and view it locally to verify that this works as expected. If you run mix docs, a doc folder will appear with an index.html file. We can open doc/index.html to view our documentation in the browser.

Adding an Image to the Moduledoc

If you navigate to the module's documentation, you will notice that the image is missing. Hex allows you to point to assets in your docs as long as they are included inside the doc directory (generated when you create docs with the mix docs command).

Usually, that command overwrites the whole doc folder. So, to ensure that our pictures are always copied there, we can use an alias in our mix.exs file. We will turn the mix docs command into one that runs mix docs and then copies all images inside the /images directory into a doc/images directory.

defmodule ChonkOMeter.MixProject do
  use Mix.Project

  def project do
    [
      ...
      aliases: aliases(),
      ...
    ]
  end

  defp aliases() do
    [docs: ["docs", &copy_pictures/1]]
  end

  defp copy_pictures(_) do
    File.cp_r(Path.expand("./images/"), Path.expand("./doc/images/"))
  end
end

If you run mix docs now, you will see that the images/ directory gets copied over to the doc folder. Open the doc/index.html file and you should now see the chonk chart appear!

Doctests in Elixir's Livebook

It's important to note that in writing our moduledoc in this way, we don't lose any of the usual capabilities ex_docs give us.

Anything you can normally do in a doctest, you can still do here. To demonstrate that, let's add a doctest to our README. First, we'll need a function to test:

defmodule ChonkOMeter do
  @moduledoc File.read!(Path.expand("./README.md"))
             |> String.split("<!-- README START -->")
             |> Enum.at(1)
             |> String.split("<!-- README END -->")

  @doc """
  Returns the Chonk rating for a given number of story points.
  """
  def story_points(points) when is_integer(points) and points >= 0 and points < 3 do
    "A Fine Boi"
  end

  def story_points(points) when is_integer(points) and points >= 3 and points < 5 do
    "He Chomnk"
  end

  def story_points(points) when is_integer(points) and points >= 5 and points < 8 do
    "A Heckin' Chonker"
  end

  def story_points(points) when is_integer(points) and points >= 8 and points < 10 do
    "H E F T Y C H O N K"
  end

  def story_points(points) when is_integer(points) and points >= 10 and points < 15 do
    "Mega Chonk"
  end

  def story_points(points) when is_integer(points) and points >= 15 do
    "Oh Lawd He Comin'"
  end
end

Now remove the boilerplate in our test file, so it looks like this:

defmodule ChonkOMeterTest do
  use ExUnit.Case
  doctest ChonkOMeter
end

Run the tests to ensure there are none for now:

mix test

Finally, in our README, we can add the usual doctest syntax:

<!-- README START -->

Chonk O Meter is a state-of-the-art size estimator. It will rate the size of anything according to the following chart:

![alt chart showing cats of various sizes](./images/chonk.jpg)

For example:

    iex> ChonkOMeter.story_points(10)
    "Mega Chonk"

<!-- README END -->

If you run the tests, you will notice that the change in the README has not triggered a re-compilation, meaning the app still thinks there are no doctests. To fix this, we just need to add an @external_resource module attribute into the main module. This tells mix to recompile when the README changes:

defmodule ChonkOMeter do
  @external_resource Path.expand("./README.md")
  # ^^ Add this line ^^
  @moduledoc File.read!(Path.expand("./README.md"))
             |> String.split("<!-- README START -->")
             |> Enum.at(1)
             |> String.split("<!-- README END -->")

  @doc """
  Returns the Chonk rating for a given number of story points.
  """
  def story_points(points) when is_integer(points) and points >= 0 and points < 3 do
    "A Fine Boi"
  end

  def story_points(points) when is_integer(points) and points >= 3 and points < 5 do
    "He Chomnk"
  end

  def story_points(points) when is_integer(points) and points >= 5 and points < 8 do
    "A Heckin' Chonker"
  end

  def story_points(points) when is_integer(points) and points >= 8 and points < 10 do
    "H E F T Y C H O N K"
  end

  def story_points(points) when is_integer(points) and points >= 10 and points < 15 do
    "Mega Chonk"
  end

  def story_points(points) when is_integer(points) and points >= 15 do
    "Oh Lawd He Comin'"
  end
end

When we run our tests, this now results in one passing doctest! We can also add a doctest to our function doc like so:

...
  @doc """
  Returns the Chonk rating for a given number of story points.

      iex> ChonkOMeter.story_points(5)
      "A Heckin' Chonker"
  """
  def story_points(points) when is_integer(points) and points >= 0 and points < 3 do
    "A Fine Boi"
  end

  def story_points(points) when is_integer(points) and points >= 3 and points < 5 do
    "He Chomnk"
  end

  def story_points(points) when is_integer(points) and points >= 5 and points < 8 do
    "A Heckin' Chonker"
  end

  def story_points(points) when is_integer(points) and points >= 8 and points < 10 do
    "H E F T Y C H O N K"
  end

  def story_points(points) when is_integer(points) and points >= 10 and points < 15 do
    "Mega Chonk"
  end

  def story_points(points) when is_integer(points) and points >= 15 do
    "Oh Lawd He Comin'"
  end
...

Adding Livebook to Elixir

So let's recap. Right now, we have a README as the source of truth for our moduledoc. We can have doctests and images and all the usual goodies that a moduledoc is allowed, but we don't have to repeat ourselves and risk copy/paste errors.

We want to keep that same energy going for our Livebook, to avoid repeating ourselves manually, but still have an interactive playground for our library on top of the usual moduledocs.

To do that, we can generate our Livebook from our module. To help with this, I've written a library we can include called livebook_helpers:

defp deps do
  [
    {:ex_doc, ">= 0.0.0", runtime: false, only: [:docs, :dev]},
    {:livebook_helpers, ">= 0.0.0", only: [:docs, :dev]},
  ]
end

Once we have fetched the deps with mix deps.get, running mix help shows one extra mix task:

...
mix create_livebook_from_module # Creates a livebook from the docs in the given module.
...

We can see from the docs that we run the mix task by providing a module and a path to a Livebook. Let's try that:

mix create_livebook_from_module ChonkOMeter "chonk_o_meter_introduction"

You should see a successful output that links to the generated Livebook! 🎉 There is one last thing we can do to make our workflow seamless. Let's add create_livebook_from_module to the end of the mix docs command.

defmodule ChonkOMeter.MixProject do
  use Mix.Project

  def project do
    [
      ...
      aliases: aliases(),
      ...
    ]
  end

  defp aliases() do
    [docs: ["docs", &copy_pictures/1, &create_livebook/1]]
  end

  defp copy_pictures(_) do
    File.cp_r(Path.expand("./images/"), Path.expand("./doc/images/"))
  end

  defp create_livebook(_) do
    Mix.Task.run("create_livebook_from_module", ["ChonkOMeter", "chonk_o_meter_introduction"])
  end
end

Whenever we run mix docs, we will copy over any static images used in the README and generate a Livebook from our main module!

Running Livebook in Elixir

So far, so good! We have a nice pipeline to create a useful Livebook, but now we need to think about running the Livebook. Start the Livebook app like so:

livebook server

By default, the Elixir sections only have access to the Elixir and Erlang standard library. If we run our generated library and then attempt to run an Elixir cell that calls the library, it will fail because the library code is not there. To solve this, we have two options — Mix.install or Livebook runtime.

Add Mix.install to Livebook

We could add a section to the beginning of the Livebook that does this:

Mix.install([:chonk_o_meter])

When called, we will get the latest version of the library from Hex. It will be made available to all subsequent Elixir cells, just like when you run Mix.install inside an IEx REPL.

You can also easily specify a version and provide live documentation for any version of a given library:

Mix.install([{chonk_o_meter: ">=0.0.1"}])

LivebookHelpers can even generate a Livebook with a Mix.install at the beginning if we supply deps to the mix task:

mix create_livebook_from_module ChonkOMeter "chonk_o_meter_introduction" "[:chonk_o_meter"]"

This works great for any library that is deployed to Hex. However, you'll run into problems if, for example, you want a Livebook for a main branch. In that case, you can do this:

Mix.install [{:chonk_o_meter, path: "./"}]

This tells mix to look in the provided path for a local version of the library. This, of course, makes some assumptions about where the Livebook will run, so it's good to make that clear. If you put the Livebook at the root of the repo and a user starts the Livebook server from there, then the path "./" will work.

If you don't want to rely on this, though, Livebook has your back with a powerful feature: runtime!

Livebook Runtime

There are three kinds of runtime, but they all let you point to code and call it in any Elixir cell within your Livebook.

The three runtime options are:

• Embedded
• Attached node
• Mix standalone

You select the runtime by clicking the cog symbol here:

Let's look at each runtime option below.

Embedded Mode

Embedded Mode lets us run the notebook code within the Livebook node itself! This is really for specific cases where there is no option to start a separate Elixir runtime — for example, on embedded devices.

Code defined in one notebook may interfere with code from another notebook. So this mode should only be used if you have no alternative and is not relevant here.

Attached Node

The attached node runtime connects a Livebook to a running Elixir app via the usual Erlang magic that we use to connect two running nodes. It looks like this:

As long as the app starts with a cookie that you know and a sname, you can give them to Livebook and connect. This is like getting a remote shell in a running app but with a more full-featured text-editing environment.

Using an attached node gives you complete control over how your app starts. You get much more control than the mix standalone and can do all sorts of things, like set env vars and start other services (like a database).

An attached node could be especially relevant for creating internal (live!) documentation for closed-source repos at work, but is not relevant for us and our library.

Mix Standalone

Finally, the Mix standalone runtime lets you point to a mix project which Livebook will compile and start (analogous to running iex -S mix in your terminal).

Mix standalone confers a great advantage over Mix.install, as you can recompile it! It's useful if you write a Livebook from scratch; in that case, you'll likely add something to the library that you'll then want to use in the Livebook.

Instead of having to kill the Livebook server and restart to access the new functions, you can add an Elixir cell with the following:

IEx.Helpers.recompile()

This will recompile the connected mix app (i.e., our library) when you call it, meaning you get access to any new functionality therein.

Livebook for Docs: Try It Yourself in Your Elixir App

That concludes our tour of Livebook for docs. You can see the chonk_o_meter library example here. For an example of these ideas in action in a real library, check out my data_schema library too.

Currently, GitHub doesn't recognize the .livemd extension, so if you play around with Livebook, I would encourage you to push to public repos. The more we do this, the more chance we have of getting GitHub to parse the files with nice syntax highlighting and markdown rendering.

Until that happens, though, we can put a magic line at the top of a Livebook to force GitHub to render it as markdown:

<!-- vim: syntax=markdown -->

The Livebook that Livebook helpers generates will include this line for you. Now you have all the knowledge you need go forth and create live docs!

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!