Dimitris Zorbas

Posted on Mar 22, 2020 • Originally published at zorbash.com on Mar 1, 2020

Writing a Command-Line Application in Elixir

#elixir #commandline #erlang #terminal

I've always been fascinated by well-made applications for the terminal. Who doesn't install htop on a new machine, am I right?

My plan was to build something that I'd use daily and other people would potentially
find useful. Therefore I decided to build a cli app for Tefter.

It's built on Elixir and Ratatouille and it's open-source.

Check out the source or download and try it or install via brew.

brew tap tefter/homebrew-cli
brew install tefter

Why Elixir

Elixir is getting popular for web and distributed applications, but these
days devs tend to write cli apps in Rust / Go / C. We use Elixir at
Tefter, so there was a case for code
reuse and at some point I stumbled upon Ratatouille.
It's an Elm inspired framework, which leverages termbox,
a C library for text-based user interfaces. Being charmed by the beautiful
API of Ratatouille and eager to overcome potential hurdles, once again I chose Elixir.

Pros

High-level language
Fault-tolerance
Optimal offline storage (ETS / DETS / Mnesia)
Some portability with Releases
A decent framework (Ratatouille)
Fantastic Concurrency (see: commands)
Code-reloading for quick debugging
It's so much fun :-)

Cons

Releases bundle the VM and could be smaller
Releases are not truly portable (a release built on a Linux machine won't work on a Mac, etc)
No trivial way to fork-exec

Demo Time!

Take a glimpse of how the app behaves:

About Tefter

Before going into more detail about the specifics of the cli, let's talk about Tefter first.
It's a tool aiming to optimise your web surfing routine, a combination of personal search-engine,
a social bookmarking tool and a place to archive stuff to read later and write notes. One would interact
with Tefter through the Web app, the browser extension, the mobile and
the desktop apps or Slack!

The App

At the moment of writing, it features three main tabs for Search, Aliases and Bookmarks.
Some of the advantages of the cli app to the rest of the available are:

You don't have to leave your terminal and keyboard
vim-style keybindings with mouse support 😎
Works offline

Shortcuts

Key	Action
Ctrl+s	Jump to Search tab
Ctrl+a	Jump to Aliases tab
Ctrl+b	Jump to Bookmarks tab
Ctrl+h	Jump to Help tab
Tab	Jump to the next tab
Home	Jump to the first tab
↑	Move up
Ctrl+k	Move up
↓	Move down
Ctrl+j	Move down
Ctrl+d	Scroll down
Ctrl+u	Scroll up
Enter	Open browser window with item under cursor
Esc	Cancel command / Quit modal
F5	Force refresh resources
Ctrl+q	Quit
/	Enter filtering mode
:	Enter command mode

Authentication

The authentication is implemented to be as seamless as possible. It
won't ask the user to type their username and password.

The first time the application is started, it looks for an authentication
token in a ~/.tefter file. This file holds a plain JSON config. If not
found it'll start a tiny web server listening on a random port. It'll
then open a browser window to a special Tefter endpoint which redirects
to the address of the local web server with the authentication token
encoded in the query params. The app then proceeds to create the
~/.tefter file.

See it in action:

Search

This work in a similar manner to the auto-complete of the Web app. The
user types and results appear in the panel below. Results can be
bookmarks, lists, domains, tags or aliases.

App Architecture

Let's have a look at how this works. At the moment, the app is structured as follows:

lib/tefter_cli
├── app
│   └── state.ex
├── app.ex
├── application.ex
├── auth_server.ex
├── authentication.ex
├── bookmarks.ex
├── cache.ex
├── command.ex
├── config.ex
├── system.ex
└── views
    ├── aliases
    │   ├── actions.ex
    │   └── state.ex
    ├── aliases.ex
    ├── authentication.ex
    ├── bookmarks
    │   ├── actions.ex
    │   └── state.ex
    ├── bookmarks.ex
    ├── components
    │   ├── bottom_bar.ex
    │   ├── cursor.ex
    │   ├── info_panel.ex
    │   ├── pagination.ex
    │   └── top_bar.ex
    ├── help.ex
    ├── helpers
    │   └── text.ex
    ├── lists.ex
    ├── search
    │   └── state.ex
    └── search.ex

In the views/ directory there is a module per tab, so we have
search.ex, aliases.ex, bookmarks.ex and help.ex. Each view has a
state management module, eg views/bookmarks/state.ex and where
applicable a module for actions. The actions handle side-effects such as
the interaction with the server and the cache.

The main entrypoint for the application is app.ex. It's rather brief
so it fits in the snippet below:

defmodule TefterCli.App do
  @behaviour Ratatouille.App

  alias Ratatouille.Runtime.{Subscription}
  alias TefterCli.App.State
  alias TefterCli.Views.{Search, Bookmarks, Lists, Aliases, Authentication, Help}

  @tabs [:search, :aliases, :bookmarks, :help]

  @impl true
  def init(_), do: State.init()

  @impl true
  def update(state, msg), do: State.update(state, msg)

  @impl true
  def render(%{token: nil} = state), do: Authentication.render(state)
  def render(%{tab: :search} = state), do: Search.render(state)
  def render(%{tab: :bookmarks} = state), do: Bookmarks.render(state)
  def render(%{tab: :aliases} = state), do: Aliases.render(state)
  def render(%{tab: :lists} = state), do: Lists.render(state)
  def render(%{tab: :help} = state), do: Help.render(state)

  @doc "Returns the available application tabs"
  def tabs, do: @tabs

  @impl true
  def subscribe(%{token: nil}), do: Subscription.interval(500, :check_token)
  def subscribe(_), do: Subscription.interval(100_000, :check_token)
end

It's placed under the supervision tree with:

{
  Ratatouille.Runtime.Supervisor,
  runtime: [app: TefterCli.App, quit_events: [{:key, Ratatouille.Constants.key(:ctrl_q)}]]
}

where we declare the "main" module of the app and that ctrl + q quits.

The most important functions of TefterCli.App are update/2 and render/1.

The update/2 receives the model - the current state of the app, as
the first argument and a message as the seconds one. The message is
usually a tuple {:event, event} where event is a termbox mouse or keyboard event like the following:

%ExTermbox.Event{
  ch: 0,
  h: 0,
  key: 27,
  mod: 0,
  type: 1,
  w: 0,
  x: 0,
  y: 0
}

The update/2 should return the updated state, but in some cases you
may have it return {model(), Command.t()}.
More about commands later.
The render/1 receives the model and must return a %Ratatouille.Element{}.
Thankfully you don't have to assemble the element structs manually and
there are macros for that. Example:

def render(model) do
  view do
    label(content: "Hello, #{model.name}!")
  end
end

In TefterCli view modules like TefterCli.Views.Bookmarks define the render/1 function and
delegate the update/2 to their state management modules like TefterCli.Views.Bookmarks.State.
At the moment, the model in TefterCli is a plain map, but will be refactored to be a struct in the future.

Anatomy of the view

Most views share the TopBar and BottomBar. Views with paginated resources have the InfoPanel which displays pagination info the permits typing commands.

Aliases

What is an alias?

Think of an alias as a dynamic shortened link. You can create a maps
alias pointing to https://www.google.com/maps/search/{{*}}?hl=en&source=opensearch and then with the browser extension installed,
type go/maps/london in the address bar to be redirected to
https://www.google.com/maps/search/london?hl=en&source=opensearch.

So with {{*}} you can set dynamic segments in your shortened links.
Dynamic segments are optional though.

In the command-line app you can:

View all the aliases you've created
Create an alias with the :c <alias> <url> command
Delete an alias with the :d command
Search for an alias by typing /
Open a browser window with the link of an alias by pressing enter

Example:

Bookmarks

The bookmarks tab is very similar to aliases. A user is likely to have way more bookmarks than aliases,
since on average a user has ~1000 bookmarks but fewer than 10 aliases.
This calls for a different pagination strategy. In aliases, there's a
sliding viewport with an offset controlled by the cursor. This doesn't
work well with bookmarks. I tested it initially with my bookmarks (I
have more that 9K bookmarks) and it was sluggish. The reason is, that on
every keyboard / mouse event, Ratatouille tries to re-render everything.
In the case of thousands of bookmarks within a viewport, it renders each
and every bookmark despite most of them being off-screen. The solution
is to only feed a slice of the bookmarks list to the viewport.

On a similar note, Ratatouille will re-render everything every
500ms, see here.

Adding Bookmarks

One can add a bookmark by typing :c <url>.

Deleting Bookmarks

To delete a bookmark, type :d and the currently selected bookmark will
be deleted.

Filtering

To filter, type /. To highlight a result, TefterCli.Views.Helpers.Text.highlight/2 is used.
Unfortunately, ExTermbox seems to drop diacritical marks from strings which would let me
render highlighted text like this "z̲o̲r̲b̲a̲s̲h̲" (see: TefterCli.Views.Helpers.Text.underline/1)
and I resorted in surrounding matching text with [ and ].

Development

Clone the repo, run it with iex -S mix, make changes and
you're welcome to submit a pull-request!

Since the app takes over your IEx session, to simplify your debugging,
most events are logged to a file in log/dev.log in development.

To drop to the IEx console, you can go to the search
tab and hit ctrl + y.

To reload the source without restarting the app hit f5.

Packaging

There are two simple bash scripts to prepare releases for Linux and MacOS
in ./bin/release_linux and ./bin/release_macos respectively.
They both use mix release to prepare a tarball which bundles the Erlang VM.
The Linux script leverages Docker to ensure that a release can be built
even on a non-Linux machine.

The framework - Ratatouille

Ratatouille is impressive. It makes you want to write something in it
and it's well documented and it's also rather simple. One can read its
source in one go.

Architecture

There's the view, which is better explained quoting the documentation.

In Ratatouille, a view is simply a tree of elements. Each element in the tree
holds an attributes map and a list of zero or more child nodes. Visually, it
looks like something this:

%Element{
  tag: :view,
  attributes: %{},
  children: [
    %Element{
      tag: :row,
      attributes: %{},
      children: [
        %Element{tag: :column, attributes: %{size: 4}, children: []},
        %Element{tag: :column, attributes: %{size: 4}, children: []},
        %Element{tag: :column, attributes: %{size: 4}, children: []}
      ]
    }
  ]
}

Then there's the runtime, which is basically this:

defp loop(state) do
  :ok = Window.update(state.window, state.app.render(state.model))

  receive do
    {:event, %{type: @resize_event} = event} ->
      state
      |> process_update({:resize, event})
      |> loop()

    {:event, event} ->
      if quit_event?(state.quit_events, event) do
        shutdown(state)
      else
        state
        |> process_update({:event, event})
        |> loop()
      end

    {:command_result, message} ->
      state
      |> process_update(message)
      |> loop()
  after
    state.interval ->
      state
      |> process_subscriptions()
      |> loop()
  end
end

I'd change Ratatouille.Runtime to be a GenServer for
plenty of reasons, introspection with :sys being one of them.

Caveats

Ratatouille is fantastic, but there are a few things that could be improved:

Performs unnecessary re-renderings
Lack of form controls

What's Next

Debian and Homebrew packages
Windows support (mention the desktop app)
Lists
Organisations
Create and delete aliases
Display notes in an overlay
Edit a bookmark Trigger the edit mode with the :e command. Then open $EDITOR with a tempfile containing the JSON representation of the bookmark. When the editor is closed update the bookmark
Import chrome / firefox bookmarks

Thank you

I want to thank ndreynolds for creating Ratatouille and I hope people will gain something
by reading this post and the source of the app and provide feedback!

DEV Community