A Slack bookmarking application in Elixir with Opus

This post describes how we used Elixir and Opus in one of our services at
Tefter, which implements bookmarking collaboration in Slack.

My relationship with Slack

I remember, when Slack started getting viral and it was set as the main
chat app at work, I was very reluctant to use it. I was quite happy with IRC and always in favour of open protocols. Since it supported an IRC / XMPP gateway, tweaking my irssi config and later finch was trivial and my overall experience was good. Later I developed my first Slack apps to experiment, accomplish trivial tasks and participate in company hackathons.

Tefter

Recently at Tefter, we released a new organisations feature. This feature, gives users the ability to collaborate within a Slack workspace.

Essential commands of the Slack app:

Create a bookmark

/tefter <url>

Create an alias

/tefter alias <alias> <url>

Search

/tefter search <query>

# Alternatively
/tefter s <query>

This is what a search looks like:

Retrieve a link by alias

/tefter <alias>

Aliases are especially useful for recurring questions concerning a link. For example
"What is the API documentation page for service x?". There you can
create a "docs" alias and point people to the link by calling /tefter docs.

Microlith

The microservice dealing with that side of our system is named Microlith. That is to contradict
its tendency to become a monolith 🙈. It is written in Elixir and it
leverages a library for railway-oriented programming called Opus.
Surprisingly I've never blogged about this tiny library of mine before, but a few others have. It incorporates some software design principles I keep close to my heart.

The main principles of Opus are:

• Each Opus pipeline module has a single entry point and returns tagged tuples {:ok, value} | {:error, error}
• A pipeline is a composition of stateless stages
• A stage returning {:error, _} halts the pipeline
• A stage may be skipped based on a condition function (:if and :unless options)
• Exceptions are converted to {:error, error} tuples by default
• An exception may be left to raise using the :raise option
• Each stage of the pipeline is instrumented. Metrics are captured automatically (but can be disabled).
• Errors are meaningful and predictable

In this post, I'll show you some code examples from Microlith where Opus is used.

The great thing about Opus is that a use-case can be described as a
series of stages. Similar to your grandma's beef stew recipe.

Creating a bookmark with Opus

So the "recipe" to create a bookmark from Slack is:

• check that the payload has the correct format
• check that the payload contains a URL to bookmark
• normalise the URL
• retrieve the Tefter account by its Slack identifier
• check that the account can create a bookmark
• create the bookmark
• respond to the user

Our next move will be to translate this to pseudo-code in Opus terms.

check :valid_payload?
check :payload_contains_url?
step :normalize_url
step :fetch_user
check :can_create_bookmark?
step :create_bookmark
step :respond

A quick rundown of the available stages of Opus.

• step: This stage processes the input value and with a success value the next stage is called with that value.
  With an error value the pipeline is halted and an {:error, any} is returned.

• check: This stage is intended for validations. It calls the stage function and unless it returns true, it halts the pipeline.

• tee: This stage is intended for side effects, such as a notification or a call to an external system where the return value is not meaningful. It never halts the pipeline.

• link: This stage is to link with another Opus.Pipeline module. It calls call/1 for the provided module. If the module is not an Opus.Pipeline it is ignored.

• skip: The skip macro can be used for linked pipelines. A linked pipeline may act as a true bypass, based on a condition,
  expressed as either :if or :unless. When skipped, none of the stages are executed and it returns the input,
  to be used by any next stages of the caller pipeline.

Now we can define our Opus.Pipeline module.

defmodule Microlith.Commands.CreateBookmark do
  @moduledoc "Pipeline which handles the bookmark command to create a bookmark"

  use Opus.Pipeline

  alias Microlith.Pipelines.FetchUser

  check :valid_payload?, error_message: :invalid_payload
  check :contains_url?, error_message: "Command called without a URL"
  step :trim_url, with: &%{&1 | url: String.trim(&1[:url])}
  step :fetch_user
  check :can_create_bookmark?
  step :create_bookmark
  step :respond
end

The module we just defined, can be used as follows:

payload = %{
  input: "https://zorbash.com",
  slack_user_id: "a Slack user identifier",
  team_id: "a Slack team identifier",
  team_domain: "whitehouse",
  response_url: "https://hooks.api.slack.com/deadbeef"
}

case Microlith.Commands.CreateBookmark.call(payload) do
  {:ok, response} ->
    json(response)
  {:error, error} ->
    Logger.warn(inspect(error))
    send_resp(conn, 422, "The request could not be accepted")
end

Concurrency

Like any decent cooking recipe, the implementation is left to the chef.
You may want to crack 6 eggs with one hand while stirring some sauce
with the other, but the end-result should be the same.

Thankfully the Elixir toolset is very well equipped with facilities to make operations in a pipeline concurrent. In most cases all we have to do is to start Tasks and pass them down to next stages. When a stage requires the result of a Task, Task.await/1 can be used.

Visualising Pipelines

I kept my favourite part for last.

With Opus you can visualise your pipelines using Opus.Graph using Opus.Graph.generate/1.

So at some point in the development of Microlith, it looked like this:

Protip: You should prefer the SVG output, where you can hover on stages and pipelines to read their documentation.

Closing Thoughts

I hope that this post gives an idea of the features of Opus and I promise to cover the next of them in a following post.
If you're using Opus, I'd be glad to hear your feedback.

Do you represent an open-source community and you're interested to try
out Tefter Organizations? Let me know and we'll add you to an unlimited
plan without any cost.