This is a very short article about using AWS Systems Manager Parameter Store to load secret configuration in a simple way. One of the motivations for the approach I'm about to describe is the desire to avoid what I consider to be an anti-pattern. If you are not interested in that discussion, then feel free to skip to the code.
The Anti-Pattern
Elixir 1.9 introduced "releases", and with it an overhaul of how configuration is done. Before this, all configuration was resolved at build time, something that was not a problem for those who ran their apps "from source", as "build time" and "run time" were effectively the same. But for those who wanted to ship a pre-built app, this caused problems.
config :my_app,
# when is this resolved?
database_url: System.get_env("DATABASE_URL")
People solved this problem by adopting a pattern: set the variable to a placeholder value and defer the lookup till later.
config :my_app,
normal_var: "I am totally normal",
deferred_var: {:system, "SEE_YOU_LATER"}
In order for this to work, the consuming code would have to recognise this pattern.
def consume_config() do
case Application.get_env(:my_app, :deferred_var) do
{:system, var} -> System.get_env(var)
value -> value
end
end
The problem with this "anti-pattern" is it confused newcomers who wrongly assumed it was a feature of the configuration system itself, rather than what it actually was: an ad-hoc workaround, that had to be implemented for every library and variable one chose to support.
Happily, this is not a problem anymore, and the story around runtime configuration is much better: just put it all in config/runtime.exs
.
Recently, I wanted to use AWS SSM to store secret configuration, and it looked like the way to go was to use Config.Provider. Looking around to see how others had solved the same problem, a familiar pattern cropped up.
config :my_app,
some_var: {:ssm, "THIS_LOOKS_FAMILIAR"}
Because Config.Provider
s run after config/runtime.exs
is loaded, you needed some placeholder value that could later be detected and replaced.
This, I thought, was an instance of the same anti-pattern.
Deciding I didn't want to re-introduce the anti-pattern, I came up with a different approach.
A different approach
First, add the required dependencies and declare a Config.Provider
.
defmodule Example.MixProject do
def project do
[
...
releases: releases()
]
end
defp releases() do
[
example: [
config_providers: [
{Example.ConfigProvider, []}
]
]
]
end
defp deps do
[
...
{:ex_aws, "2.0"},
{:ex_aws_ssm, "2.0"}
]
end
end
And then, implement it as follows.
defmodule Example.ConfigProvider do
@behaviour Config.Provider
@impl true
def init([]) do
[]
end
@impl true
def load(config, _state) do
{:ok, _} = Application.ensure_all_started(:hackney)
{:ok, _} = Application.ensure_all_started(:ex_aws)
params = get_parameters(config[:ex_aws]) |> Enum.into(%{})
Config.Reader.merge(config,
example: [
{ExampleWeb.Endpoint, secret_key_base: params["secret_key_base"]},
{Example.Repo, url: params["database_url"]}
]
)
end
defp get_parameters(config) do
ExAws.SSM.get_parameters_by_path(prefix(), with_decryption: true)
|> ExAws.request(config)
|> case do
{:ok, %{"Parameters" => params}} ->
params
|> Enum.map(fn param ->
{String.trim_leading(param["Name"], prefix()), param["Value"]}
end)
error ->
raise "Failed to fetch parameters: #{inspect(error)}"
end
end
defp prefix() do
"/example/prod/"
end
end
Essentially, instead of using a generic Config.Provider
that relies on placeholder values to do its job, I wrote a one-off Config.Provider
that added the configuration I wanted.
Regaining what was lost
This worked fine for me in the project I was working on, but looking back at it, I think it suffers from two problems.
There was something nice about the "wrong" approach: it was declarative and the configuration was all in one place.
I thought about an alternative that would help the code regain the clarity that was lost and this is what I came up with.
First, I started with what I thought was the simplest possible API.
config :my_app,
some_var: System.get_env("SOME_VAR"),
database_url: Secret.get_env("DATABASE_URL")
Just replace System.get_env(...)
with Secret.get_env(...)
.
And got the following (admittedly rough) code working.
defmodule Secret do
def get_env(key, default \\ nil) do
Map.get(get_all_env(), key, default)
end
defp get_all_env() do
case :ets.whereis(__MODULE__) do
:undefined ->
:ets.new(__MODULE__, [:set, :public, :named_table])
{:ok, _} = Application.ensure_all_started(:hackney)
{:ok, _} = Application.ensure_all_started(:ex_aws)
env = fetch_env()
:ets.insert(__MODULE__, {:env, env})
env
ref ->
[{:env, env}] = :ets.lookup(ref, :env)
env
end
end
defp fetch_env() do
config = [
region: System.fetch_env!("AWS_REGION"),
access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY")
]
get_parameters(config) |> Enum.into(%{})
end
defp get_parameters(config) do
# same as above
end
end
$ aws ssm put-parameter \
--name "/example/prod/DATABASE_URL" \
--value "ecto://postgres:postgres@localhost/example_prod" \
--type SecureString \
--overwrite
> Secret.get_env("DATABASE_URL")
"ecto://postgres:postgres@localhost/example_prod"
I have yet to switch to this approach, but it is something I will consider next time I need to accomplish the same task.
Top comments (0)