How to use Ansible with Terraform

Recently, I’ve started using Terraform for creating a cloud test rig and it’s pretty dope. In a matter of a few days, I went from “never used AWS” to the “I have a declarative way to create an isolated infrastructure in the cloud”. I’m spinning a couple of instances in a dedicated subnet inside a VPC with a security group and dedicated SSH keypair and all of this is coded in a mere few hundred lines.

It’s all nice and dandy but after creating an instance from some basic AMI I need to provision it. My go-to tool for this is Ansible but, unfortunately, Terraform doesn’t support it natively as it does for Chef and Salt. This is unlike Packer that has ansible (remote) and ansible-local that I’ve used for creating a Docker image.

So I’ve spent some time and found a few ways to marry Terraform with Ansible that I’ll describe hereafter. But first, let’s talk about provisioning.

Do we really need provisioning in the cloud?

Instead of using the empty AMIs you could bake your own AMI and skip the whole provisioning part completely but I see a giant flaw in this setup. Every change, even a small one, requires recreation of the whole instance. If it’s a change somewhere on the base level then you’ll need to recreate your whole fleet. It quickly becomes unusable in case of deployment, security patching, adding/removing a user, changing config and other simple things.

Even more so if you bake your own AMIs then you should again provision it somehow and that’s where things like Ansible appears again. My recommendation here is again to use Packer with Ansible.

So in most cases, I’m strongly for the provisioning because it’s unavoidable anyway.

How to use Ansible with Terraform

Now, returning to the actual provisioning I found 3 ways to use Ansible with Terraform after reading the heated discussion at this GitHub issue. Read on to find the one that’s most suitable for you.

Inline inventory with instance IP

One of the most obvious yet hacky solutions is to invoke Ansible within local-exec provisioner. Here is how it looks like:

provisioner "local-exec" {
    command = "ansible-playbook -i '${self.public_ip},' --private-key ${var.ssh_key_private} provision.yml"
}

Nice and simple, but there is a problem here. local-exec provisioner starts without waiting for an instance to launch, so in most cases, it will fail because by the time it will try to connect there is nobody listening.

As a nice workaround, you can use preliminary remote-exec provisioner that will wait until the connection to the instance is established and then invoke thelocal-exec provisioner.

As a result, I have this thingy that plays the role of “Ansible provisioner”

provisioner "remote-exec" {
    inline = ["sudo dnf -y install python"]

    connection {
      type = "ssh"
      user = "fedora"
      private_key = "${file(var.ssh_key_private)}"
    }
  }

  provisioner "local-exec" {
    command = "ansible-playbook -u fedora -i '${self.public_ip},' --private-key ${var.ssh_key_private} provision.yml" 
  }

To make ansible-playbook work you have to have an Ansible code in the same directory with Terraform code like this:

$ ll infra
drwxrwxr-x. 3 avd avd 4.0K Mar 5 15:54 roles/
-rw-rw-r--. 1 avd avd 367 Mar 5 15:19 ansible.cfg
-rw-rw-r--. 1 avd avd 2.5K Mar 7 18:54 main.tf
-rw-rw-r--. 1 avd avd 454 Mar 5 15:27 variables.tf
-rw-rw-r--. 1 avd avd 38 Mar 5 15:54 provision.yml

This inline inventory will work in most cases, except when you need multiple hosts in inventory. For example, when you setup Consul agent you need a list of Consul servers for rendering a config and that is usually found in the usual inventory. So but it won’t work here because you have a single host in your inventory.

Anyway, I’m using this approach for the basic things like setting up users and installing some basic packages.

Dynamic inventory after Terraform

Another simple solution for provisioning infrastructure created by Terraform is just don’t tie Terraform and Ansible together. Create infrastructure with Terraform and then use Ansible with dynamic inventory regardless of how your instances were created.

So you first create an infra with terraform apply and then you invoke ansible-playbook -i inventory site.yml, where inventory dir contains dynamic inventory scripts.

This will work great but has a little drawback – if you need to increase the number of instances you must remember to launch Ansible after Terraform.

That’s what I use complementary to the previous approach.

Inventory from Terraform state

There is another interesting thing that might work for you – generate static inventory from Terraform state.

When you work with Terraform it maintains the state of the infrastructure that contains everything including your instances. With a local backend, this state is stored in a JSON file that can be easily parsed and converted to the Ansible inventory.

Here are 2 projects with examples that you can use if you want to go this way.

https://github.com/adammck/terraform-inventory

$ terraform-inventory -inventory terraform.tfstate
[all]
52.51.215.84

[all:vars]

[server]
52.51.215.84

[server.0]
52.51.215.84

[type_aws_instance]
52.51.215.84

[name_c10k server]
52.51.215.84

[%_1]
52.51.215.84

https://github.com/express42/terraform-ansible-example/blob/master/ansible/terraform.py

$ ~/soft/terraform.py --root . --hostfile
## begin hosts generated by terraform.py ##
52.51.215.84 C10K Server
## end hosts generated by terraform.py ##

IMHO, I don’t see a point in this approach.

Ansible plugin for Terraform that didn’t work for me

Finally, there are few projects that try to make a native looking Ansible provisioner for Terraform like builtin Chef provisioner.

https://github.com/jonmorehouse/terraform-provisioner-ansible – this was the first attempt to make such plugin but, unfortunately, it’s not currently maintained and moreover it’s not supported by the current Terraform plugin system.

https://github.com/radekg/terraform-provisioner-ansible – this one is more recent and currently maintained. It enables this kind of provisioning:

...
provisioner "ansible" {
    plays {
        playbook = "./provision.yml"
        hosts = ["${self.public_ip}"]
    }
    become = "yes"
    local = "yes"
}
...

Unfortunately, I wasn’t able to make it work so I blew it off because first 2 solutions cover all of my cases.

Conclusion

Terraform and Ansible is a powerful combo that I use for provisioning cloud infrastructure. For basic cloud instances setup, I invoke Ansible withlocal-exec and later I invoke Ansible separately with dynamic inventory.

You can find an example of how I do it at c10k/infrastructure

Thanks! Until next time!

Comments

[Fred Richards]

I haven't tried this yet, but I like the idea of using terraform to update ~/.ssh/config, then dynamic inventory from ansible to use the same list of hosts. This has the added benefit of keeping a current list of hosts in your ssh-config for your user, for manual access.

Terraform: registry.terraform.io/modules/mcgr...

Ansible: github.com/ansible/ansible/blob/de...