Distributing go binaries with fpm

Go has a nice tooling — build system, cross-compilation, dependency management and even formatting tools. And in the end, you get a single binary.

Now having a single binary how to distribute it on servers? I mean, how you solve the following problems:

1. Identifying what version is currently run in prod?
2. Upgrading the binary?
3. Downgrading to the known version?
4. Distributing extra stuff with a binary — data files, service definitions and stuff?

The common to all of the problems above is versioning. You need to assign and track the version of your Go program to keep the sanity in the prod.

One of the solutions is docker — you put the binary into the scratch image, put anything you want along with the binary, tag the image, upload it to the registry and then use it on the server with docker tools.

It sounds reasonable and trendy. But operating docker is not an easy walk. Networking with docker is hard, docker is breaking on upgrades, etc. Though in the long run, it could pay off because it’ll allow you to transition to some nice platform like Kubernetes.

But what if you don’t want to use docker? What if you don’t want to install the docker tools and keep the docker daemon running on your production just for the single binary?

If you don’t use docker then in case of golang you’re entering a hostile place. Go tooling gives you a solution in the form of go get. But go get only fetches from HEAD and requires you to manually use git to switch version and then invoke go build to rebuild the program. Also, keeping dev environment on the production infrastructure is stupid.

Instead, I have a much simpler and battle-tested solution — packages. Yes, the simple and familiar distro packages like “deb” and “rpm”. It has versions, it has good tooling allowing you to query, upgrade and downgrade packages, supply any extra data and even script the installations with things like postinst.

So the idea is to package the go binary as a package and install it on your infrastructure with package management utilities. Though building packages sometimes get scary, packaging a single file (with metadata) is really simple with the help of an amazing tool called fpm.

fpm allows you to create target package like “deb” or “rpm” from various sources like a plain directory, tarballs or other packages. Here is the list of sources and targets from github:

Sources:

• gem (even autodownloaded for you)
• python modules (autodownload for you)
• pear (also downloads for you)
• directories
• tar(.gz) archives
• rpm
• deb
• node packages (npm)
• pacman (ArchLinux) packages

Targets:

• deb
• rpm
• solaris
• freebsd
• tar
• directories
• Mac OS X .pkg files (osxpkg)
• pacman (ArchLinux) packages

To package Go binaries we’ll use “directory” source and package it as “deb” and “rpm”.

Let’s start with “rpm”:

$ fpm -s dir -t rpm -n mypackage $GOPATH/bin/packer
Created package {:path=>"mypackage-1.0-1.x86_64.rpm"}

And that’s a valid package!

$ rpm -qipl mypackage-1.0-1.x86_64.rpm
Name : mypackage
Version : 1.0
Release : 1
Architecture: x86_64
Install Date: (not installed)
Group : default
Size : 87687286
License : unknown
Signature : (none)
Source RPM : mypackage-1.0-1.src.rpm
Build Date : Mon 06 Nov 2017 07:54:47 PM MSK
Build Host : airblade
Relocations : / 
Packager : <avd@airblade>
Vendor : avd@airblade
URL : http://example.com/no-uri-given
Summary : no description given
Description :
no description given
/home/avd/go/bin/packer

You can see, though, that it put the file with the path as is, in my case under my $GOPATH. We can tell fpm where to put it on the target system like this:

$ fpm -f -s dir -t rpm -n mypackage $GOPATH/bin/packer=/usr/local/bin/
Force flag given. Overwriting package at mypackage-1.0-1.x86_64.rpm {:level=>:warn}
Created package {:path=>"mypackage-1.0-1.x86_64.rpm"}

$ rpm -qpl mypackage-1.0-1.x86_64.rpm
/usr/local/bin/packer

Now, that’s good.

By the way, because we made it as rpm package we got a 80% reduction in size due to package compression:

$ stat -c '%s' $GOPATH/bin/packer mypackage-1.0-1.x86_64.rpm
87687286
16097515

If you’re using deb-based distro all you have to do is change the target to thedeb:

$ fpm -f -s dir -t deb -n mypackage $GOPATH/bin/packer=/usr/local/
bin/
Debian packaging tools generally labels all files in /etc as config files, as mandated by policy, so fpm defaults to this behavior for deb packages. You can disable this default behavior with --deb-no-default-config-files flag {:level=>:warn}
Created package {:path=>"mypackage_1.0_amd64.deb"}

$ dpkg-deb -I mypackage_1.0_amd64.deb
 new debian package, version 2.0.
 size 16317930 bytes: control archive=430 bytes.
     248 bytes, 11 lines control              
     126 bytes, 2 lines md5sums              
 Package: mypackage
 Version: 1.0
 License: unknown
 Vendor: avd@airblade
 Architecture: amd64
 Maintainer: <avd@airblade>
 Installed-Size: 85632
 Section: default
 Priority: extra
 Homepage: http://example.com/no-uri-given
 Description: no description given

$ dpkg-deb -c mypackage_1.0_amd64.deb
drwxrwxr-x 0/0 0 2017-11-06 20:05 ./
drwxr-xr-x 0/0 0 2017-11-06 20:05 ./usr/
drwxr-xr-x 0/0 0 2017-11-06 20:05 ./usr/share/
drwxr-xr-x 0/0 0 2017-11-06 20:05 ./usr/share/doc/
drwxr-xr-x 0/0 0 2017-11-06 20:05 ./usr/share/doc/mypackage/
-rw-r--r-- 0/0 135 2017-11-06 20:05 ./usr/share/doc/mypackage/changelog.gz
drwxr-xr-x 0/0 0 2017-11-06 20:05 ./usr/local/
drwxr-xr-x 0/0 0 2017-11-06 20:05 ./usr/local/bin/
-rwxrwxr-x 0/0 87687286 2017-09-06 20:06 ./usr/local/bin/packer

Note, that I’m creating deb package on Fedora which is rpm-based distro!

Now you just upload the binary to your repo and you’re good to go.