How to Deploy rest-server(Restic) on Docker Swarm Behind Traefik Reverse Proxy

As you might know Restic, a simple backup manager. I have been using restic for a while now, backing up all my data easily and securely. For example...

# Initialize a backup repository in the specified local directory:
$ restic init --repo path/to/repository

# Backup a directory to the repository:
$ restic --repo path/to/repository backup path/to/directory

# Show backup snapshots currently stored in the repository:
$ restic --repo path/to/repository snapshots

# Restore a specific backup snapshot to a target directory:
$ restic --repo path/to/repository restore latest|snapshot_id --target path/to/target

When it comes to managing remote backups, restic supports sftp

$ restic -r sftp:user@host:/srv/restic-repo init
enter password for new repository:
enter password again:
created restic repository f1c6108821 at sftp:user@host:/srv/restic-repo
Please note that knowledge of your password is required to access the repository.
Losing your password means that your data is irrecoverably lost.

There is one more option to manage remote backups, that is via http with rest-server tool.

Basically, you run rest-server as a systemd daemon or in docker to expose http endpoint and restic client can interact with this http to manage backups on remote locations.

Then you might ask, why use this http rest-server, when we have sftp?

As rest-server READM.md says

• Compared to the SFTP backend, the REST backend has better performance.
• REST protocol should be faster and more scalable, due to some inefficiencies of the SFTP protocol.
• Rest Server adds is the optional ability to run in append-only mode.
• Rest Server implementation is really simple and as such could be used on the low-end devices, no problem.

Deployment

👉 Github repo contains docker stack files.

I considered this tool rest-server for my home server to manage backups, that's why I deployed rest-server behind Traefik reverse proxy with HTTPS. Here is the diagram below.

$ git clone https://github.com/veerendra2/raspberrypi-homeserver.git
$ cd raspberrypi-homeserver/services/rest-server
$ tree -a .
.
├── .env_rest-server
├── docker-stack.yml
└── secrets
    └── htpasswd.txt

1 directory, 3 files

So, here is my docker swarm service for reference

version: "3.8"

networks:
  network_monitoring:
    external: true
  network_public:
    external: true

secrets:
  htpasswd:
    file: secrets/htpasswd.txt

services:
  rest-server:
    image: restic/rest-server:latest
    deploy:
      replicas: 1
      placement:
        constraints: [node.role == manager]
      restart_policy:
        condition: on-failure
        delay: 30s
        max_attempts: 3
      labels:
        - traefik.enable=true
        - traefik.docker.network=network_public
        - traefik.http.routers.rest-server.tls=true
        - traefik.http.routers.rest-server.rule=Host(`veeru.duckdns.com`) && PathPrefix(`/restic`)
        - traefik.http.services.rest-server.loadbalancer.server.port=8000
    hostname: rest-server
    user: 1000:1003
    env_file:
      - .env_rest-server
    volumes:
      - /media/disk2/backups:/restic:rw
    networks:
      - network_public
      - network_monitoring
    secrets:
      - htpasswd

▶️If you want to know more about how I deployed Traefik reverse proxy, refer my github repo
▶️If you want to know more about how I used duckdns sub-domain as my home server's hostname, refer my medium blog post

and the .env_rest-server

DISABLE_AUTHENTICATION=1
OPTIONS=--prometheus --prometheus-no-auth --append-only --private-repos
DATA_DIRECTORY=/restic
PASSWORD_FILE=/run/secrets/htpasswd

I prefer expose the service(rest-server) with sub-path(/restic), that's why you see the traefik label like below

- traefik.http.routers.rest-server.rule=Host(`veeru.duckdns.com`) && PathPrefix(`/restic`)

Or if you want to expose the service with subdomain, you can set

- traefik.http.routers.rest-server.rule=Host(`restic.yourdomain.com`)

By the way, if don't have personal domain, you can check my other write-up Traefik HTTPS Config with DuckDNS for Local Homeserver

Authentication

I disabled authentication as you can see DISABLE_AUTHENTICATION=1. But still, you need to create user to interact with rest-server

$ htpasswd -B -c .htpasswd restic
New password:
Re-type new password:
Adding password for user restic

# create htpasswd
$ cat .htpasswd
restic:[REDACTED]

# save htpasswd content in secrets directory to use it as docker secrets
cat .htpasswd > secrets/htpasswd.txt

Data directory

DATA_DIRECTORY=/restic

This option tells rest-server where our restic repositories should be stored. Since we are running in Docker, we can mount this data directory outside of docker in the desired location as you can see in docker stack file.

...
  volumes:
    - /media/disk2/backups:/restic:rw
...

Prometheus monitoring

You can also enable Prometheus monitoring, by setting option --prometheus and --prometheus-no-auth is for no authentication for Prometheus scrap

Other Options

• --append-only

The --append-only mode allows creation of new backups but prevents deletion and modification of existing backups. This can be useful when backing up systems that have a potential of being hacked.
and --private-repos

• --private-repos

To prevent your users from accessing each others' repositories, you may use the --private-repos flag which grants access only when a subdirectory with the same name as the user is specified in the repository URL. For example, user "foo" using the repository URLs rest:https://foo:pass@host:8000/foo or rest:https://foo:pass@host:8000/foo/ would be granted access, but the same user using repository URLs rest:https://foo:pass@host:8000/ or rest:https://foo:pass@host:8000/foobar/ would be denied access. Users can also create their own subrepositories, like /foo/bar/.

This is the reason we need to create a user. Each user gets a separate directory inside the data directory(/restic) to store their backups. In above section, we created user called restic, the restic user can store backups on server at /restic/restic location(So, the path is like [DATA_DIRECTORY]/[USER]/). If you create a user "ramu", the user "ramu" can store backup at location /restic/ramu/ on the server.

Testing

📰https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server

Prepare restic repository(A restic repository is nothing but a folder containing your backup in encrypted format and its metadata).

$ restic init --repo rest:https://veeru.duckdns.org/restic/my-projects
enter password for new repository:
enter password again:
created restic repository 9266e74132 at rest:https://veeru.duckdns.org/restic/my-projects/

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

restic client intracts with rest-server (with endpoint https://veeru.duckdns.org/restic) and creates repository on remote location as you can below

$ cd /media/disk2/backups/
# restic directory is created
$ ls
restic

# check files inside restic directory
$ tree -L 2 restic/
restic/
└── my-projects
    ├── config
    ├── data
    ├── index
    ├── keys
    ├── locks
    └── snapshots

6 directories, 1 file

Note that /restic location is the sub-path we created for Traefik. Traefik will route traffic to rest-server when it sees /restic in the URL path.

Run backup

$ restic -r rest:https://veeru.duckdns.org/restic/my-projects backup my-projects/
enter password for repository:
repository 9266e741 opened (version 2, compression level auto)
created new cache in /Users/veerendra.k/Library/Caches/restic
no parent snapshot found, will read all files
[0:01] 2370 files 421.600 MiB, total 13206 files 1.562 GiB, 0 errors
...

Files:       105887 new,     0 changed,     0 unmodified
Dirs:        33890 new,     0 changed,     0 unmodified
Added to the repository: 3.807 GiB (2.988 GiB stored)

processed 105887 files, 4.461 GiB in 2:01
snapshot 9c7a6616 saved

That's it!, restic will backup(with encrption) all files from source path to remote rest-server location.

List snapshots

$ restic -r rest:https://veeru.duckdns.org/restic/my-projects snapshots
enter password for repository:
repository 9266e741 opened (version 2, compression level auto)
ID        Time                 Host          Tags        Paths
-----------------------------------------------------------------------------------------------
9c7a6616  2023-07-18 23:22:11  C02F66HKML85              /Users/veerendra.k/my-projects
-----------------------------------------------------------------------------------------------
1 snapshots

This simple rest-server allows me to manage my backups on remote locations.