Skip to content

Restic: test the restore, not just the backup

Backups · VPS
Not studied

Added 2026-10-11. The example assumes Linux with Restic installed.

Problem

Cron finished successfully, but after losing the VPS it turns out the configs never made it into the backup, the repository password is unavailable, or the restored application won't start.

Solution

restic is "a backup program that is fast, efficient and secure". Keep the backup outside the source VPS and regularly restore a chosen snapshot into a separate directory. restic check verifies the repository structure; --read-data additionally reads all stored data. Neither of these checks confirms that the application works after a restore.

Example

First create a remote repository, set up SSH access and a password file with 0600 permissions. The password and SSH key need their own backup. Replace the address and paths below with yours; restic init is only needed when creating a new repository.

export RESTIC_REPOSITORY='sftp:backup@backup.example.com:/srv/restic/devops-kb'
export RESTIC_PASSWORD_FILE='/etc/restic/devops-kb.password'

# The job's user must have access to these directories.
restic backup /srv/devops-kb /etc/nginx --tag devops-kb
restic snapshots --tag devops-kb
restic check --read-data

The restore drill is a separate script. Pick a snapshot ID from the list above and check its date, paths and host. A full repository read takes time and outbound traffic.

#!/usr/bin/env bash
set -euo pipefail
: "${SNAPSHOT_ID:?Set the ID of the Restic snapshot to restore}"
: "${RESTIC_REPOSITORY:?Set the repository}"
: "${RESTIC_PASSWORD_FILE:?Set the password file}"

umask 077
restore_dir=$(mktemp -d /var/tmp/devops-kb-restore.XXXXXX)
started_at=$(date +%s)
restic restore "$SNAPSHOT_ID" --target "$restore_dir"
test -s "$restore_dir/srv/devops-kb/mkdocs.yml"
test -d "$restore_dir/etc/nginx"
printf 'Files restored in %s s to %s\n' \
  "$(( $(date +%s) - started_at ))" "$restore_dir"

This checks that the files are there, but not the whole result: build the site from the restored project and check the Nginx configuration in an isolated environment. A database first needs a consistent dump or a suitable backup method; copying the files of a running database doesn't guarantee that on its own. Keep the drill directory until the result is checked, then delete it by hand.

When to use

For VPS configuration, sources and data that can't be restored by a single deploy from Git. For devops-kb, decide separately whether .business/ and other files outside Git need backups.

Source

How to apply it yourself

  1. Define the acceptable data loss (RPO) and recovery time (RTO).
  2. Run the drill on another host with access to the backup of the keys.
  3. Record the snapshot ID and age, the time to a working site, and any missing files.
  4. Mark it "Studied" after a successful restore and scheduling the next drill. The file copy time from the example is only part of the RTO.