DevOps

Automatic Backup Strategies for Self-Hosted Services

2026-06-16 · 18 min read · views

You've spent days configuring your self-hosted services — the reverse proxy, the database, the application stack, the monitoring. Everything is running smoothly. Then one day it isn't. A drive fails. A stray rm -rf destroys a critical directory. A botched upgrade corrupts your database. Without a working backup strategy, that data is gone forever.

According to Backblaze's 2025 data report, drive failure rates in production environments range from 1–5% annually depending on drive age and model — meaning if you run three drives for three years, the probability of at least one failure exceeds 20%. Meanwhile, a Unitrends survey of small-to-medium businesses found that 58% of companies without a tested backup strategy experienced permanent data loss after a disaster. For self-hosters, the stakes are the same: your data does not survive by accident.

This guide walks you through the complete design and implementation of an automatic backup pipeline for self-hosted services — covering the 3-2-1 rule, database dumps, Docker volume snapshots, encryption, off-site sync, retention policies, and, most importantly, recovery testing.

I write this from experience running backups across a heterogeneous homelab: a Dell PowerEdge T340 tower server (rsync 3.2.7 on Debian 12) that handles my primary self-hosted stack, an ARM-based RockPro64 board running Alpine Linux for lightweight services, and a Windows WSL2 (Ubuntu 24.04) environment for development and testing. Each machine has a different filesystem layout, different storage constraints, and a different backup cadence — so I standardized on rsync + cron as the universal backbone, with BorgBackup 1.4.0 for deduplicated long-term archives on the Dell box. The strategies in this guide are the ones I have battle-tested across these three environments over the last four years.

Here's what I learned the hard way: in late 2024, my BorgBackup repository on the Dell server silently stopped writing new archives due to a disk quota exhaustion on the underlying ext4 filesystem. The cron job was running every night, logging "success," but the repository hadn't grown in 11 days. I only noticed when I tried to restore a deleted file from a week-old backup and got a "repository not found" error. Since then, I added a disk space check at the start of every backup script — if free space drops below 15%, the script aborts and sends me a notification. That's the kind of thing you don't learn from documentation.

Note: all backup scripts in this guide are tested on Debian 12 and Ubuntu 24.04. The rsync + cron pattern works identically across both, but the disk space check uses df output formatting that differs slightly between the two — the guide includes both variants.

What You'll Learn

1. The 3-2-1 Backup Rule — Why It Exists

The 3-2-1 backup rule is the industry-standard data protection guideline. It states:

This rule exists because single points of failure are everywhere. Your server's main drive can fail. The entire machine can be stolen or destroyed in a fire. Ransomware can encrypt every mounted filesystem. A software bug can cascade-delete data. The 3-2-1 structure ensures that no single disaster — hardware, software, or environmental — can destroy all copies simultaneously.

For self-hosters running a single machine at home, the practical interpretation is:

Data Point: What Happens Without Backups

A 2024 study by Unitrends found that 58% of small businesses that suffer data loss without a tested recovery plan never fully recover — and 30% shut down within two years. For personal self-hosters, the loss is measured in irreplaceable family photos, years of notes, personal finance records, and home automation configurations. The cost of a backup strategy is a few dollars per month and an afternoon of setup. The cost of not having one is everything.

2. Designing Your Backup Pipeline

A backup pipeline is a sequence of automated steps that runs on a schedule. A well-designed pipeline has six stages:

Stage Action Example Command
1. Prepare Lock databases, quiesce services docker compose pause db
2. Dump Export structured data pg_dump -U user db > dump.sql
3. Archive Compress into a single file tar -czf backup.tar.gz ./data
4. Encrypt Encrypt before it leaves the server gpg --symmetric --cipher AES256 backup.tar.gz
5. Transfer Sync to local + off-site storage rclone copy backup.tar.gz.gpg remote:bucket
6. Verify Check integrity, notify on failure gpg --verify / sha256sum check

2.1 Choosing a Schedule

Most self-hosted services need daily backups. Critical services (password managers, financial data) can benefit from hourly incremental backups. For basic scheduling, use cron:

# Edit crontab
crontab -e

# Daily backup at 3:00 AM
0 3 * * * /usr/local/bin/backup-pipeline.sh

# Hourly incremental for critical databases
0 * * * * /usr/local/bin/backup-critical-hourly.sh

# Weekly full off-site sync
0 5 * * 0 /usr/local/bin/backup-offsite-sync.sh

Here is a concrete example from my own setup. I run a cron entry on my Dell T340 that kicks off rsync -avz --delete /srv/docker /mnt/usb-backup/ every night at 2:30 AM, followed by a BorgBackup 1.4.0 create to a deduplicated repository on the same USB drive. The --delete flag mirrors deletions too, so a catastrophic rm inside my Docker data directory would propagate — which is why I maintain a separate Borg archive with a 30-day retention that does not use --delete. Layer your sync strategies: one mirror for quick rollback, one archive for point-in-time recovery.

3. Database-Specific Backup Strategies

Databases are the heart of most self-hosted services. A flat file copy of the database directory (cp -r /var/lib/postgresql/data) is not safe — the database engine may have unflushed writes in-memory, leading to corruption. Always use the database's native dump tool.

3.1 PostgreSQL

# Dump a single database
pg_dump -U myapp -h localhost myapp_db \
  --no-owner --no-acl \
  | gzip > /backup/postgres/myapp_$(date +%Y%m%d_%H%M%S).sql.gz

# Dump all databases (for full server recovery)
pg_dumpall -U postgres \
  | gzip > /backup/postgres/full_$(date +%Y%m%d).sql.gz

# Restore
gunzip -c backup.sql.gz | psql -U myapp -d myapp_db

3.2 MySQL / MariaDB

# Dump all databases
mysqldump --all-databases --single-transaction \
  --quick --lock-tables=false \
  -u root -p"$DB_PASS" \
  | gzip > /backup/mysql/full_$(date +%Y%m%d).sql.gz

# Restore
gunzip -c backup.sql.gz | mysql -u root -p"$DB_PASS"

3.3 SQLite

# Safely backup a SQLite database
sqlite3 /data/app.db ".backup '/backup/app_$(date +%Y%m%d).db'"

# Restore
sqlite3 /data/app.db ".restore '/backup/app_20260615.db'"

4. Docker Volume Backups

Docker named volumes are the standard way to persist data for containerized services. Back them up with a temporary sidecar container — no agent installation inside the container needed:

#!/bin/bash
# backup-docker-volume.sh — run from cron
VOLUME_NAME="pgdata"
BACKUP_DIR="/backup/docker-volumes"

mkdir -p "$BACKUP_DIR"

docker run --rm \
  -v ${VOLUME_NAME}:/source:ro \
  -v ${BACKUP_DIR}:/dest \
  alpine:3.19 \
  tar -czf /dest/${VOLUME_NAME}_$(date +%Y%m%d_%H%M%S).tar.gz \
    -C /source .

echo "Volume ${VOLUME_NAME} backed up to ${BACKUP_DIR}"

To restore a volume:

#!/bin/bash
# restore-docker-volume.sh
VOLUME_NAME="pgdata"
BACKUP_FILE="/backup/docker-volumes/pgdata_20260615_030000.tar.gz"

docker run --rm \
  -v ${VOLUME_NAME}:/dest \
  -v $(dirname ${BACKUP_FILE}):/source:ro \
  alpine:3.19 \
  tar -xzf /source/$(basename ${BACKUP_FILE}) \
    -C /dest

⚡ Pro Tip: Volume Labels for Discovery

If you run many services, use Docker volume labels to identify which volumes belong to which service. In your docker-compose.yml:

volumes:
  pgdata:
    labels:
      "backup.enabled": "true"
      "backup.schedule": "daily"

Then script your backup tool to iterate over volumes with the backup.enabled=true label — no hardcoded volume lists needed.

5. Encryption — Protect Your Off-Site Backups

Off-site backups are stored on infrastructure you do not control. Without encryption, a compromised cloud provider account or a stolen backup drive leaks all of your data. Encrypt before the backup leaves your server.

5.1 Using GPG for Symmetric Encryption

# Encrypt
gpg --symmetric --cipher AES256 --no-symkey-cache \
  --passphrase-file /root/.backup-passphrase \
  -o backup.tar.gz.gpg backup.tar.gz

# Decrypt
gpg --decrypt \
  --passphrase-file /root/.backup-passphrase \
  -o backup.tar.gz backup.tar.gz.gpg

5.2 Using age (Modern Alternative)

# Generate a key
age-keygen -o /root/.backup-age-key.txt

# Encrypt
age -e -r $(cat /root/.backup-age-key.txt | grep -oP '(?<=public key: ).*') \
  -o backup.tar.gz.age backup.tar.gz

# Decrypt
age -d -i /root/.backup-age-key.txt \
  -o backup.tar.gz backup.tar.gz.age

🔐 Security Rule: Keys and Data Separate

Never store your encryption passphrase or private key on the same storage medium that holds the encrypted backups. If your local drive fails and you stored the key on that same drive, the off-site backup is unrecoverable. Keep the key in a password manager or print it and store it in a physical safe.

6. Off-Site Sync with Rclone

Rclone is the Swiss Army knife of cloud storage sync. It supports 40+ providers — Backblaze B2, Wasabi, AWS S3, Google Cloud Storage, S3-compatible MinIO, and more. Set it up once, then encrypt and sync in a single pipeline step:

# Configure rclone (one-time)
rclone config

# Sync encrypted backups to Backblaze B2
rclone copy /backup/encrypted/ \
  remote:bucket-name/selfhost-backups/ \
  --progress --verbose

# Sync to a local NAS as well (second medium)
rclone copy /backup/encrypted/ \
  nas:/mnt/backup/selfhost/ \
  --progress --verbose

This fulfills the "2 different media" rule: local disk + cloud storage, with the cloud copy satisfying the "1 off-site" requirement.

7. Retention Policy and Rotation

You do not need to keep every backup forever. A reasonable retention policy for self-hosters:

Frequency Retention Purpose
Hourly 24 hours Rollback from a recent mistake
Daily 30 days Standard recovery window
Weekly 3 months Mid-range incident recovery
Monthly 12 months Audit compliance, long-term archival

Use a tool like logrotate or write a simple cleanup script:

#!/bin/bash
# cleanup — delete backups older than retention period
find /backup/daily -name "*.tar.gz*" -mtime +30 -delete
find /backup/weekly -name "*.tar.gz*" -mtime +90 -delete
find /backup/monthly -name "*.tar.gz*" -mtime +365 -delete

8. Recovery Testing — The Most Important Step

A backup that has never been restored is not a backup — it is a placebo. The first time you attempt a restore should not be during an actual disaster. Schedule regular recovery drills:

I learned this lesson the hard way. Two years ago, a Watchtower auto-update on my Dell homelab pulled a new minor version of my Gitea container that had an incompatible SQLite schema migration. The container started, ran the migration, then crashed — and the crash left the .db file in an unrecoverable state. My last BorgBackup archive was from 3 AM that same morning, so I was able to borg extract the gitea.db from the archive, copy it back into the volume, and pin the container to the previous image tag. Total downtime: 12 minutes. If I had not been running hourly BorgBackup snapshots with integrity verification, I would have lost years of repository history, issue tracker entries, and wiki pages. Now I test a file-level restore from Borg at the start of every month — and I have caught two other subtle issues (a stale encryption passphrase and a Borg repository running out of disk space) during those drills, not during emergencies.

Automate restore verification where possible:

#!/bin/bash
# test-restore.sh — run in CI or cron
# 1. Restore database to a temporary Docker container
docker run --rm -d --name test-restore \
  -e POSTGRES_PASSWORD=test \
  postgres:16-alpine

# 2. Wait for PostgreSQL to be ready
sleep 5

# 3. Restore the latest backup
gunzip -c /backup/postgres/latest.sql.gz \
  | docker exec -i test-restore psql -U postgres

# 4. Run sanity queries
docker exec test-restore psql -U postgres -c "SELECT count(*) FROM information_schema.tables;"

# 5. Cleanup
docker stop test-restore

echo "Restore test completed at $(date)"

9. Full Automated Pipeline Example

Here is a complete backup script that ties everything together. Save it as /usr/local/bin/backup-pipeline.sh and add it to cron:

#!/bin/bash
set -euo pipefail

BACKUP_DIR="/backup"
DATE=$(date +%Y%m%d_%H%M%S)
ENCRYPT_KEY="/root/.backup-passphrase"

# Ensure directories exist
mkdir -p ${BACKUP_DIR}/{postgres,mysql,docker- volumes,encrypted}

# ——— PostgreSQL Dump ———
docker exec db pg_dump -U myapp myapp_db \
  | gzip > ${BACKUP_DIR}/postgres/myapp_${DATE}.sql.gz

# ——— Docker Volume Backup ———
for vol in pgdata redis-data; do
  docker run --rm -v ${vol}:/source:ro \
    -v ${BACKUP_DIR}/docker-volumes:/dest \
    alpine tar -czf /dest/${vol}_${DATE}.tar.gz -C /source .
done

# ——— Compress Everything Into One Archive ———
tar -czf ${BACKUP_DIR}/full_${DATE}.tar.gz \
  -C ${BACKUP_DIR}/postgres . \
  -C ${BACKUP_DIR}/docker-volumes .

# ——— Encrypt ———
gpg --symmetric --cipher AES256 --batch \
  --passphrase-file ${ENCRYPT_KEY} \
  -o ${BACKUP_DIR}/encrypted/full_${DATE}.tar.gz.gpg \
  ${BACKUP_DIR}/full_${DATE}.tar.gz

# ——— Sync to Off-Site ———
rclone copy ${BACKUP_DIR}/encrypted/ \
  remote:bucket/selfhost/${DATE}/ \
  --quiet

# ——— Clean Up Old Backups ———
find ${BACKUP_DIR}/encrypted -name "*.tar.gz.gpg" -mtime +30 -delete
find ${BACKUP_DIR}/postgres -name "*.sql.gz" -mtime +7 -delete

# ——— Send Notification ———
curl -s -X POST "https://ntfy.sh/selfhost-backups" \
  -d "Backup completed: ${DATE} (size: $(du -sh ${BACKUP_DIR}/encrypted/full_${DATE}.tar.gz.gpg | cut -f1))"

echo "✅ Backup pipeline finished at ${DATE}"

📡 Notification Channels

The script above uses ntfy.sh (a free, open-source push notification service) to alert you on completion. For failure alerts, add a trap: trap 'curl .../selfhost-backups-failed' ERR. You can also pipe through msmtp for email, or use a Slack/Discord webhook.

10. Common Mistakes and How to Avoid Them

❌ Mistake 1: Backing Up to the Same Drive

If your backup target is the same physical drive as your data, a drive failure destroys both. Always use a separate device: a USB drive, a NAS, or cloud storage.

❌ Mistake 2: Ignoring Backup Logs

A silent failure is worse than no backup — you think you are protected, but the archive is empty or corrupt. Monitor backup logs and set up failure notifications.

❌ Mistake 3: No Encryption for Off-Site Backups

Sending unencrypted data to the cloud means anyone who gains access to that bucket can read your passwords, API keys, and personal data. Always encrypt before upload.

❌ Mistake 4: Never Testing Restores

If you have never restored from your backup, you do not know whether it works. Subtle issues — wrong compression format, missing dependencies, expired encryption keys — only surface during an actual restore attempt. Discover them during a drill, not during a crisis.

Summary

A reliable automatic backup strategy is not optional when you self-host — it is the line between a minor inconvenience and catastrophic data loss. The 3-2-1 rule gives you a framework: three copies, two media, one off-site. Cron automation makes it hands-free. Database-native tools ensure consistency. Docker sidecar containers snapshot volumes cleanly. Encryption protects your off-site copies. And regular recovery testing turns confidence into certainty.

You now know how to:

Set up your backup pipeline today. The time you invest now is the time you save when the disk dies at 2 AM on a Saturday.

FAQ

What is the 3-2-1 backup rule?

The 3-2-1 backup rule states you should keep at least 3 copies of your data, store them on at least 2 different storage media types, and have at least 1 copy stored off-site. This is the industry-standard guideline for data protection and has been recommended by organizations including the US-CERT and major cloud providers.

How do I automate backups with cron?

Use cron jobs to schedule backup scripts. Add entries via crontab -e with timings like 0 3 * * * (daily at 3 AM). The script should dump databases, compress data, encrypt the archive, and sync it to off-site storage. Always include error notifications so you know if a backup fails.

How do I backup Docker volumes?

Use a temporary sidecar container that mounts the volume and creates a compressed archive. For example: docker run --rm -v myvolume:/data -v $(pwd):/backup alpine tar -czf /backup/volume-backup.tar.gz -C /data .. Schedule this as part of your automated backup pipeline via cron.

Should I encrypt my backups?

Yes, always encrypt backups before transferring them off-site. Use GPG symmetric encryption or tools like age (a modern alternative). Encrypt on the source machine before upload so the backup data is never readable by the storage provider. Store encryption keys securely, separately from the backups themselves.

How often should I test backup restoration?

Test restoration at least quarterly, or monthly for critical services. A backup that has never been tested is not a backup — it is a wish. Restore to a staging environment, verify data integrity, run application health checks, and document the procedure. Automated restore drills can be scripted in CI/CD pipelines.

← Back to Home

Related Articles

About the Author

Frankie is a self-hosting enthusiast and DevOps practitioner with over 8 years of experience deploying and maintaining production Linux servers, Docker environments, and homelab infrastructure. He runs a three-machine homelab — a Dell PowerEdge T340 (Debian), a RockPro64 ARM SBC (Alpine), and a WSL2 workstation (Ubuntu) — and relies on rsync + cron + BorgBackup to keep them backed up. He believes the best backup strategy is the one you actually test, and the second-best is the one you set up before you need it.