Backup Solutions

Backup Solutions

Introduction

Regular backups are essential for protecting your data from hardware failures, accidental deletions, or system corruption. MidnightBSD provides several built-in backup tools and supports many third-party solutions through mports. This guide covers various backup methods to help you implement a robust backup strategy tailored to your needs.

Backup solutions range from simple file copying to sophisticated, encrypted, deduplicated backups to remote servers. The right approach depends on your data volume, sensitivity, and recovery requirements.

Backup Strategy Basics

Follow the 3-2-1 rule for critical data:

Consider these factors when choosing backup tools:

Built-in Tools

dump and restore

The dump and restore utilities are traditional Unix tools for backing up entire file systems. They are included in the MidnightBSD base system and work at the file system level.

Key features:

Basic usage:

# Full backup of a file system
# dump -0 -f backupfile.dump /dev/ada0s1a

# Incremental backup (levels 1-9)
# dump -1 -f backupfile.dump /dev/ada0s1a

# Restore from backup
# cd /destination
# restore -r -f backupfile.dump

For more details, see the Dump & Restore documentation.

cpdup

cpdup is a fast file copying utility designed for efficient directory synchronization. It's particularly useful for maintaining mirrors of file systems or creating backup copies with minimal overhead.

Key features:

cpdup is included in the MidnightBSD base system, so no package installation is needed. Confirm it is available with command -v cpdup.

Basic usage:

# Local backup
cpdup /source/directory /backup/directory

# Backup to a remote server via SSH
cpdup -v /local/directory user@remotehost:/backup/directory

# Dry run (show what would be copied)
cpdup -v -n /source /backup

Common options:

Automated backup script with cpdup:

#!/bin/sh
# Backup home directories to external drive
BACKUP_SRC="/home"
BACKUP_DEST="/mnt/backup/home"
LOG_FILE="/var/log/cpdup_backup.log"

# Ensure backup destination is mounted
if ! mount | grep -q "$BACKUP_DEST"; then
    mount /dev/da1s1 /mnt/backup
    if [ $? -ne 0 ]; then
        echo "Failed to mount backup drive" | logger
        exit 1
    fi
fi

# Run the backup
echo "Starting cpdup backup: $(date)" >> "$LOG_FILE"
cpdup -v "$BACKUP_SRC" "$BACKUP_DEST" >> "$LOG_FILE" 2>&1

# Unmount backup drive
umount /mnt/backup

echo "Backup completed: $(date)" >> "$LOG_FILE"
rsync

rsync is a versatile file synchronization tool that efficiently transfers and synchronizes files between directories, local drives, or remote servers. It's widely used for backups due to its efficiency and flexibility.

Key features:

Installation:

# mport install rsync

Basic usage:

# Local directory sync
rsync -av /source/directory/ /backup/directory/

# Remote backup via SSH
rsync -av -e ssh /local/source/ user@remote:/backup/destination/

# Dry run (show what would be transferred)
rsync -avn /source/ /backup/

# Compressed transfer
rsync -avz /source/ remote:/backup/

Common options:

Excluding files and directories:

# Exclude specific files
rsync -av --exclude='*.tmp' --exclude='*.bak' /source/ /backup/

# Exclude using a file
rsync -av --exclude-from='/etc/rsync_exclude.txt' /source/ /backup/

Incremental backup with rsync:

#!/bin/sh
# Incremental backup with date-stamped directories
BACKUP_SRC="/home"
BACKUP_DEST="/backup/home"
DATE=$(date +%Y%m%d_%H%M%S)

# Create date-stamped backup directory
mkdir -p "${BACKUP_DEST}/${DATE}"

# Run rsync with compression and progress
rsync -avzP --delete "${BACKUP_SRC}/" "${BACKUP_DEST}/${DATE}/"

# Create latest symlink for easy access
rm -f "${BACKUP_DEST}/latest"
ln -s "${BACKUP_DEST}/${DATE}" "${BACKUP_DEST}/latest"

Backup to external drive with rsync:

#!/bin/sh
EXTERNAL_MOUNT="/mnt/external"
BACKUP_SRC="/home"
LOG_FILE="/var/log/rsync_backup.log"

# Mount external drive
mount /dev/da0s1 "$EXTERNAL_MOUNT"
if [ $? -ne 0 ]; then
    echo "Failed to mount external drive" | logger
    exit 1
fi

# Run backup
echo "Starting rsync backup: $(date)" >> "$LOG_FILE"
rsync -av --delete --stats "$BACKUP_SRC/" "$EXTERNAL_MOUNT/backup/" >> "$LOG_FILE" 2>&1

# Unmount external drive
umount "$EXTERNAL_MOUNT"

echo "Backup completed: $(date)" >> "$LOG_FILE"

Third-Party Tools via mports

Restic

restic is a modern backup program that is fast, efficient, and secure. It supports deduplication, compression, and encrypted backups to various storage backends.

Key features:

Installation:

# mport install restic

Initializing a repository:

# Create a local repository
restic -r /backup/restic-repo init

# Create a repository on a remote server via SFTP
restic -r sftp:user@remotehost:/backup/restic-repo init

# Create a repository on Amazon S3
restic -r s3:s3.amazonaws.com/bucket-name init

Basic backup and restore:

# Backup a directory
restic -r /backup/restic-repo backup /home

# List snapshots
restic -r /backup/restic-repo snapshots

# Restore latest snapshot to /restore
restic -r /backup/restic-repo restore latest --target /restore

# Mount a repository for browsing
restic -r /backup/restic-repo mount /mnt/restic &

Automating with restic:

#!/bin/sh
# Automated restic backup script
REPO="/backup/restic-repo"
BACKUP_DIRS="/home /etc /var"
PASSWORD_FILE="/root/.restic-password"

# Set environment variables for password
export RESTIC_PASSWORD_FILE="$PASSWORD_FILE"

# Run backup for each directory
for dir in $BACKUP_DIRS; do
    restic -r "$REPO" backup "$dir" --verbose
    if [ $? -ne 0 ]; then
        echo "Backup failed for $dir" | logger
    fi
done

# Forget old snapshots (keep last 30 days)
restic -r "$REPO" forget --keep-daily 30 --group-by paths,tags

# Check repository integrity
restic -r "$REPO" check --read-data-subset=10%

Backup to remote server:

# Backup to a remote server via SFTP
restic -r sftp:user@backup-server:/backups/midnightbsd backup /home /etc

# Store the password in a root-only file instead of shell history
chmod 600 /root/.restic-password
export RESTIC_PASSWORD_FILE=/root/.restic-password
restic -r sftp:user@backup-server:/backups/midnightbsd backup /home
Tarsnap

tarsnap is a secure, efficient online backup service for Unix-like operating systems. It uses Amazon S3 for storage and provides client-side deduplication, compression, and encryption.

Key features:

Installation:

# mport install tarsnap

Setup and configuration:

# Register and create account at https://www.tarsnap.com/
# After registration, you'll receive an account key

# Generate the key directly at the protected destination
mkdir -p /root/.tarsnap
tarsnap-keygen --keyfile /root/.tarsnap/keyfile \
    --user your-email@example.com --machine mymidnightbsd
chmod 600 /root/.tarsnap/keyfile

Basic usage:

# Create a backup archive
# tarsnap -c -f backup-$(date +%Y%m%d) /etc /home /var

# More efficient: use cache directory for deduplication
tarsnap --cachedir /var/cache/tarsnap -c -f backup-$(date +%Y%m%d) /etc /home /var

# List all archives
tarsnap --list-archives

# Restore files
tarsnap -x -f backup-20250101 -C /restore etc/passwd

# Delete old archives
tarsnap -d -f backup-20241201

Automated backup script:

#!/bin/sh
# Automated tarsnap backup
CACHE_DIR="/var/cache/tarsnap"
LOG_FILE="/var/log/tarsnap_backup.log"
BACKUP_NAME="midnightbsd-$(date +%Y%m%d-%H%M%S)"

# Create cache directory if it doesn't exist
mkdir -p "$CACHE_DIR"

# Run backup
echo "Starting tarsnap backup: $(date)" >> "$LOG_FILE"
tarsnap --cachedir "$CACHE_DIR" -c -f "$BACKUP_NAME" /etc /home /usr/local/etc >> "$LOG_FILE" 2>&1

# List archives
echo "Current archives:" >> "$LOG_FILE"
tarsnap --list-archives >> "$LOG_FILE" 2>&1

echo "Backup completed: $(date)" >> "$LOG_FILE"
Duplicity and Duply

duplicity creates encrypted, bandwidth-efficient incremental backups. duply is a profile-based front end that makes duplicity configuration and recurring operations easier.

Key features:

Installation:

# mport install duplicity duply

Basic Duplicity usage:

# Create a full backup, followed by incrementals on later runs
duplicity full /home file:///backup/duplicity
duplicity incremental /home file:///backup/duplicity

# Inspect, verify, restore, and expire old backup sets
duplicity collection-status file:///backup/duplicity
duplicity verify file:///backup/duplicity /home
duplicity restore file:///backup/duplicity /restore/home
duplicity remove-older-than 30D --force file:///backup/duplicity

Using Duply profiles:

# Create ~/.duply/home/conf, then edit source, target, and GPG settings
duply home create
chmod 600 ~/.duply/home/conf

duply home backup
duply home status
duply home restore /restore/home

Keep the profile and any referenced passphrase file readable only by the backup account. Do not put passphrases directly in scripts, command lines, or source control.

BorgBackup

borg is a deduplicating backup program that supports compression, authentication, and encryption. It's optimized for fast and efficient backups with powerful search and restore capabilities.

Key features:

Installation:

# mport install py312-borgbackup

Basic usage:

# Initialize a repository
borg init --encryption=repokey /backup/borg-repo

# Create a backup
borg create --stats /backup/borg-repo::home-$(date +%Y%m%d) /home

# List archives
borg list /backup/borg-repo

# Extract files from an archive
borg extract /backup/borg-repo::home-20250101

# Mount repository for browsing
borg mount /backup/borg-repo /mnt/borg

Backup with encryption:

# Create encrypted repository
borg init --encryption=keyfile /backup/borg-repo

# Read the passphrase from a protected file
chmod 600 /root/.borg-passphrase
BORG_PASSCOMMAND="cat /root/.borg-passphrase" borg create /backup/borg-repo::mybackup /home

Automated borg backup:

#!/bin/sh
# Automated borg backup
REPO="/backup/borg-repo"
PASS_FILE="/root/.borg-passphrase"
LOG_FILE="/var/log/borg_backup.log"
ARCHIVE_NAME="midnightbsd-{now:%Y-%m-%d_%H-%M-%S}"

# Run backup
echo "Starting borg backup: $(date)" >> "$LOG_FILE"
BORG_PASSCOMMAND="cat $PASS_FILE" borg create --stats --progress \
    "$REPO::$ARCHIVE_NAME" \
    /home /etc /usr/local/etc >> "$LOG_FILE" 2>&1

# Prune old archives (keep daily for 7 days, weekly for 4 weeks, monthly for 6 months)
BORG_PASSCOMMAND="cat $PASS_FILE" borg prune --stats \
    --keep-daily 7 --keep-weekly 4 --keep-monthly 6 \
    "$REPO" >> "$LOG_FILE" 2>&1

# Check repository integrity
BORG_PASSCOMMAND="cat $PASS_FILE" borg check "$REPO" >> "$LOG_FILE" 2>&1

echo "Backup completed: $(date)" >> "$LOG_FILE"
Other backup packages

The current mports tree also provides these packages:

# Install only the tools needed for your backup design
mport install flashback zetaback
mport install bacula9-client bacula9-server
mport install backuppc

Consult each package's installed documentation before enabling services; Bacula and BackupPC require site-specific access controls, storage paths, and retention policies.

Offsite Backup Strategies

Offsite backups are crucial for disaster recovery. Here are several approaches for backing up data to remote locations:

SSH-based Backups

Using SSH for secure remote backups is a common and reliable approach:

# Using rsync over SSH
rsync -av -e "ssh -p 2222" /home/user/ user@remote:/backup/home

# Using cpdup over SSH
cpdup -v /local/source user@remote:/backup/destination

# Using tar over SSH
tar czf - /home | ssh user@remote "cat > /backup/home-$(date +%Y%m%d).tar.gz"

SSH key-based authentication:

# Generate SSH key on backup server
ssh-keygen -t ed25519 -f /root/.ssh/backup_key

# Copy public key to remote server
ssh-copy-id -i /root/.ssh/backup_key.pub user@remote

# Use specific key for backup scripts
rsync -av -e "ssh -i /root/.ssh/backup_key" /home user@remote:/backup
Cloud Storage Backups

Many tools support cloud storage providers for offsite backups:

Restic with cloud providers:

# Amazon S3
restic -r s3:s3.amazonaws.com/bucket-name backup /home

# Backblaze B2
restic -r b2:bucket-name backup /home

# Microsoft Azure
restic -r azure:container-name backup /home

# Google Cloud Storage
restic -r gs:bucket-name backup /home

Configuration for cloud providers:

# Configure credentials with the provider's authenticated CLI or a root-only file,
# then export them from that protected store before invoking restic.
restic -r s3:s3.amazonaws.com/bucket-name backup /home
Network Attached Storage (NAS)

Using NAS devices for backups:

# Mount NAS share
mount -t nfs nas-server:/volume1/backups /mnt/nas

# Backup using rsync
rsync -av /home /mnt/nas/midnightbsd-backup

# Unmount after backup
umount /mnt/nas
External Hard Drives

Rotating external drives provide a simple offsite solution:

# List available disks
dmesg | grep da

# Mount external drive
mount /dev/da0s1 /mnt/backup

# Run backup
rsync -av --delete /home /mnt/backup/home-$(date +%Y%m%d)

# Unmount when done
umount /mnt/backup

Automated external drive backup:

#!/bin/sh
# Rotating drive backup script
MOUNT_POINT="/mnt/backup"
LOG_FILE="/var/log/external_backup.log"

# Check for external drive
if ! ls /dev/da*; then
    echo "No external drive found" | logger
    exit 1
fi

# Get the first external drive
DRIVE=$(ls /dev/da* | head -1)

# Create mount point and mount
mkdir -p "$MOUNT_POINT"
mount "$DRIVE" "$MOUNT_POINT"
if [ $? -ne 0 ]; then
    echo "Failed to mount $DRIVE" | logger
    exit 1
fi

# Run backup
echo "Starting external drive backup: $(date)" >> "$LOG_FILE"
rsync -av --delete /home "$MOUNT_POINT/backup-$(date +%Y%m%d)" >> "$LOG_FILE" 2>&1

# Sync and unmount
sync
umount "$MOUNT_POINT"

echo "External backup completed: $(date)" >> "$LOG_FILE"

Automating Backups

Automating backups ensures they happen regularly and consistently. Here are several approaches:

Cron Jobs

Use cron for scheduled backups:

# Edit root's crontab
crontab -e

# Daily backup at 2:00 AM
0 2 * * * /usr/local/bin/daily-backup.sh >> /var/log/daily-backup.log 2>&1

# Weekly full backup on Sundays at 3:00 AM
0 3 * * 0 /usr/local/bin/weekly-backup.sh >> /var/log/weekly-backup.log 2>&1

# Monthly backup on 1st at 4:00 AM
0 4 1 * * /usr/local/bin/monthly-backup.sh >> /var/log/monthly-backup.log 2>&1

Cron examples for different backup tools:

# Daily rsync backup
0 2 * * * rsync -av --delete /home /mnt/backup/home >> /var/log/rsync.log 2>&1

# Weekly restic backup
0 3 * * 0 restic -r /backup/restic-repo backup /home /etc >> /var/log/restic.log 2>&1

# Monthly tarsnap backup
0 4 1 * * tarsnap -c -f monthly-$(date +%Y%m) /home /etc >> /var/log/tarsnap.log 2>&1
Backup Wrapper Script

A comprehensive backup wrapper script that handles multiple backup methods:

#!/bin/sh
# Comprehensive backup wrapper
BACKUP_CONFIG="/etc/backup.conf"
LOG_DIR="/var/log/backups"

# Load a root-owned configuration file (chmod 600 /etc/backup.conf)
[ -r "$BACKUP_CONFIG" ] || exit 1
. "$BACKUP_CONFIG"

# Create log directory
mkdir -p "$LOG_DIR"

# Function for logging
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "${LOG_DIR}/backup-$(date +%Y%m%d).log"
}

# Function to check if command exists
command_exists() {
    command -v "$1" >/dev/null 2>&1
}

# Run backups based on configuration
log "Starting backup sequence"

# If using restic
if [ "$USE_RESTIC" = "yes" ] && command_exists restic; then
    log "Running restic backup"
    set -- "$RESTIC_SOURCE_HOME" "$RESTIC_SOURCE_ETC" "$RESTIC_SOURCE_VAR"
    restic -r "$RESTIC_REPO" backup "$@" >> "${LOG_DIR}/restic.log" 2>&1
    restic -r "$RESTIC_REPO" forget --keep-daily "$RESTIC_KEEP_DAILY" >> "${LOG_DIR}/restic.log" 2>&1
fi

# If using rsync
if [ "$USE_RSYNC" = "yes" ] && command_exists rsync; then
    log "Running rsync backup"
    rsync -av --delete "$RSYNC_SOURCES" "$RSYNC_DEST" >> "${LOG_DIR}/rsync.log" 2>&1
fi

# If using cpdup
if [ "$USE_CPDUP" = "yes" ] && command_exists cpdup; then
    log "Running cpdup backup"
    cpdup "$CPDUP_SOURCES" "$CPDUP_DEST" >> "${LOG_DIR}/cpdup.log" 2>&1
fi

log "Backup sequence completed"

Sample configuration file (/etc/backup.conf):

# Enable/disable different backup methods
USE_RESTIC="yes"
USE_RSYNC="yes"
USE_CPDUP="no"

# Restic configuration
RESTIC_REPO="/backup/restic-repo"
RESTIC_SOURCE_HOME="/home"
RESTIC_SOURCE_ETC="/etc"
RESTIC_SOURCE_VAR="/var"
RESTIC_KEEP_DAILY=30

# Rsync configuration
RSYNC_SOURCES="/home"
RSYNC_DEST="/mnt/backup/home"

# Cpdup configuration
CPDUP_SOURCES="/home"
CPDUP_DEST="/mnt/backup/home"
Email Notifications

Set up email notifications for backup success/failure:

#!/bin/sh
# Backup with email notifications
LOG_FILE="/var/log/backup.log"
EMAIL="admin@example.com"

# Run backup
rsync -av /home /backup/home >> "$LOG_FILE" 2>&1

# Check if backup succeeded
if [ $? -eq 0 ]; then
    SUBJECT="Backup Success: $(date +%Y%m%d)"
    echo "Backup completed successfully" | mail -s "$SUBJECT" "$EMAIL"
else
    SUBJECT="Backup Failed: $(date +%Y%m%d)"
    echo "Backup failed. See $LOG_FILE for details" | mail -s "$SUBJECT" "$EMAIL"
    cat "$LOG_FILE" | mail -s "Backup Log: $(date +%Y%m%d)" "$EMAIL"
fi

Backup Verification and Testing

Regular verification and testing of backups is crucial to ensure data can be restored when needed.

Verifying Backup Integrity

Most backup tools provide commands to verify repository integrity:

# Restic repository check
restic -r /backup/restic-repo check

# Borg repository check
borg check /backup/borg-repo

# Duplicity backup verification
duplicity verify file:///backup/duplicity /home

# Tarsnap integrity check (list archives)
tarsnap --list-archives
Testing Restores

Regularly test restoring files to ensure your backups work:

# Test restore with restic
restic -r /backup/restic-repo restore latest --target /tmp/test-restore
ls -la /tmp/test-restore
rm -rf /tmp/test-restore

# Test restore with borg (extract runs relative to the current directory)
mkdir -p /tmp/test-restore/borg
(cd /tmp/test-restore/borg && borg extract /backup/borg-repo::latest)
ls -la /tmp/test-restore
rm -rf /tmp/test-restore

# Test restore with rsync (verify files exist)
rsync -av --dry-run /backup/home/ /tmp/test-restore/

Automated verification script:

#!/bin/sh
# Backup verification script
VERIFY_LOG="/var/log/backup-verify.log"
TEST_DIR="/tmp/backup-verify-$(date +%Y%m%d)"

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$VERIFY_LOG"
}

mkdir -p "$TEST_DIR"

# Test restic restore
if command -v restic >/dev/null 2>&1; then
    log "Testing restic restore"
    if restic -r /backup/restic-repo restore latest --target "$TEST_DIR/restic" >> "$VERIFY_LOG" 2>&1; then
        log "Restic restore: SUCCESS"
    else
        log "Restic restore: FAILED"
    fi
fi

# Test borg restore
if command -v borg >/dev/null 2>&1; then
    log "Testing borg restore"
    mkdir -p "$TEST_DIR/borg"
    if (cd "$TEST_DIR/borg" && borg extract /backup/borg-repo::latest) >> "$VERIFY_LOG" 2>&1; then
        log "Borg restore: SUCCESS"
    else
        log "Borg restore: FAILED"
    fi
fi

# Clean up
rm -rf "$TEST_DIR"
log "Verification completed"
Backup Log Analysis

Regularly review backup logs to catch issues early:

# Check recent backup logs
tail -n 50 /var/log/backup.log

# Search for errors
grep -i "error\|failed\|warning" /var/log/backup*.log

# Check disk space usage
du -sh /backup/*

# Check when backups were last run
ls -lt /var/log/backup*.log

Recovery Procedures

Knowing how to restore data is just as important as creating backups. Here are recovery procedures for different scenarios:

Single File Recovery

Recovering individual files from various backup types:

# From restic
restic -r /backup/restic-repo mount /mnt/restic
# Browse /mnt/restic to find your file, then copy it
cp /mnt/restic/latest/home/user/important-file /home/user/
umount /mnt/restic

# From borg
borg mount /backup/borg-repo /mnt/borg
cp /mnt/borg/latest/home/user/document.txt /home/user/
umount /mnt/borg

# From dump/restore
# Find the archive containing the file
ls -la /backup/dump/
# Mount dump archive and extract
# This typically requires knowing which dump file contains the file

# From rsync backups
cp /backup/home/user/missing-file /home/user/
Directory Recovery

Recovering entire directories:

# Using restic
restic -r /backup/restic-repo restore latest --target /restore --include /home/user

# Using borg (paths are extracted beneath the current directory)
mkdir -p /restore
(cd /restore && borg extract /backup/borg-repo::latest home/user)

# Using rsync
rsync -av /backup/home/user/ /restore/user/

# Using cpdup
cpdup /backup/home/user /restore/user
Full System Recovery

For complete system recovery, you'll need a recovery environment:

  1. Boot from MidnightBSD installation media or live CD
  2. Mount your backup destination (external drive, network share, etc.)
  3. Restore critical system files first (/etc, /root, etc.)
  4. Restore user data
  5. Reinstall any packages if needed
  6. Verify system configuration

Full system restore using restic:

# Boot from live media
# Mount backup repository (adjust for your setup)
mount /dev/da0s1 /mnt

# Restore system files
restic -r /mnt/restic-repo restore latest --target / --host midnightbsd

# For ZFS systems, restore to alternate location first
restic -r /mnt/restic-repo restore latest --target /mnt/restored

# Then manually handle ZFS pool imports and data restoration

Full system restore using dump:

# Boot from live media
# Mount backup drive
mount /dev/da0s1 /mnt

# Restore each file system
cd /mnt
restore -r -f /mnt/dump/root.dump
restore -r -f /mnt/dump/var.dump
restore -r -f /mnt/dump/usr.dump
restore -r -f /mnt/dump/home.dump

Disaster Recovery Checklist:

Recovery Documentation Template:

# System Recovery Documentation
# System: midnightbsd-server
# Last Updated: $(date)

## Backup Locations
- Local: /backup
- Remote: backup-server:/backups/midnightbsd
- Cloud: s3://my-backups/midnightbsd

## Backup Tools Used
- Primary: restic
- Secondary: rsync
- Configuration: /etc/backup.conf

## Recovery Procedures
1. Boot from MidnightBSD installation USB
2. Mount backup repository: mount /dev/da0s1 /mnt
3. Run: restic -r /mnt/restic-repo restore latest --target /restore
4. Copy critical files: cp -R /restore/etc /etc
5. Reboot and verify

## Important Files to Restore First
- /etc/rc.conf
- /etc/fstab
- /etc/ssh/sshd_config
- /etc/passwd
- /etc/group
- /home/*/.ssh/authorized_keys

## Contacts for Recovery Assistance
- Admin: admin@example.com (555-1234)
- Backup Provider: support@backup-service.com