Every system and every device needs backups! Proxmox has a stunning server Proxmox Backup Server (later PBS). It's highly recommended and a great tool for automation of critical backups. The PBS makes use of the power of ZFS.

Installation

The installation is pretty easy on real HW, it's not recommended to run it as a VM for production nodes. You can test it as a VM on Proxmox it will work.

The installation process is straight forward and well documented.

Your main task is to create Backup Strategies for different groups of VM's running in the cluster. There might be legal or organizational need to take into account. Many workflows dictate the frequency of backups needed.

Planning

The most important task is the planning 👈

ZFS in the best choice for small to medium sized clusters

• Backup strategy - where are the backups stored onsite and off site link
• Backup tactics - which VM/CT's will be backed up and when, why, how
• Housekeeping strategy: garbage collection, pruning and verification
• Storage size, depends on number and size of VM's and frequency and retention
• FS and its options will determine many things a bow

The backup frequency

For some applications you need very rapid backups that usually are stored on the node that deleted files can be quickly restored or a roll back easily performed.

Other applications do only change at update and need mostly a manual backup job but a weekly or monthly run just in case.

You can group backups by type and create one job doing the hourly ond another to do the weekly and one for the daily ones. Whatever your use case is.

Backup Retention of snapshots

The retention of old backups depends on how often data changes and how important an older state may be in a specific workload.

For this example, we assume that you are doing daily backups, have a retention period of 10 years, and the period between backups stored gradually grows.

• keep-last: 3 - even if only daily backups, an admin may want to create an extra one just before or after a big upgrade. Setting keep-last ensures this.
• keep-hourly: not set - for daily backups this is not relevant. You cover extra manual backups already, with keep-last.
• keep-daily: 13 - together with keep-last, which covers at least one day, this ensures that you have at least two weeks of backups.
• keep-weekly: 8 - ensures that you have at least two full months of weekly backups.
• keep-monthly: 11 - together with the previous keep settings, this ensures that you have at least a year of monthly backups.
• keep-yearly: 9 - this is for the long term archive. As you covered the current year with the previous options, you would set this to nine for the remaining ones, giving you a total of at least 10 years of coverage.

Backup Schedule

Make a schedule for different backups to run. It's not nice tho have your backup starting during peek hours. Your network and your backup server can't do everything at once. Remember to keep the network clean for production apps to run freely, cap PBS use of the network accordingly.

Backup Proxmox nodes

It's possible to backup the nodes by PBS, it's setup is a bit manual. Do you need it - maybe, maybe not. Host backups are explained in the documentation.
A quick and dirty way is to backup your /etc/pve folder and /etc/network/interfaces file. Should be enough to personalize a new install in no time.

Hardware

• 1G NIC's you should bond them for speed ....
• 10G NIC's - Excellent
• SAS drives - Excellent
• Spinning rust - Good for large storage pool, backups are large
• SSD's needed for L2ARC and ZIL. For backup hmmmm

Time Synchronization

The Proxmox VE cluster stack itself relies heavily on the fact that all the nodes have precisely synchronized time. Some other components, like Ceph, also won’t work properly if the local time on all nodes is not in sync.

See my post Start using Proxmox for how to.

Mail alerts - they are mandatory  

Backup servers are critical components, more so then the servers they serve. It's mandatory to activate email alerts. You need to setup email on the server.

See my post Start using Proxmox for how to setup email alerts.

Configuring the Proxmox Backup Server

Your focus will be on setting up the housekeeping tasks for the server: Prune Garbage Collection and Verification. It is done per Datastore.

You need to create a strategy on database level how to run backups.
Ansering the W -method: what to backup, when to backup and why  to keep backups and how long to keep them

Prune and Garbage Collection

These are the first two housekeeping tasks to setup

• GC Schedule: 22:00
• Prune Schedule: 21:00
  - Set the Namespace and depth as needed and use the retention strategy

Verification

Verification of backups is an extremely important housekeeping job.
You need to know if the backup is OK or is it affected by bit-rot.

• Verify Job Schedule: 20:00

Cluster backup Jobs

Cluster (or for a standalone PVE) Backup Jobs are declared at the Datacenter level. You create the backup jobs for your cluster by defining which nodes and which VM's to backup and when and how to do it.

Per Job declare which nodes All or by Include, Exclude or as Pool based. What Storage to use and what is the schedule. Set the backup mode Snapshot of the VM, or to Suspend or Stop the VM first. The Job is Enabled as default.

Remember to:

• Comment your jobs so you know what they are years from now
  {{cluster}}, {{guestname}}, {{node}}, {{vmid}} and/or free text.
• Configure email allerts to Who and On failure only or Notify always
• Retention: Without any keep option, the storage's configuration or node's vzdump.conf is used as fallback
• Write a Note for the template for other or future you

Host or File-level backups

For Debian based systems using the Proxmox Backup Client. For basic backups and to access files in the backup on your laptop.

sudo nano /etc/apt/sources.list.d/pbs-client-list
# Add this line
deb http://download/proxmox.com/debian/pbs-client bullseye main

sudo wget http;//enterprise.proxmox.com/debian/proxmox-release-bullseye.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg

sudo apt update && sudo apt install proxmox-backup-client -y

Add an environment variable and an alias

It's good to add an environment variable to your .bashrc: PBS:datastore
export PBS_REPOSITORY=192.168.1.100:backup otherwise you need to write a long string. And also add an alias alias pbs='proxmox-backup-client'

Example

This example shows having the environment variable and the alias in the .bashrc. Run as root to backup everything from a VM or PC (/) to a PBS datastore (backup). You will need the PBS password for root@pam.

pbs backup root.pxar:/ --repository 192.168.1.100:backup

Using file/level backups on your PC

You may want to mount a file-level backup to a computer at /mnt/mybackup.
This way you can get to single files in the backup.

• List your backups proxmox-backup-server list or pbs list
• copy the last snapshot string for the host you want to mount  
• mount it by pbs mount <last snapshot string> root.pxar /mnt/mybackup

Use a remote PBS

You can run several PBS systems for your cluster. By syncing remotes to the main unit you have redundancy. You can also make a redundant backup cluster in the form of a X type  (A -> B and B -> A) or O type (A -> B -> C -> A) .

References

Proxmox documentation ^[1] PBS wiki page for more info ^[2] Backup nodes or other machines using the client ^[3] ZFS on Linux, requrements and howto ^[4] Proxmox Backup Server and Client Git ^[5]