Beta. This feature works and is being tested on more setups. Report anything that does not behave as described here.
On an S3-compatible destination, VMStack can lock every backup with S3 Object Lock for a number of days. Until the lock ends, nobody can delete or overwrite the backup through the panel or with the nodes' key, which protects backups against a stolen node key or ransomware on the panel.
This feature is beta: it works and is being tested with more S3 providers and setups. Report any problem you find.
Object Lock is available on S3 destinations only. AWS S3, Wasabi, Backblaze B2, Hetzner Object Storage and MinIO implement it; Cloudflare R2 does not (the check fails with object_lock_off).
Prepare the bucket#
- Create a bucket with Object Lock enabled. Most providers allow this only when the bucket is created; Object Lock also turns on versioning. Do not set a default retention on the bucket: VMStack sets the lock on each object.
- Create an access key limited to the bucket with the usual S3 permissions (see S3-compatible storage) plus
s3:ListBucketVersions,s3:GetBucketObjectLockConfiguration,s3:PutObjectRetentionands3:DeleteObjectVersion. - Never give that key
s3:BypassGovernanceRetention. The panel never sends a bypass, and the key the nodes hold should not be able to.
Turn it on#
-
Edit the S3 destination (or add one) under Backups › Destinations.
-
In the Immutable backups section, tick Lock every backup (S3 Object Lock).
-
Choose the Mode:
Mode What it means Governance (recommended) The nodes' key cannot delete a backup or shorten its lock. An account allowed to bypass governance retention (often the bucket's owner, in the provider's console) still can, so a backup kept by mistake can be removed Compliance Nobody can delete a backup or shorten its lock before it ends: not the bucket's owner, not the provider's support. Its storage is paid for until then. Choose it only when rules require it -
Set Locked for (days), 1 to 3650 (14 by default).
-
Save, then press Check. With the option on, the check adds the lines
object_lock(the bucket has Object Lock),versioning(on),retention(a small test object locked for one day, which stays in the bucket until that day ends) andlifecycle.
Every object of a backup (each disk and the manifest) gets the same retain-until date: the time of its first upload plus the days set. Changing the mode or the days applies to later backups only. The option can be turned on at any time but not turned off while the destination holds backups; to stop locking, add a new destination.
What the panel does with locked backups#
- The server's Backups tab and Backups › Restore show Locked with the date. Delete is greyed out with the reason, for administrators too: "This backup is immutable: its destination locks it (S3 Object Lock, governance mode) until … It can be deleted after that." The API answers 409
refusedwithlocked_until. - When retention, the maximum age or a deleted server's clean-up would delete a locked backup, it waits instead (audited once as
backup.delete_deferred). The hourly prune deletes it after the lock has ended. The card says "Expires when its lock ends, …" with the date. - Deletes remove every version and delete marker of the backup's objects, so the space is freed.
- A backup that failed or was cancelled during its upload is locked as well; its clean-up waits for the lock to end.
- With incremental backups, each incremental extends the lock of the earlier backups of its chain to its own date, so nothing an incremental needs can be deleted before it.
Restores and verifies of locked backups work as usual. The panel records the version id of every uploaded object and restores exactly those versions, so a newer version or a delete marker written later with a stolen key does not affect a restore.
Requirements#
Backups, checks and deletes on an immutable destination run only on nodes whose agent reports Object Lock support; other nodes are refused by name ("runs an agent that does not lock backups"). Restores and verifies have no such requirement. See Updating the agent.
Not supported: legal holds, extending the lock of existing backups, deleting locked backups early from the panel, and PBS "protected" snapshots.