Archive Plugins

11.9.1 Archive Plugins

Available since: replication-manager v3.1.45

An archive is where a cluster's backups go once they leave the backup directory: their versions, their retention and the way back for a restore or a reseed. In v3.1.45 a cluster no longer has one restic repository wired into its settings. It has a list of archives, each one made from an archive plugin with its own settings and secrets.

Each archive plugin declares:

• typed settings: string, int, bool, enum or secret. The add and edit forms of the Maintenance tab, and the checks of the API, are drawn from them.
• capabilities: what can be done with an archive of that plugin.

The plugins

Enterprise plugins can be added or attached on the Partner and Support plans. The Developer plan also gets them, for testing.

11.9.2 Archives of a cluster

• Names. An archive has a name you choose (archive-bso-s3, archive-rdem-pbs) and can rename: lowercase letters, digits, - and _, at most 64 characters, unique in the cluster. An archive added without a name gets archive-<type>-<n>, for example archive-local-1.
• Attached archive. A cluster sends its backups to one archive, named by backup-archive-name (empty for none). Switching archives means changing that one setting. The archive left is detached: it stays in the list with its content and its size, and restoring from it needs it attached again.
• Remove deletes the content. Removing an archive deletes its content, then takes it out of the list. Content left behind would still be invoiced and no longer visible. The confirmation shows the size deleted. Removing the attached archive detaches it first.
• Replacing a heavy archive: add archive-local-2, attach it, then remove archive-local-1.

Configuration

The archive list is a TOML array of tables in the cluster section. A plugin's settings and secrets are keys of its entry; secrets are stored encrypted and are never sent back by the API.

[mycluster]
backup-archive-name = "archive-bso-s3"

[[mycluster.archive]]
name   = "archive-local-1"
plugin = "archive-restic-local"

[[mycluster.archive]]
name   = "archive-bso-s3"
plugin = "archive-restic-s3"
aws-bucket   = "backups-paris"
aws-endpoint = "https://s3.fr-par.example"
aws-access-key-id = "..."
# password and aws-access-secret are secrets: set them from the GUI or the API

Migration from backup-restic-*

When it loads a cluster that has no archive list, replication-manager builds archive 1 from backup-archive-mode and the backup-restic-* settings:

• The password and the S3 secret move to that archive.
• With mode none and customised settings, the archive is kept but not attached, so nothing is lost.

The save writes the list instead of the legacy keys. Downgrading to v3.1.44 is not supported, and an active/standby pair must be upgraded together.

backup-archive-mode remains as a shortcut over the list: none detaches; another mode attaches an archive of that type, or creates one.

Retention

Retention is the cluster's, backup-keep-*: by default the last 10, plus 1 hourly, 1 daily, 4 weekly, 12 monthly and 2 yearly. The plugin only says how it applies it, through its purge-* settings.

• Tagged policies. backup-keep-tag-policies keeps snapshots that carry a tag by their own rules, counted apart from the backups. The default is trigger:binlogs keep-within=1d, for the binlog-only snapshots taken between backups. Snapshots taken since the last backup are always kept.
• Per-archive policies. An archive's purge-policies setting replaces the cluster's tagged policies for that archive.
• When the purge runs. backup-archive-purge-trigger is after-backup (each backup the archive receives) or schedule (on scheduler-archive-purge-cron).

Keeping the backup in the archive only

With backup-archive-remove-original = true, a backup is deleted from the backup directory once the archive's snapshot is checked to hold it. A restore or a reseed fetches it back from the archive first.

11.9.3 External plugins

An external archive plugin is a program named archiveplugin-<name>, run once per operation:

• the request goes on its stdin as one JSON document;
• the answer comes back on its stdout as one JSON document;
• progress is written on stderr;
• secrets are passed only in its environment (REPMAN_ARCHIVEPLUGIN_SECRET_*), never logged nor written in a file.

archiveplugin-<name> --manifest prints its manifests.

replication-manager looks for these programs in <working dir>/plugins, and for a source build in the plugins directory next to its binary. When plugin-signing-public-key is set, a program without a valid signature is left out, as for the log plugins (see Plugin Architecture).

The operations are probe, push, list, restore, forget, purge, size, remove and sync, as the plugin's capabilities allow.

plakar (archiveplugin-plakar)

Encrypted, deduplicated plakar archives, built on plakar's own engine (kloset).

Deleting a snapshot only marks it as deleted. The purge reclaims the space of packfiles that were marked more than 7 days earlier, so space comes back about a week later, on a later purge.

Proxmox Backup Server (archiveplugin-pbs)

Each cluster's backups go to one namespace of a PBS datastore.

• Encrypted by default. When a new archive is added without a key, its probe generates one, and replication-manager stores it as the archive's secret. Keep a copy of it with the configuration backup: without the key, the snapshots cannot be restored. An archive with no key and without no-encrypt raises WARN0263.
• One token per cluster. The plugin's scripts/pbs-cluster-token.sh creates the cluster's namespace and a token limited to it, so one cluster never reads or deletes another cluster's backups. pbs-cluster-allow-purge.sh lets that token prune its own backups.
• Retention. PBS prunes by counts. A backup-keep-within* window has no PBS equivalent, so that purge is refused rather than run with less than the policy. Space comes back with the datastore's garbage collection.

11.9.4 API

See Archiving for the restic settings and disk space management.