Adds error-correction (parity) data for remote data volumes so they can be
repaired after bit-rot or corruption on the backend, addressing issue #314
(and reviving the intent of the stale draft PR #4574, redesigned for the
current architecture).
A pluggable IParity module system is introduced, mirroring the existing
compression/encryption modules (IParity + ParityLoader + a new
Duplicati.Library.Parity project). The default Par2Parity implementation
shells out to the "par2" (par2cmdline) program. par2 emits an index file
plus recovery volume files; these are concatenated into a single parity
file (the PAR2 format is a stream of self-describing packets, so this is a
valid PAR2 file). All par2 operations run in a temp working directory using
a fixed canonical name so create and repair stay symmetric despite differing
temp file names.
Behavior:
- Upload (Put): after a data volume (dblock/dlist) is uploaded, a parity
companion "<name>.par2" is generated from the final encrypted bytes and
uploaded unencrypted and untracked, inline on the held backend. dindex is
skipped (regenerable). Generated post-upload so it always keys off the
settled filename (retry/rename safe).
- Download (Get): on a hash/size mismatch, the parity companion is fetched
and used to repair the file in place before decryption. This single choke
point covers Restore/Compact/Test/Repair.
- Delete: the parity companion is removed (best-effort) with its data volume.
- Parsing: parity files are recognized by ParsedVolume.Parse (avoiding the
greedy encryption-group trap) and excluded from remote-list reconciliation
so they are never flagged as extra.
Parity is opt-in via --parity-redundancy-level (>0), --parity-module
(default par2), with --no-parity and par2-specific options. It is best-effort:
if the par2 program is absent the backup still succeeds with a single warning,
and repair falls through to the normal hash-mismatch error. Not tracked in the
database and requires no schema migration; the parity name is always derivable
from the data volume name.
Adds NUnit tests: parity filename generation/parsing and loader discovery
(no external dependency), plus end-to-end Par2Parity create/verify/repair
tests that are skipped when the par2 program is not available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
