This PR is a security hardning change and will break some setups.
The hardning is done to ensure the datafolder is not compromised and particularly that it is not maliciously seeded.
The changes now **requires** that the datafolder is locked down with exact permissions, or Duplicati will refuse to use it. This is a change to prior functionality where the folder would simply be locked down if it was not already locked.
Previously, a file named `insecure-permissions.txt` could be placed in the folder to instruct Duplicati not to set permissions. With this change that file is no longer supported.
The only way to opt-out of the permission check is to supply `--allow-insecure-datafolder` or the environment variable `DUPLICATI__ALLOW_INSECURE_DATAFOLDER`. Once either of those are set, Duplicati will no longer check permissions and accept folders with other permissions.
The setting can be applied to `preload.json`, but the file needs to be in a trusted location, which is either the directory with the binaries or the file pointed to by `DUPLICATI_PRELOAD_SETTINGS`. The setting cannot be placed in a `preload.json` file that is in an insecure folder as the folder is not read if the permissions are not correct.
The previous paths `/usr/local/share/Duplicati/preload.json` and `C:\ProgramData\Duplicati\preload.json` are no longer supported as they cannot be guaranteed to be locked down.
A `preload.json` in the datafolder is still supported, provided the folder passes the permission check (or the `--allow-insecure-datafolder` is passed to the executable).
For most users, this should not cause any problems as Duplicati has been locking down the folders, so they should be correctly locked down already.
The configuretool has been updated with a `secure-datafolder` command that can be used to force the correct permissions on a folder, in case the folder did not have the correct permissions.
For some Docker setups it may not be possible to set the permissions and the check will fail. These setup will need to apply `DUPLICATI__ALLOW_INSECURE_DATAFOLDER=true` in the image to run without the protections.
This PR has a large blast radius because it takes the final step and bumps up the Controller to be fully async.
We have historically done a piece-by-piece update, so all operations were already async but the controller interface was kept synchronous.
With this update, the controller is now fully async and all tests are updated.
Most places where the new C# compiler warns about function names not ending in `Async` were also adressed, giving a massive refactor change.
Functionally, no changes are done.
Adds two new commands to the database CLI tool:
- verify: Lists all databases with their status (Found, Missing, Orphaned)
across dbconfig.json, server database, and filesystem
- cleanup: Removes orphaned database files not referenced in dbconfig.json
or server database, with --dry-run and --force options
Includes unit tests for both commands.
Kenneth Skovhede