When creating a new controller, the options dictionary was used by-reference.
This meant that code could update the options dictionary *after* creating the controller, which would not be thread-safe.
It does not have much effect on regular operations as the normal method is to create the dictionary, pass it, and not modify it.
But in the tests, we frequently re-use the dictionary and update it with new values to test various cases.
Due to the dictionary being a reference, the controller would update the dictionary to fit the operations, but this would reflect back into the tests, meaning that some tests were being run with options that were not supposed to be there.
This meant that at least two edge cases that we test for were never found to have errors, even though they do.
The first issue is the listing of files using a non-default blocksize. The second issue is recreating the database with a non-default file-hash algorithm.
This PR removes the reference passing, and fixes the issues that were uncovered.
This PR has a large blast radius because it takes the final step and bumps up the Controller to be fully async.
We have historically done a piece-by-piece update, so all operations were already async but the controller interface was kept synchronous.
With this update, the controller is now fully async and all tests are updated.
Most places where the new C# compiler warns about function names not ending in `Async` were also adressed, giving a massive refactor change.
Functionally, no changes are done.
When performing queries with large inputs, there is a possibility that we reach the maximum number of possible parameters. If the query exceeds the maximum number of parameters the query fails.
This change uses a temporary table that will be created in the case the inputs exceed the total number available parameters, making sure the calls proceed as before with small inputs, but uses temporary tables on larger inputs.
Some tests were added to ensure the update works as expected, even with large inputs.
Renamed the options to be more descriptive.
Rewrote the dependency query to be more readable and also check for metadata dependency.
Fixed an edge case where there were no modified files, and no new backup is created.
Fixed some issues with local/utc datetime compares.
Fixed logic for deleting filesets to only look at the filesets themselves. This makes the version lock follow the dlist file, and can create "dangling" volumes that have no filesets, but these will be picked up by compaction.
Updated tests to work correctly.
Added lock-info update option for the backup recreate call, so the UI can also re-create lock information on database rebuilds.
This PR adds support for locking files if the backend supports it.
To activate locking, set the option `--file-lock-duration=30D` and the backup will lock the files.
If the database is rebuilt with the intention of continuing the backups, use the option `--repair-refresh-lock-info` which will update lock information in the database after recreating the database.
The locking works by asking the backend to lock files after a backup has completed.
The implementation keeps track of which files are currently assigned a lock and prevents attempting to delete the files that are currently locked.
Note that the bucket should not have a default lock policy as Duplicati needs to finish the backup before the locking is applied.
In this initial version, Azure Blob Storage, B2, S3 and iDrive are supported with locking.
The CLI is updated to allow setting locks on a specific version. The backend tool is updated to allow setting locks on specific files.
Kenneth Skovhede