This PR adds settings to choose a Power Mode provider, which will detect if the system is in a paused/suspended state and avoid starting tasks while being suspended.
This currently only works for Windows, where the previous inplementation is named `NET` (for .NET) and the new mode is named `Native` and using the Windows documented approach with a hidden window that listens for `WM_POWERBROADCAST` messages.
The default is `Native` but can now also be disabled by choosing the `None` provider.
This updates VSS to default use Vanara in favor of AlphaVSS which is no longer maintained.
The build for Vanara requires targeting `net8.0-windows7.0`, which will cause significant build overhead and complexity for cross platform builds.
To counter this, the setup is to have a single project, `Duplicati.Library.WindowsModules`, that is targeting `net8.0-windows7.0`.
The output from this project is then hoisted into the TrayIcon project for Windows builds so the files are available when debugging on Windows.
A top-level dummy executable project is added to ensure the project always builds.
The built modules are then loaded with reflection when requested.
With the use of Vanara there is now also support for using BackupRead to read files without making a VSS snapshot.
With BackupRead, it is possible to read locked files, but it still requires the SeBackupPrivilege as VSS does as well.
Unfortunately, the `vssapi.dll` file is not shipped for Arm64 on Windows, so even with Vanara this will not work, and only WMIC is supported on Arm64.
A workaround is to run Duplicati with x64 emulation if more advanced VSS features are needed (HyperV and MSSQL support).
This PR also updates options and filters out unsupported options for each operating system, so options that are not supported by the current OS are not reported and will give warnings if they are used, as opposed to just being ignored.
The release builder project has been updated to exclude the unused project, and purge unwanted outputs.
This PR fixes the issues with not correctly loading and disposing modules.
Prior to this PR, there would be only a single instance of each module loaded, which would be configured and later disposed, meaning that the same instance would be used after being disposed, and disposed multiple times.
This mostly worked, but it could carry configuration details over between unrelated runs.
This fixes#6314
This PR moves the check from the FTP backend into shared code, so all backends will now check for read/write permissions when using the "Test" button.
For situations where write permissions are not required (like restore or verify) there is a flag on the v2 API that indicates if the connection was established, so the UI can show a message suggesting that it is possible to proceed, if the current UI operation does not require write permissions.
This fixes#2473
This repair will leave the database in a still-broken state, but if this is run prior to purge-broken-files, less data will be lost.
Also updated the purge-broken-files operation to keep files and directories in the set, if they are only missing metadata.
This commit adds the `DateTime` option to support validation of the `--time` parameter, which can both be an absolute date and time, or a relative value.
This fixes#6107
The code was using a two-way reference linking the database to the results and the results to the database.
However, this was not really used so it ended up just cluttering the code. With this commit, the results and databases are no longer keeping track of each other, and the only interaction is when the results are written to the database at the end of the operation.
To ensure result data is always written, this code has been moved into the controller, which will now flush the results for all operations. Prior to this, only some operations would write result data to the log.
The controller now also handles logic for cleaning old log data and invoking the vacuum command.
Fixed a bug due to using System.Text.Json where some attributes from NewtonSoft.Json were being applied and not picked up.
This enables a fail-early approach, where the remote source loads the initial results.
By storing the `IAsyncEnumerable` it is possible to not perform any additional remote calls, even with the initial probing call.
This makes it possible to detect the absence of sources, so the backup can be aborted if sources are missing.
This fixes the restore issues as all sources will now appear as if they were originally part of the source file system.
The backend implementation has now been simplified so the backends only return names of entries with no path information. Most backends already do this, but the S3 backends did not, so this was fixed as well.
This also clears up the interfaces, so the backend does not know about the `ISourceProviderEntry`. All mapping is done by the `BackendSourceProvider`, making it simpler to support other backends as sources.
This PR is the final change, where all backends are now purely implemented with async interfaces.
The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.
In many places, the backeds were already prepared for async output, in other some glue was needed.
To reduce the scope of this change, some backends simply report a synchronous result as an async list.
With this interface it is possible to wrap the current support for snapshots and USN and simplify the enumeration.
This makes a number of assumptions into fields such as `IsMetaEntry` and `IsRootEntry` that removes code that was checking for various properties.
This first commit has support for VSS and USN as well as "naked" files, keeping feature parity.
There is also an experimental support for loading a backend-as-a-source. The support for this is currently limited as the backends need to be extended to support a folder concept.
This will make listing buckets with large count of files more reliable, if a 429 response is received it will conform to retry-after headers or if not present (as is the case today) will default to a 5 second delay.
* Rewrite the backend manager.
This creates a new more logic backend manager that handles all backend operations.
For each top-level operation, there is now a single backend manager instance that is passed to sub-commands.
The internals of the backend manager are now rewritten to use async logic instead of threading.
The design uses a single task runner that dispatches operations and honors concurrent settings and limits from one place.
The logic for each operation has been moved into a separate files/classes to make it easier to understand each operation.
This fixes#5804
* Fixed a few issues with not awaiting tasks
* Fixed not creating empty databases
* Fixed an issue with hashes not being recorded on download
* Reworked the download logic to avoid attempting to decrypt the file if it has been modified.
* Review fixes
* Renamed db collector to better reflect the purpose.
* Review fixes
* Added test to verify that compression modules will mask duplicate entries, if present
* Because there are likely many, otherwise valid, dindex zip archives with duplicated blocklists out there, the log message is now verbose.
To make sure we no longer create archives with duplicated entries, the `--unittest-mode` option makes this a warning that is checked in most tests.
* Added guard in `IndexVolumeCreator` to prevent creating zip files with duplicated block lists.
* Added debug check for duplicates
* Fixed the duplicate test
* Fixed issues with timeout in msgraph
Fixed a place where the cancellation token was not applied.
Fixed a problem with msgraph not sending correct DNS information.
Fixed a problem with msgraph downloading the entire file before streaming it, bypassing throttle and tripping the timeout.
Added `ITimeoutExemptBackend` to selectively disable timeouts for backends that do not support it.
Updated the backendmanager to support `ITimeoutExemptBackend`.
Updated the backendmanager to abort as soon as a timeout happens.
* Updated BackendTester to use the `TimeoutObservingStream` and support the `--read-write-timeout` setting.
Updated backend tester code to use `var`
* Fixed a bug reading the `read-write-timeout`
* Updated to using constants with names instead of `-1`.
* Simplified task control by having all logic in a single class and using async control mechanism
* Fixed the tests
* Made the marking of a backup partial explicitly communicated if the enumeration process is stopped.
* Added support for pausing active transfers
* Introduced transfer token to abort operations.
* Updated the way LiveControl is emitting events.
* Retains the paused state of the server across reboots, this fixes#5760.
* Reworked the way throttle speeds are propagated through LiveControls.
* Added option to pause transfers in the UI.
* Removed throttle settings from the LiveControls class so it is exclusively handled by the ApplicationSettings.
* Removed `thread-priority` as it does not work at all when tasks are used.
* Fixed the stop dialog and API to only support stop and abort.
* Removed unused variables
None of these changes have impact on code execution, they are mainly part of the effort to remove all build warnings.
The removal of Thread.ResetAbort leaves a TODO for us to integrate CancelationToken when it gets propagated down here on future changes.
Kept the infrastructure to allow re-introducing it later if a better source for a device key is found.
Added support for not having any valid settings encryption key.
Added warnings if the user is running without a settings encryption key.
Added additional checks during startup to help with troubleshooting and configuring a setup with a settings encryption key.
Docker builds will now warn if no encryption key is set.
This fixes#5518
Kenneth Skovhede