This adds a common way to catch stop exceptions and handles them so error resilient logic does not continue when a stop exception is thrown.
Previously, the logic would only check for `ThreadAbortException` which was used to signal a stop prior to .NET8.
This PR adds a unified way of supplying timeouts to each of the backends.
It also adds an opt-in way of adding commonly used options, so parsing and usage can be standardized across multiple backends.
This adds a retry count and a retry delay for posting backup reports via HTTP requests. This will make the report delivery more reliable if the network is unstable or server is briefly unreachable.
Initially the idea was to allow scanning not only the certificate but scan the whole certificate chain for a explicitly supplied hash to be accepted unconditionally.
A review of the validation revealed it was always missing a fallback to the certificate.verify()
Also decided that even when a hash is supplied and matched, the validity date supersedes other criteria except the acceptAll.
It should be noted that I was unable to completely remove IPv4 on Windows 11 to a state where it would not provide the 127.0.0.1 loopback, even when disabling Ipv4 in various ways, including with registry flags.
On ubuntu/linux, after completely disabling ipv4 the ipv6 loopback was returned.
This removes the use of the `CallContext` substitute, which was a compatibility class from .NET4 times, and replaces it with `AsyncLocal` which retains the call state.
This enables a fail-early approach, where the remote source loads the initial results.
By storing the `IAsyncEnumerable` it is possible to not perform any additional remote calls, even with the initial probing call.
This makes it possible to detect the absence of sources, so the backup can be aborted if sources are missing.
This fixes the restore issues as all sources will now appear as if they were originally part of the source file system.
The backend implementation has now been simplified so the backends only return names of entries with no path information. Most backends already do this, but the S3 backends did not, so this was fixed as well.
This also clears up the interfaces, so the backend does not know about the `ISourceProviderEntry`. All mapping is done by the `BackendSourceProvider`, making it simpler to support other backends as sources.
Made the SMBShareConnection slightly more robust.
Introduced v2 List API to S3 backend, currently it retains v1 as the default.
Changed S3 backends to not recurse, potentially making smaller listings. This option can be disabled if it breaks things.
Added support for relative paths in the SSH backend.
With this interface it is possible to wrap the current support for snapshots and USN and simplify the enumeration.
This makes a number of assumptions into fields such as `IsMetaEntry` and `IsRootEntry` that removes code that was checking for various properties.
This first commit has support for VSS and USN as well as "naked" files, keeping feature parity.
There is also an experimental support for loading a backend-as-a-source. The support for this is currently limited as the backends need to be extended to support a folder concept.
Added support for `--portable-mode` and `--server-datafolder` for the CLI, ServerUtil, and Agent.
Fixed a few places where file reads were not cached properly.
Moved some responsibilities of UpdaterManager into DataFolderManager.
This now supports storing `machineid.txt` and `installation.txt` inside the path pointed to by `--server-datafolder` or alternatively with `--portable-mode`.
It is possible that some fringe backwards compatibility is lost with this update.
Fixed an issue with AutoUpdater crashing on check/download.
This fixes#5902
* Updated reporting values.
The reporter module can now emit the following extra values:
- operating-system
- installation-type
- destination-type
- next-scheduled-run
When running from the built-in scheduler, it will automatically set `--next-scheduled-run` so it is applied to the reporting data.
* Implemented an improved machine name reader.
This fixes#5907
* Added a guard against sending too much of the destination url as the destination.
* Fixed reading Windows Machine name
* Removed extra Windows logic as it has no effect
* Rewrite the backend manager.
This creates a new more logic backend manager that handles all backend operations.
For each top-level operation, there is now a single backend manager instance that is passed to sub-commands.
The internals of the backend manager are now rewritten to use async logic instead of threading.
The design uses a single task runner that dispatches operations and honors concurrent settings and limits from one place.
The logic for each operation has been moved into a separate files/classes to make it easier to understand each operation.
This fixes#5804
* Fixed a few issues with not awaiting tasks
* Fixed not creating empty databases
* Fixed an issue with hashes not being recorded on download
* Reworked the download logic to avoid attempting to decrypt the file if it has been modified.
* Review fixes
* Renamed db collector to better reflect the purpose.
* Review fixes
All CLI tools and executables now accept a unified set of help command variants:
"help", "/help","--help", "usage", "/usage", "--usage", "/h", "-h"
Affected executables:
Duplicati.Agent
Duplicati.CommandLine
Duplicati.CommandLine.AutoUpdater
Duplicati.CommandLine.BackendTester
Duplicati.CommandLine.BackendTool
Duplicati.CommandLine.ConfigurationImporter
Duplicati.CommandLine.SecretTool
Duplicati.CommandLine.Snapshots
Duplicati.GUI.TrayIcon
Duplicati.Server
Duplicati.Service
Duplicati.WindowsService
Note that SharpAesCrypt command is not included in this because the parameter parsing is part of the SharpAesCrypt code base.
* Fixed nullability warnings in `Utility.cs`
* Fixed a number of nullability issues and warnings
* More nullable fixes
* Fixed the overwrite issue
* Fixed additional nullability issues
* Removed additional nullability issues
* More nullability fixes and added some xmldoc
* Added more xmldocs and removed an unused argument
* Removed more warnings
* Simplified task control by having all logic in a single class and using async control mechanism
* Fixed the tests
* Made the marking of a backup partial explicitly communicated if the enumeration process is stopped.
* Added support for pausing active transfers
* Introduced transfer token to abort operations.
* Updated the way LiveControl is emitting events.
* Retains the paused state of the server across reboots, this fixes#5760.
* Reworked the way throttle speeds are propagated through LiveControls.
* Added option to pause transfers in the UI.
* Removed throttle settings from the LiveControls class so it is exclusively handled by the ApplicationSettings.
* Removed `thread-priority` as it does not work at all when tasks are used.
* Fixed the stop dialog and API to only support stop and abort.
* Removed unused variables
* Implemented support for emitting intermediate certificates from pfx files
* Added certificate check to crash earlier
* Fixed loading certificate on Windows
Kenneth Skovhede