This enables a fail-early approach, where the remote source loads the initial results.
By storing the `IAsyncEnumerable` it is possible to not perform any additional remote calls, even with the initial probing call.
This makes it possible to detect the absence of sources, so the backup can be aborted if sources are missing.
This fixes the restore issues as all sources will now appear as if they were originally part of the source file system.
The backend implementation has now been simplified so the backends only return names of entries with no path information. Most backends already do this, but the S3 backends did not, so this was fixed as well.
This also clears up the interfaces, so the backend does not know about the `ISourceProviderEntry`. All mapping is done by the `BackendSourceProvider`, making it simpler to support other backends as sources.
This PR is the final change, where all backends are now purely implemented with async interfaces.
The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.
In many places, the backeds were already prepared for async output, in other some glue was needed.
To reduce the scope of this change, some backends simply report a synchronous result as an async list.
This change will allow an environment variable READ_WRITE_TIMEOUT_SECONDS to control the read write timeout used during the tests on streaming backends.
-1 will set it to infinite.
Additionally all console logs during test execution have a timestamp with millisecond precision to aid in benchmarking.
Added support for `--portable-mode` and `--server-datafolder` for the CLI, ServerUtil, and Agent.
Fixed a few places where file reads were not cached properly.
Moved some responsibilities of UpdaterManager into DataFolderManager.
This now supports storing `machineid.txt` and `installation.txt` inside the path pointed to by `--server-datafolder` or alternatively with `--portable-mode`.
It is possible that some fringe backwards compatibility is lost with this update.
Fixed an issue with AutoUpdater crashing on check/download.
This fixes#5902
All CLI tools and executables now accept a unified set of help command variants:
"help", "/help","--help", "usage", "/usage", "--usage", "/h", "-h"
Affected executables:
Duplicati.Agent
Duplicati.CommandLine
Duplicati.CommandLine.AutoUpdater
Duplicati.CommandLine.BackendTester
Duplicati.CommandLine.BackendTool
Duplicati.CommandLine.ConfigurationImporter
Duplicati.CommandLine.SecretTool
Duplicati.CommandLine.Snapshots
Duplicati.GUI.TrayIcon
Duplicati.Server
Duplicati.Service
Duplicati.WindowsService
Note that SharpAesCrypt command is not included in this because the parameter parsing is part of the SharpAesCrypt code base.
* Added options to insert delays in backend tester
* Added wait delays to Azure and GoogleDrive to see it the CI is more stable with small delays
* Added more debug output to the backend tester
* Fixed dropbox reporting OAuth quota errors, when it is likely Dropbox API quota errors
* Implemented exception detection on AzureBlob backend
* Also showing sleep times in output
* Fixed a bug with the dropbox quota response
* Fixed error message to report correct number of files
* Fixed suppressed warning
* Fixed issues with timeout in msgraph
Fixed a place where the cancellation token was not applied.
Fixed a problem with msgraph not sending correct DNS information.
Fixed a problem with msgraph downloading the entire file before streaming it, bypassing throttle and tripping the timeout.
Added `ITimeoutExemptBackend` to selectively disable timeouts for backends that do not support it.
Updated the backendmanager to support `ITimeoutExemptBackend`.
Updated the backendmanager to abort as soon as a timeout happens.
* Updated BackendTester to use the `TimeoutObservingStream` and support the `--read-write-timeout` setting.
Updated backend tester code to use `var`
* Fixed a bug reading the `read-write-timeout`
* Updated to using constants with names instead of `-1`.
* Implemented support for emitting intermediate certificates from pfx files
* Added certificate check to crash earlier
* Fixed loading certificate on Windows
Avoids setting deprecated `Thread.CurrentCulture` and `Thread.CurrentUICulture`, using `CultureInfo.CurrentCulture` instead.
Fixed a few legacy places where incorrect string compares were used.
Kenneth Skovhede