This PR adds a unified way of supplying timeouts to each of the backends.
It also adds an opt-in way of adding commonly used options, so parsing and usage can be standardized across multiple backends.
This implementation ensures the exception for FolderMissingException is treated properly in async enum circumventing the limitation of not having a yield within a try catch block.
- Removed obsolete Microsoft.Azure.* references which were deprecated and replaced with Azure.Storage.*
- Cleaned up unused references
- Changes to code due to obsolete classes
- Patched some missing ConfigureAwait(false) on calls that could freeze the UI
- Merged in new async List changes.
This fixes the restore issues as all sources will now appear as if they were originally part of the source file system.
The backend implementation has now been simplified so the backends only return names of entries with no path information. Most backends already do this, but the S3 backends did not, so this was fixed as well.
This also clears up the interfaces, so the backend does not know about the `ISourceProviderEntry`. All mapping is done by the `BackendSourceProvider`, making it simpler to support other backends as sources.
Made the SMBShareConnection slightly more robust.
Introduced v2 List API to S3 backend, currently it retains v1 as the default.
Changed S3 backends to not recurse, potentially making smaller listings. This option can be disabled if it breaks things.
Added support for relative paths in the SSH backend.
This PR is the final change, where all backends are now purely implemented with async interfaces.
The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.
In many places, the backeds were already prepared for async output, in other some glue was needed.
To reduce the scope of this change, some backends simply report a synchronous result as an async list.
With this interface it is possible to wrap the current support for snapshots and USN and simplify the enumeration.
This makes a number of assumptions into fields such as `IsMetaEntry` and `IsRootEntry` that removes code that was checking for various properties.
This first commit has support for VSS and USN as well as "naked" files, keeping feature parity.
There is also an experimental support for loading a backend-as-a-source. The support for this is currently limited as the backends need to be extended to support a folder concept.
This will make listing buckets with large count of files more reliable, if a 429 response is received it will conform to retry-after headers or if not present (as is the case today) will default to a 5 second delay.
* Removing timeout from Webdav
The timeout will be controlled externally by the TimeoutObservingStream and or the cancellation token.
Timeout for List set to default of 10 minutes
Since filesystems may take quite sometime to list large number of files, the timeout was set to a high value.
Previous timeout of 30 seconds was triggering in a case reported by user.
Timeout for CreateFolder and DeleteAsync was kept to default of 30 seconds.
Minor formmating/refactoring was done.
* Removing timeout from Webdav
The timeout will be controlled externally by the TimeoutObservingStream and or the cancellation token.
Timeout for List set to default of 10 minutes
Since filesystems may take quite sometime to list large number of files, the timeout was set to a high value.
Previous timeout of 30 seconds was triggering in a case reported by user.
Timeout for CreateFolder and DeleteAsync was kept to default of 30 seconds.
Minor formating/refactoring was done.
* Fixes to Integrated Authentication
This is a fix to the webdav when using integrated authentication, the hint for the underlying issue was found on an old [thread] (https://github.com/dotnet/wcf/issues/2923). By setting up a vm environment with Windows 2022 Server+IIS/Webdav it was possible to reproduce the [issue](https://github.com/duplicati/duplicati/issues/5872) and fix it.
Change to validation script is to avoid the requirement of filling the username when integrated authentication is true.
Modernize HTTP client implementation across backend services by:
- Replace legacy HttpWebClient with modern HttpClient
- Implement proper async/await patterns throughout the codebase
- Add cancellation token support for improved request handling
- Refactor related code for better maintainability
This change is part of our broader initiative to modernize the codebase
and improve its resilience through proper asynchronous operations.
* Fixed nullability warnings in `Utility.cs`
* Fixed a number of nullability issues and warnings
* More nullable fixes
* Fixed the overwrite issue
* Fixed additional nullability issues
* Removed additional nullability issues
* More nullability fixes and added some xmldoc
* Added more xmldocs and removed an unused argument
* Removed more warnings
Adds support for user-configurable read and write buffer sizes while enforcing protocol-negotiated maximums. When not specified, operations default to using the maximum sizes negotiated during the SMB session setup.
Tests revealed that by not using a single connection throttling issues appeared when connecting to Windows shares (possibly due to multiple authentications in a short period of time)
The 'no-reuse-connection' option controls whether SMB/CIFS connections are reused:
When enabled, creates a new connection and authentication for each file operation
When not specified, defaults to reusing the same connection across operations
* Primary functionality implemented for CIFS
This will add CIFS support to Duplicati.
One of the advantages is that the resource does not need to be mounted as the backend connects and authenticates directly to the resource via TCP/or Netbios over TCP.
The authentication supported is domain/username/password. At present integrated Windows authentication is not supported.
This implementation relies on SMBLibrary library (https://github.com/TalAloni/SMBLibrary).
* Fixes to XML documentation
* Implementing single CIFS connection + Fix on PutAsync
Single CIFS connection to avoid re-authentication on every operation.
Fix to PutAsync which only because apparent after file sizes were larger than _smb2Client.MaxWriteSize
* Adding CI tests for CIFS
The CI test for CIFS is self contained, requiring no environment variables. Also updated README.
This introduces a few constants in favor of multiple strings with the same value.
The supported arguments in the module have been augmented to better report possible values and default values.
The options parser has been updated to work better with CLI where the options are not default filled.
Kenneth Skovhede