This PR is the final change, where all backends are now purely implemented with async interfaces.
The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.
In many places, the backeds were already prepared for async output, in other some glue was needed.
To reduce the scope of this change, some backends simply report a synchronous result as an async list.
* Removing timeout from Webdav
The timeout will be controlled externally by the TimeoutObservingStream and or the cancellation token.
Timeout for List set to default of 10 minutes
Since filesystems may take quite sometime to list large number of files, the timeout was set to a high value.
Previous timeout of 30 seconds was triggering in a case reported by user.
Timeout for CreateFolder and DeleteAsync was kept to default of 30 seconds.
Minor formmating/refactoring was done.
* Removing timeout from Webdav
The timeout will be controlled externally by the TimeoutObservingStream and or the cancellation token.
Timeout for List set to default of 10 minutes
Since filesystems may take quite sometime to list large number of files, the timeout was set to a high value.
Previous timeout of 30 seconds was triggering in a case reported by user.
Timeout for CreateFolder and DeleteAsync was kept to default of 30 seconds.
Minor formating/refactoring was done.
* Fixes to Integrated Authentication
This is a fix to the webdav when using integrated authentication, the hint for the underlying issue was found on an old [thread] (https://github.com/dotnet/wcf/issues/2923). By setting up a vm environment with Windows 2022 Server+IIS/Webdav it was possible to reproduce the [issue](https://github.com/duplicati/duplicati/issues/5872) and fix it.
Change to validation script is to avoid the requirement of filling the username when integrated authentication is true.
This is a proposal for replacing the deprecated WebClient with the HttpClient.
This commit implements the core functionality following the HttpClientFactory pattern as suggested by the framework. It also supports an alternative approach in areas where the HttpClient pattern is not desirable.
The first backend component to be migrated is WebDav, and the HttpClient pattern is also applied to the update downloader.
Once merged, other backend components can be migrated, eventually removing the deprecated WebClient from the codebase.
Additionally, the debug-profind-file option from webdav, which was a remnant from early development, is removed as it is no longer useful for debugging different webdav server responses.
Without await, the using statement can dispose the Stream before the
call to PutAsync completes, resulting in an ObjectDisposedException.
This fixes#4556.
By changing to IEnumerable, it is possible to iterate only a portion of the list, which is useful when not all entries are needed (e.g., when testing a connection).
All existing backends have been updated, and any which were able to be changed to yield return results in a straightforward way now do.
Many backends had a try/catch in the List() method. Due to the fact that yield returns can't be placed within a try/catch block, these have been refactored to either scope the try/catch to the parts that (should) be the only places throwing exceptions, so that exceptions are still caught and handled.
Note that lazy evaluation may cause some changes in behavior - exceptions that were previously thrown at the point of invokation of List() may now be thrown while it is being enumerated.
I believe this will not be problematic though, as the only well-known exception seems to be FolderMissingException, which should be thrown by Test(), but TestList() attempts to enumerate the list to force this exception.
Any places that require the legacy behavior can get it by simply converting the lazy enumerable to a List()
InvariantCulture is useful when comparing / sorting human language strings in a culturely correct way. It handles things like accented letters in a way that makes sense to humans (e.g., 'a' should be sorted next to 'á', rather than after 'z').
Ordinal looks just at the raw code points of the characters. As such, it is recommended for use in cases when comparing system strings (file paths, command line parameters, config settings, etc.). Since it doesn't need to use the culture specific sorting rules, this method can often be faster.
For more information, see https://stackoverflow.com/questions/492799/difference-between-invariantculture-and-ordinal-string-comparison (and other related questions)
Kenneth Skovhede