This PR is the final change, where all backends are now purely implemented with async interfaces.
The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.
In many places, the backeds were already prepared for async output, in other some glue was needed.
To reduce the scope of this change, some backends simply report a synchronous result as an async list.
* Added options to insert delays in backend tester
* Added wait delays to Azure and GoogleDrive to see it the CI is more stable with small delays
* Added more debug output to the backend tester
* Fixed dropbox reporting OAuth quota errors, when it is likely Dropbox API quota errors
* Implemented exception detection on AzureBlob backend
* Also showing sleep times in output
* Fixed a bug with the dropbox quota response
* Fixed error message to report correct number of files
* Fixed suppressed warning
Without await, the using statement can dispose the Stream before the
call to PutAsync completes, resulting in an ObjectDisposedException.
This fixes#4556.
In pull request #4443, we renamed some of the Azure and S3 options to
use dashes '-' instead of underscores '_' for consistency. However, our
treatment of the options in the backend constructors still required that
the underscore versions be present even if the dashed version was
provided.
Now, we allow for either option to be present, but the value for the
dashed version will take precedence.
Allow users to authenticate using a Shared access signature (SAS) token. SAS tokens are more secure because they:
* have a start and expiry date/time
* can be restricted to specified IP addresses
* can limit access scope to a single Storage blob container
The lines with null-coalescing operator (??) throw an exception when the dictionary does not contain the first key (eg azure_account_name). Exception thrown: "The given key was not present in the dictionary".
By changing to IEnumerable, it is possible to iterate only a portion of the list, which is useful when not all entries are needed (e.g., when testing a connection).
All existing backends have been updated, and any which were able to be changed to yield return results in a straightforward way now do.
Many backends had a try/catch in the List() method. Due to the fact that yield returns can't be placed within a try/catch block, these have been refactored to either scope the try/catch to the parts that (should) be the only places throwing exceptions, so that exceptions are still caught and handled.
Note that lazy evaluation may cause some changes in behavior - exceptions that were previously thrown at the point of invokation of List() may now be thrown while it is being enumerated.
I believe this will not be problematic though, as the only well-known exception seems to be FolderMissingException, which should be thrown by Test(), but TestList() attempts to enumerate the list to force this exception.
Any places that require the legacy behavior can get it by simply converting the lazy enumerable to a List()
Kenneth Skovhede