This PR updates existing backends to implement the `IRenameEnabledBackend` interface, allowing for more efficient file renaming operations. The implementation varies by backend, with some using native rename operations and others falling back to copy-and-delete patterns.
This PR adds support for locking files if the backend supports it.
To activate locking, set the option `--file-lock-duration=30D` and the backup will lock the files.
If the database is rebuilt with the intention of continuing the backups, use the option `--repair-refresh-lock-info` which will update lock information in the database after recreating the database.
The locking works by asking the backend to lock files after a backup has completed.
The implementation keeps track of which files are currently assigned a lock and prevents attempting to delete the files that are currently locked.
Note that the bucket should not have a default lock policy as Duplicati needs to finish the backup before the locking is applied.
In this initial version, Azure Blob Storage, B2, S3 and iDrive are supported with locking.
The CLI is updated to allow setting locks on a specific version. The backend tool is updated to allow setting locks on specific files.
This PR adds a dynamic property so a backend can signal if it supports streaming, based on the settings.
This is currently used for the File backend, so that toggling `--use-move-for-put` will disable streaming on the backend instead of relying on the `--disable-streaming-transfers` flag.
This PR moves the check from the FTP backend into shared code, so all backends will now check for read/write permissions when using the "Test" button.
For situations where write permissions are not required (like restore or verify) there is a flag on the v2 API that indicates if the connection was established, so the UI can show a message suggesting that it is possible to proceed, if the current UI operation does not require write permissions.
This fixes#2473
This adds support for Azure Access Tiers, similar to AWS Storage Class.
This update adds the option to set a custom storage class, as well as the option to define which storage classes are considered archive classes.
- Removed obsolete Microsoft.Azure.* references which were deprecated and replaced with Azure.Storage.*
- Cleaned up unused references
- Changes to code due to obsolete classes
- Patched some missing ConfigureAwait(false) on calls that could freeze the UI
- Merged in new async List changes.
This PR is the final change, where all backends are now purely implemented with async interfaces.
The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.
In many places, the backeds were already prepared for async output, in other some glue was needed.
To reduce the scope of this change, some backends simply report a synchronous result as an async list.
* Added options to insert delays in backend tester
* Added wait delays to Azure and GoogleDrive to see it the CI is more stable with small delays
* Added more debug output to the backend tester
* Fixed dropbox reporting OAuth quota errors, when it is likely Dropbox API quota errors
* Implemented exception detection on AzureBlob backend
* Also showing sleep times in output
* Fixed a bug with the dropbox quota response
* Fixed error message to report correct number of files
* Fixed suppressed warning
Without await, the using statement can dispose the Stream before the
call to PutAsync completes, resulting in an ObjectDisposedException.
This fixes#4556.
In pull request #4443, we renamed some of the Azure and S3 options to
use dashes '-' instead of underscores '_' for consistency. However, our
treatment of the options in the backend constructors still required that
the underscore versions be present even if the dashed version was
provided.
Now, we allow for either option to be present, but the value for the
dashed version will take precedence.
Allow users to authenticate using a Shared access signature (SAS) token. SAS tokens are more secure because they:
* have a start and expiry date/time
* can be restricted to specified IP addresses
* can limit access scope to a single Storage blob container
The lines with null-coalescing operator (??) throw an exception when the dictionary does not contain the first key (eg azure_account_name). Exception thrown: "The given key was not present in the dictionary".
By changing to IEnumerable, it is possible to iterate only a portion of the list, which is useful when not all entries are needed (e.g., when testing a connection).
All existing backends have been updated, and any which were able to be changed to yield return results in a straightforward way now do.
Many backends had a try/catch in the List() method. Due to the fact that yield returns can't be placed within a try/catch block, these have been refactored to either scope the try/catch to the parts that (should) be the only places throwing exceptions, so that exceptions are still caught and handled.
Note that lazy evaluation may cause some changes in behavior - exceptions that were previously thrown at the point of invokation of List() may now be thrown while it is being enumerated.
I believe this will not be problematic though, as the only well-known exception seems to be FolderMissingException, which should be thrown by Test(), but TestList() attempts to enumerate the list to force this exception.
Any places that require the legacy behavior can get it by simply converting the lazy enumerable to a List()
Kenneth Skovhede