Add Storj DCS backend and deprecated Tardigrade DCS backend.
Storj renamed their "Tardigrade Decentralized Cloud Storage" product to "Storj DCS". This adds a new Storj DCS backend and deprecates the old Tardigrade one. A warning is issued to inform Tardigrade users to modify their configurations to use the new backend.
This also adds a passphrase verification field to ensure that the user has provided the correct encryption passphrase.
This adds a method to get a default instance of AmazonS3Config. This
accomplishes two key things:
1. Previously, the documentation for the default buffer size (8192) was
inconsistent with the actual default value (65536). This was due to
differences in the property values betwen the instance used to
generate the documentation, and that used in the actual backend
client.
2. Disables the use of Amazon Resource Names (ARN). This does not seem
critical to our application. Moreover, if the UseArnRegion property
is not set, accessing it triggers an expensive operation (~30
seconds) to get the region endpoint. This causes the loading of the
backend modules to take a long time and run afoul of HTTP request
timeouts, causing the application to think that no backends are
available.
The previous version of the AWSSDK.S3 library that we used did not
have this property, so this was not an issue until we started
migrating away from .NET Framework.
This fixes#4543.
Without await, the using statement can dispose the Stream before the
call to PutAsync completes, resulting in an ObjectDisposedException.
This fixes#4556.
Before performing test write, reset stream Position to 0, otherwise test performs zero-byte write.
When performing test read, verify that file was written correctly.
Previously, we would catch any Exception thrown by VerifyRemoteList and
perform an auto-cleanup (if specified in the options). However, we
should only perform the auto-cleanup if VerifyRemoteList detected an
issue with the uploaded files. Otherwise, an unrelated exception can
cause the database repair to be performed unnecessarily. In the case of
a connection issue, this can leave the database in a corrupted state.
This fixes#4516.
.NET 5 moves the responsibility to the operating system, and since
SSH.NET currently relies on the ECDsaCng class, we will simply interpret
all exceptions to mean lack of ECDSA support.
It's possible that our test will no longer be valid (e.g., due to
changes in SSH.NET), so we don't want the backend to fail if our test
is no longer valid.
In pull request #4443, we renamed some of the Azure and S3 options to
use dashes '-' instead of underscores '_' for consistency. However, our
treatment of the options in the backend constructors still required that
the underscore versions be present even if the dashed version was
provided.
Now, we allow for either option to be present, but the value for the
dashed version will take precedence.
Allow users to authenticate using a Shared access signature (SAS) token. SAS tokens are more secure because they:
* have a start and expiry date/time
* can be restricted to specified IP addresses
* can limit access scope to a single Storage blob container
The lines with null-coalescing operator (??) throw an exception when the dictionary does not contain the first key (eg azure_account_name). Exception thrown: "The given key was not present in the dictionary".
By default, the dot (.) element matches every character _except_ the
newline character. This causes issues with files that have a newline
character in the name. For example, they are excluded when restoring
the containing directory.
By specifying the RegexOptions.Singleline option, the dot (.) element
will match every character, including the newline character.
This fixes#4508.
SSH.NET relies on the System.Security.Cryptography.ECDsaCng class for
ECDSA algorithms, which is not implemented in Mono (as of 6.12.0.144).
This prevents clients from connecting if one of the ECDSA algorithms is
chosen as the host key algorithm. In the event that this causes a
connection failure, we will prevent the client from advertising support
for ECDSA algorithms and make another connection attempt.
Related forum discussion:
https://forum.duplicati.com/t/release-2-0-6-1-beta-2021-sftp-failure-synology/12358
