Commit Graph
1002 Commits
Author SHA1 Message Date
Valentin c2dc922866 Mega Strings const review 2021-07-05 21:10:23 +02:00
Valentin eda3666bc7 Add support for mega.nz two fa authentication 2021-06-26 13:39:05 +02:00
Serge 5404fed634 Do not use StreamWriter 2021-06-02 19:35:40 +10:00
Serge 955dbebc03 Check contents of written test file
Before performing test write, reset stream Position to 0, otherwise test performs zero-byte write.
When performing test read, verify that file was written correctly.
2021-06-01 02:51:42 +10:00
warwickmmandGitHub ed86cca3e4 Merge pull request #4509 from sergethedev17/feature/azure-storage-sastoken
Add ability to provide SAS token for Azure backend
2021-05-30 10:15:43 -07:00
Kenneth SkovhedeandGitHub 8ee90fdf64 Merge pull request #4506 from warwickmm/mono_ssh_avoid_ecdsa_algorithms
Avoid ECDSA algorithms when using SFTP with Mono
2021-05-29 12:22:52 +02:00
Kenneth SkovhedeandGitHub 368a3a9a9b Merge branch 'master' into feature/azure-storage-sastoken 2021-05-29 12:20:30 +02:00
Kenneth SkovhedeandGitHub 2bcb0f511c Merge pull request #4513 from warwickmm/fix_bug_handlings_deprecated_options
Fix bug in handling deprecated Azure and S3 options
2021-05-29 12:17:44 +02:00
Kenneth Hsu a5fbee0335 Simplify exception handling when testing for ECDSA support.
.NET 5 moves the responsibility to the operating system, and since
SSH.NET currently relies on the ECDsaCng class, we will simply interpret
all exceptions to mean lack of ECDSA support.
2021-05-18 12:19:49 -07:00
Kenneth Hsu c246976fbb Cache knowledge of ECDSA support using static field.
This is simpler, and would potentially allow us to query this without an
instance.
2021-05-18 10:18:14 -07:00
Kenneth Hsu 22564bb2a3 Cache knowledge of ECDSA support. 2021-05-18 10:07:33 -07:00
Kenneth Hsu bfa240c8ec Assume we have ECDSA support if our test throws exception.
It's possible that our test will no longer be valid (e.g., due to
changes in SSH.NET), so we don't want the backend to fail if our test
is no longer valid.
2021-05-18 09:06:00 -07:00
Kenneth Hsu 90dfce81b2 Construct instance of ECDsaCng to test for ECDSA support.
This more closely reflects the SSH.NET code.
2021-05-18 09:00:40 -07:00
Kenneth Hsu 3d8817608e Test for ECDSA support explicitly.
The previous implementation could possibly fail for reasons unrelated to
ECDSA support.
2021-05-16 17:29:00 -07:00
marodev a156c511da Reorder parameters of previously wrongly refactored string interpolation 2021-05-16 09:12:00 +02:00
Kenneth Hsu 29f1d2c3cb Fix bug handling deprecated Azure and S3 options.
In pull request #4443, we renamed some of the Azure and S3 options to
use dashes '-' instead of underscores '_' for consistency.  However, our
treatment of the options in the backend constructors still required that
the underscore versions be present even if the dashed version was
provided.

Now, we allow for either option to be present, but the value for the
dashed version will take precedence.
2021-05-15 14:37:53 -07:00
marodev e87688d9d3 Fix ReSharper's "Use string interpolation expression" / Code Smell 2021-05-15 17:02:57 +02:00
Serge 5b3d85fc25 Add authentication using Shared access signature token
Allow users to authenticate using a Shared access signature (SAS) token. SAS tokens are more secure because they:
 * have a start and expiry date/time
 * can be restricted to specified IP addresses
 * can limit access scope to a single Storage blob container
2021-05-15 21:22:50 +10:00
Serge 5172119864 Bugfix read non-present key from dictionary
The lines with null-coalescing operator (??) throw an exception when the dictionary does not contain the first key (eg azure_account_name). Exception thrown: "The given key was not present in the dictionary".
2021-05-15 21:03:26 +10:00
Kenneth Hsu b80845a176 Enumerate items to list before modifying collection. 2021-05-11 15:40:33 -07:00
Kenneth Hsu 3e2103335b SSH: Avoid advertising support for ECDSA algorithms with Mono.
SSH.NET relies on the System.Security.Cryptography.ECDsaCng class for
ECDSA algorithms, which is not implemented in Mono (as of 6.12.0.144).
This prevents clients from connecting if one of the ECDSA algorithms is
chosen as the host key algorithm.  In the event that this causes a
connection failure, we will prevent the client from advertising support
for ECDSA algorithms and make another connection attempt.

Related forum discussion:

	https://forum.duplicati.com/t/release-2-0-6-1-beta-2021-sftp-failure-synology/12358
2021-05-11 12:42:34 -07:00
Kenneth SkovhedeandGitHub 8ce34c084b Merge pull request #4488 from warwickmm/file_backend_should_truncate
Make File backend truncate existing files when overwriting
2021-05-08 11:40:23 +02:00
Kenneth Hsu c14ace0d68 Make File backend truncate existing files when overwriting.
The implementations of ISystemIO.FileOpenWrite call File.OpenWrite,
which does not truncate the existing file.  As such, if we overwrite a
larger file with a smaller one, tail remnants of the existing file will
remain.

Since most of the put operations involve unique filenames, this is only
a problem when overwriting existing files (e.g.,
duplicati-verification.json).

This fixes a regression introduced in revision 71c6aca8a7 ("Fixed IO
calls in Filebackend") from pull request #3456.  The previous
implementation called File.Open with FileMode.Create, which truncates
existing files.

This fixes #4486.
2021-04-20 16:16:00 -07:00
Tyler Gill b64c2f1df0 Move Retry-After logic to RetryAfterHelper to make it easier for other backends to incorporate it. 2021-03-23 17:28:39 -06:00
Tyler Gill dadd30380c Add some logging to Microsoft Graph backends. 2021-03-23 17:08:58 -06:00
Tyler Gill 1ec1e18634 OAuthHttpClient should behave the same as the WebRequest code path and treat no http-operation-timeout override as meaning to use an infinite timeout. 2021-03-22 10:47:25 -06:00
Tyler Gill 06c28387d5 Address some more feedback 2021-03-20 16:09:23 -06:00
Tyler Gill 9a4632d551 Remove SubStream and add functionality to ReadLimitLengthStream. Switch to using that.
Also add some hardening around the Position and make sure the stream starts at relative Position 0.
2021-03-17 20:37:46 -06:00
Tyler Gill b0793d3e49 Refactor the Retry-After logic. Now it tracks Retry-After headers on all requests, and blocks any future requests until the time from the prior request has passed. 2021-03-16 11:40:38 -06:00
Tyler Gill 7894152489 A few more improvements to the MicrosoftGraphBackends.
First, when performing an upload using an upload session, data is streamed directly from the input stream to the request stream. (It is no longer read into an intermediate buffer first.)
Hopefully this will help these uploads respond to throttling behavior better.
Second, these backends now check for and respect a Retry-After header if it is present.
On 5xx errors, the existing exponential backoff is used for the intermediate retries.
A few other minor things have also been cleaned up (e.g., using Task.Delay instead of Thread.Sleep)
2021-03-15 18:04:39 -06:00
Kenneth Hsu c8261320e8 Do not alias S3 username and password to deprecated options.
The Controller class iterates over all supported options and will issue
a warning if any of the provided options are deprecated (or aliased to a
deprecated option).  If the provided auth-username option is aliased to
the deprecated aws_access_key_id, then a warning notification will
appear, even if the user hasn't explicitly provided the deprecated
value.

This fixes #4462.
2021-03-07 15:38:13 -08:00
Kenneth SkovhedeandGitHub 973307cefc Merge pull request #4443 from julianohubel/3748_Underscores_instead_of_dashes
Use dashes in Azure and S3 options for consistency
2021-03-06 14:36:13 +01:00
Kenneth SkovhedeandGitHub 834ebd78b9 Merge pull request #4406 from warwickmm/update_ssh_net
Update SSH.NET to version 2020.0.1
2021-03-06 14:20:59 +01:00
Kenneth SkovhedeandGitHub 29e598b704 Merge pull request #4447 from warwickmm/remove_unused_uplink_libraries
Remove unnecessary Tardigrade libraries from build output
2021-03-06 14:07:55 +01:00
Kenneth Hsu 69b9585421 Remove Telegram backend.
The TLSharp library has some issues and the behavior of the Telegram
backend is not reliable.

Some alternative libraries for communicating with the Telegram API are
TgSharp, TDLib and TdSharp.

This fixes #4424.

Revert "Merge pull request #4324 from martikyan/feature/telegram_backend"

This reverts commit c7f9f8f556, reversing
changes made to 85139f86f2.
2021-02-21 11:37:58 -08:00
Kenneth Hsu 841bcce389 Remove unnecessary Tardigrade libraries from build output.
The required libraries are already included in
Duplicati/Library/Backend/Tardigrade and copied to the output directory.
The extra storj_uplink.dll file in the output directory (caused when
releases are built on macOS) was causing spurious
BadImageFormatExceptions.

This fixes #4434.
2021-02-19 21:39:47 -08:00
Juliano Hubel 8933d6cd2a #3748|Spelling Correction 2021-02-16 17:10:02 +00:00
Juliano Hubel 6a569b80d2 #3748|Adding_Deprecated_messages_to_old_variables 2021-02-16 10:41:55 +00:00
Juliano Hubel 9047de86e1 #3748_Underscores_insted_of_dashes 2021-02-15 15:53:13 +00:00
Kenneth Hsu 88f6b1e55e Update SSH.NET to version 2020.0.1.
From the release notes:

Even though RFC 4253 requires that an identification string MUST be
terminated by a carriage return and line feed, this fix restores support
for banners and identification strings that are only terminated by a
line feed.

This is a workaround for an issue in version 7.4 of OpenSSH which was
fixed in version 7.5.
2021-01-24 10:14:02 -08:00
Kenneth Hsu e50d011a1d Update SSH.NET to version 2020.0.0.
This includes support for additional key exchange algorithms, host key
algorithms, and private key formats.

https://github.com/sshnet/SSH.NET/releases/tag/2020.0.0

This fixes #2808, #2950, #3360, and #4378.
2020-12-31 09:30:49 -08:00
Karen Martikyan 0f138a1cfb default literal fix for build check pass 2020-11-09 22:11:10 +04:00
Karen Martikyan 74ec1e078f Fixed problems while using few different accounts for storing files. 2020-11-09 22:05:43 +04:00
warwickmmandGitHub c7f9f8f556 Merge pull request #4324 from martikyan/feature/telegram_backend
Add support for Telegram channels as a backend destination.
2020-11-08 17:59:37 -08:00
Karen Martikyan 812d698aba Removed SystemExtensions.cs, using Utility.Utility 2020-11-09 00:28:10 +04:00
Karen Martikyan 1feb75002d Fixed bug connected with finding minimal element in an empty list 2020-11-09 00:16:00 +04:00
Karen Martikyan e1b2a190ad Retrieval of epoch moved to extensions 2020-11-08 23:27:29 +04:00
Karen Martikyan 1bc6e20a57 Fixed possible bug connected with retrieval more than 100 messages from a channel 2020-11-08 23:14:39 +04:00
Karen Martikyan 1d106e8163 PR comments fixed: minor changes 2020-11-08 16:50:29 +04:00
Karen Martikyan 8187f66c60 Removed redundant package dependency 2020-11-07 18:45:45 +04:00