Commit Graph
1364 Commits
Author SHA1 Message Date
marceloduplicatiandGitHub 99ee234640 Primary functionality implemented for CIFS (#5780)
* Primary functionality implemented for CIFS

This will add CIFS support to Duplicati.

One of the advantages is that the resource does not need to be mounted as the backend connects and authenticates directly to the resource via TCP/or Netbios over TCP.

The authentication supported is domain/username/password. At present integrated Windows authentication is not supported.

This implementation relies on SMBLibrary library (https://github.com/TalAloni/SMBLibrary).

* Fixes to XML documentation

* Implementing single CIFS connection + Fix on PutAsync

Single CIFS connection to avoid re-authentication on every operation.
Fix to PutAsync which only because apparent after file sizes were larger than _smb2Client.MaxWriteSize

* Adding CI tests for CIFS

The CI test for CIFS is self contained, requiring no environment variables. Also updated README.
2025-01-06 14:22:23 +01:00
Kenneth SkovhedeandGitHub 2cbf62d02b Update Storj options and docs (#5838)
This introduces a few constants in favor of multiple strings with the same value.
The supported arguments in the module have been augmented to better report possible values and default values.
The options parser has been updated to work better with CLI where the options are not default filled.
2025-01-03 17:21:00 +01:00
Kenneth SkovhedeandGitHub 228a472b07 Changed Sia redundancy to decimal (#5837) 2025-01-03 17:20:37 +01:00
Kenneth SkovhedeandGitHub d0924c3e01 Auto-select version for OpenStack (#5836)
This adds a check to see if the give url contains the standard `/v3` suffix, and if so selects the v3 protocol.
2025-01-03 17:20:19 +01:00
Kenneth SkovhedeandGitHub c70b593fed Corrected documentation for OpenStack (#5835)
The region was not described correctly.
2025-01-03 17:19:57 +01:00
Kenneth SkovhedeandGitHub 78da4527be Removed unused region from Aliyun (#5833) 2025-01-03 14:08:29 +01:00
Kenneth SkovhedeandGitHub 2fac31bd60 Clean Azure Blob options (#5829)
This removes Azure Blob options that has been deprecated for years.
The option `--azure-blob-container-name` was not used, so it was removed as well.
2025-01-02 18:51:00 +01:00
Kenneth SkovhedeandGitHub 6fdb01b52a This commit ensures the client connects on first use, instead of relying on the library to connect. (#5828) 2025-01-01 14:06:35 +01:00
Kenneth SkovhedeandGitHub 31503ea595 Added option to log FTP diagnostics data to regular log (#5821) 2024-12-31 16:47:28 +01:00
Kenneth SkovhedeandGitHub ba645ddb5f Clean up options for file backend (#5820)
This removes unsupported arguments from non-windows clients, so the reported options fit what is supported.
2024-12-31 16:37:07 +01:00
Kenneth SkovhedeandGitHub 8f853cfa7f Google drive and GCS do not work with timeout (#5812) 2024-12-26 10:42:58 +01:00
Kenneth SkovhedeandGitHub 394b876985 Enable noop daemon for FTP (#5813)
This fixes an issue with timeouts on large file.
2024-12-26 10:41:06 +01:00
Kenneth SkovhedeandGitHub 447c2b3238 Feature/improve ci testing (#5781)
* Added options to insert delays in backend tester

* Added wait delays to Azure and GoogleDrive to see it the CI is more stable with small delays

* Added more debug output to the backend tester

* Fixed dropbox reporting OAuth quota errors, when it is likely Dropbox API quota errors

* Implemented exception detection on AzureBlob backend

* Also showing sleep times in output

* Fixed a bug with the dropbox quota response

* Fixed error message to report correct number of files

* Fixed suppressed warning
2024-12-20 16:10:08 +01:00
Kenneth SkovhedeandGitHub 0023dfd19f Removed the use of WebClient in MicrosoftGraphBackend in favor of HttpClient. (#5799) 2024-12-19 22:40:18 +01:00
Kenneth SkovhedeandGitHub 8593887184 Rename OneDrive v2 (#5795)
* Change displayname of OneDrive v2

Since OneDrive v1 is removed, there is no need to name the current backend `v2`.

* Renamed everything but the protocol key
2024-12-19 22:38:35 +01:00
Kenneth SkovhedeandGitHub 86f3f6b178 Fixed issues with timeout in msgraph (#5790)
* Fixed issues with timeout in msgraph

Fixed a place where the cancellation token was not applied.

Fixed a problem with msgraph not sending correct DNS information.

Fixed a problem with msgraph downloading the entire file before streaming it, bypassing throttle and tripping the timeout.

Added `ITimeoutExemptBackend` to selectively disable timeouts for backends that do not support it.

Updated the backendmanager to support `ITimeoutExemptBackend`.

Updated the backendmanager to abort as soon as a timeout happens.

* Updated BackendTester to use the `TimeoutObservingStream` and support the `--read-write-timeout` setting.

Updated backend tester code to use `var`

* Fixed a bug reading the `read-write-timeout`

* Updated to using constants with names instead of `-1`.
2024-12-19 09:31:06 +01:00
Marcelo C. c8c79d09eb Fix to re-introduce Unscaping directory names for FTP Client
The need for this was detected by forum post: https://forum.duplicati.com/t/ftp-without-ssl-fails-with-2-1-0-2-beta-2024-11-29/19691/12
2024-12-17 10:11:45 -03:00
Kenneth Skovhede 4dfc4f586e Adding e2 url format to description.
This fixes #5755
2024-12-11 08:38:06 +01:00
Kenneth Skovhede ac61b2afbe Updated FluentFTP to latest.
This should fix #5715
2024-12-09 10:20:27 +01:00
Kenneth Skovhede e738221bb2 Removed unused dependency 2024-12-05 14:42:53 +01:00
Kenneth Skovhede 6156024e8f Bumped SSH.NET to latest
This fixes #4674
2024-12-05 14:33:44 +01:00
Kenneth SkovhedeandGitHub f3a71fa1c5 Merge pull request #5732 from marceloduplicati/feature/fix-pcloud-subfolders
Implement nested folder support and improve file operations on pCloud Backend
2024-12-05 14:03:20 +01:00
Marcelo C. ce73e0f1f6 Implementation of fixes suggested on code review
Post changes tests:

- Folder creation and discovery checked.
- Backup and restore operations checked.
- Tests with backend tests performed.
2024-12-04 09:29:14 -03:00
Kenneth SkovhedeandGitHub 471ff7ede8 Merge pull request #5734 from duplicati/feature/fix-quota-info-on-file-backend
Fixed issue with quota info.
2024-12-04 09:39:42 +01:00
Kenneth Skovhede 17cc382325 The legacy flag --use-ssl toggles explicit TLS 2024-12-04 08:34:09 +01:00
Kenneth Skovhede 356498414c Fixed issue with quota info.
This fixes #5733
2024-12-03 22:12:38 +01:00
Marcelo C. 356d0c1bfd Implement nested folder support and improve file operations on pCloud Backend
- Add support for subfolders
- Implement robust file selection using fileID for Download/Delete operations instead of concatenation of path + filename
- Improved validation for folder existence during backend connectivity tests supporting subfolders.
2024-12-03 12:16:17 -03:00
Marcelo C. 2eab4611de Fix for the streaming error on BackendTester interaction
This issue does not affect the pCloud backend per se and its functioning as expected.

The issue is related to a symptom the BackendTester revealed as discussed here:

https://github.com/duplicati/duplicati/pull/5709

It turns out the backends should assume the caller has set the input stream at the correct position and take it from there.

Therefore the fix is simply to remove the .Position = 0; which was well intended as a way to make sure the stream was at the beginning, but  thats not the pattern to follow when integrating backends.

A future blog post/tutorial on new backend integrations will mention these premises.
2024-11-27 19:34:18 +01:00
Kenneth SkovhedeandGitHub 79d6dc0cad Merge branch 'master' into feature/backends-add-pcloud 2024-11-25 11:08:55 +01:00
Marcelo C. 068ae42aa1 Implementation of changes requested in review 2024-11-25 11:01:59 +01:00
Kenneth Skovhede c915b70697 Added support for ftp/aftp treating url paths as before the update 2024-11-21 14:37:25 +01:00
Kenneth Skovhede 9dadd514bf Inverted the flag for relative paths, as that is more compatible with the previous library. 2024-11-21 13:13:44 +01:00
Kenneth Skovhede 5712264e6a Updated some parsing to be simpler.
Changed the strategy to use absolute paths for all operations, to avoid cases where a silent reconnect would end up in a different folder.
Add backwards compatible option to use CWD style relative operations.
Added support for detecting the SSL certificate hash when there is a certificate error.
2024-11-19 13:55:23 +01:00
Kenneth Skovhede 2828729a59 Changed logic for how error messages are constructed 2024-11-19 06:52:17 +01:00
Marcelo Ceccon 4beafc0cc7 Added pCloud Integration
Added pCloud backend with custom interface for editing destination.
2024-11-18 13:07:01 +01:00
Kenneth Skovhede 88ae1d18b1 Fixed output from S3 options to mimic previous version 2024-11-15 14:18:39 +01:00
Kenneth Skovhede 38a255eb00 Fixed startup delay on Windows.
Delay was caused by AWS library performing network queries when reading properties on the configuration object.
On top of this, the properties were read twice.
Unified reading+application of options between iDrive and AWS S3.
2024-11-15 14:04:42 +01:00
Kenneth Skovhede 81c0b2209e Fixed a warning when using S3 2024-11-11 21:20:21 +01:00
Kenneth SkovhedeandGitHub c96d2e0b7c Merge pull request #5655 from marceloduplicati/feature/warnings-cleanup-lowimpactwarnings
Further progress on removal of build warnings.
2024-11-08 10:54:57 +01:00
Marcelo Ceccon d4d9c47824 Further progress on removal of build warnings.
None of these changes have impact on code execution, they are mainly part of the effort to remove all build warnings.

The removal of Thread.ResetAbort leaves a TODO for us to integrate CancelationToken when it gets propagated down here on future changes.
2024-11-07 18:02:15 +01:00
Marcelo Ceccon 706e490f79 Upgrading AWS SDK to latest
This also resolves a series of warnings produced by unequalized versions between AWSSDK.Core and AWSSDK.S3 references.

The decision to not upgrade Minio (which is rather dated) is because starting at 4.0.0 several calls were made obsolete including the constructor. A specific feature branch will be opened to integrate the new version and run CI tests.
2024-11-07 17:33:44 +01:00
Kenneth Skovhede a9cd488bf2 Change the s3-client option to an enumeration-type as it has two possible values.
Fixed the parser to handle empty string and null as the same.
2024-11-06 22:07:27 +01:00
Kenneth SkovhedeandGitHub 2c8be00bab Merge pull request #5619 from duplicati/feature/reuse-retry-after-information
Fixed MSGraph backend not obeying the retry-after header.
2024-11-04 09:23:58 +01:00
Kenneth Skovhede 4ef0497d0e Reverted the message severity, it is not actionable by the user. 2024-11-01 09:24:28 +01:00
Kenneth Skovhede e62f1639b3 Implemented a way to share retry timeouts between multiple instances of the same backend. 2024-11-01 09:20:11 +01:00
Kenneth Skovhede faa210df63 Updated rclone backend to handle empty options the same as missing options (apply default values).
Made the local repo optional in the UI because it has a default value.
2024-11-01 09:02:40 +01:00
marceloduplicatiandGitHub bc4f569325 Merge branch 'duplicati:master' into feature/remove-legacy-ftp 2024-10-15 13:01:20 +02:00
Marcelo Ceccon 3857d0356f Precedence of options match to documentation 2024-10-15 12:58:37 +02:00
Marcelo Ceccon 34f108f127 Implementing backwards compatibility on ftp parameters
With this change, ftp-passive, ftp-regular and use-ssl parameters found on existing FTP Backend configurations will be respected and exhibit the same behavior. They are marked as deprecated so they will issue warning messages when used prompting users to refer to new parameters.
2024-10-15 12:17:51 +02:00
Marcelo Ceccon a4482a4d03 Removing the legacy FTP Backend
For this change, what was known as the alternate FTP client which uses FluentFTP library now officially replaces the FTP Backend which still used the deprecated FtpWebrequest.

To mantain compatibility with existing configurations, both ftp and aftp targets and configuration prefixes are accepted.
2024-10-11 11:21:54 +02:00