Commit Graph
210 Commits
Author SHA1 Message Date
Kenneth Skovhede 0f9669b273 Final conversion to async backends
This PR is the final change, where all backends are now purely implemented with async interfaces.

The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.

In many places, the backeds were already prepared for async output, in other some glue was needed.

To reduce the scope of this change, some backends simply report a synchronous result as an async list.
2025-02-17 16:45:51 +01:00
marceloduplicatiandGitHub 3a604998e2 Merge pull request #5921 from duplicati/feature/backend-tester-improv
Change timeout configuration for BackendTester
2025-02-03 17:44:17 -03:00
marceloduplicatiandGitHub 03b7fac65c Merge pull request #5889 from marceloduplicati/feature/unified-help-arguments
Help Command unification
2025-01-30 10:23:10 -03:00
Marcelo C. 0fca2ba96b Changing default streaming timeout from 10 to 60 seconds 2025-01-29 20:37:40 -03:00
Marcelo C. 36db29dc77 Change timeout configuration for BackendTester
This change will allow an environment variable READ_WRITE_TIMEOUT_SECONDS to control the read write timeout used during the tests on streaming backends.

-1 will set it to infinite.

Additionally all console logs during test execution have a timestamp with millisecond precision to aid in benchmarking.
2025-01-28 10:35:32 -03:00
Kenneth SkovhedeandGitHub 997891525a Updated license-updater to update csproj files (#5898)
* Updated license-updater to update csproj files

* License updates
2025-01-22 21:04:19 +01:00
Marcelo C. cee1b8428d Centralizing the check on Helper function. 2025-01-21 12:45:53 -03:00
Marcelo C. 2e502ea7be Help Command unification
All CLI tools and executables now accept a unified set of help command variants:

"help", "/help","--help", "usage", "/usage", "--usage", "/h", "-h"

Affected executables:

Duplicati.Agent
Duplicati.CommandLine
Duplicati.CommandLine.AutoUpdater
Duplicati.CommandLine.BackendTester
Duplicati.CommandLine.BackendTool
Duplicati.CommandLine.ConfigurationImporter
Duplicati.CommandLine.SecretTool
Duplicati.CommandLine.Snapshots
Duplicati.GUI.TrayIcon
Duplicati.Server
Duplicati.Service
Duplicati.WindowsService

Note that SharpAesCrypt command is not included in this because the parameter parsing is part of the SharpAesCrypt code base.
2025-01-17 14:00:14 -03:00
Marcelo C. adc4bfa650 Added timers to backend testing tool
Also changed logic around test result, ensuring it fails when any unexpected behavior happens.
2025-01-17 09:45:49 -03:00
Kenneth SkovhedeandGitHub 9464caf622 Feature/update license 2025 (#5851)
* Fixed some minor whitespace issues

* Updated all copyright to 2025
2025-01-07 09:40:39 +01:00
Kenneth SkovhedeandGitHub 447c2b3238 Feature/improve ci testing (#5781)
* Added options to insert delays in backend tester

* Added wait delays to Azure and GoogleDrive to see it the CI is more stable with small delays

* Added more debug output to the backend tester

* Fixed dropbox reporting OAuth quota errors, when it is likely Dropbox API quota errors

* Implemented exception detection on AzureBlob backend

* Also showing sleep times in output

* Fixed a bug with the dropbox quota response

* Fixed error message to report correct number of files

* Fixed suppressed warning
2024-12-20 16:10:08 +01:00
Kenneth SkovhedeandGitHub 86f3f6b178 Fixed issues with timeout in msgraph (#5790)
* Fixed issues with timeout in msgraph

Fixed a place where the cancellation token was not applied.

Fixed a problem with msgraph not sending correct DNS information.

Fixed a problem with msgraph downloading the entire file before streaming it, bypassing throttle and tripping the timeout.

Added `ITimeoutExemptBackend` to selectively disable timeouts for backends that do not support it.

Updated the backendmanager to support `ITimeoutExemptBackend`.

Updated the backendmanager to abort as soon as a timeout happens.

* Updated BackendTester to use the `TimeoutObservingStream` and support the `--read-write-timeout` setting.

Updated backend tester code to use `var`

* Fixed a bug reading the `read-write-timeout`

* Updated to using constants with names instead of `-1`.
2024-12-19 09:31:06 +01:00
Kenneth Skovhede 61ea36091a Added DNSNames as async 2024-10-02 09:37:52 +02:00
Kenneth Skovhede 586c08522e Implemented async method for Test, CreateFolder and Quota in all backends.
Still missing the List and DNSHostnames.
The AsyncRequest class is also preventing many backends from fully upgrading.
2024-10-01 09:30:02 +02:00
Kenneth Skovhede 8bdb85e261 Implemented RenameAsync on all backends supporting rename 2024-09-29 23:21:50 +02:00
Kenneth Skovhede ecf2e8b551 Implemented DeleteAsync on all backends 2024-09-29 23:04:54 +02:00
Kenneth Skovhede e3e3d1305f Implemented async version of Get for all backends. 2024-09-29 22:00:57 +02:00
Kenneth Skovhede 3548475360 Implemented preload support.
This commit adds the option to provide preload/preset options that change the default values for Duplicati's commandline or environment variables.
This fixes #5480
2024-08-28 00:18:55 +02:00
Suguru Hirahara a6b8f61646 Remove conjugation more 2024-08-13 08:09:04 -04:00
Grzegorz Łagocki 9b19bac7da Add Copyright Property to *.csproj files 2024-07-09 00:37:18 +02:00
Kenneth Skovhede 8e25c6e2ab Reduced dynamic scanning with static linking.
Cleaned up some `proj` files.
Added caching to options.
2024-04-26 14:32:41 +02:00
Kenneth Skovhede f77f77ad3a Fixed types to allow building release executables 2024-03-15 15:20:28 +01:00
Kenneth Skovhede 115e887802 Implemented new updater logic that supports multi-arch distributions. 2024-03-15 14:18:56 +01:00
Kenneth Skovhede 473c6cbbb8 Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade 2024-03-04 12:21:53 +01:00
Kenneth Skovhede aa20088c39 Tool based upgrade of all projects with some manual update 2024-03-01 14:29:54 +01:00
Kenneth Skovhede 35cc847df7 Merge branch 'master' into feature/upgrade-to-sdk-style-projects 2024-03-01 14:28:55 +01:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Kenneth Skovhede 3bf6b162ca Upgraded utility projects.
Removed redundant packages files.
Removed `mozroots` in prepare of switch to non-mono releases.
2024-02-26 08:24:48 +01:00
Kenneth Skovhede d7b6dd90be Updated all projects to use SDK-style projects using upgrade-assist 2024-02-25 16:19:25 +01:00
gpatel-frandGitHub f985af4a5a removes obsolete backend HubiC - fixes #4836 (#5018) 2023-08-30 13:45:55 +02:00
gpatel-fr cf17f488d1 update Newtonsoft lib to 13.02 2023-03-24 23:17:11 +01:00
Thomas Suckow 69ccc12e7d Reorganize the backends to be referenced via a single csproj
For my sanity
2022-11-18 21:55:46 -08:00
Thomas Suckow 78197453a1 Merge remote-tracking branch master into feature/net5-avalonia
# Conflicts:
#	Duplicati CommandLine Only.sln
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/UpdaterManager.cs
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/Utility.cs
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/app.config
#	Duplicati/UnitTest/BackendToolTests.cs
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	guiTests/guiTest.py
2022-11-11 21:40:08 -08:00
Muhammad Abrar 713fce2acb Adding idrivee2 reference in BackendTester, BackendTool and Duplicati.GUI.TrayIcon csproj as well 2022-06-13 10:59:08 +05:00
Thomas Suckow 8c518f7d3a .net6 2022-04-12 21:43:50 -07:00
Thomas Suckow d528703fe7 Merge branch 'master' into experiment/net5-split
# Conflicts:
#	.github/workflows/tests.yml
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/Library/Backend/AzureBlob/AzureBlobWrapper.cs
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/Mega/packages.config
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Tardigrade/Properties/AssemblyInfo.cs
#	Duplicati/Library/Backend/Tardigrade/packages.config
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Common/IO/SystemIOWindows.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/packages.config
#	Duplicati/Library/Utility/Utility.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/packages.config
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
2022-02-22 19:04:01 -08:00
warwickmmandGitHub 184dfeae78 Merge pull request #4492 from TopperDEL/storj-dcs-update
Add Storj DCS backend and deprecated Tardigrade DCS backend.

Storj renamed their "Tardigrade Decentralized Cloud Storage" product to "Storj DCS".  This adds a new Storj DCS backend and deprecates the old Tardigrade one.  A warning is issued to inform Tardigrade users to modify their configurations to use the new backend.

This also adds a passphrase verification field to ensure that the user has provided the correct encryption passphrase.
2021-07-20 08:28:54 -07:00
TopperDEL 0c9435fe7b fix: Use fresh GUID for Tardigrade-Backend-Project 2021-07-19 09:21:08 +02:00
TopperDEL 0981c66465 feat: Create a copy of old Tardigrade-Backend with reuse of new Storj-Backend 2021-07-15 17:15:03 +02:00
TopperDEL a3d9d4c6ba feat: Rename C# and Project-files to Storj 2021-05-03 09:50:29 +02:00
Kenneth Skovhede b550e61d9b Re-added compatibility analyzers 2021-04-03 21:49:51 +02:00
Kenneth Skovhede 557415c9ad Changed implementation to target netstandard2.0.
Added builds for .Net 4.8
2021-04-03 14:13:32 +02:00
Kenneth Skovhede 8892ff7df3 Cleaned up all .csproj files.
Moved things around to make cleaner build files.
Implementing everything with `netstandard2.1` and then thin wrapper executables using .net5
2021-04-03 13:57:02 +02:00
Kenneth Skovhede 4f565a004f Merge branch 'netstandard' of https://github.com/mnaiman/duplicati into netstandard
# Conflicts:
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SSHv2/packages.config
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Telegram/Duplicati.Library.Backend.Telegram.csproj
#	Duplicati/Library/Backend/Telegram/Properties/AssemblyInfo.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
2021-04-01 15:08:42 +02:00
Tyler Gill 3dce0df106 Fix typo 2021-03-23 17:40:34 -06:00
Tyler Gill a064d66603 Parse throttle speeds in backend tester as sizes. 2021-03-22 10:47:44 -06:00
Tyler Gill da71d28422 Allow backend tester to test throttling. 2021-03-22 09:57:56 -06:00
Tyler Gill 7f1b6bc7a2 Fix some bugs in ReadLimitLengthStream changes, and add a unit test for some of the new behavior.
Also make NonSeekableStream block seeks via the Position setter.
2021-03-18 12:24:11 -06:00
Kenneth Hsu 69b9585421 Remove Telegram backend.
The TLSharp library has some issues and the behavior of the Telegram
backend is not reliable.

Some alternative libraries for communicating with the Telegram API are
TgSharp, TDLib and TdSharp.

This fixes #4424.

Revert "Merge pull request #4324 from martikyan/feature/telegram_backend"

This reverts commit c7f9f8f556, reversing
changes made to 85139f86f2.
2021-02-21 11:37:58 -08:00
Thomas Suckow acea2fe10f upstream
Update Telegram SharpAESCrypt dep
Update SystemIOWindows to split the difference

# Conflicts:
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/AutoUpdateBuilder/packages.config
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/DependencyFinder/packages.config
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	BuildTools/UpdateVersionStamp/app.config
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/RecoveryTool/packages.config
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/packages.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/AutoUpdater/app.config
#	Duplicati/Library/AutoUpdater/packages.config
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AlternativeFTP/app.config
#	Duplicati/Library/Backend/AlternativeFTP/packages.config
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/AzureBlob/packages.config
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Backblaze/packages.config
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/Box/packages.config
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/CloudFiles/app.config
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/Dropbox/packages.config
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/FTP/app.config
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/File/app.config
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/GoogleServices/packages.config
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/Mega/packages.config
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OAuthHelper/packages.config
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OneDrive/packages.config
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/OpenStack/packages.config
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/Rclone/packages.config
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/S3/app.config
#	Duplicati/Library/Backend/S3/packages.config
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SSHv2/packages.config
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/SharePoint/app.config
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Sia/packages.config
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/TahoeLAFS/packages.config
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Tardigrade/packages.config
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Backend/WEBDAV/app.config
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Common/IO/SystemIOWindows.cs
#	Duplicati/Library/Common/app.config
#	Duplicati/Library/Common/packages.config
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/Compression/app.config
#	Duplicati/Library/Compression/packages.config
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/DynamicLoader/app.config
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Encryption/packages.config
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Localization/packages.config
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Logging/app.config
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/app.config
#	Duplicati/Library/Main/packages.config
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/packages.config
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/Snapshots/app.config
#	Duplicati/Library/Snapshots/packages.config
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/UsageReporter/packages.config
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/app.config
#	Duplicati/Library/Utility/packages.config
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/packages.config
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Server/packages.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/RestoreHandlerTests.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/UnitTest/packages.config
#	Duplicati/WindowsService/App.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Tools/SQLiteTool/App.config
#	Tools/SQLiteTool/SQLiteTool.csproj
#	thirdparty/TlsTest/TlsTest.csproj
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/mozroots/mozroots.csproj
2020-12-29 09:59:42 -08:00