Commit Graph
130 Commits
Author SHA1 Message Date
Kenneth Skovhede 61ea36091a Added DNSNames as async 2024-10-02 09:37:52 +02:00
Kenneth Skovhede 586c08522e Implemented async method for Test, CreateFolder and Quota in all backends.
Still missing the List and DNSHostnames.
The AsyncRequest class is also preventing many backends from fully upgrading.
2024-10-01 09:30:02 +02:00
Kenneth Skovhede 8bdb85e261 Implemented RenameAsync on all backends supporting rename 2024-09-29 23:21:50 +02:00
Kenneth Skovhede ecf2e8b551 Implemented DeleteAsync on all backends 2024-09-29 23:04:54 +02:00
Kenneth Skovhede e3e3d1305f Implemented async version of Get for all backends. 2024-09-29 22:00:57 +02:00
a289093eec Update Duplicati/Library/Backend/S3/S3Backend.cs
Co-authored-by: Suguru Hirahara <luixxiul@users.noreply.github.com>
2024-08-01 08:04:28 +02:00
Kenneth Skovhede 2fe5115dd6 Added Sakura Cloud to list of S3 providers 2024-07-29 16:03:23 +02:00
Suguru Hirahara 254bcc63a7 Rename DescriptionS3ClientLong to S3ClientDescriptionLong 2024-07-26 22:25:40 -04:00
PoliandGitHub 9d7fad14cf Update S3Backend.cs - Poli Systems GmbH
Updated the S3 backends of Poli Systems.
S3 became S3-02 and there is a new backend S3-03.
2024-06-15 01:57:45 +02:00
Kenneth SkovhedeandGitHub f20d7b9321 Merge pull request #5222 from duplicati/feature/some-s3-option-cleanup
Removed S3 options that have been deprecated for years.
2024-06-11 10:42:32 +02:00
Kenneth Skovhede b11c558ba2 Removed options that have been deprecated for years.
Simplified the parsing of options slightly.
2024-06-04 20:54:57 +02:00
Kenneth Skovhede c0a49dec2d Removed unused US region, and fixed mapping of location names 2024-06-04 20:13:02 +02:00
Kenneth Skovhede dbdd9a0022 Added additional AWS S3 regions:
- Africa (Cape Town): `af-south-1`
- Asia Pacific (Hyderabad): `ap-south-2`
- Asia Pacific (Jakarta): `ap-southeast-3`
- Asia Pacific (Melbourne): `ap-southeast-4`
- Canada West (Calgary): `ca-west-1`
- Europe (Spain): `eu-south-2`
- Europe (Zurich): `eu-central-2`
- Israel (Tel Aviv): `il-central-1`
- Middle East (UAE): `me-central-1`
- AWS GovCloud (US-East): `us-gov-east-1`
- AWS GovCloud (US-West): `us-gov-west-1`

List of region endpoints is now generated from region table.
2024-06-04 20:00:04 +02:00
Kenneth Skovhede 473c6cbbb8 Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade 2024-03-04 12:21:53 +01:00
Thomas Suckow a3bbf63561 Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel
# Conflicts:
#	.appveyor.yml
#	.github/workflows/tests.yml
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	Duplicati CommandLine Only.sln
#	Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
#	Duplicati.Library.RestAPI/Runner.cs
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/HashAlgorithmHelper.cs
#	Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/GeneralBlackBoxTesting.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Installer/debian/docker/debian/control
#	Tools/SQLiteTool/SQLiteTool.csproj
#	Tools/ZipFileDebugger/ZipFileDebugger.csproj
#	guiTests/guiTest.py
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/alphavss/Homepage.txt
#	thirdparty/alphavss/License.txt
#	thirdparty/alphavss/licensedata.json
2024-02-29 21:55:17 -08:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
JanandGitHub 3a0667dc64 Add option to disable chunked encoding for AWS S3. (#4999)
Some S3 providers do not support MultiChunkedEncoding (#4994).
2023-09-25 02:02:55 +02:00
adithajayandGitHub 16c13f4fef Update S3Backend.cs
Added endpoints for S3 compatible storage from Scaleway. I feel adding this provider is important as they provide 75GB of free storage and many users would utilize that.
2023-04-12 02:31:15 +05:30
Daniel BakerandGitHub a8053ce702 Update S3Backend.cs
Added Impossible Cloud US endpoint.
2023-03-15 16:27:58 +01:00
Thomas Suckow 78197453a1 Merge remote-tracking branch master into feature/net5-avalonia
# Conflicts:
#	Duplicati CommandLine Only.sln
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/UpdaterManager.cs
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/Utility.cs
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/app.config
#	Duplicati/UnitTest/BackendToolTests.cs
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	guiTests/guiTest.py
2022-11-11 21:40:08 -08:00
linitio 1ee2214b8d FIx missing ',' on S3Backend.cs 2022-06-14 16:34:36 +02:00
linitio 7560a65f04 Add all Infomaniak clusters Swiss Backup & Public Cloud 2022-06-14 16:20:29 +02:00
linitio 8ade3abe8c Add all Infomaniak clusters Swiss Backup & Public Cloud 2022-06-14 16:17:16 +02:00
Kenneth Skovhede 63e7c4e693 Minor naming fix 2022-06-12 20:25:22 +02:00
Thomas Suckow d528703fe7 Merge branch 'master' into experiment/net5-split
# Conflicts:
#	.github/workflows/tests.yml
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/Library/Backend/AzureBlob/AzureBlobWrapper.cs
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/Mega/packages.config
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Tardigrade/Properties/AssemblyInfo.cs
#	Duplicati/Library/Backend/Tardigrade/packages.config
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Common/IO/SystemIOWindows.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/packages.config
#	Duplicati/Library/Utility/Utility.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/packages.config
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
2022-02-22 19:04:01 -08:00
Mickael Asseline 81d6d1e40a Add Infomaniak SwissBackup to backend 2022-01-31 15:17:52 +01:00
PoliandGitHub 8fda8caafb Update S3Backend.cs 2021-09-19 15:11:58 +02:00
Kenneth Hsu f2bcf470d2 Add factory method to get default AmazonS3Config instance.
This adds a method to get a default instance of AmazonS3Config.  This
accomplishes two key things:

1. Previously, the documentation for the default buffer size (8192) was
   inconsistent with the actual default value (65536).  This was due to
   differences in the property values betwen the instance used to
   generate the documentation, and that used in the actual backend
   client.

2. Disables the use of Amazon Resource Names (ARN).  This does not seem
   critical to our application.  Moreover, if the UseArnRegion property
   is not set, accessing it triggers an expensive operation (~30
   seconds) to get the region endpoint.  This causes the loading of the
   backend modules to take a long time and run afoul of HTTP request
   timeouts, causing the application to think that no backends are
   available.

   The previous version of the AWSSDK.S3 library that we used did not
   have this property, so this was not an issue until we started
   migrating away from .NET Framework.

This fixes #4543.
2021-06-28 09:52:13 -07:00
Kenneth Hsu de2a651763 Fix bug in PutAsync implementations.
Without await, the using statement can dispose the Stream before the
call to PutAsync completes, resulting in an ObjectDisposedException.

This fixes #4556.
2021-06-12 15:12:13 -07:00
Kenneth Hsu 29f1d2c3cb Fix bug handling deprecated Azure and S3 options.
In pull request #4443, we renamed some of the Azure and S3 options to
use dashes '-' instead of underscores '_' for consistency.  However, our
treatment of the options in the backend constructors still required that
the underscore versions be present even if the dashed version was
provided.

Now, we allow for either option to be present, but the value for the
dashed version will take precedence.
2021-05-15 14:37:53 -07:00
Kenneth Hsu c8261320e8 Do not alias S3 username and password to deprecated options.
The Controller class iterates over all supported options and will issue
a warning if any of the provided options are deprecated (or aliased to a
deprecated option).  If the provided auth-username option is aliased to
the deprecated aws_access_key_id, then a warning notification will
appear, even if the user hasn't explicitly provided the deprecated
value.

This fixes #4462.
2021-03-07 15:38:13 -08:00
Juliano Hubel 8933d6cd2a #3748|Spelling Correction 2021-02-16 17:10:02 +00:00
Juliano Hubel 6a569b80d2 #3748|Adding_Deprecated_messages_to_old_variables 2021-02-16 10:41:55 +00:00
Juliano Hubel 9047de86e1 #3748_Underscores_insted_of_dashes 2021-02-15 15:53:13 +00:00
Kenneth Hsu 7c59b3d531 Add Storadera S3 provider.
This fixes #4313.
2020-09-10 16:44:31 -07:00
Kenneth SkovhedeandGitHub 53f9b8bddc Merge pull request #4237 from warwickmm/add_s3_me_bahrain
Fix endpoint for Middle East (Bahrain) S3 region
2020-07-09 13:15:54 +02:00
Kenneth SkovhedeandGitHub 59cf5886d3 Merge pull request #4226 from warwickmm/add_s3_eu_south
Add support for Europe (Milan) S3 region
2020-07-09 13:15:00 +02:00
Jon Schewe 7294befad9 Remove try/catch that is hiding errors
The try/catch here was hiding timeouts fetching the test files. This
logic appears to be for handling a very old version of S3. They should
not be needed anymore.

Fixes duplicati/duplicati#4225
2020-07-08 07:52:04 -05:00
Kenneth Hsu b0a3763fdf Fix endpoint for Middle East (Bahrain) S3 region.
This fixes issue #4235.
2020-06-27 12:21:59 -07:00
Kenneth Hsu 9e05c5a982 Add support for Europe (Milan) S3 region.
This fixes #4224.
2020-06-14 11:23:45 -07:00
Cedric Hauber 0276eca34e Added MyCloudyPlace.com S3 provider 2020-05-06 19:05:22 +02:00
verhoek 3f3de17d1c Fix codacy complaints. 2019-11-30 23:14:17 +01:00
verhoek 274bcfc437 Removed forgotten m_options. 2019-11-27 21:49:45 +01:00
verhoek 8d782bd0c8 Make codacy happier. 2019-11-26 22:19:11 +01:00
verhoek 707ee587af Use same exception handling cases for Minio as for the AWS client. 2019-11-26 21:59:53 +01:00
verhoek f93562931d Adjusted static command line doc for s3-client. 2019-11-25 23:46:50 +01:00
verhoek 259dd75953 Added listbucket for minio client 2019-11-22 22:37:23 +01:00
verhoek a0b20bfd92 Created s3 client interface. 2019-11-21 22:37:32 +01:00
Kenneth Hsu 2aa76881a5 Mark fields that don't need to be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2019-10-19 13:59:22 -07:00
Kenneth Skovhede d8ef349c3e Changed lookup to use dictionary lookup instead of looping over the keys. 2019-09-17 22:01:33 +02:00