Commit Graph
3969 Commits
Author SHA1 Message Date
Kenneth Skovhede f395da68d7 Hacky support for 7z 2024-03-01 14:30:28 +01:00
Kenneth Skovhede d3d4fc1434 More cleanup after using Mono.Posix 2024-03-01 14:30:28 +01:00
Kenneth Skovhede c08571efba Updating S3 to use the async interface 2024-03-01 14:30:28 +01:00
Kenneth Skovhede 5c1d6bda76 Fixes for OAuth 2024-03-01 14:30:28 +01:00
Kenneth Skovhede cfbfdad2ab Fixes for Azure Blob 2024-03-01 14:30:28 +01:00
Kenneth Skovhede fb863ea14e Updated the common library, using the Nuget Posix package 2024-03-01 14:30:28 +01:00
Kenneth Skovhede 97d89bb46f Disable app-domains 2024-03-01 14:30:28 +01:00
Kenneth Skovhede c168849dff Hand-picked changes from #3124 2024-03-01 14:30:28 +01:00
Kenneth Skovhede aa20088c39 Tool based upgrade of all projects with some manual update 2024-03-01 14:29:54 +01:00
Kenneth Skovhede 35cc847df7 Merge branch 'master' into feature/upgrade-to-sdk-style-projects 2024-03-01 14:28:55 +01:00
Thomas Suckow a3bbf63561 Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel
# Conflicts:
#	.appveyor.yml
#	.github/workflows/tests.yml
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	Duplicati CommandLine Only.sln
#	Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
#	Duplicati.Library.RestAPI/Runner.cs
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/HashAlgorithmHelper.cs
#	Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/GeneralBlackBoxTesting.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Installer/debian/docker/debian/control
#	Tools/SQLiteTool/SQLiteTool.csproj
#	Tools/ZipFileDebugger/ZipFileDebugger.csproj
#	guiTests/guiTest.py
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/alphavss/Homepage.txt
#	thirdparty/alphavss/License.txt
#	thirdparty/alphavss/licensedata.json
2024-02-29 21:55:17 -08:00
Kenneth Skovhede d16d702975 More fixes for stray using directives 2024-02-28 20:42:33 +01:00
Kenneth Skovhede 1f3b00d7ff Fixed a using directive that was caught in the license region 2024-02-28 15:57:06 +01:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Kenneth Skovhede 72e47e419c Fixed referencing Windows.Forms via reflection for cross-platform build 2024-02-26 14:50:43 +01:00
Kenneth Skovhede 3bf6b162ca Upgraded utility projects.
Removed redundant packages files.
Removed `mozroots` in prepare of switch to non-mono releases.
2024-02-26 08:24:48 +01:00
Kenneth Skovhede 22acb8bdda Fixed compile issues after upgrade 2024-02-26 06:59:30 +01:00
Kenneth Skovhede d7b6dd90be Updated all projects to use SDK-style projects using upgrade-assist 2024-02-25 16:19:25 +01:00
trueai-org 076ce156cb fix: aliyun oss code format, zh 2024-02-23 17:54:47 +08:00
trueai-org 35cb8056f6 add aliyun oss, and chinese readme 2024-02-23 11:12:41 +08:00
Jojo-1000 929e61f6b4 Catch RetiredException in BackupHandler to avoid StopNow race condition. 2024-02-22 18:48:44 +01:00
gpatel-fr b12b2603ca update storj to 2.12 2024-02-21 23:10:49 +01:00
gpatel-fr f7def9355b update sshnet to 2023.0.1 2024-02-16 11:39:45 +01:00
Jojo-1000 a0f9785f5c Check that the database exists before writing error logs.
Closes #5091

Fixes that any fatal error in test-filters created an INVALID! file in the current working dir.
2024-02-09 17:10:05 +01:00
gpatel-frandGitHub 396f7dfebe don't report backup as failed if all tests problems are 'extra' (#5074)
these problems don't block database rebuilding so it's not
a good thing to fail the backup.
Also fix erroneous message for ListAndIndexes
2023-12-23 14:25:28 +01:00
gpatel-frandGitHub e2c77b5310 do a vacuum of bug report database (#5069) 2023-12-13 09:27:51 +01:00
gpatel-frandGitHub b08b59228c report all filesets in error in VerifyConsistency (#5052)
only reporting the first one could incite to deleting
it while many filesets can be impacted by a file error
2023-12-12 18:25:14 +01:00
gpatel-frandGitHub 79b87566dd set default log level to Dryrun if dryrun option is set (#5059) 2023-12-11 14:42:45 +01:00
gpatel-frandGitHub 11d6364b4a remove agsXmpp, replace it with Sharp.Xmpp (#5046)
also add a missing space in default body (fixes a small bug under Windows)
2023-12-10 00:26:26 +01:00
gpatel-frandGitHub 0de4cb7968 allow full-remote-verification to target only list and index files (#5037)
The option is turned into an enum with values True, False, ListIndexes
True and False are for compatibility with the existing backups.
ListIndexes is equivalent to True but don't handle block volumes.
This is to make more workable a strategy of full remote testing
after each backup.
2023-12-10 00:24:07 +01:00
gpatel-frandGitHub af4fd5af05 change error reporting for full backup remote verification (#5004)
Before the change, it could happen that when the postbackup test
was finding problems in the backend, the job would be reported
as having completed successfully, the found problems being displayed
only in the complete log. After the change, an error is logged
that cause the job to be failed.
To repro, the following procedure can be used in a test backup:
change a hash for a block in the block table database and run a backup having set
--full-remote-verification and --full-block-verification to true and
--backup-test-percentage to 100.
2023-12-10 00:23:16 +01:00
Jojo-1000 f7e435b1dd Improve file backend list speed.
Directly list FileInfo instead of lookups by filename. This greatly speeds up backup verify on exFAT.

Closes #5061
2023-11-19 20:04:37 +01:00
Jojo-1000 a45ce610df Fix URI parsing for IPv6 addresses and add tests.
Closes #5057
2023-11-10 13:53:10 +01:00
gpatel-frandGitHub ec9d49d684 Experiment/rebuildfaster (#5043)
* experimental faster db rebuild (#4955)

* try run update db query for every block file only when needed
add repair-force-block-use option
more appropriate naming for some routines
more tracing
more comments
2023-11-05 18:19:08 +01:00
gpatel-frandGitHub 43dea88b4f enable to perform tests without checking file list (#5054)
following the option --disable-filelist-consistency-checks
2023-11-04 14:25:59 +01:00
JanandGitHub 3a0667dc64 Add option to disable chunked encoding for AWS S3. (#4999)
Some S3 providers do not support MultiChunkedEncoding (#4994).
2023-09-25 02:02:55 +02:00
gpatel-frandGitHub 4012e7f608 add console log to the Alternative FTP backend for debugging purposes (#5034) 2023-09-24 10:52:57 +02:00
taz-ilandgpatel-fr f791066b7a FluentFTP version update - Fix handling of missing base folder (#4946)
* Improve errors specificity by:
a. Avoiding unnecesary wrapping in AggregateException - Using Library.Utility.Await instead of Wait / Result
b. Using messages from InnerException during TestConnection

* Fix detection of missing base directory:
Depend on error code while trying to SetWorkingDirectory, rather than very wide try/catch and a hardcoded error string (Different FTP servers return different error strings. E.g. hardcoded here was "Directory not found." while Filezilla returns "Can't open file or directory").

* Fix indentation

Intentionally seperated from the previous commit for easier reviewing (otherwise Github text diff display is confused by the removal of the large try / catch blocks)

* Fix creation of missing base directory:
(previously CreateClient would fail since it would try to SetWorkingDirectory to the not-yet-existing directory)
2023-09-24 09:28:19 +02:00
taz-ilandgpatel-fr dbd8cdb86a Update FluentFTP to latest version (46.0.2) (#4935)
* Upgrade FluentFTP to v42 in an attempt to resolve TLS session issues:
duplicati/duplicati#4831

Note v40 introduced breaking changes, specifically seperating between sync and async clients.

Using a different approach this time, single async client.

* Bump FluentFTP to latest (42.0.0 --> 46.0.2)
2023-09-24 09:28:19 +02:00
gpatel-frandGitHub b19f6f3743 update restsharp to 106.10.1 (#5029)
to make Dependabot happy - have to update minio as well
2023-09-23 14:57:27 +02:00
JanandGitHub 91eace3ab2 Include exception type and message in job log. (#4976)
If an exception is provided for a log message, display the type and message (excluding stack trace) even if withExceptionDetails=false. This helps to diagnose bugs if the live log was not saved.

Closes #4843
2023-09-21 23:13:15 +02:00
Janandgpatel-fr b894a197a9 Fix missing file error caused by interrupted compact.
Before the dindex files are deleted on the remote, update the state in the database to Deleting. Previously, only dblock files were handled this way.
Patch by warwickmm
Closes #4129
2023-09-21 22:56:52 +02:00
Janandgpatel-fr 71eb03130a Improve result reporting for interrupted/failed backups (Fix #4829)
Direct import of PR#4978
Update web UI for new result reports.
For operations with fatal errors, write logs to same operation ID.
Test that Interrupted flag is correct in RunScriptTests.
Update backup log display for new result reporting.
Hide file statistics for fatal errors and change fatal icon.
2023-09-21 20:01:17 +02:00
gpatel-frandGitHub 91f1fc6a60 update microsoft.rest.clientruntime to 2.3.24 (#5020) 2023-08-31 10:54:50 +02:00
gpatel-frandGitHub f985af4a5a removes obsolete backend HubiC - fixes #4836 (#5018) 2023-08-30 13:45:55 +02:00
gpatel-frandGitHub 729a84fdc1 update mail libraries (#5014)
known side effect: the mail server user password will no longer be displayed in the verbose log
2023-08-28 11:50:37 +02:00
Andrew RuthvenandAndrew Ruthven dc609e297d Add Catalyst Cloud as an OpenStack target
I successfully have backups running to Catalyst Cloud and I'm able to restore.
2023-05-12 16:02:00 +12:00
Russell Smith 6fb3206b22 Use 1MB default block as suggested by users.
This still allows 50 blocks for the default upload size.
It reduces the blocks to track by a factor of 10.
If you are using larger backups, you want a larger block size.

If you have small backups and it's a problem, recreating those
will take less time.
2023-04-22 16:50:48 +10:00
adithajayandGitHub 16c13f4fef Update S3Backend.cs
Added endpoints for S3 compatible storage from Scaleway. I feel adding this provider is important as they provide 75GB of free storage and many users would utilize that.
2023-04-12 02:31:15 +05:30
gpatel-fr cf17f488d1 update Newtonsoft lib to 13.02 2023-03-24 23:17:11 +01:00