Commit Graph
194 Commits
Author SHA1 Message Date
Kenneth Skovhede 1428d8cc6f Preserve order of input sources 2025-02-24 12:53:48 +01:00
Kenneth Skovhede 9dfe977d3f Changed logic so the source providers are now logically mounted into the source directory structure.
This fixes the restore issues as all sources will now appear as if they were originally part of the source file system.

The backend implementation has now been simplified so the backends only return names of entries with no path information.  Most backends already do this, but the S3 backends did not, so this was fixed as well.

This also clears up the interfaces, so the backend does not know about the `ISourceProviderEntry`. All mapping is done by the `BackendSourceProvider`, making it simpler to support other backends as sources.
2025-02-21 16:06:55 +01:00
Kenneth Skovhede 1716d09b51 Fixed source providers for S3, CIFS, and SSH.
Made the SMBShareConnection slightly more robust.

Introduced v2 List API to S3 backend, currently it retains v1 as the default.

Changed S3 backends to not recurse, potentially making smaller listings. This option can be disabled if it breaks things.

Added support for relative paths in the SSH backend.
2025-02-21 13:20:52 +01:00
Kenneth Skovhede 3e72768eec Added warning when using the Mega backend 2025-02-06 21:53:51 +01:00
Kenneth Skovhede fac12ba772 Further consolidated the data folder.
Added support for `--portable-mode` and `--server-datafolder` for the CLI, ServerUtil, and Agent.

Fixed a few places where file reads were not cached properly.

Moved some responsibilities of UpdaterManager into DataFolderManager.

This now supports storing `machineid.txt` and `installation.txt` inside the path pointed to by `--server-datafolder` or alternatively with `--portable-mode`.

It is possible that some fringe backwards compatibility is lost with this update.

Fixed an issue with AutoUpdater crashing on check/download.

This fixes #5902
2025-01-28 13:01:35 +01:00
Kenneth SkovhedeandGitHub 039e1a08fb Merge branch 'master' into feature/prevent-windows-subfolder-storage 2025-01-28 08:58:26 +01:00
Kenneth SkovhedeandGitHub d99f74ed94 Rewrite the backend manager. (#5896)
* Rewrite the backend manager.

This creates a new more logic backend manager that handles all backend operations.

For each top-level operation, there is now a single backend manager instance that is passed to sub-commands.

The internals of the backend manager are now rewritten to use async logic instead of threading.

The design uses a single task runner that dispatches operations and honors concurrent settings and limits from one place.

The logic for each operation has been moved into a separate files/classes to make it easier to understand each operation.

This fixes #5804

* Fixed a few issues with not awaiting tasks

* Fixed not creating empty databases

* Fixed an issue with hashes not being recorded on download

* Reworked the download logic to avoid attempting to decrypt the file if it has been modified.

* Review fixes

* Renamed db collector to better reflect the purpose.

* Review fixes
2025-01-28 08:54:50 +01:00
Kenneth Skovhede 27f0bb4ea0 Unified code to find the data folder.
The `machineid.txt` and `installation.txt` files are now default located in same folder where the database is default located.
2025-01-27 22:51:45 +01:00
Kenneth Skovhede 97d62163c5 Renamed methods for database path to better distinguish what they do 2025-01-27 11:46:24 +01:00
Kenneth SkovhedeandGitHub 9464caf622 Feature/update license 2025 (#5851)
* Fixed some minor whitespace issues

* Updated all copyright to 2025
2025-01-07 09:40:39 +01:00
Kenneth SkovhedeandGitHub ff1b2b7990 Simplified task control (#5753)
* Simplified task control by having all logic in a single class and using async control mechanism
* Fixed the tests
* Made the marking of a backup partial explicitly communicated if the enumeration process is stopped.
* Added support for pausing active transfers
* Introduced transfer token to abort operations.
* Updated the way LiveControl is emitting events.
* Retains the paused state of the server across reboots, this fixes #5760.
* Reworked the way throttle speeds are propagated through LiveControls.
* Added option to pause transfers in the UI.
* Removed throttle settings from the LiveControls class so it is exclusively handled by the ApplicationSettings.
* Removed `thread-priority` as it does not work at all when tasks are used.
* Fixed the stop dialog and API to only support stop and abort.
* Removed unused variables
2024-12-18 08:27:59 +01:00
Kenneth Skovhede a72bf27aa3 Merge remote-tracking branch 'origin/master' into feature/add-secret-provider 2024-11-05 11:48:30 +01:00
Kenneth Skovhede 8250a5befa Refactored SecretProviderHelper to replace in Uri instances instead of raw string.
This fixes some issues with URL encoding/decoding.
Due to the legacy class `Duplicati.Library.Utility.Uri` there are two replacements done.
2024-11-04 15:57:56 +01:00
Kenneth Skovhede 3f3144218f Removed 7z from the source as it has been marked experimental for 7 years.
This fixes #5630
2024-11-01 17:40:09 +01:00
Kenneth Skovhede 34d13a20e2 Initial version of the external secret provider 2024-10-24 16:06:59 +02:00
Kenneth Skovhede f3e2a0a902 Avoid setting Thread.CurrentCulture
Avoids setting deprecated `Thread.CurrentCulture` and `Thread.CurrentUICulture`, using `CultureInfo.CurrentCulture` instead.
Fixed a few legacy places where incorrect string compares were used.
2024-09-18 09:24:55 +02:00
Suguru Hirahara ae57dee0f0 Remove redundant space characters from *.cs and *.html 2024-07-26 00:10:31 -04:00
Jojo-1000 930f1c8bd3 Fix most unsupported platform warnings. 2024-05-30 01:50:02 +02:00
Jojo-1000 e2e8adf02f Merge branch 'master' into fix-force-locale 2024-05-12 00:01:42 +02:00
Kenneth Skovhede 8e25c6e2ab Reduced dynamic scanning with static linking.
Cleaned up some `proj` files.
Added caching to options.
2024-04-26 14:32:41 +02:00
Kenneth Skovhede c2a37d0cd8 Merge remote-tracking branch 'origin/master' into feature/multi-arch-build 2024-04-18 14:09:53 +02:00
Kenneth Skovhede 9f0810612a Changed logic for reporting exceptions, so all statistics are reported despite an exception stopping the backup 2024-04-02 22:31:55 +02:00
Kenneth Skovhede c49e5609a9 Merge remote-tracking branch 'origin/master' into feature/kestrel-avalonia-upgrade 2024-03-07 06:35:15 +01:00
Kenneth SkovhedeandGitHub cbb2b6f295 Merge pull request #5096 from Jojo-1000/fix-invalid-file
Check that the database exists before writing error logs
2024-03-04 13:27:20 +01:00
Kenneth Skovhede 473c6cbbb8 Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade 2024-03-04 12:21:53 +01:00
Thomas Suckow a3bbf63561 Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel
# Conflicts:
#	.appveyor.yml
#	.github/workflows/tests.yml
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	Duplicati CommandLine Only.sln
#	Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
#	Duplicati.Library.RestAPI/Runner.cs
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/HashAlgorithmHelper.cs
#	Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/GeneralBlackBoxTesting.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Installer/debian/docker/debian/control
#	Tools/SQLiteTool/SQLiteTool.csproj
#	Tools/ZipFileDebugger/ZipFileDebugger.csproj
#	guiTests/guiTest.py
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/alphavss/Homepage.txt
#	thirdparty/alphavss/License.txt
#	thirdparty/alphavss/licensedata.json
2024-02-29 21:55:17 -08:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Jojo-1000 a0f9785f5c Check that the database exists before writing error logs.
Closes #5091

Fixes that any fatal error in test-filters created an INVALID! file in the current working dir.
2024-02-09 17:10:05 +01:00
Janandgpatel-fr 71eb03130a Improve result reporting for interrupted/failed backups (Fix #4829)
Direct import of PR#4978
Update web UI for new result reports.
For operations with fatal errors, write logs to same operation ID.
Test that Interrupted flag is correct in RunScriptTests.
Update backup log display for new result reporting.
Hide file statistics for fatal errors and change fatal icon.
2023-09-21 20:01:17 +02:00
Jojo-1000 708715e705 Fix force-locale option for CLI and GUI.
- Localization culture is set from multiple threads on the server, so use current local context for every new call instead of global context
- Change locale before executing help commands if flag is present
- When force-locale is set, also change locale for log messages

Closes #4176
2023-06-23 13:30:46 +02:00
Thomas Suckow bf5b9537d8 Merge remote-tracking branch 'thomas/feature/net5-freebsd' into feature/net5-unsplit
# Conflicts:
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/CLI/Commands.cs
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/ImageLoader.cs
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SSHv2/packages.config
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/TestUtils.cs
#	Duplicati/UnixSupport/UnixSupport.csproj
#	Installer/Makefile/Makefile
#	guiTests/guiTest.py
2022-03-03 21:20:34 -08:00
TopperDEL fc55b16992 fix: More info in deprecation-warning 2021-07-16 08:34:01 +02:00
TopperDEL 5d36791aa5 feat: Inform user about deprecated Tardigrade-Backend with a warning 2021-07-15 17:37:22 +02:00
Thomas Suckow 360d819c0d Threads are not longer abortable 2021-01-03 08:54:09 -08:00
Kenneth Hsu fea55b9981 Avoid removing files unrelated to backup being deleted.
This fixes issues #3845 and #4244.
2020-07-21 12:59:28 -07:00
Sean Templeton d5e064ae45 Fix a few more issues with stop now
When a backup is interrupted it will now show as a partial backup in the list of backups on the restore files page.

Temporary filelist volumes are protected from deletion in several places. All but the RestoreHandler
only protect the most recent filelist. The RestoreHandler protects all temporary filelist volumes as the user
might be restoring from one that would be deleted.

Prevent the CoCoL BackendRequests channel is retired exception from appearing when the backup is stopped by checking
if the channel is retired before trying to add the FlushRequest to the channel. Any exceptions from the BackendUploader
task will still be thrown.
2020-03-01 21:19:07 -06:00
Kenneth Hsu 83ac03843d Fix spelling errors in comments.
In doing so, we also normalized some line endings.
2019-12-14 09:52:55 -08:00
Kenneth Hsu 53a35e6e06 Remove support for Amazon Cloud Drive.
Amazon discontinued access to the Cloud Drive API.

See https://forum.duplicati.com/t/amazon-cloud-drive-discontinued/7391.
2019-12-01 10:36:36 -08:00
Kenneth Hsu afab61a05b Use pattern matching to simplify casts. 2019-10-19 10:15:38 -07:00
BlueBlock 9ae3f88dd5 Merge remote-tracking branch 'upstream/master' into fix-stop-after-current-file 2019-08-16 23:46:55 -04:00
drwtsn32x cfeabfb2dd Add auto vacuum interval option 2019-07-29 17:56:43 -07:00
drwtsn32x a64c7ee2b9 clean up LastCompact property 2019-07-25 19:51:45 -07:00
BlueBlock ba1c9bac21 resolve codacy issues 2019-07-23 11:14:06 -04:00
BlueBlock 29a1a7c618 fix stop after the current file
'stop after upload' is now
stop after the current file' and the operation will cleanly allow the current file to complete.
2019-07-23 10:35:33 -04:00
drwtsn32x ba8e964b39 Fix comparison check 2019-07-21 09:54:06 -07:00
drwtsn32x 2a24de8676 tweaks 2019-07-21 08:11:42 -07:00
drwtsn32x 3e4495cfdf Add auto-compact-interval option to allow delay between automatic compactions 2019-07-21 07:25:32 -07:00
Kenneth Skovhede 8dae320a7a Added warnings about AmazonCD shutting down 2019-06-25 13:16:18 +02:00
Kenneth Hsu 457254d9e1 Remove unused local variables. 2019-04-17 20:10:53 -07:00
Kenneth Hsu df32525a47 Make parameters to Controller.ListControlFiles method required.
There is only one usage of this method, and it provides both parameters.
2019-01-25 20:19:33 -08:00