Commit Graph
344 Commits
Author SHA1 Message Date
Carl Johnsen 6afe959c56 Changed the default number of file processors to match the other concurrent counts 2024-12-23 06:50:24 +01:00
Carl Johnsen a566d1a49f Merge remote-tracking branch 'upstream/master' into fileprocessors 2024-12-23 06:45:43 +01:00
Kenneth SkovhedeandGitHub 86f3f6b178 Fixed issues with timeout in msgraph (#5790)
* Fixed issues with timeout in msgraph

Fixed a place where the cancellation token was not applied.

Fixed a problem with msgraph not sending correct DNS information.

Fixed a problem with msgraph downloading the entire file before streaming it, bypassing throttle and tripping the timeout.

Added `ITimeoutExemptBackend` to selectively disable timeouts for backends that do not support it.

Updated the backendmanager to support `ITimeoutExemptBackend`.

Updated the backendmanager to abort as soon as a timeout happens.

* Updated BackendTester to use the `TimeoutObservingStream` and support the `--read-write-timeout` setting.

Updated backend tester code to use `var`

* Fixed a bug reading the `read-write-timeout`

* Updated to using constants with names instead of `-1`.
2024-12-19 09:31:06 +01:00
Kenneth SkovhedeandGitHub ff1b2b7990 Simplified task control (#5753)
* Simplified task control by having all logic in a single class and using async control mechanism
* Fixed the tests
* Made the marking of a backup partial explicitly communicated if the enumeration process is stopped.
* Added support for pausing active transfers
* Introduced transfer token to abort operations.
* Updated the way LiveControl is emitting events.
* Retains the paused state of the server across reboots, this fixes #5760.
* Reworked the way throttle speeds are propagated through LiveControls.
* Added option to pause transfers in the UI.
* Removed throttle settings from the LiveControls class so it is exclusively handled by the ApplicationSettings.
* Removed `thread-priority` as it does not work at all when tasks are used.
* Fixed the stop dialog and API to only support stop and abort.
* Removed unused variables
2024-12-18 08:27:59 +01:00
Kenneth Skovhede 5363bf0d96 Implemented read-write timeout.
This fixes #5725
2024-12-10 22:12:20 +01:00
Carl Johnsen 95887b5841 Set the default cache eviction ratio to be 4gb.
Also set the default value to be parsed from a string.
2024-12-10 06:02:51 +01:00
Carl Johnsen 405e73b2d2 Changed the 'restore-cache-evict' option to be an integer representing percentage instead of a float between 0 and 1 2024-12-10 05:38:57 +01:00
Carl Johnsen f491fa93fc Changed restore scaling parameters to use the default value to produce the default string, rather than a string literal. 2024-12-10 05:32:40 +01:00
Carl Johnsen 9440565706 Changed from hardcoded restore scaling constants to scale with the number of processors on the target machine. 2024-12-10 05:17:59 +01:00
Carl Johnsen 345701576c Merge branch 'master' into performance/restoreopt 2024-12-05 12:33:30 +01:00
Kenneth Skovhede f5cabd4c7f Remove --use-block-cache
The option has no effect.
This closes #3566
The block cache was added originally to work around perceived slow hash table lookups in the database.
By having an in-memory dictionary, lookups would be much faster.
It has no effect on disk wear, as the lookups are read-only.
The impact of such a cache was measured to be minimal with recent versions of SQLite.
2024-12-04 10:06:49 +01:00
Carl Johnsen 3e7b505ed2 Merge branch 'master' into performance/restoreopt 2024-12-03 09:21:14 +01:00
Carl Johnsen 76240550ed Add preallocation option for restored files and update related documentation 2024-12-03 09:10:36 +01:00
Carl Johnsen 7aa19f851e Add configurable volume downloaders for restore process 2024-12-02 15:16:21 +01:00
Carl Johnsen 533088684b Consistent naming of decryptors 2024-12-02 15:12:50 +01:00
Carl Johnsen cdf0c28fd8 Added internal profiling as an option. Currently only implemented in the restore process. 2024-11-29 05:49:03 +01:00
Carl Johnsen c26c54c5f7 Fixed naming inconsistencies in the new restore options 2024-11-25 15:17:10 +01:00
Carl Johnsen 5625c4c1ce Add configurable options for scaling the number of restore file processors, decryptors, and decompressors 2024-11-22 16:53:16 +01:00
Carl Johnsen fe6f91dc9a Add option for using legacy restore method 2024-11-22 16:40:54 +01:00
Carl Johnsen 5436b2f9f5 Add eviction ratio option for restore cache management 2024-11-22 16:32:53 +01:00
Carl Johnsen de24a9f15a Added the option to control the size of the data block cache during restore 2024-11-22 16:24:27 +01:00
Kenneth Skovhede d2f615f442 Added possible values to cpu-intensity for better documentation 2024-11-07 08:10:32 +01:00
Kenneth Skovhede a72bf27aa3 Merge remote-tracking branch 'origin/master' into feature/add-secret-provider 2024-11-05 11:48:30 +01:00
Carl Johnsen 8927dbb81a Added the commandline option for controlling the CPU intensity 2024-11-01 13:32:18 +01:00
Kenneth Skovhede 9a89ca5a15 Cleaned up the use of the default pattern to use a single const value.
Added support for inheriting the secret pattern as it is more logical that all settings are carried through to the application.
2024-10-26 08:37:26 +02:00
Kenneth Skovhede 4b84135622 Merge remote-tracking branch 'origin/master' into feature/add-secret-provider 2024-10-24 16:11:41 +02:00
Kenneth Skovhede 34d13a20e2 Initial version of the external secret provider 2024-10-24 16:06:59 +02:00
Kenneth Skovhede f251d34b57 Implemented support for including the machine hostname in reports for simpler identification. 2024-10-16 16:22:47 +02:00
Kenneth SkovhedeandGitHub dec473eab3 Merge pull request #5588 from marceloduplicati/feature/webdav-ignore-invalid-certificates
Solves Webdav issue with self signed certificates and accepts appropriate options
2024-10-15 11:10:25 +02:00
Marcelo Ceccon fd591d9af2 Fix to Webdav issue around certificate validation
Fix to Webdav to process the options accept-specified-ssl-hash and accept-any-ssl-certificate

Minor refactoring
2024-10-10 10:36:47 +02:00
Kenneth Skovhede 91ea81f537 Added commandline option to toggle locks 2024-10-08 09:35:00 +02:00
Kenneth Skovhede 3c144f5481 Merge remote-tracking branch 'origin/master' into feature/remove-thread-culture-dependence 2024-10-02 14:56:11 +02:00
Kenneth Skovhede 627bccbb94 Removed piped download as it was slower than not piping.
It can be re-added later if a better way is found.
2024-09-26 08:43:01 +02:00
Kenneth Skovhede f3e2a0a902 Avoid setting Thread.CurrentCulture
Avoids setting deprecated `Thread.CurrentCulture` and `Thread.CurrentUICulture`, using `CultureInfo.CurrentCulture` instead.
Fixed a few legacy places where incorrect string compares were used.
2024-09-18 09:24:55 +02:00
Suguru Hirahara 13ca8f6fca Add ConsoleloglevelLong to console-log-level 2024-07-30 02:39:00 -04:00
Suguru Hirahara 6a7bf8fd34 Add LogfileloglevelLong to log-file-log-level 2024-07-30 02:38:52 -04:00
Suguru Hirahara ed0a9c746b Add ListverifyuploadsLong to list-verify-uploads 2024-07-30 02:38:39 -04:00
Suguru Hirahara 5b65562ea6 Update strings to indicate deprecation
- DisablefilepathcacheDeprecated
- FilereadbuffersizeDeprecated

This also makes "DEPRECATED: " translatable.
2024-07-24 00:56:05 -04:00
Kenneth SkovhedeandGitHub 17e2266a69 Merge pull request #5231 from duplicati/feature/increase-default-test-coverage
Increase the default test coverage to be 0.1%
2024-06-25 09:58:50 +02:00
Kenneth Skovhede d264dbd329 Introducing decimal commandline type 2024-06-11 06:50:25 +02:00
Kenneth Skovhede f4dee8a305 Increase the default test coverage to be 0.1% 2024-06-10 11:49:43 +02:00
Kenneth Skovhede f9db217af0 Changed logic for local blocks to be opt-in 2024-06-10 11:37:58 +02:00
Jojo-1000 50808c866b Merge branch 'master' into quota-disable 2024-05-11 23:46:19 +02:00
Kenneth SkovhedeandGitHub 0b566ecf1f Merge pull request #5132 from duplicati/feature/1mib-default-blocksize
1 MiB default blocksize
2024-04-21 09:37:02 +02:00
Kenneth Skovhede 7dfa19d48b Merge branch 'master' into feature/1mib-default-blocksize 2024-03-18 15:19:42 +01:00
Kenneth Skovhede 115e887802 Implemented new updater logic that supports multi-arch distributions. 2024-03-15 14:18:56 +01:00
Kenneth Skovhede b27323c705 Merge remote-tracking branch 'origin/master' into feature/kestrel-avalonia-upgrade 2024-03-08 11:40:33 +01:00
Kenneth Skovhede 6b9165a587 Added support for machine and backup ids in reports 2024-03-07 22:11:49 +01:00
Kenneth Skovhede 473c6cbbb8 Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade 2024-03-04 12:21:53 +01:00
Thomas Suckow a3bbf63561 Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel
# Conflicts:
#	.appveyor.yml
#	.github/workflows/tests.yml
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	Duplicati CommandLine Only.sln
#	Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
#	Duplicati.Library.RestAPI/Runner.cs
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/HashAlgorithmHelper.cs
#	Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/GeneralBlackBoxTesting.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Installer/debian/docker/debian/control
#	Tools/SQLiteTool/SQLiteTool.csproj
#	Tools/ZipFileDebugger/ZipFileDebugger.csproj
#	guiTests/guiTest.py
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/alphavss/Homepage.txt
#	thirdparty/alphavss/License.txt
#	thirdparty/alphavss/licensedata.json
2024-02-29 21:55:17 -08:00