Kenneth Skovhede
7031bc57b6
Adding ConfigureAwait(false) to all async enumerations
2025-02-18 09:18:36 +01:00
Kenneth Skovhede
536e62275f
Changed the logic in recreate to avoid opening the same zip file multiple times.
2025-02-06 21:34:02 +01:00
Kenneth Skovhede and GitHub
d99f74ed94
Rewrite the backend manager. ( #5896 )
...
* Rewrite the backend manager.
This creates a new more logic backend manager that handles all backend operations.
For each top-level operation, there is now a single backend manager instance that is passed to sub-commands.
The internals of the backend manager are now rewritten to use async logic instead of threading.
The design uses a single task runner that dispatches operations and honors concurrent settings and limits from one place.
The logic for each operation has been moved into a separate files/classes to make it easier to understand each operation.
This fixes #5804
* Fixed a few issues with not awaiting tasks
* Fixed not creating empty databases
* Fixed an issue with hashes not being recorded on download
* Reworked the download logic to avoid attempting to decrypt the file if it has been modified.
* Review fixes
* Renamed db collector to better reflect the purpose.
* Review fixes
2025-01-28 08:54:50 +01:00
Kenneth Skovhede and GitHub
6c493186a7
Changed MissingFileDetected message to Warning ( #5893 )
...
* Changed `MissingFileDetected` message to `Warning`
* Updated tests to match errors/warning counts
2025-01-22 21:18:33 +01:00
Kenneth Skovhede and GitHub
d64c7e721f
Handle duplicated missing blocks ( #5858 )
...
* Implemented test for #5845
* Removed unittests not executing cleanup.
* Cleanup commit
* Fixed the query that causes the constraint violation
2025-01-10 09:24:54 +01:00
Kenneth Skovhede and GitHub
9464caf622
Feature/update license 2025 ( #5851 )
...
* Fixed some minor whitespace issues
* Updated all copyright to 2025
2025-01-07 09:40:39 +01:00
Kenneth Skovhede and GitHub
ff1b2b7990
Simplified task control ( #5753 )
...
* Simplified task control by having all logic in a single class and using async control mechanism
* Fixed the tests
* Made the marking of a backup partial explicitly communicated if the enumeration process is stopped.
* Added support for pausing active transfers
* Introduced transfer token to abort operations.
* Updated the way LiveControl is emitting events.
* Retains the paused state of the server across reboots, this fixes #5760 .
* Reworked the way throttle speeds are propagated through LiveControls.
* Added option to pause transfers in the UI.
* Removed throttle settings from the LiveControls class so it is exclusively handled by the ApplicationSettings.
* Removed `thread-priority` as it does not work at all when tasks are used.
* Fixed the stop dialog and API to only support stop and abort.
* Removed unused variables
2024-12-18 08:27:59 +01:00
Kenneth Skovhede
dedee8fd35
Hand-picked the recreate fixes from #4982 .
...
This fixes that the recreate will move unused blocks into the `DeletedBlock` table after the database has been recreated.
Due to this, the disruption test can now be verified.
2024-11-01 15:56:14 +01:00
Kenneth Skovhede
7d43687090
Reverted to materializing the blocklist as we cannot check the hash before, and we need to throw the exception before we insert any entries.
2024-09-09 17:13:59 +02:00
Kenneth Skovhede
ff6757877f
Fixing the inconsistency check on repair.
...
Update the SQL to also group on size (just in case).
Renamed the normal read method to signal that it is unverified.
Made the verified reader use enumeration state machine to avoid materializing the list.
`var`-ified some of the test code.
2024-09-09 15:27:37 +02:00
Kenneth Skovhede
17d9b89fbb
Merge branch 'master' into fix-repair-index-consistency
...
# Conflicts:
# Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
2024-09-09 11:49:34 +02:00
Suguru Hirahara
8e85c10754
Set arguments for strings
2024-07-29 11:30:43 -04:00
Kenneth Skovhede
c890e824ee
Implemented test for reproducing issue #5196
...
Fixed issue by changing order options are written and verified.
Some whitespace changes due to renaming a method
2024-05-21 15:48:29 +02:00
Kenneth Skovhede
473c6cbbb8
Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade
2024-03-04 12:21:53 +01:00
Thomas Suckow
a3bbf63561
Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel
...
# Conflicts:
# .appveyor.yml
# .github/workflows/tests.yml
# BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
# BuildTools/DependencyFinder/DependencyFinder.csproj
# BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
# BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
# Duplicati CommandLine Only.sln
# Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
# Duplicati.Library.RestAPI/Runner.cs
# Duplicati.sln
# Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
# Duplicati/CommandLine/BackendTester/app.config
# Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
# Duplicati/CommandLine/BackendTool/app.config
# Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
# Duplicati/CommandLine/Duplicati.CommandLine.csproj
# Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
# Duplicati/CommandLine/app.config
# Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
# Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
# Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
# Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
# Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
# Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
# Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
# Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
# Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
# Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
# Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
# Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
# Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
# Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
# Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
# Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
# Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
# Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
# Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
# Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
# Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
# Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
# Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
# Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
# Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
# Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
# Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
# Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
# Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
# Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
# Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
# Duplicati/Library/Common/Duplicati.Library.Common.csproj
# Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
# Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
# Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
# Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
# Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
# Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
# Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
# Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
# Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
# Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
# Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
# Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
# Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
# Duplicati/Library/Utility/HashAlgorithmHelper.cs
# Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
# Duplicati/License/Duplicati.License.csproj
# Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
# Duplicati/Server/Duplicati.Server.csproj
# Duplicati/Server/app.config
# Duplicati/Service/Duplicati.Service.csproj
# Duplicati/Service/app.config
# Duplicati/Tools/Duplicati.Tools.csproj
# Duplicati/UnitTest/Duplicati.UnitTest.csproj
# Duplicati/UnitTest/GeneralBlackBoxTesting.cs
# Duplicati/UnitTest/app.config
# Duplicati/WindowsService/WindowsService.csproj
# Installer/debian/docker/debian/control
# Tools/SQLiteTool/SQLiteTool.csproj
# Tools/ZipFileDebugger/ZipFileDebugger.csproj
# guiTests/guiTest.py
# thirdparty/UnixSupport/UnixSupport.csproj
# thirdparty/alphavss/Homepage.txt
# thirdparty/alphavss/License.txt
# thirdparty/alphavss/licensedata.json
2024-02-29 21:55:17 -08:00
Kenneth Skovhede
40dd3c6816
Updated all license mentions to MIT
2024-02-28 15:45:30 +01:00
Jojo-1000
6a85544fa5
Check blocklist consistency in recreate to prevent broken filesets.
...
Compact used broken LocalDatabase.GetBlocklists without a hash check, so some backups may contain invalid index files.
2023-12-12 21:28:15 +01:00
gpatel-fr and GitHub
ec9d49d684
Experiment/rebuildfaster ( #5043 )
...
* experimental faster db rebuild (#4955 )
* try run update db query for every block file only when needed
add repair-force-block-use option
more appropriate naming for some routines
more tracing
more comments
2023-11-05 18:19:08 +01:00
Thomas Suckow
ecb0249259
dotnet no longer provides a method for a string to hashalgorithm
...
* Also fix up not diposing of hashers
2021-04-04 11:17:13 -07:00
Max
41a5591532
upstream
2019-12-25 16:34:56 +01:00
Kenneth Skovhede and GitHub
153a972853
Merge branch 'master' into fix-stop-after-current-file
2019-09-17 10:06:36 +02:00
drwtsn32x
4df61495c2
change wording
2019-09-11 17:52:57 -07:00
drwtsn32x
76c61699eb
Improve logging during database recreation
2019-09-11 03:43:54 -07:00
BlueBlock
605efac4e7
retrieve filesetData to get IsFullBackup state
2019-09-10 12:51:42 -04:00
BlueBlock
8f93e446b2
re-created filesets are set to full backups
2019-09-02 18:20:21 -04:00
Michal Naiman
1bc1e58737
upstream
2019-04-28 13:45:26 +02:00
Kenneth Skovhede
03722fcba1
Merge branch 'master' into feature/fix_path_storage2
...
# Conflicts:
# Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
2018-10-23 11:33:35 +02:00
Kenneth Hsu
b824c86c7e
Avoid unnecessary enumeration of collections.
...
This improves performance by using Any() to avoid unnecessary
enumeration of collections.
2018-10-06 16:20:18 -07:00
Max
a22a7a7d24
remove of FasterHashing component
2018-09-09 14:42:40 +02:00
Kenneth Skovhede
61060e05b8
Merge branch 'master' into feature/fix_path_storage2
...
# Conflicts:
# Duplicati/Library/Main/Database/LocalBackupDatabase.cs
2018-08-07 22:44:41 +02:00
Kenneth Skovhede
355ef920ea
Fixed the code to allow reading various parameters from the manifest when restoring the database.
...
This change ensures that the update happens, even when creating a partial database as the GUI does.
This fixes #2323
2018-06-30 17:28:23 +02:00
Kenneth Skovhede
cbc18970d5
Re-applied the path-storage fix as the merge failed somehow
2018-06-14 10:12:24 +02:00
Kenneth Skovhede
68c36edbf7
Added a new unittest option that disables auto-fixes to make sure we do not mask errors by auto-fixing them during unittests
2018-06-12 09:31:39 +02:00
Kenneth Skovhede
4b3389f4b6
Fix the missing block references prior to looking for missing block info
2018-06-12 08:29:55 +02:00
Kenneth Skovhede
7081af9785
Added protection against missing or defect files when attempting to restore
2018-06-11 14:42:13 +02:00
Kenneth Skovhede
bce1db123a
Merge branch 'master' into feature/speedup_backup_index_files
...
# Conflicts:
# Duplicati/Library/Utility/Utility.cs
2018-06-08 10:31:27 +02:00
Kenneth Hsu
20f2f37806
Remove unnecessary calls to String.Format.
2018-06-02 12:02:55 -07:00
Kenneth Hsu
f12858e6a5
Mark fields that shouldn't be reassigned as readonly.
...
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2018-05-25 10:04:52 -07:00
Kenneth Skovhede
632b4ff0e2
Added support for cases where the recreate finds missing block files that are not needed.
2018-05-10 00:57:51 +02:00
Kenneth Skovhede
13c33c6f89
Improved logging to isolate the cause of missing remote files
2018-05-09 18:09:20 +02:00
Kenneth Skovhede
79f7b22e0f
Merge branch 'master' into concurrent_processing
...
Changed a few things on the way, removing the log process
# Conflicts:
# Duplicati/Library/Main/Controller.cs
# Duplicati/Library/Main/Database/ExtensionMethods.cs
# Duplicati/Library/Main/Database/LocalBackupDatabase.cs
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
# Duplicati/Library/Main/Operation/BackupHandler.cs
# Duplicati/Library/Main/Operation/FilelistProcessor.cs
# Duplicati/Library/Main/Operation/PurgeBrokenFilesHandler.cs
# Duplicati/Library/Main/Operation/TestFilterHandler.cs
# Duplicati/Library/Main/Options.cs
# Duplicati/Library/Main/ResultClasses.cs
# Duplicati/UnitTest/BasicSetupHelper.cs
2018-04-11 23:02:47 +02:00
Kenneth Skovhede
842fd96543
Implemented a new logging system that is more transparent and allows a more granular way of picking log messages.
...
Added ID's to each log message and each exception to allow later introduction of a Knowledgebase service that explains each error in more detail.
2018-03-15 09:12:34 +01:00
Kenneth Skovhede
75bf6857c1
Merge branch 'master' into concurrent_processing
...
# Conflicts:
# .travis.yml
# Duplicati/Library/Main/Database/LocalBackupDatabase.cs
# Duplicati/Library/Main/Operation/BackupHandler.cs
# Duplicati/Library/Main/ResultClasses.cs
# Duplicati/Library/Main/Volumes/VolumeWriterBase.cs
# Duplicati/Library/Utility/TempFile.cs
2018-01-20 15:02:41 +01:00
Kenneth Hsu
5da12a704d
Convert if statement to switch statement.
...
When the comparison is performed on an enum, using a switch statement
can allow for additional compiler optimizations.
2017-09-26 20:13:40 -07:00
Kenneth Hsu
c082b18389
Declare variable outside of if statement.
...
This will allow us to more easily convert the if statement to a switch
statement.
2017-09-26 20:10:19 -07:00
Kenneth Skovhede
683b04741d
Merge branch 'master' into concurrent_processing
...
# Conflicts:
# Duplicati/Library/Main/Operation/BackupHandler.cs
Manually fixed the HashAlgorithmHelper case
2017-09-20 13:56:37 +02:00
Kenneth Skovhede
6d401fbd8d
Changed all hash algorithms to use the Cng version.
...
This fixes #2621
2017-09-20 09:58:43 +02:00
Kenneth Skovhede
66cf98a8b3
Merge branch 'master' into concurrent_processing
...
# Conflicts:
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
2017-06-20 15:17:02 +02:00
Kenneth Skovhede
a0d9ad5d4e
Changed --backup-prefix hint texts to --prefix.
...
This fixes #2548
2017-06-19 22:06:35 +02:00
Kenneth Skovhede
d18431b385
Merge branch 'master' into concurrent_processing
...
# Conflicts:
# Duplicati/Library/Main/Controller.cs
# Duplicati/Library/Main/Database/ExtensionMethods.cs
# Duplicati/Library/Main/Database/LocalBackupDatabase.cs
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
# Duplicati/Library/Main/Operation/BackupHandler.cs
# Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
# Duplicati/Library/Main/ResultClasses.cs
First attempt at merging all changes from 2016-10-08 to now :/
2017-05-30 00:35:31 +02:00
Kenneth Skovhede