haeaeae/duplicati
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,690 Commits 140 Branches 416 Tags
043b9230b5f74cede96f99eff6cc299cffa3e31c
Commit Graph
577 Commits
Author SHA1 Message Date
Kenneth SkovhedeandGitHub f7b7555e87 Fixes for tests (#5918) 
* Two small test fixes

* Ignore exceptions when comitting an empty transaction

* Attempt to fix FTP connection issue

* Unified test by having a single download function.
Added DNS probing call to attempt fixing sporadic DNS errors.

* Fixed argument order

* Avoid creating temp file during tests, if not needed
 2025-01-28 08:55:28 +01:00
Kenneth SkovhedeandGitHub d99f74ed94 Rewrite the backend manager. (#5896) 
* Rewrite the backend manager.

This creates a new more logic backend manager that handles all backend operations.

For each top-level operation, there is now a single backend manager instance that is passed to sub-commands.

The internals of the backend manager are now rewritten to use async logic instead of threading.

The design uses a single task runner that dispatches operations and honors concurrent settings and limits from one place.

The logic for each operation has been moved into a separate files/classes to make it easier to understand each operation.

This fixes #5804

* Fixed a few issues with not awaiting tasks

* Fixed not creating empty databases

* Fixed an issue with hashes not being recorded on download

* Reworked the download logic to avoid attempting to decrypt the file if it has been modified.

* Review fixes

* Renamed db collector to better reflect the purpose.

* Review fixes
 2025-01-28 08:54:50 +01:00
Kenneth SkovhedeandGitHub 52a73c3418 Fixed issue 5845 (#5880) 
* Fixed issue 5845

This commit introduces the test that makes the error.

* Fixed query cleaning up duplicates, in the case that where a block has multiple valid source candidates.

* Reduced number of replicas for the test
 2025-01-16 14:27:15 +01:00
Kenneth Skovhede fb1d15fb9d Avoid optimize after vacuum 2025-01-14 19:42:03 +01:00
Kenneth SkovhedeandGitHub 6bbd64a655 Fixed nullability warnings in Utility.cs (#5861) 
* Fixed nullability warnings in `Utility.cs`

* Fixed a number of nullability issues and warnings

* More nullable fixes

* Fixed the overwrite issue

* Fixed additional nullability issues

* Removed additional nullability issues

* More nullability fixes and added some xmldoc

* Added more xmldocs and removed an unused argument

* Removed more warnings
 2025-01-14 14:03:48 +01:00
Kenneth SkovhedeandGitHub d64c7e721f Handle duplicated missing blocks (#5858) 
* Implemented test for #5845

* Removed unittests not executing cleanup.

* Cleanup commit

* Fixed the query that causes the constraint violation
 2025-01-10 09:24:54 +01:00
Kenneth SkovhedeandGitHub 9464caf622 Feature/update license 2025 (#5851) 
* Fixed some minor whitespace issues

* Updated all copyright to 2025
 2025-01-07 09:40:39 +01:00
Carl Johnsen 198e4d8e6e Refactor SQL query formatting 2025-01-03 11:06:49 +01:00
Carl Johnsen 661a73ff2a Merge remote-tracking branch 'upstream/master' into SlowBigRestore 2025-01-03 10:11:50 +01:00
Carl Johnsen 345701576c Merge branch 'master' into performance/restoreopt 2024-12-05 12:33:30 +01:00
Kenneth Skovhede f5cabd4c7f Remove --use-block-cache 
The option has no effect.
This closes #3566
The block cache was added originally to work around perceived slow hash table lookups in the database.
By having an in-memory dictionary, lookups would be much faster.
It has no effect on disk wear, as the lookups are read-only.
The impact of such a cache was measured to be minimal with recent versions of SQLite.
 2024-12-04 10:06:49 +01:00
Carl Johnsen 6ba956dfcd Moved all database operations to the LocalRestoreDatabase. 
Also changed the old changes, so that the functions related to the legacy restore remains untouched.
 2024-12-03 08:35:14 +01:00
Carl Johnsen f85ffabe3e Linux SQLite doesn't want values in quotes 2024-12-02 10:45:52 +01:00
Carl Johnsen 079598c2e8 Added database function for getting files alongside symlinks to restore 2024-11-29 03:41:41 +01:00
Carl Johnsen 7aa9db533e Added preliminary metadata restoration 2024-11-26 20:01:19 +01:00
Carl Johnsen ebac24ce04 Rather than specifying multiple guids for a restore, use the one that the restore database has. 2024-11-25 08:52:51 +01:00
Carl Johnsen 0cd9374bed Added comments and docstrings 2024-11-22 15:26:04 +01:00
Carl Johnsen 43a392f139 The tempfiletable name is also required externally from the database. 2024-11-22 11:26:25 +01:00
Carl Johnsen 878780dfe4 Added Database.ExecuteReaderEnumerable for prepared statements 2024-11-19 10:32:54 +01:00
Carl Johnsen 50c6fb434c Got the list of block IDs per file 2024-11-08 05:56:48 +01:00
Kenneth SkovhedeandGitHub 3e3258b081 Merge pull request #5615 from duplicati/feature/fix-issue-5038 
Feature/fix issue 5038
 2024-11-05 10:07:00 +01:00
Kenneth Skovhede cae5b218d4 Potential fix for the missing blocklist hash 2024-11-04 21:46:12 +01:00
Kenneth SkovhedeandGitHub e959542536 Merge pull request #5623 from duplicati/feature/fix-compact-disruption-test 
Fix issues with extra blocks and files being recorded after an interrupted `Compact()`
 2024-11-04 16:59:23 +01:00
Kenneth SkovhedeandGitHub ababf2f84e Merge pull request #5610 from duplicati/feature/fix-leftover-index 
Feature/fix leftover index
 2024-11-04 16:50:05 +01:00
Kenneth Skovhede 448b6e0e03 Fixed typo 2024-11-01 16:54:39 +01:00
Kenneth Skovhede a7e57c2ffb Added DuplicateBlock to the list of items purged. 2024-11-01 16:06:41 +01:00
Kenneth Skovhede 1809ffac09 Added a fix from #4982 that deletes additional information from the database when removing broken files. 2024-11-01 16:06:20 +01:00
Kenneth Skovhede dedee8fd35 Hand-picked the recreate fixes from #4982. 
This fixes that the recreate will move unused blocks into the `DeletedBlock` table after the database has been recreated.
Due to this, the disruption test can now be verified.
 2024-11-01 15:56:14 +01:00
Kenneth Skovhede 1dc2acb2bf Ensure repair also works with the index 2024-11-01 15:02:48 +01:00
Kenneth Skovhede db517a300b Implemented a fix for compaction issues that could cause inconsistent views of the remote storage. 2024-11-01 14:50:26 +01:00
Kenneth Skovhede 30a3410180 Changed strategy to check if block is known in a previous blockset. 
If this fileset is the first containing the block in a blockset, we add it to the index file.
 2024-10-31 15:30:25 +01:00
Kenneth Skovhede e9c256c7f2 Merge remote-tracking branch 'origin/master' into feature/fix-leftover-index 2024-10-29 16:40:06 +01:00
Kenneth Skovhede 300752652f Added detection of the SQLite journal file for the active database. 
This fixes #5603
 2024-10-28 09:00:40 +01:00
Kenneth Skovhede be0e4d2ed1 Fixed an incorrect string quote 2024-10-05 17:36:42 +02:00
Kenneth Skovhede ff6757877f Fixing the inconsistency check on repair. 
Update the SQL to also group on size (just in case).
Renamed the normal read method to signal that it is unverified.
Made the verified reader use enumeration state machine to avoid materializing the list.
`var`-ified some of the test code.
 2024-09-09 15:27:37 +02:00
Kenneth Skovhede 17d9b89fbb Merge branch 'master' into fix-repair-index-consistency 
# Conflicts:
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
 2024-09-09 11:49:34 +02:00
Suguru Hirahara ae57dee0f0 Remove redundant space characters from *.cs and *.html 2024-07-26 00:10:31 -04:00
Kenneth SkovhedeandGitHub bfc3d61bc2 Merge branch 'master' into fix-leftover-index 2024-07-19 13:53:06 +02:00
Kenneth Skovhede 3f29331dc6 Fixed a case where listing would fail with empty filters 2024-06-21 07:52:07 +02:00
Jojo-1000 e316944a50 Fix compare on linux showing 0 bytes when exclude filters are used. 
Closes #5229
 2024-06-10 16:00:39 +02:00
Kenneth SkovhedeandGitHub fdc891aa36 Merge pull request #5208 from Jojo-1000/fix-sql-quotes 
Fix SQL string quotes
 2024-06-04 20:58:20 +02:00
Jojo-1000 917e2cd0c4 Remove obsolete custom platform detection. 2024-05-30 02:14:05 +02:00
Jojo-1000 55284410c7 Fix sql queries that use double quoted strings. 
Closes #5202
 2024-05-30 00:00:48 +02:00
Jojo-1000 0b92f19572 Merge branch 'master' into fix-leftover-index 2024-05-11 23:37:21 +02:00
Kenneth Skovhede 6362dd3cf8 Updated query to remove warning from SQLite logging 2024-04-16 14:19:15 +02:00
Kenneth Skovhede 49fb45859c Fixed some incorrectly quoted literals in SQL statements 2024-04-15 08:24:01 +02:00
Kenneth Skovhede c49e5609a9 Merge remote-tracking branch 'origin/master' into feature/kestrel-avalonia-upgrade 2024-03-07 06:35:15 +01:00
Kenneth SkovhedeandGitHub cbb2b6f295 Merge pull request #5096 from Jojo-1000/fix-invalid-file 
Check that the database exists before writing error logs
 2024-03-04 13:27:20 +01:00
Kenneth Skovhede 473c6cbbb8 Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade 2024-03-04 12:21:53 +01:00
Thomas Suckow a3bbf63561 Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel 
# Conflicts:
#	.appveyor.yml
#	.github/workflows/tests.yml
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	Duplicati CommandLine Only.sln
#	Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
#	Duplicati.Library.RestAPI/Runner.cs
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/HashAlgorithmHelper.cs
#	Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/GeneralBlackBoxTesting.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Installer/debian/docker/debian/control
#	Tools/SQLiteTool/SQLiteTool.csproj
#	Tools/ZipFileDebugger/ZipFileDebugger.csproj
#	guiTests/guiTest.py
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/alphavss/Homepage.txt
#	thirdparty/alphavss/License.txt
#	thirdparty/alphavss/licensedata.json
 2024-02-29 21:55:17 -08:00