Commit Graph
659 Commits
Author SHA1 Message Date
Kenneth Skovhede 11a233c20a Merge commit '4a6bf8aa019d86cc9a10eefa55aced14e8a13128' into feature/improve-block-reuse 2025-04-06 09:25:10 +02:00
Kenneth Skovhede 1095850965 Reworked the way deleted blocks are detected.
This should have a much lower performance impact.
2025-04-06 08:41:59 +02:00
Kenneth Skovhede cfad020ef2 Merge commit '61892a40d9aff663da86eda273505970268f6384' into feature/improve-block-reuse 2025-04-06 08:36:24 +02:00
Kenneth SkovhedeandGitHub 15c2079cdc Merge branch 'master' into feature/improve-compact-block-use-check 2025-04-06 08:28:03 +02:00
Kenneth Skovhede 2deb147e71 Fixed change stats and added a test 2025-04-05 22:58:10 +02:00
Kenneth Skovhede 8b72528030 Re-implemented the change statistics calculation. 2025-04-05 21:42:05 +02:00
Kenneth Skovhede bb24036602 Fixed performance degradation after parameter queries 2025-04-05 09:25:10 +02:00
Kenneth Skovhede 52fc144580 Merge branch 'master' into feature/improve-compact-block-use-check 2025-04-04 14:41:58 +02:00
Kenneth Skovhede 0ed672ff33 Fixed missing @Type parameter 2025-04-03 22:33:09 +02:00
Kenneth Skovhede 2060e9067d Clear databases fileset entries prior to recreate 2025-04-03 22:20:42 +02:00
Kenneth Skovhede 457349fd2d More cleanup 2025-04-03 22:15:19 +02:00
Kenneth Skovhede 0b735d79ff More cleanup and a few bug fixes 2025-04-03 16:38:51 +02:00
Kenneth Skovhede 2006556778 Fixed some more nullable support in the database layer 2025-04-03 15:46:20 +02:00
Kenneth Skovhede 0638afb8b8 Fixed some nullable support 2025-04-03 15:25:20 +02:00
Kenneth Skovhede ef449bc569 Some nullable updates 2025-04-03 14:32:47 +02:00
Kenneth Skovhede 35682bd4fe Cleaned up some nullability issues 2025-04-03 14:20:00 +02:00
Kenneth Skovhede e96e769ba6 Removed support for positional arguments in extension methods 2025-04-03 13:59:39 +02:00
Kenneth Skovhede 6db50381cb Additional fixes 2025-04-03 13:58:06 +02:00
Kenneth Skovhede 9e3aa72d00 Added LocalBackupDatabase 2025-04-03 11:31:36 +02:00
Kenneth Skovhede 7c8571ed33 Added LocalTestDatabase 2025-04-03 09:03:22 +02:00
Kenneth Skovhede defb5a385b Added LocalRestoreDatabase 2025-04-03 09:00:55 +02:00
Kenneth Skovhede 424cc6f3ad Added LocalRepairDatabase 2025-04-02 23:34:30 +02:00
Kenneth Skovhede 7b948a2f1d Fixed issue with SQLite bindings not supporting arrays.
Added helper class to wrap values in a temporary table if they exceeed the maximum number of parameters.
2025-04-02 23:26:20 +02:00
Kenneth Skovhede ba8866f7d8 Added LocalRecreateDatabase 2025-04-02 16:54:16 +02:00
Kenneth Skovhede d7ccfa378d Added LocalPurgeDatabase 2025-04-02 16:20:12 +02:00
Kenneth Skovhede c1d83a4e59 Removed unused parameters 2025-04-02 14:51:31 +02:00
Kenneth Skovhede 6a4e0a989d Added LocalListDatabase 2025-04-02 14:46:38 +02:00
Kenneth Skovhede b0cab485ef Added LocalListChangesDatabase 2025-04-02 14:32:39 +02:00
Kenneth Skovhede 53a1d2a723 Added LocalListBrokenFilesDatabase 2025-04-02 14:06:06 +02:00
Kenneth Skovhede 1a1761b041 Added LocalListAffectedDatabase 2025-04-02 13:57:23 +02:00
Kenneth Skovhede a6a6790395 Updated LocalDeleteDatabase 2025-04-02 13:49:21 +02:00
Kenneth Skovhede 834436cebe LocalBugReportDatabase 2025-04-02 13:31:31 +02:00
Kenneth Skovhede d593398779 Fixed merge issue 2025-04-01 22:07:51 +02:00
Kenneth Skovhede f5061476c0 Merge remote-tracking branch 'origin/master' into feature/improve-block-reuse 2025-04-01 22:02:58 +02:00
Kenneth Skovhede 8753e5dc85 Merge remote-tracking branch 'origin/master' into feature/improve-compact-block-use-check 2025-04-01 15:54:54 +02:00
Carl Johnsen a4cf24e854 Merge branch 'master' into bugfix/restore-folder-metadata 2025-03-26 12:07:27 +01:00
Kenneth Skovhede 08cb034acf Added custom version of the validation to support not failing validation prior to running the repair that fixes the error. 2025-03-25 11:56:52 +01:00
Kenneth SkovhedeandGitHub 85f5ddde1d Merge branch 'master' into feature/guard-against-missing-files 2025-03-25 11:39:13 +01:00
Carl Johnsen f73d3390cd The FileProcessor now works in two phases: first it restores the files, then they restore the folder metadata, unless specified otherwise. 2025-03-24 20:24:37 +01:00
Carl Johnsen c19af329f4 Removed whitespace 2025-03-24 14:00:39 +01:00
Kenneth Skovhede bf1ef0e794 Fixed nullability warnings 2025-03-24 10:31:52 +01:00
Kenneth Skovhede 3c741cbcfc Delete empty index files
If there are index files with no references, they will now be deleted by the repair process.

Such files are likely leftovers from an earlier version that would leave such files if various errors happened.
2025-03-20 22:21:57 +01:00
Kenneth Skovhede d40301b2b7 Guard against fileset errors
This PR adds a few extra consistency checks to the database, so any failure in the operations is detected earlier than before.

Especially when cleaning temporary and partial files, it is important that the deletes do not remove unexpected entries.

Also, after a recreate, repair or purge command, the database consistency is verified, so it becomes easier to identify which operation caused the failure.

To assist in trouble shooting, there is also a more elaborate error message if a fileset discrepancy is detected, that pinpoints which versions are affected.

Finally, the repair command has been extended to support recreating a fileset from the remote data.
2025-03-20 15:59:14 +01:00
Kenneth SkovhedeandGitHub 146a2ace51 Merge branch 'master' into feature/improve-sqlite-config 2025-03-20 12:05:28 +01:00
Kenneth SkovhedeandGitHub 2e3f867fa6 Merge pull request #6061 from duplicati/feature/untangle-db-and-results
Untangle the results and database
2025-03-20 12:02:55 +01:00
Kenneth Skovhede fbe79b1ca3 Improve SQLite loading
This commit changes the way the new restore flow loads the extra database connections so these will also apply the temporary folder settings and custom pragmas.

This also changes the logic around the SQLite temporary folder. Previously, this folder would unconditionally be set to the general temporary folder, allowing only an override with a custom pragma.

No the setup checks if the system has the environment variable `SQLITE_TMPDIR` set already. If this variable is already set, it will NOT be overwritten.
Actual effects of this change depends on how SQLite internally resolves the temporary folder.
2025-03-20 10:05:10 +01:00
Kenneth SkovhedeandGitHub 7d8b0bd96b Merge pull request #6054 from duplicati/feature/prefer-named-parameters-in-sql
Move away from positional parameters in SQL statements.
2025-03-20 09:52:17 +01:00
Kenneth Skovhede 7f008341df Untangle the results and database
The code was using a two-way reference linking the database to the results and the results to the database.

However, this was not really used so it ended up just cluttering the code. With this commit, the results and databases are no longer keeping track of each other, and the only interaction is when the results are written to the database at the end of the operation.

To ensure result data is always written, this code has been moved into the controller, which will now flush the results for all operations. Prior to this, only some operations would write result data to the log.

The controller now also handles logic for cleaning old log data and invoking the vacuum command.

Fixed a bug due to using System.Text.Json where some attributes from NewtonSoft.Json were being applied and not picked up.
2025-03-20 09:51:49 +01:00
Kenneth Skovhede fac135d5a5 Updated restore database to not use positional arguments 2025-03-19 22:23:57 +01:00
Kenneth Skovhede 47afbbbdad Fixed a transitional issue 2025-03-19 21:42:08 +01:00