Commit Graph
2423 Commits
Author SHA1 Message Date
Kenneth SkovhedeandGitHub 168afe4a85 Merge pull request #6082 from duplicati/feature/improve-manifest-error-message
Better error message on manifest version mismatch.
2025-03-25 08:49:45 +01:00
Kenneth Skovhede 0841e884db Better error message on manifest version mismatch.
This changes the error message reported if the manifest or encoding fileds contain values that are not supported.

This fixes #2065
2025-03-24 22:49:05 +01:00
Kenneth Skovhede bf1ef0e794 Fixed nullability warnings 2025-03-24 10:31:52 +01:00
Kenneth SkovhedeandGitHub ae59ed294f Merge pull request #6067 from duplicati/feature/handle-stop-exceptions
Handle stop exceptions
2025-03-21 10:05:52 +01:00
Kenneth Skovhede a163f53dba Handle stop exceptions
This adds a common way to catch stop exceptions and handles them so error resilient logic does not continue when a stop exception is thrown.

Previously, the logic would only check for `ThreadAbortException` which was used to signal a stop prior to .NET8.
2025-03-20 22:32:04 +01:00
Kenneth Skovhede 3c741cbcfc Delete empty index files
If there are index files with no references, they will now be deleted by the repair process.

Such files are likely leftovers from an earlier version that would leave such files if various errors happened.
2025-03-20 22:21:57 +01:00
Kenneth SkovhedeandGitHub 146a2ace51 Merge branch 'master' into feature/improve-sqlite-config 2025-03-20 12:05:28 +01:00
Kenneth SkovhedeandGitHub 2e3f867fa6 Merge pull request #6061 from duplicati/feature/untangle-db-and-results
Untangle the results and database
2025-03-20 12:02:55 +01:00
Kenneth Skovhede fbe79b1ca3 Improve SQLite loading
This commit changes the way the new restore flow loads the extra database connections so these will also apply the temporary folder settings and custom pragmas.

This also changes the logic around the SQLite temporary folder. Previously, this folder would unconditionally be set to the general temporary folder, allowing only an override with a custom pragma.

No the setup checks if the system has the environment variable `SQLITE_TMPDIR` set already. If this variable is already set, it will NOT be overwritten.
Actual effects of this change depends on how SQLite internally resolves the temporary folder.
2025-03-20 10:05:10 +01:00
Kenneth SkovhedeandGitHub 7d8b0bd96b Merge pull request #6054 from duplicati/feature/prefer-named-parameters-in-sql
Move away from positional parameters in SQL statements.
2025-03-20 09:52:17 +01:00
Kenneth Skovhede 7f008341df Untangle the results and database
The code was using a two-way reference linking the database to the results and the results to the database.

However, this was not really used so it ended up just cluttering the code. With this commit, the results and databases are no longer keeping track of each other, and the only interaction is when the results are written to the database at the end of the operation.

To ensure result data is always written, this code has been moved into the controller, which will now flush the results for all operations. Prior to this, only some operations would write result data to the log.

The controller now also handles logic for cleaning old log data and invoking the vacuum command.

Fixed a bug due to using System.Text.Json where some attributes from NewtonSoft.Json were being applied and not picked up.
2025-03-20 09:51:49 +01:00
Kenneth Skovhede fac135d5a5 Updated restore database to not use positional arguments 2025-03-19 22:23:57 +01:00
Kenneth Skovhede 47afbbbdad Fixed a transitional issue 2025-03-19 21:42:08 +01:00
Kenneth Skovhede 833a77bcf6 Fixed a transition issue 2025-03-19 17:21:16 +01:00
Kenneth Skovhede f509c174d3 Fixed a missing = 2025-03-19 15:32:05 +01:00
Kenneth Skovhede 6c7843b332 Unified options
This PR adds a unified way of supplying timeouts to each of the backends.

It also adds an opt-in way of adding commonly used options, so parsing and usage can be standardized across multiple backends.
2025-03-19 15:27:02 +01:00
Kenneth Skovhede fe1cef3be8 Move away from positional parameters in SQL statements.
This is a move away from using positional parameters in SQL statements and rok towards only using named parameters.
2025-03-18 22:41:11 +01:00
Kenneth Skovhede 4a46918760 Additional cleanup in database files 2025-03-18 20:30:27 +01:00
Kenneth Skovhede 902bd22185 Making comparison in local time 2025-03-17 13:32:39 +01:00
Kenneth Skovhede bb1aa8e1f7 Cosmetic cleanup
Removed superfluous namespaces.
Using `var` most places.
Using extension methods for common setup of commands.
2025-03-14 14:34:56 +01:00
Kenneth SkovhedeandGitHub e038a23c44 Merge pull request #6036 from duplicati/feature/invariant-format-sql-strings
Implemented forced invariant locale on all SQL strings.
2025-03-14 13:35:10 +01:00
Kenneth Skovhede 7234d43961 Fixed all SQL strings to use interpolation and forced them into the FormattableString handler using the invariant formatting. 2025-03-14 12:13:17 +01:00
Kenneth Skovhede c1bdaa069e Async operation handlers
Rewrote all operation handlers to be async.
This moves the async/sync threshold out into the controller.

Fixed all the uses of cancellation token in the handlers to use the provided cancellation token.

Introduced a new ReusableTransaction that can be passed around, avoiding a ref parameter passing of the transaction instance.
2025-03-14 10:41:14 +01:00
Kenneth Skovhede f342d70819 Implemented forced invariant locale on all SQL strings.
This fixes #6023
2025-03-13 23:06:32 +01:00
Kenneth Skovhede 30ec975469 Relaxed size requirement to better work with tests 2025-03-13 13:34:55 +01:00
Kenneth Skovhede 5edfcc5c6a Prevent blocksize near remote volume size
If the user selects the blocksize to be the same as the remote volume size, this will cause each block to be in a separate volume and will trigger repeated compact calls.

This commit adds a check that ensures the volume size is large enough to hold at least two blocks.

Also fixed an issue where the compact could create volumes larger than the set limit (at most one block larger).
2025-03-13 12:09:57 +01:00
Kenneth SkovhedeandGitHub eac394f0a3 Merge pull request #6011 from duplicati/feature/improved-failure-resilience
Improved failure resilience
2025-03-12 14:48:14 +01:00
Carl Johnsen ebb04ddd08 Made waking up a waiting block request on the BlockManagers cache async to lower time spent by the volume consumer task 2025-03-11 14:23:37 +01:00
Carl Johnsen b61d594180 Added higher resolution internal profiling timers for the BlockManager 2025-03-11 14:22:49 +01:00
Kenneth Skovhede a0c1676bca Added option to force repair with an older database. 2025-03-11 10:45:22 +01:00
Kenneth Skovhede ceae46f395 Fixed a long-standing bug that could cause errors on delete due to pending delete of filesets 2025-03-10 22:01:14 +01:00
Kenneth Skovhede 086d98ee4b Detect non-attached files in filesets 2025-03-10 21:02:16 +01:00
Kenneth Skovhede d500097e05 Added additional consistency checks to local database 2025-03-10 14:41:36 +01:00
Kenneth Skovhede 55fc5d2e6c Added support for recreating filesets and remote lists in some scenarios 2025-03-10 14:41:12 +01:00
Kenneth Skovhede be16323f24 General cleanup, common database validation exception 2025-03-10 14:38:45 +01:00
Carl-Johannes JohnsenandGitHub f0167885e0 Merge pull request #6026 from carljohnsen/bugfix/restore-deadlock
Bugfix/restore deadlock
2025-03-10 13:21:12 +01:00
Kenneth Skovhede 32762bec34 Fixed comment and added warning message 2025-03-10 10:59:53 +01:00
Kenneth Skovhede 9a82beaf8b Fixed log messages to correspond to actual operation performed 2025-03-10 10:55:07 +01:00
Kenneth Skovhede 6f7f2757a9 Rewrote timeout monitoring 2025-03-10 10:22:34 +01:00
Carl Johnsen f77fcc186c Merge remote-tracking branch 'upstream/master' into bugfix/restore-deadlock 2025-03-10 05:50:26 +01:00
Carl Johnsen 83dc334069 Improve error handling in restore operations by retiring inputs and outputs on exceptions 2025-03-10 05:44:31 +01:00
Carl Johnsen 63464342cd Changed the request timeout to issue a warning, rather than an error 2025-03-09 08:16:10 +01:00
Kenneth Skovhede ceff365ff8 Fixed compact count to use sample count, instead of file count. 2025-03-08 15:30:17 +01:00
Kenneth Skovhede 12cabf8da7 Merge branch 'feature/improved-failure-resilience' of github.com:duplicati/duplicati into feature/improved-failure-resilience 2025-03-08 15:28:24 +01:00
Kenneth Skovhede 1de0a92413 Strategy is now to disallow compact/delete/purge if the remote state contains partial data.
Backup will auto-repair, and user otherwise need to call "repair" to recover the state, before invoking compact/delete/purge.
This is done to prevent cases where the remote state contiains partial data and the automatic cleanup process would remove remote files.
2025-03-08 15:28:02 +01:00
Kenneth Skovhede 5f8ff74bd5 Streamlined handling of previous backup's fileset and dlist file if it failed to upload. 2025-03-08 15:25:55 +01:00
Kenneth Skovhede fce09596f3 Removed unused parameter and redundant namespace prefixes 2025-03-08 15:18:19 +01:00
Kenneth Skovhede 9952e31f49 Fixed a bug where retention time was incorrectly parsed when reading from the database.
This effectively caused the retention time to be ignored when reading from the database.
Also moved some code related to finding an unfinished fileset into the common database class.
2025-03-08 15:13:03 +01:00
Kenneth Skovhede f6f0cf8b96 More docs 2025-03-07 15:45:51 +01:00
Kenneth Skovhede c3762a57ca Added docs 2025-03-07 15:23:06 +01:00