Commit Graph
5034 Commits
Author SHA1 Message Date
Carl Johnsen c19af329f4 Removed whitespace 2025-03-24 14:00:39 +01:00
Kenneth SkovhedeandGitHub ae59ed294f Merge pull request #6067 from duplicati/feature/handle-stop-exceptions
Handle stop exceptions
2025-03-21 10:05:52 +01:00
Kenneth Skovhede a163f53dba Handle stop exceptions
This adds a common way to catch stop exceptions and handles them so error resilient logic does not continue when a stop exception is thrown.

Previously, the logic would only check for `ThreadAbortException` which was used to signal a stop prior to .NET8.
2025-03-20 22:32:04 +01:00
Kenneth Skovhede 3c741cbcfc Delete empty index files
If there are index files with no references, they will now be deleted by the repair process.

Such files are likely leftovers from an earlier version that would leave such files if various errors happened.
2025-03-20 22:21:57 +01:00
Kenneth SkovhedeandGitHub b0d20638f2 Merge pull request #6064 from duplicati/feature/reduce-serialized-results
Update serialized results to omit task control
2025-03-20 20:57:48 +01:00
Kenneth Skovhede ddf90196cd Update serialized results to omit task control
The `TaskControl` and `TaskReader` properties contain no information that can be used in serialized form, but they were added to the output anyway.

With this commit, the unwanted extra data is no longer emitted.
2025-03-20 16:06:11 +01:00
Kenneth SkovhedeandGitHub 146a2ace51 Merge branch 'master' into feature/improve-sqlite-config 2025-03-20 12:05:28 +01:00
Kenneth Skovhede d8a15ac7ed Fixed review comments 2025-03-20 12:05:16 +01:00
Kenneth SkovhedeandGitHub 2e3f867fa6 Merge pull request #6061 from duplicati/feature/untangle-db-and-results
Untangle the results and database
2025-03-20 12:02:55 +01:00
Kenneth Skovhede fbe79b1ca3 Improve SQLite loading
This commit changes the way the new restore flow loads the extra database connections so these will also apply the temporary folder settings and custom pragmas.

This also changes the logic around the SQLite temporary folder. Previously, this folder would unconditionally be set to the general temporary folder, allowing only an override with a custom pragma.

No the setup checks if the system has the environment variable `SQLITE_TMPDIR` set already. If this variable is already set, it will NOT be overwritten.
Actual effects of this change depends on how SQLite internally resolves the temporary folder.
2025-03-20 10:05:10 +01:00
Kenneth SkovhedeandGitHub 7d8b0bd96b Merge pull request #6054 from duplicati/feature/prefer-named-parameters-in-sql
Move away from positional parameters in SQL statements.
2025-03-20 09:52:17 +01:00
Kenneth Skovhede 7f008341df Untangle the results and database
The code was using a two-way reference linking the database to the results and the results to the database.

However, this was not really used so it ended up just cluttering the code. With this commit, the results and databases are no longer keeping track of each other, and the only interaction is when the results are written to the database at the end of the operation.

To ensure result data is always written, this code has been moved into the controller, which will now flush the results for all operations. Prior to this, only some operations would write result data to the log.

The controller now also handles logic for cleaning old log data and invoking the vacuum command.

Fixed a bug due to using System.Text.Json where some attributes from NewtonSoft.Json were being applied and not picked up.
2025-03-20 09:51:49 +01:00
Kenneth SkovhedeandGitHub fb62da8fa6 Merge pull request #6059 from duplicati/feature/honor-insecure-permissions
Honor insecure permissions.
2025-03-19 23:31:16 +01:00
Kenneth Skovhede fac135d5a5 Updated restore database to not use positional arguments 2025-03-19 22:23:57 +01:00
Kenneth Skovhede 47afbbbdad Fixed a transitional issue 2025-03-19 21:42:08 +01:00
Kenneth Skovhede fef0431144 Honor insecure permissions.
This commit changes the logic so permissions are only set on files, if the `insecure-permissions.txt` file is not found in the folder.

Prior to this commit, new files would be locked down without condition.
2025-03-19 21:20:07 +01:00
Kenneth Skovhede 833a77bcf6 Fixed a transition issue 2025-03-19 17:21:16 +01:00
Kenneth Skovhede 8f38c82f1e Changed order of function arguments for improved readability 2025-03-19 15:48:01 +01:00
Kenneth Skovhede f509c174d3 Fixed a missing = 2025-03-19 15:32:05 +01:00
Kenneth SkovhedeandGitHub 49e68292c5 Merge branch 'master' into feature/unified-timeout-and-options 2025-03-19 15:27:47 +01:00
Kenneth SkovhedeandGitHub af7bbb4164 Merge pull request #6055 from duplicati/feature/guard-livecontrol-startup-range
Add guards to startup interval
2025-03-19 15:27:26 +01:00
Kenneth Skovhede 6c7843b332 Unified options
This PR adds a unified way of supplying timeouts to each of the backends.

It also adds an opt-in way of adding commonly used options, so parsing and usage can be standardized across multiple backends.
2025-03-19 15:27:02 +01:00
Kenneth Skovhede 10359cb5b3 Add guards to startup interval
In some cases the stored database state would be misinterpreted causing abnormal wait periods that crashes the program on startup.

This change converts all time compares to UTC asn the database value was already UTC.

It also adds a clamping function to prevent crashes on extreme values (using 24h pause instead of crash).
2025-03-19 09:14:34 +01:00
Kenneth Skovhede fe1cef3be8 Move away from positional parameters in SQL statements.
This is a move away from using positional parameters in SQL statements and rok towards only using named parameters.
2025-03-18 22:41:11 +01:00
Kenneth Skovhede 4a46918760 Additional cleanup in database files 2025-03-18 20:30:27 +01:00
Kenneth SkovhedeandGitHub b1f705f71a Merge pull request #6048 from duplicati/feature/timezone-aware-compare
Making comparison in local time
2025-03-17 14:21:47 +01:00
Kenneth Skovhede 902bd22185 Making comparison in local time 2025-03-17 13:32:39 +01:00
Kenneth Skovhede e37f20aedd Added retries for HTTP messages
This adds a retry count and a retry delay for posting backup reports via HTTP requests. This will make the report delivery more reliable if the network is unstable or server is briefly unreachable.
2025-03-16 16:52:30 +01:00
marceloduplicatiandGitHub 35da99c64c Merge pull request #6041 from marceloduplicati/feature/scan-root-certificates
Re-work of SSL Validation + Root Certificate feature
2025-03-14 17:44:41 -03:00
Marcelo C. 12f53513d0 Re-work of SSL Validation + Root Certificate feature
Initially the idea was to allow scanning not only the certificate but scan the whole certificate chain for a explicitly supplied hash to be accepted unconditionally.

A review of the validation revealed it was always missing a fallback to the certificate.verify()

Also decided that even when a hash is supplied and matched, the validity date supersedes other criteria except the acceptAll.
2025-03-14 12:09:01 -03:00
Kenneth Skovhede bb1aa8e1f7 Cosmetic cleanup
Removed superfluous namespaces.
Using `var` most places.
Using extension methods for common setup of commands.
2025-03-14 14:34:56 +01:00
Kenneth SkovhedeandGitHub e038a23c44 Merge pull request #6036 from duplicati/feature/invariant-format-sql-strings
Implemented forced invariant locale on all SQL strings.
2025-03-14 13:35:10 +01:00
Kenneth Skovhede 7234d43961 Fixed all SQL strings to use interpolation and forced them into the FormattableString handler using the invariant formatting. 2025-03-14 12:13:17 +01:00
Kenneth Skovhede c1bdaa069e Async operation handlers
Rewrote all operation handlers to be async.
This moves the async/sync threshold out into the controller.

Fixed all the uses of cancellation token in the handlers to use the provided cancellation token.

Introduced a new ReusableTransaction that can be passed around, avoiding a ref parameter passing of the transaction instance.
2025-03-14 10:41:14 +01:00
Kenneth Skovhede f342d70819 Implemented forced invariant locale on all SQL strings.
This fixes #6023
2025-03-13 23:06:32 +01:00
Kenneth Skovhede 30ec975469 Relaxed size requirement to better work with tests 2025-03-13 13:34:55 +01:00
Kenneth Skovhede 5edfcc5c6a Prevent blocksize near remote volume size
If the user selects the blocksize to be the same as the remote volume size, this will cause each block to be in a separate volume and will trigger repeated compact calls.

This commit adds a check that ensures the volume size is large enough to hold at least two blocks.

Also fixed an issue where the compact could create volumes larger than the set limit (at most one block larger).
2025-03-13 12:09:57 +01:00
marceloduplicatiandGitHub 4268eda62d Merge pull request #6030 from marceloduplicati/feature/use-ipv4-v6-auto
Integrating the IPv6 & IPv4 detection
2025-03-12 12:19:04 -03:00
Kenneth SkovhedeandGitHub eac394f0a3 Merge pull request #6011 from duplicati/feature/improved-failure-resilience
Improved failure resilience
2025-03-12 14:48:14 +01:00
Marcelo C. b035c69e4d Integrating the IPv6 & IPv4 detection
It should be noted that I was unable to completely remove IPv4 on Windows 11 to a state where it would not provide the 127.0.0.1 loopback, even when disabling Ipv4 in various ways, including with registry flags.

On ubuntu/linux, after completely disabling ipv4 the ipv6 loopback was returned.
2025-03-11 16:24:15 -03:00
Carl Johnsen ebb04ddd08 Made waking up a waiting block request on the BlockManagers cache async to lower time spent by the volume consumer task 2025-03-11 14:23:37 +01:00
Carl Johnsen b61d594180 Added higher resolution internal profiling timers for the BlockManager 2025-03-11 14:22:49 +01:00
Kenneth Skovhede a0c1676bca Added option to force repair with an older database. 2025-03-11 10:45:22 +01:00
Kenneth Skovhede ceae46f395 Fixed a long-standing bug that could cause errors on delete due to pending delete of filesets 2025-03-10 22:01:14 +01:00
Kenneth Skovhede 086d98ee4b Detect non-attached files in filesets 2025-03-10 21:02:16 +01:00
Kenneth Skovhede d500097e05 Added additional consistency checks to local database 2025-03-10 14:41:36 +01:00
Kenneth Skovhede 55fc5d2e6c Added support for recreating filesets and remote lists in some scenarios 2025-03-10 14:41:12 +01:00
Kenneth Skovhede be16323f24 General cleanup, common database validation exception 2025-03-10 14:38:45 +01:00
marceloduplicatiandGitHub 899d8a259b Merge pull request #6008 from marceloduplicati/feature/azure-upgrade-and-fixes
Azure Library Upgrade and Fixes
2025-03-10 09:36:32 -03:00
Carl-Johannes JohnsenandGitHub f0167885e0 Merge pull request #6026 from carljohnsen/bugfix/restore-deadlock
Bugfix/restore deadlock
2025-03-10 13:21:12 +01:00