Kenneth Skovhede
51bcec3efc
Style cleanup, named tuples.
...
Fix indentation.
Move metadata check up.
2025-05-09 17:34:09 +02:00
Kenneth Skovhede
183ee39d6e
Fixed a bug in list-changes
2025-05-09 11:13:10 +02:00
Kenneth Skovhede
b947f7a7a0
Simplified a query
2025-05-09 11:12:45 +02:00
Kenneth Skovhede
fe866ceb9f
Implemented support for repairing the database when only parts of the source data is available.
...
This repair will leave the database in a still-broken state, but if this is run prior to purge-broken-files, less data will be lost.
Also updated the purge-broken-files operation to keep files and directories in the set, if they are only missing metadata.
2025-05-09 11:12:29 +02:00
Kenneth Skovhede
d22ce5370e
Rewrote the repair for scenarios where remote files are missing.
...
This adds a number of additional features to the repair process so it can actually fully recover if all data is present.
Also added a several tests to ensure that the functionality works in multiple scenarios.
This fixes #5987
2025-05-08 10:31:38 +02:00
Kenneth Skovhede and GitHub
9b3328daef
Merge pull request #6210 from duplicati/feature/improve-backup-listing-speed
...
Improve backup listing speed for restore
2025-04-30 12:47:13 +02:00
Kenneth Skovhede
39dca1eb79
Improve list-broken-files output
...
This hand-picks the changes from #5049 by @gpatel-fr.
The change is to use a left-join to include all broken files, even if there are no entries in the Blockset table.
The update is only visual but makes the results consistent with the actual operation performed by purge-broken-files.
This close #5049
2025-04-30 09:22:24 +02:00
Kenneth Skovhede
f533e3d300
Moved fragment back
2025-04-29 15:49:38 +02:00
Kenneth Skovhede
a3bc9b80ca
Implemented 4 new list methods that are optimized for a single operation
2025-04-29 10:03:57 +02:00
Kenneth Skovhede and GitHub
2837a105ee
Merge pull request #6186 from duplicati/feature/add-sqlite-cache-value
...
Added option to set the SQLite page cache
2025-04-25 14:45:43 +02:00
Kenneth Skovhede and GitHub
4515e13342
Merge pull request #6199 from duplicati/feature/add-downgrade-db-tool
...
Added a database downgrade tool
2025-04-25 14:43:59 +02:00
Kenneth Skovhede and GitHub
1bbe04a782
Merge branch 'master' into feature/add-sqlite-cache-value
2025-04-25 13:58:24 +02:00
Kenneth Skovhede
ba298df915
Added an upgrade command to the database tool
2025-04-25 12:30:03 +02:00
Kenneth Skovhede
ebaf725598
Fixed a case where ArchiveTime could be created as nullable
2025-04-24 21:26:24 +02:00
Kenneth Skovhede
2ef04e43ee
Simplified new ArchiveTime column to not allow null
2025-04-24 20:30:08 +02:00
Kenneth Skovhede
75f7582014
Fixed passing transaction to more places
2025-04-24 20:28:26 +02:00
Kenneth Skovhede
46f5efd178
Merge branch 'master' into feature/support-glacier-style-non-readable-files
2025-04-23 22:37:31 +02:00
Kenneth Skovhede
e0475f3a9c
Ignore any error on pragma optimize.
...
Previous logic would only catch SQLite exceptions, but it is just a performance thing, so we can ignore any errors (they are still logged)
2025-04-23 16:40:36 +02:00
Kenneth Skovhede and GitHub
171eb1bb12
Merge branch 'master' into feature/add-sqlite-cache-value
2025-04-23 16:34:55 +02:00
Kenneth Skovhede
18477c572d
Fixed some nullability warnings
2025-04-23 15:40:16 +02:00
Kenneth Skovhede
3ff868b0f9
Removed tracking of files sizes, and implemented it with a database lookup
2025-04-23 15:39:32 +02:00
Kenneth Skovhede
0a7a1cd689
Added option to set the SQLite page cache
...
This adds an extra option to allow setting the SQLite page cache size as a regular option.
Prior to this commit it was only possible to set the SQLite page cache size via environment variables.
The option to use environment variables is preserved, and as options from the environment variable are applied after the new setting, environment variables take precedence.
The default value for the new option is to use 1% of the system memory for the page cache. For the restore process, a connection per worker may be made which defaults to half the number of cores, so the maximum amount of memory is 1% * half the CPU cores.
This is just the upper limit, and SQLite may choose not to use all of it.
This fixes #6178
2025-04-22 14:26:46 +02:00
Kenneth Skovhede
0945657a29
Fixed transaction issue with test handler
2025-04-15 23:24:35 +02:00
Kenneth Skovhede
c3fe98c09c
Disable the begin immediate as it seems the library does it automatically
2025-04-14 22:57:44 +02:00
Kenneth Skovhede
4773914639
Rewrote testing to use an externally supplied transaction, to avoid temporary transactions from clearing the active transaction.
...
This is needed because SQLite does not support multiple active transactions.
2025-04-14 12:05:10 +02:00
Kenneth Skovhede
faede65d8d
Changed transactions to use a wrapper that ensures a transaction is allocated.
2025-04-14 12:04:00 +02:00
Kenneth Skovhede
6e18f7a4d9
Added SQLite tests
...
Fixed the temporary db value list
2025-04-14 11:24:13 +02:00
Kenneth Skovhede
11a233c20a
Merge commit '4a6bf8aa019d86cc9a10eefa55aced14e8a13128' into feature/improve-block-reuse
2025-04-06 09:25:10 +02:00
Kenneth Skovhede
1095850965
Reworked the way deleted blocks are detected.
...
This should have a much lower performance impact.
2025-04-06 08:41:59 +02:00
Kenneth Skovhede
cfad020ef2
Merge commit '61892a40d9aff663da86eda273505970268f6384' into feature/improve-block-reuse
2025-04-06 08:36:24 +02:00
Kenneth Skovhede and GitHub
15c2079cdc
Merge branch 'master' into feature/improve-compact-block-use-check
2025-04-06 08:28:03 +02:00
Kenneth Skovhede
2deb147e71
Fixed change stats and added a test
2025-04-05 22:58:10 +02:00
Kenneth Skovhede
8b72528030
Re-implemented the change statistics calculation.
2025-04-05 21:42:05 +02:00
Kenneth Skovhede
bb24036602
Fixed performance degradation after parameter queries
2025-04-05 09:25:10 +02:00
Kenneth Skovhede
52fc144580
Merge branch 'master' into feature/improve-compact-block-use-check
2025-04-04 14:41:58 +02:00
Kenneth Skovhede
0ed672ff33
Fixed missing @Type parameter
2025-04-03 22:33:09 +02:00
Kenneth Skovhede
2060e9067d
Clear databases fileset entries prior to recreate
2025-04-03 22:20:42 +02:00
Kenneth Skovhede
457349fd2d
More cleanup
2025-04-03 22:15:19 +02:00
Kenneth Skovhede
0b735d79ff
More cleanup and a few bug fixes
2025-04-03 16:38:51 +02:00
Kenneth Skovhede
2006556778
Fixed some more nullable support in the database layer
2025-04-03 15:46:20 +02:00
Kenneth Skovhede
0638afb8b8
Fixed some nullable support
2025-04-03 15:25:20 +02:00
Kenneth Skovhede
ef449bc569
Some nullable updates
2025-04-03 14:32:47 +02:00
Kenneth Skovhede
35682bd4fe
Cleaned up some nullability issues
2025-04-03 14:20:00 +02:00
Kenneth Skovhede
e96e769ba6
Removed support for positional arguments in extension methods
2025-04-03 13:59:39 +02:00
Kenneth Skovhede
6db50381cb
Additional fixes
2025-04-03 13:58:06 +02:00
Kenneth Skovhede
9e3aa72d00
Added LocalBackupDatabase
2025-04-03 11:31:36 +02:00
Kenneth Skovhede
7c8571ed33
Added LocalTestDatabase
2025-04-03 09:03:22 +02:00
Kenneth Skovhede
defb5a385b
Added LocalRestoreDatabase
2025-04-03 09:00:55 +02:00
Kenneth Skovhede
424cc6f3ad
Added LocalRepairDatabase
2025-04-02 23:34:30 +02:00
Kenneth Skovhede
7b948a2f1d
Fixed issue with SQLite bindings not supporting arrays.
...
Added helper class to wrap values in a temporary table if they exceeed the maximum number of parameters.
2025-04-02 23:26:20 +02:00
Kenneth Skovhede