Kenneth Skovhede
0df9d67a6a
Implemented handling for cases where the previous fileset ends up being empty and removed.
...
This fixes #5862
2025-05-14 23:36:48 +02:00
Kenneth Skovhede
0c3d74c580
Removed unused code
2025-05-14 20:40:25 +02:00
Kenneth Skovhede and GitHub
164e0dbd66
Merge pull request #6220 from duplicati/feature/fix-restore-duplicate
...
Fix restore duplicate
2025-05-12 14:45:13 +02:00
Kenneth Skovhede
d0fa3b7d87
Merge commit '2f0e9dc6f131e6473b9a1ed8e41e2eefb893c8d6' into feature/fix-restore-duplicate
2025-05-12 11:15:57 +02:00
Kenneth Skovhede and GitHub
fe42c53ad3
Merge branch 'master' into feature/fix-db-schema
2025-05-12 11:07:29 +02:00
Kenneth Skovhede
547ad8437c
Revert "Ignore filesets that are in the deleting state"
...
This reverts commit 513e67418d
2025-05-11 15:39:27 +02:00
Kenneth Skovhede
424ef6c5bf
Rolled in index from #4706
...
This closes #4706
2025-05-10 22:30:05 +02:00
Kenneth Skovhede
513e67418d
Ignore filesets that are in the deleting state
2025-05-10 22:12:45 +02:00
Kenneth Skovhede
2f12473398
Added missing index on remotevolume name
2025-05-10 10:02:08 +02:00
Kenneth Skovhede
33a21c782d
Fixed incorrect transaction use
2025-05-10 09:55:13 +02:00
Kenneth Skovhede
91cfdebd95
Fixed issue with repair leaving old filesets in the database.
...
Fixed issue with the full/partial state not being replicated for re-created filesets
2025-05-10 09:12:11 +02:00
Kenneth Skovhede
51bcec3efc
Style cleanup, named tuples.
...
Fix indentation.
Move metadata check up.
2025-05-09 17:34:09 +02:00
Kenneth Skovhede
183ee39d6e
Fixed a bug in list-changes
2025-05-09 11:13:10 +02:00
Kenneth Skovhede
b947f7a7a0
Simplified a query
2025-05-09 11:12:45 +02:00
Kenneth Skovhede
fe866ceb9f
Implemented support for repairing the database when only parts of the source data is available.
...
This repair will leave the database in a still-broken state, but if this is run prior to purge-broken-files, less data will be lost.
Also updated the purge-broken-files operation to keep files and directories in the set, if they are only missing metadata.
2025-05-09 11:12:29 +02:00
Kenneth Skovhede
d22ce5370e
Rewrote the repair for scenarios where remote files are missing.
...
This adds a number of additional features to the repair process so it can actually fully recover if all data is present.
Also added a several tests to ensure that the functionality works in multiple scenarios.
This fixes #5987
2025-05-08 10:31:38 +02:00
Kenneth Skovhede
a271183b44
Merge branch 'fix-restore-duplicate' of github.com:Jojo-1000/duplicati into Jojo-1000-fix-restore-duplicate
2025-04-30 18:32:43 +02:00
Kenneth Skovhede and GitHub
9b3328daef
Merge pull request #6210 from duplicati/feature/improve-backup-listing-speed
...
Improve backup listing speed for restore
2025-04-30 12:47:13 +02:00
Kenneth Skovhede
39dca1eb79
Improve list-broken-files output
...
This hand-picks the changes from #5049 by @gpatel-fr.
The change is to use a left-join to include all broken files, even if there are no entries in the Blockset table.
The update is only visual but makes the results consistent with the actual operation performed by purge-broken-files.
This close #5049
2025-04-30 09:22:24 +02:00
Kenneth Skovhede
f533e3d300
Moved fragment back
2025-04-29 15:49:38 +02:00
Kenneth Skovhede
a3bc9b80ca
Implemented 4 new list methods that are optimized for a single operation
2025-04-29 10:03:57 +02:00
Kenneth Skovhede and GitHub
2837a105ee
Merge pull request #6186 from duplicati/feature/add-sqlite-cache-value
...
Added option to set the SQLite page cache
2025-04-25 14:45:43 +02:00
Kenneth Skovhede and GitHub
4515e13342
Merge pull request #6199 from duplicati/feature/add-downgrade-db-tool
...
Added a database downgrade tool
2025-04-25 14:43:59 +02:00
Kenneth Skovhede and GitHub
1bbe04a782
Merge branch 'master' into feature/add-sqlite-cache-value
2025-04-25 13:58:24 +02:00
Kenneth Skovhede
ba298df915
Added an upgrade command to the database tool
2025-04-25 12:30:03 +02:00
Kenneth Skovhede
ebaf725598
Fixed a case where ArchiveTime could be created as nullable
2025-04-24 21:26:24 +02:00
Kenneth Skovhede
2ef04e43ee
Simplified new ArchiveTime column to not allow null
2025-04-24 20:30:08 +02:00
Kenneth Skovhede
75f7582014
Fixed passing transaction to more places
2025-04-24 20:28:26 +02:00
Kenneth Skovhede
46f5efd178
Merge branch 'master' into feature/support-glacier-style-non-readable-files
2025-04-23 22:37:31 +02:00
Kenneth Skovhede
e0475f3a9c
Ignore any error on pragma optimize.
...
Previous logic would only catch SQLite exceptions, but it is just a performance thing, so we can ignore any errors (they are still logged)
2025-04-23 16:40:36 +02:00
Kenneth Skovhede and GitHub
171eb1bb12
Merge branch 'master' into feature/add-sqlite-cache-value
2025-04-23 16:34:55 +02:00
Kenneth Skovhede
18477c572d
Fixed some nullability warnings
2025-04-23 15:40:16 +02:00
Kenneth Skovhede
3ff868b0f9
Removed tracking of files sizes, and implemented it with a database lookup
2025-04-23 15:39:32 +02:00
Kenneth Skovhede
0a7a1cd689
Added option to set the SQLite page cache
...
This adds an extra option to allow setting the SQLite page cache size as a regular option.
Prior to this commit it was only possible to set the SQLite page cache size via environment variables.
The option to use environment variables is preserved, and as options from the environment variable are applied after the new setting, environment variables take precedence.
The default value for the new option is to use 1% of the system memory for the page cache. For the restore process, a connection per worker may be made which defaults to half the number of cores, so the maximum amount of memory is 1% * half the CPU cores.
This is just the upper limit, and SQLite may choose not to use all of it.
This fixes #6178
2025-04-22 14:26:46 +02:00
Kenneth Skovhede
0945657a29
Fixed transaction issue with test handler
2025-04-15 23:24:35 +02:00
Kenneth Skovhede
c3fe98c09c
Disable the begin immediate as it seems the library does it automatically
2025-04-14 22:57:44 +02:00
Kenneth Skovhede
4773914639
Rewrote testing to use an externally supplied transaction, to avoid temporary transactions from clearing the active transaction.
...
This is needed because SQLite does not support multiple active transactions.
2025-04-14 12:05:10 +02:00
Kenneth Skovhede
faede65d8d
Changed transactions to use a wrapper that ensures a transaction is allocated.
2025-04-14 12:04:00 +02:00
Kenneth Skovhede
6e18f7a4d9
Added SQLite tests
...
Fixed the temporary db value list
2025-04-14 11:24:13 +02:00
Kenneth Skovhede
11a233c20a
Merge commit '4a6bf8aa019d86cc9a10eefa55aced14e8a13128' into feature/improve-block-reuse
2025-04-06 09:25:10 +02:00
Kenneth Skovhede
1095850965
Reworked the way deleted blocks are detected.
...
This should have a much lower performance impact.
2025-04-06 08:41:59 +02:00
Kenneth Skovhede
cfad020ef2
Merge commit '61892a40d9aff663da86eda273505970268f6384' into feature/improve-block-reuse
2025-04-06 08:36:24 +02:00
Kenneth Skovhede and GitHub
15c2079cdc
Merge branch 'master' into feature/improve-compact-block-use-check
2025-04-06 08:28:03 +02:00
Kenneth Skovhede
2deb147e71
Fixed change stats and added a test
2025-04-05 22:58:10 +02:00
Kenneth Skovhede
8b72528030
Re-implemented the change statistics calculation.
2025-04-05 21:42:05 +02:00
Kenneth Skovhede
bb24036602
Fixed performance degradation after parameter queries
2025-04-05 09:25:10 +02:00
Kenneth Skovhede
52fc144580
Merge branch 'master' into feature/improve-compact-block-use-check
2025-04-04 14:41:58 +02:00
Kenneth Skovhede
0ed672ff33
Fixed missing @Type parameter
2025-04-03 22:33:09 +02:00
Kenneth Skovhede
2060e9067d
Clear databases fileset entries prior to recreate
2025-04-03 22:20:42 +02:00
Kenneth Skovhede
457349fd2d
More cleanup
2025-04-03 22:15:19 +02:00
Kenneth Skovhede