This hand-picks the changes from #5049 by @gpatel-fr.
The change is to use a left-join to include all broken files, even if there are no entries in the Blockset table.
The update is only visual but makes the results consistent with the actual operation performed by purge-broken-files.
This close#5049
If there are extra unknown files, this can be a sign that multiple backups are using the same folder.
It can also be a sign that an older database is used for some reason.
This PR updates the error message to mention these two common cases, and spell out what the repair process will do.
This ensures that files cannot appear on the remote destination, despite terminating the host program.
This fixes#4485
There is still work needed to handle the repair and compact operations so they use transactions and can commit changes during transfer progress.
This moves the throttle support to the StreamUtil throttled stream, which allows multiple streams to share a throttle value, such that the throttle setting is applied to the entire communication from Duplicati, and not for each connection.
This also cleans up a bit in the progress reporting, where the backend manager now keeps track of the "active" transfer and only emits progress on that. Once it finishes, a new transfer is promoted a "active".
This adds an extra option to allow setting the SQLite page cache size as a regular option.
Prior to this commit it was only possible to set the SQLite page cache size via environment variables.
The option to use environment variables is preserved, and as options from the environment variable are applied after the new setting, environment variables take precedence.
The default value for the new option is to use 1% of the system memory for the page cache. For the restore process, a connection per worker may be made which defaults to half the number of cores, so the maximum amount of memory is 1% * half the CPU cores.
This is just the upper limit, and SQLite may choose not to use all of it.
This fixes#6178
The delete call would select the same fileset multiple times if multiple retention options were present.
This fixes the selection to a distinct set.
This fixes#6127
While investigating issue [#6103](https://github.com/duplicati/duplicati/issues/6103), found that connections were not disposed due to the backend pool in BackendManager's handler. Set up a FileZilla server to test the issue before and after the change, confirming that connections now close as expected with this fix.
