Commit Graph
22 Commits
Author SHA1 Message Date
Kenneth SkovhedeandGitHub eac394f0a3 Merge pull request #6011 from duplicati/feature/improved-failure-resilience
Improved failure resilience
2025-03-12 14:48:14 +01:00
Carl Johnsen f77fcc186c Merge remote-tracking branch 'upstream/master' into bugfix/restore-deadlock 2025-03-10 05:50:26 +01:00
Kenneth Skovhede 58da574130 Moved shutdown logic into separate method so it can be called for both uploads and downloads 2025-03-07 15:19:40 +01:00
Kenneth Skovhede 4e60a7ad49 Merge remote-tracking branch 'origin/master' into feature/improved-failure-resilience 2025-03-06 12:09:42 +01:00
Kenneth Skovhede 1321488527 Improve failure detection
This commit adds extra logic to prevent cases where the backup reports success but leaves behind partial files.

A new flag is recorded to track if partial files are possible. If no partial files are possible, the verification will fail if any are found.
2025-03-06 12:02:11 +01:00
Kenneth Skovhede 4f62a8a413 Fixed off-by-one for retry count 2025-03-05 09:04:46 +01:00
Carl Johnsen 3596e0ebd3 When profiling a get, it's worthwhile knowing where the remote file is being stored locally 2025-03-04 19:54:40 +01:00
Kenneth Skovhede 1f7415cc8e Simplified backend manager handler code 2025-03-04 15:42:31 +01:00
Kenneth Skovhede f77e529eb6 Merge commit '2339aeff28adba15442d0be60c56568ae642ad0b' into feature/make-backends-async 2025-02-26 15:23:48 +01:00
Carl Johnsen 98e572714d Added a comment about downloaders in the teardown of BackendManager.Handler 2025-02-21 10:08:21 +01:00
Carl Johnsen 7612b5cce3 Ensured proper synchronization of operations in BackendManager.Handler 2025-02-21 10:07:41 +01:00
Carl Johnsen c164181ec2 BackendManager.GetOperation's were unintentionally still synchronized. 2025-02-21 10:07:09 +01:00
Carl Johnsen beefbf2a46 Removed some unused variables from the BackendManager 2025-02-19 16:12:58 +01:00
Carl Johnsen 374741fc6d Added BackendManager.GetDirectAsync(), which downloads a file without decrypting it 2025-02-19 16:11:55 +01:00
Carl Johnsen 2c06820281 Made the BackendManager.GetOperations' decryption methods static and public so the restores VolumeDecryptor can use the same code. 2025-02-19 16:09:32 +01:00
Carl Johnsen e2e930b2d7 Reworked BackendManager.Handler to handle multiple downloads in parallel 2025-02-19 15:55:59 +01:00
Carl Johnsen 864ce59c4a Reworded pending uploads to active to better reflect that the uploads are in progress 2025-02-19 15:55:13 +01:00
Carl Johnsen 9f3f15615d Exposed spawning multiple backends from the BackendManager 2025-02-18 09:10:13 +01:00
Kenneth Skovhede 0f9669b273 Final conversion to async backends
This PR is the final change, where all backends are now purely implemented with async interfaces.

The code changes here replace the LIST call with an async version. This affect all backends (and then some) but the changes are mostly mechanical rewrites of the code.

In many places, the backeds were already prepared for async output, in other some glue was needed.

To reduce the scope of this change, some backends simply report a synchronous result as an async list.
2025-02-17 16:45:51 +01:00
Marcelo C. 11643c2413 Removing TimeoutObservingStream from backend manager 2025-02-06 11:07:17 -03:00
Kenneth Skovhede b1e23da40c Implemented correct shutdown of backend manager handler 2025-01-29 11:04:13 +01:00
Kenneth SkovhedeandGitHub d99f74ed94 Rewrite the backend manager. (#5896)
* Rewrite the backend manager.

This creates a new more logic backend manager that handles all backend operations.

For each top-level operation, there is now a single backend manager instance that is passed to sub-commands.

The internals of the backend manager are now rewritten to use async logic instead of threading.

The design uses a single task runner that dispatches operations and honors concurrent settings and limits from one place.

The logic for each operation has been moved into a separate files/classes to make it easier to understand each operation.

This fixes #5804

* Fixed a few issues with not awaiting tasks

* Fixed not creating empty databases

* Fixed an issue with hashes not being recorded on download

* Reworked the download logic to avoid attempting to decrypt the file if it has been modified.

* Review fixes

* Renamed db collector to better reflect the purpose.

* Review fixes
2025-01-28 08:54:50 +01:00