Commit Graph
33 Commits
Author SHA1 Message Date
Kenneth Skovhede 9dfe977d3f Changed logic so the source providers are now logically mounted into the source directory structure.
This fixes the restore issues as all sources will now appear as if they were originally part of the source file system.

The backend implementation has now been simplified so the backends only return names of entries with no path information.  Most backends already do this, but the S3 backends did not, so this was fixed as well.

This also clears up the interfaces, so the backend does not know about the `ISourceProviderEntry`. All mapping is done by the `BackendSourceProvider`, making it simpler to support other backends as sources.
2025-02-21 16:06:55 +01:00
Kenneth Skovhede 6bbc93b8df Merge remote-tracking branch 'origin/master' into feature/alternate-source-provider 2025-02-16 16:37:12 +01:00
Kenneth Skovhede 8c935a7344 This creates a new ISourceProvider interface for describing a data source.
With this interface it is possible to wrap the current support for snapshots and USN and simplify the enumeration.

This makes a number of assumptions into fields such as `IsMetaEntry` and `IsRootEntry` that removes code that was checking for various properties.

This first commit has support for VSS and USN as well as "naked" files, keeping feature parity.

There is also an experimental support for loading a backend-as-a-source. The support for this is currently limited as the backends need to be extended to support a folder concept.
2025-02-09 16:14:22 +01:00
Kenneth Skovhede 1690871d10 Implemented fix that pushes updated timestamps to previous version, in the case that no files were changed, and no new backup is recorded in remote storage. 2025-02-06 15:31:20 +01:00
Kenneth SkovhedeandGitHub 9464caf622 Feature/update license 2025 (#5851)
* Fixed some minor whitespace issues

* Updated all copyright to 2025
2025-01-07 09:40:39 +01:00
Kenneth SkovhedeandGitHub ff1b2b7990 Simplified task control (#5753)
* Simplified task control by having all logic in a single class and using async control mechanism
* Fixed the tests
* Made the marking of a backup partial explicitly communicated if the enumeration process is stopped.
* Added support for pausing active transfers
* Introduced transfer token to abort operations.
* Updated the way LiveControl is emitting events.
* Retains the paused state of the server across reboots, this fixes #5760.
* Reworked the way throttle speeds are propagated through LiveControls.
* Added option to pause transfers in the UI.
* Removed throttle settings from the LiveControls class so it is exclusively handled by the ApplicationSettings.
* Removed `thread-priority` as it does not work at all when tasks are used.
* Fixed the stop dialog and API to only support stop and abort.
* Removed unused variables
2024-12-18 08:27:59 +01:00
Carl Johnsen b2288cb804 Removed named CoCoL channels in favor of passing around instances. 2024-12-09 13:55:22 +01:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Kenneth Hsu a39a24beae Catch and log exceptions encountered when getting symlink target.
Since the GetSymlinkTarget method can throw an exception, we should log
the failure case and avoid further processing of the symlink target.

This concerns #2171, which describes issues with symlinks created using
Windows Subsystem for Linux.
2020-12-06 10:08:46 -08:00
BlueBlock e52cef304b codacy fixes applied 2 2019-08-02 13:54:30 -04:00
BlueBlock 4407d24f5d codacy fixes applied 2019-08-02 13:48:50 -04:00
BlueBlock b416c6c78b fix - do not throw exception in case where user has asked to cancel 2019-07-24 15:02:30 -04:00
Kenneth Skovhede 03722fcba1 Merge branch 'master' into feature/fix_path_storage2
# Conflicts:
#	Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
2018-10-23 11:33:35 +02:00
Kenneth Hsu 1c8bb276b4 Allow tasks to complete on any thread when context is irrelevant.
When the code following the await can be executed on any thread, it's
recommended to use ConfigureAwait(false) to avoid unnecessary context
switching and potential deadlocks.
2018-09-19 16:57:17 -07:00
Kenneth Skovhede a8cb4af389 Merge branch 'master' into feature/fix_path_storage2
# Conflicts:
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Operation/Backup/BackupDatabase.cs
#	Duplicati/Library/Main/Operation/Backup/MetadataPreProcess.cs
2018-09-07 09:44:23 +02:00
Rune HenriksenandGitHub b8e53cc839 Merge branch 'master' into remove_unnecessary_async 2018-09-04 21:27:11 +02:00
Kenneth Skovhede 52cb9fb1f9 Removed the deprecated --store-metadata option.
Rewrote the logic for pre-filtering files based on timestamps/metadata to be more verbose and well documented.
Fixed an issue where `--check-filetime-only` also checked the file size, which was not present due to the reduced query used to grab only the lastmodified information.
Fixed a problem where the check for existing metadata was wrong and produced a warning.

This fixes #3204
This handles some of #3194
2018-08-11 11:48:19 +02:00
Kenneth Hsu 1a2ee3a5ef Simplify return type of synchronous methods.
Since these methods no longer have the async modifier, they do not need
to return a Task, and do not need to be awaited.
2018-07-04 15:35:14 -07:00
Kenneth Hsu 6c38651f19 Rename methods to clarify their synchronous behavior.
Since these methods no longer have the async modifier, we should not
include 'Async' in their names.
2018-07-04 15:35:14 -07:00
Kenneth Skovhede 8ac99e83b7 Merge branch 'master' into feature/fix_path_storage2 2018-06-28 10:51:11 +02:00
Kenneth Hsu 54c4f9224d Allow tasks to complete on any thread when context is irrelevant.
When the code following the await can be executed on any thread, it's
recommended to use ConfigureAwait(false) to avoid unnecessary context
switching and potential deadlocks.
2018-06-21 19:28:27 -07:00
Kenneth Skovhede cbc18970d5 Re-applied the path-storage fix as the merge failed somehow 2018-06-14 10:12:24 +02:00
Kenneth Skovhede 79f7b22e0f Merge branch 'master' into concurrent_processing
Changed a few things on the way, removing the log process

# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/FilelistProcessor.cs
#	Duplicati/Library/Main/Operation/PurgeBrokenFilesHandler.cs
#	Duplicati/Library/Main/Operation/TestFilterHandler.cs
#	Duplicati/Library/Main/Options.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/UnitTest/BasicSetupHelper.cs
2018-04-11 23:02:47 +02:00
Kenneth Skovhede d18431b385 Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/Library/Main/ResultClasses.cs

First attempt at merging all changes from 2016-10-08 to now :/
2017-05-30 00:35:31 +02:00
Kenneth Skovhede c1c8c9a1aa Merge branch 'master' into concurrent_processing
Some changes were required for this merge,
now the StreamBlockSplitter process will take in any stream, store the blocks and return the blockset ID.
This makes it compatible with the changes for metadata handling where they support more than a single block of metadata.

# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRepairDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
2016-10-18 13:27:36 +02:00
Kenneth Skovhede 46342075c1 Fixed error from last merge 2016-08-10 09:27:13 +02:00
Kenneth Skovhede f0c0fc3a2e Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	thirdparty/CoCoL/CoCoL.dll
2016-08-08 15:03:42 +02:00
Kenneth Skovhede 537fe20838 Added a lot of comments.
Fixed a few issues that caused parallel uploads.
Fixed the use of the encryption module so the encryption module is not re-used for multiple jobs.
2016-02-25 19:15:51 +01:00
Kenneth Skovhede 20f7b42945 Fixed channels to use type-safe initializations.
Updated logging channel to use a wrapper for later early-filter optimizations.
Various bug fixes and optimizations.
2016-02-24 23:48:42 +01:00
Kenneth Skovhede 371965d88c Work towards using the processed based concurrent execution flow 2016-02-19 00:02:38 +01:00
Kenneth Skovhede ec877696b0 More work on concurrent processing 2016-02-18 16:46:53 +01:00
Kenneth Skovhede 01c3b3f5c0 More work on a concurrent implementation of the backup process 2016-02-09 09:17:31 +01:00
Kenneth Skovhede 0a26e263f1 Work on concurrent processing 2016-01-28 22:24:51 +01:00