Commit Graph
64 Commits
Author SHA1 Message Date
Kenneth Skovhede faede65d8d Changed transactions to use a wrapper that ensures a transaction is allocated. 2025-04-14 12:04:00 +02:00
Kenneth Skovhede 0b735d79ff More cleanup and a few bug fixes 2025-04-03 16:38:51 +02:00
Kenneth Skovhede 2006556778 Fixed some more nullable support in the database layer 2025-04-03 15:46:20 +02:00
Kenneth Skovhede 6db50381cb Additional fixes 2025-04-03 13:58:06 +02:00
Kenneth Skovhede 424cc6f3ad Added LocalRepairDatabase 2025-04-02 23:34:30 +02:00
Kenneth SkovhedeandGitHub 85f5ddde1d Merge branch 'master' into feature/guard-against-missing-files 2025-03-25 11:39:13 +01:00
Kenneth Skovhede 3c741cbcfc Delete empty index files
If there are index files with no references, they will now be deleted by the repair process.

Such files are likely leftovers from an earlier version that would leave such files if various errors happened.
2025-03-20 22:21:57 +01:00
Kenneth Skovhede d40301b2b7 Guard against fileset errors
This PR adds a few extra consistency checks to the database, so any failure in the operations is detected earlier than before.

Especially when cleaning temporary and partial files, it is important that the deletes do not remove unexpected entries.

Also, after a recreate, repair or purge command, the database consistency is verified, so it becomes easier to identify which operation caused the failure.

To assist in trouble shooting, there is also a more elaborate error message if a fileset discrepancy is detected, that pinpoints which versions are affected.

Finally, the repair command has been extended to support recreating a fileset from the remote data.
2025-03-20 15:59:14 +01:00
Kenneth Skovhede 833a77bcf6 Fixed a transition issue 2025-03-19 17:21:16 +01:00
Kenneth Skovhede bb1aa8e1f7 Cosmetic cleanup
Removed superfluous namespaces.
Using `var` most places.
Using extension methods for common setup of commands.
2025-03-14 14:34:56 +01:00
Kenneth Skovhede 7234d43961 Fixed all SQL strings to use interpolation and forced them into the FormattableString handler using the invariant formatting. 2025-03-14 12:13:17 +01:00
Kenneth Skovhede f342d70819 Implemented forced invariant locale on all SQL strings.
This fixes #6023
2025-03-13 23:06:32 +01:00
Kenneth Skovhede 55fc5d2e6c Added support for recreating filesets and remote lists in some scenarios 2025-03-10 14:41:12 +01:00
Kenneth SkovhedeandGitHub 9464caf622 Feature/update license 2025 (#5851)
* Fixed some minor whitespace issues

* Updated all copyright to 2025
2025-01-07 09:40:39 +01:00
Kenneth Skovhede 473c6cbbb8 Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade 2024-03-04 12:21:53 +01:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Thomas Suckow ecb0249259 dotnet no longer provides a method for a string to hashalgorithm
* Also fix up not diposing of hashers
2021-04-04 11:17:13 -07:00
Kenneth Skovhede 4f565a004f Merge branch 'netstandard' of https://github.com/mnaiman/duplicati into netstandard
# Conflicts:
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SSHv2/packages.config
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Telegram/Duplicati.Library.Backend.Telegram.csproj
#	Duplicati/Library/Backend/Telegram/Properties/AssemblyInfo.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
2021-04-01 15:08:42 +02:00
Kenneth Hsu 534b71e409 Fix bug with repairing multiple missing blocklist hashes.
If there is more than one missing blocklist hash, we must reset the
offset index before copying hashes from the next blocklist into the
buffer.

This fixes #4397.
2020-12-26 21:37:52 -08:00
Michal Naiman 1bc1e58737 upstream 2019-04-28 13:45:26 +02:00
Rune HenriksenandGitHub f5f353fe19 Merge branch 'master' into feature/fix_path_storage2 2019-01-25 23:37:57 +01:00
Kenneth Hsu b5d9451b33 Remove redundant casts. 2019-01-21 19:30:52 -08:00
Kenneth Skovhede 03722fcba1 Merge branch 'master' into feature/fix_path_storage2
# Conflicts:
#	Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
2018-10-23 11:33:35 +02:00
Kenneth Hsu f4f676a1de Remove unnecessary variable assignment.
These variables are assigned values that are never read by any
subsequent instruction.
2018-09-28 20:13:45 -07:00
Max a22a7a7d24 remove of FasterHashing component 2018-09-09 14:42:40 +02:00
Kenneth Skovhede cbc18970d5 Re-applied the path-storage fix as the merge failed somehow 2018-06-14 10:12:24 +02:00
Kenneth Hsu f12858e6a5 Mark fields that shouldn't be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2018-05-25 10:04:52 -07:00
Kenneth Skovhede 79f7b22e0f Merge branch 'master' into concurrent_processing
Changed a few things on the way, removing the log process

# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/FilelistProcessor.cs
#	Duplicati/Library/Main/Operation/PurgeBrokenFilesHandler.cs
#	Duplicati/Library/Main/Operation/TestFilterHandler.cs
#	Duplicati/Library/Main/Options.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/UnitTest/BasicSetupHelper.cs
2018-04-11 23:02:47 +02:00
Kenneth Skovhede 842fd96543 Implemented a new logging system that is more transparent and allows a more granular way of picking log messages.
Added ID's to each log message and each exception to allow later introduction of a Knowledgebase service that explains each error in more detail.
2018-03-15 09:12:34 +01:00
Kenneth Skovhede 75bf6857c1 Merge branch 'master' into concurrent_processing
# Conflicts:
#	.travis.yml
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/Library/Main/Volumes/VolumeWriterBase.cs
#	Duplicati/Library/Utility/TempFile.cs
2018-01-20 15:02:41 +01:00
Kenneth Hsu 604b6e4337 Dispose enumerators after manually iterating over elements.
Since the IEnumerator<T> interface implements IDisposable, we should
make sure to dispose of the enumerator when manually iterating over the
elements.
2017-10-07 12:27:11 -07:00
Kenneth Skovhede 683b04741d Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Operation/BackupHandler.cs

Manually fixed the HashAlgorithmHelper case
2017-09-20 13:56:37 +02:00
Kenneth Skovhede 6d401fbd8d Changed all hash algorithms to use the Cng version.
This fixes #2621
2017-09-20 09:58:43 +02:00
Kenneth Skovhede d18431b385 Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/Library/Main/ResultClasses.cs

First attempt at merging all changes from 2016-10-08 to now :/
2017-05-30 00:35:31 +02:00
Kenneth Skovhede a029890409 Implemented better default error messsages 2017-01-09 11:35:38 +01:00
Kenneth Skovhede c1c8c9a1aa Merge branch 'master' into concurrent_processing
Some changes were required for this merge,
now the StreamBlockSplitter process will take in any stream, store the blocks and return the blockset ID.
This makes it compatible with the changes for metadata handling where they support more than a single block of metadata.

# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRepairDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
2016-10-18 13:27:36 +02:00
Kenneth Skovhede 309bf1648d Merge branch 'master' into feature/fix_for_different_blockhash_and_filehash_plus_multiblocks_metadata
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/UnitTest/BorderTests.cs
2016-10-17 13:28:27 +02:00
Kenneth Skovhede 94c5e10480 Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/RemoteVolumeEntry.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/Library/Main/Operation/RepairHandler.cs
#	Duplicati/Library/Main/Operation/RestoreControlFilesHandler.cs
2016-09-16 10:33:07 +02:00
Kenneth Skovhede 0c68daff6c Fixed all whitespace to be 4 spaces instead of tabs 2016-09-15 11:39:27 +02:00
Kenneth Skovhede f0c0fc3a2e Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	thirdparty/CoCoL/CoCoL.dll
2016-08-08 15:03:42 +02:00
Kenneth Skovhede 742eab0913 Fixed multiple cases where connections and command instances were not disposed.
This should fix all cases of "File is locked" on windows.
2016-04-06 20:40:34 +02:00
Kenneth Skovhede d101f91c94 Fixed support for metadata larger than blocksize. Now supports one full blocklist of metadata, appx 300mb with default settings.
Added correct handling of metadata hashes, such that a metadata blockset is now handled the same was as a regular file blockset.
The query that creates the filesets does not support more than a single blocklist of metadata, hence the limitiation.
2016-04-04 18:11:48 +02:00
Kenneth Skovhede 317fd493d8 Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
2016-03-30 08:30:09 +02:00
Kenneth Skovhede 0ebbef0b97 Implemented feature that does not delete extra index files if their content fits the database. 2016-03-17 21:51:42 +01:00
Kenneth Skovhede 537fe20838 Added a lot of comments.
Fixed a few issues that caused parallel uploads.
Fixed the use of the encryption module so the encryption module is not re-used for multiple jobs.
2016-02-25 19:15:51 +01:00
Kenneth Skovhede 02a1d7131e Implemented a sanity check to see if the repair process makes damages. 2015-11-28 21:53:46 +01:00
Kenneth Skovhede e6e3ddd29a Spell fix 2015-11-28 21:52:55 +01:00
Kenneth Skovhede e65e7d234e Fixed an issue where a restored database was missing some files.
This fixes #1410.
2015-07-12 21:36:48 +02:00
Kenneth Skovhede 74bbf0f1f8 Added support for handling null values in the remotevolume table.
This fixes #1334.
2015-05-17 13:38:25 +02:00
Kenneth Skovhede a14fceaa95 Fixed an issue with readers being closed prematurely during the repair process. 2015-03-06 13:54:17 +01:00