Kenneth Skovhede
2ef04e43ee
Simplified new ArchiveTime column to not allow null
2025-04-24 20:30:08 +02:00
Kenneth Skovhede
75f7582014
Fixed passing transaction to more places
2025-04-24 20:28:26 +02:00
Kenneth Skovhede
46f5efd178
Merge branch 'master' into feature/support-glacier-style-non-readable-files
2025-04-23 22:37:31 +02:00
Kenneth Skovhede
e0475f3a9c
Ignore any error on pragma optimize.
...
Previous logic would only catch SQLite exceptions, but it is just a performance thing, so we can ignore any errors (they are still logged)
2025-04-23 16:40:36 +02:00
Kenneth Skovhede
18477c572d
Fixed some nullability warnings
2025-04-23 15:40:16 +02:00
Kenneth Skovhede
faede65d8d
Changed transactions to use a wrapper that ensures a transaction is allocated.
2025-04-14 12:04:00 +02:00
Kenneth Skovhede
2060e9067d
Clear databases fileset entries prior to recreate
2025-04-03 22:20:42 +02:00
Kenneth Skovhede
2006556778
Fixed some more nullable support in the database layer
2025-04-03 15:46:20 +02:00
Kenneth Skovhede
ef449bc569
Some nullable updates
2025-04-03 14:32:47 +02:00
Kenneth Skovhede
35682bd4fe
Cleaned up some nullability issues
2025-04-03 14:20:00 +02:00
Kenneth Skovhede
7b948a2f1d
Fixed issue with SQLite bindings not supporting arrays.
...
Added helper class to wrap values in a temporary table if they exceeed the maximum number of parameters.
2025-04-02 23:26:20 +02:00
Kenneth Skovhede
08cb034acf
Added custom version of the validation to support not failing validation prior to running the repair that fixes the error.
2025-03-25 11:56:52 +01:00
Kenneth Skovhede
d40301b2b7
Guard against fileset errors
...
This PR adds a few extra consistency checks to the database, so any failure in the operations is detected earlier than before.
Especially when cleaning temporary and partial files, it is important that the deletes do not remove unexpected entries.
Also, after a recreate, repair or purge command, the database consistency is verified, so it becomes easier to identify which operation caused the failure.
To assist in trouble shooting, there is also a more elaborate error message if a fileset discrepancy is detected, that pinpoints which versions are affected.
Finally, the repair command has been extended to support recreating a fileset from the remote data.
2025-03-20 15:59:14 +01:00
Kenneth Skovhede and GitHub
2e3f867fa6
Merge pull request #6061 from duplicati/feature/untangle-db-and-results
...
Untangle the results and database
2025-03-20 12:02:55 +01:00
Kenneth Skovhede
7f008341df
Untangle the results and database
...
The code was using a two-way reference linking the database to the results and the results to the database.
However, this was not really used so it ended up just cluttering the code. With this commit, the results and databases are no longer keeping track of each other, and the only interaction is when the results are written to the database at the end of the operation.
To ensure result data is always written, this code has been moved into the controller, which will now flush the results for all operations. Prior to this, only some operations would write result data to the log.
The controller now also handles logic for cleaning old log data and invoking the vacuum command.
Fixed a bug due to using System.Text.Json where some attributes from NewtonSoft.Json were being applied and not picked up.
2025-03-20 09:51:49 +01:00
Kenneth Skovhede
f509c174d3
Fixed a missing =
2025-03-19 15:32:05 +01:00
Kenneth Skovhede
fe1cef3be8
Move away from positional parameters in SQL statements.
...
This is a move away from using positional parameters in SQL statements and rok towards only using named parameters.
2025-03-18 22:41:11 +01:00
Kenneth Skovhede
dd07dabee1
Merge commit 'b1f705f71a39e4ea79c7dae56bfd0fb345af5110' into feature/support-glacier-style-non-readable-files
2025-03-18 20:44:05 +01:00
Kenneth Skovhede
bb1aa8e1f7
Cosmetic cleanup
...
Removed superfluous namespaces.
Using `var` most places.
Using extension methods for common setup of commands.
2025-03-14 14:34:56 +01:00
Kenneth Skovhede
7234d43961
Fixed all SQL strings to use interpolation and forced them into the FormattableString handler using the invariant formatting.
2025-03-14 12:13:17 +01:00
Kenneth Skovhede
f342d70819
Implemented forced invariant locale on all SQL strings.
...
This fixes #6023
2025-03-13 23:06:32 +01:00
Kenneth Skovhede
086d98ee4b
Detect non-attached files in filesets
2025-03-10 21:02:16 +01:00
Kenneth Skovhede
d500097e05
Added additional consistency checks to local database
2025-03-10 14:41:36 +01:00
Kenneth Skovhede
be16323f24
General cleanup, common database validation exception
2025-03-10 14:38:45 +01:00
Kenneth Skovhede
12cabf8da7
Merge branch 'feature/improved-failure-resilience' of github.com:duplicati/duplicati into feature/improved-failure-resilience
2025-03-08 15:28:24 +01:00
Kenneth Skovhede
9952e31f49
Fixed a bug where retention time was incorrectly parsed when reading from the database.
...
This effectively caused the retention time to be ignored when reading from the database.
Also moved some code related to finding an unfinished fileset into the common database class.
2025-03-08 15:13:03 +01:00
Kenneth Skovhede
f6f0cf8b96
More docs
2025-03-07 15:45:51 +01:00
Kenneth Skovhede
ca236f03e3
Renamed flag to more precisely describe the purpose
2025-03-06 14:56:35 +01:00
Kenneth Skovhede
7beeb27ade
Extended tests and fixed a case where a left-over upload could cause errors.
2025-03-06 14:53:46 +01:00
Kenneth Skovhede
1321488527
Improve failure detection
...
This commit adds extra logic to prevent cases where the backup reports success but leaves behind partial files.
A new flag is recorded to track if partial files are possible. If no partial files are possible, the verification will fail if any are found.
2025-03-06 12:02:11 +01:00
Kenneth Skovhede
1690871d10
Implemented fix that pushes updated timestamps to previous version, in the case that no files were changed, and no new backup is recorded in remote storage.
2025-02-06 15:31:20 +01:00
Kenneth Skovhede
9806f5cf74
Support for un-marking an archived file
2025-01-29 21:44:06 +01:00
Kenneth Skovhede
71b3152f71
Implemented support for not testing archived files
2025-01-29 11:36:45 +01:00
Kenneth Skovhede and GitHub
f7b7555e87
Fixes for tests ( #5918 )
...
* Two small test fixes
* Ignore exceptions when comitting an empty transaction
* Attempt to fix FTP connection issue
* Unified test by having a single download function.
Added DNS probing call to attempt fixing sporadic DNS errors.
* Fixed argument order
* Avoid creating temp file during tests, if not needed
2025-01-28 08:55:28 +01:00
Kenneth Skovhede
fb1d15fb9d
Avoid optimize after vacuum
2025-01-14 19:42:03 +01:00
Kenneth Skovhede and GitHub
9464caf622
Feature/update license 2025 ( #5851 )
...
* Fixed some minor whitespace issues
* Updated all copyright to 2025
2025-01-07 09:40:39 +01:00
Kenneth Skovhede
db517a300b
Implemented a fix for compaction issues that could cause inconsistent views of the remote storage.
2024-11-01 14:50:26 +01:00
Kenneth Skovhede
e9c256c7f2
Merge remote-tracking branch 'origin/master' into feature/fix-leftover-index
2024-10-29 16:40:06 +01:00
Kenneth Skovhede
ff6757877f
Fixing the inconsistency check on repair.
...
Update the SQL to also group on size (just in case).
Renamed the normal read method to signal that it is unverified.
Made the verified reader use enumeration state machine to avoid materializing the list.
`var`-ified some of the test code.
2024-09-09 15:27:37 +02:00
Kenneth Skovhede
17d9b89fbb
Merge branch 'master' into fix-repair-index-consistency
...
# Conflicts:
# Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
2024-09-09 11:49:34 +02:00
Kenneth Skovhede and GitHub
bfc3d61bc2
Merge branch 'master' into fix-leftover-index
2024-07-19 13:53:06 +02:00
Kenneth Skovhede
3f29331dc6
Fixed a case where listing would fail with empty filters
2024-06-21 07:52:07 +02:00
Jojo-1000
0b92f19572
Merge branch 'master' into fix-leftover-index
2024-05-11 23:37:21 +02:00
Kenneth Skovhede
49fb45859c
Fixed some incorrectly quoted literals in SQL statements
2024-04-15 08:24:01 +02:00
Kenneth Skovhede
c49e5609a9
Merge remote-tracking branch 'origin/master' into feature/kestrel-avalonia-upgrade
2024-03-07 06:35:15 +01:00
Kenneth Skovhede and GitHub
cbb2b6f295
Merge pull request #5096 from Jojo-1000/fix-invalid-file
...
Check that the database exists before writing error logs
2024-03-04 13:27:20 +01:00
Kenneth Skovhede
473c6cbbb8
Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade
2024-03-04 12:21:53 +01:00
Thomas Suckow
a3bbf63561
Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel
...
# Conflicts:
# .appveyor.yml
# .github/workflows/tests.yml
# BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
# BuildTools/DependencyFinder/DependencyFinder.csproj
# BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
# BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
# Duplicati CommandLine Only.sln
# Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
# Duplicati.Library.RestAPI/Runner.cs
# Duplicati.sln
# Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
# Duplicati/CommandLine/BackendTester/app.config
# Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
# Duplicati/CommandLine/BackendTool/app.config
# Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
# Duplicati/CommandLine/Duplicati.CommandLine.csproj
# Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
# Duplicati/CommandLine/app.config
# Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
# Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
# Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
# Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
# Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
# Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
# Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
# Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
# Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
# Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
# Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
# Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
# Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
# Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
# Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
# Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
# Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
# Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
# Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
# Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
# Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
# Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
# Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
# Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
# Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
# Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
# Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
# Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
# Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
# Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
# Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
# Duplicati/Library/Common/Duplicati.Library.Common.csproj
# Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
# Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
# Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
# Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
# Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
# Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
# Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
# Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
# Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
# Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
# Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
# Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
# Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
# Duplicati/Library/Utility/HashAlgorithmHelper.cs
# Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
# Duplicati/License/Duplicati.License.csproj
# Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
# Duplicati/Server/Duplicati.Server.csproj
# Duplicati/Server/app.config
# Duplicati/Service/Duplicati.Service.csproj
# Duplicati/Service/app.config
# Duplicati/Tools/Duplicati.Tools.csproj
# Duplicati/UnitTest/Duplicati.UnitTest.csproj
# Duplicati/UnitTest/GeneralBlackBoxTesting.cs
# Duplicati/UnitTest/app.config
# Duplicati/WindowsService/WindowsService.csproj
# Installer/debian/docker/debian/control
# Tools/SQLiteTool/SQLiteTool.csproj
# Tools/ZipFileDebugger/ZipFileDebugger.csproj
# guiTests/guiTest.py
# thirdparty/UnixSupport/UnixSupport.csproj
# thirdparty/alphavss/Homepage.txt
# thirdparty/alphavss/License.txt
# thirdparty/alphavss/licensedata.json
2024-02-29 21:55:17 -08:00
Kenneth Skovhede
40dd3c6816
Updated all license mentions to MIT
2024-02-28 15:45:30 +01:00
Jojo-1000
a0f9785f5c
Check that the database exists before writing error logs.
...
Closes #5091
Fixes that any fatal error in test-filters created an INVALID! file in the current working dir.
2024-02-09 17:10:05 +01:00
Kenneth Skovhede