Commit Graph
530 Commits
Author SHA1 Message Date
Kenneth Skovhede 6362dd3cf8 Updated query to remove warning from SQLite logging 2024-04-16 14:19:15 +02:00
Kenneth Skovhede 49fb45859c Fixed some incorrectly quoted literals in SQL statements 2024-04-15 08:24:01 +02:00
Kenneth Skovhede c49e5609a9 Merge remote-tracking branch 'origin/master' into feature/kestrel-avalonia-upgrade 2024-03-07 06:35:15 +01:00
Kenneth SkovhedeandGitHub cbb2b6f295 Merge pull request #5096 from Jojo-1000/fix-invalid-file
Check that the database exists before writing error logs
2024-03-04 13:27:20 +01:00
Kenneth Skovhede 473c6cbbb8 Merge commit '4f577c65a4d8806f79637c50c21ca3a683c5a07a' into feature/kestrel-avalonia-upgrade 2024-03-04 12:21:53 +01:00
Thomas Suckow a3bbf63561 Merge remote-tracking branch 'origin/feature/upgrade-to-sdk-style-projects' into feature/net5-kestrel
# Conflicts:
#	.appveyor.yml
#	.github/workflows/tests.yml
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	Duplicati CommandLine Only.sln
#	Duplicati.Library.RestAPI/RESTMethods/RequestInfo.cs
#	Duplicati.Library.RestAPI/Runner.cs
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Assets/icons/normal-warning.png
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Program.cs
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Idrivee2/Duplicati.Library.Backend.Idrivee2.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Storj/Duplicati.Library.Backend.Storj.csproj
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/TencentCOS/Duplicati.Library.Backend.TencentCOS.csproj
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Modules/Builtin/CheckMonoSSL.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/SendJabberMessage.cs
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/HashAlgorithmHelper.cs
#	Duplicati/Library/Utility/Power/WindowsPowerSupplyState.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/GeneralBlackBoxTesting.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Installer/debian/docker/debian/control
#	Tools/SQLiteTool/SQLiteTool.csproj
#	Tools/ZipFileDebugger/ZipFileDebugger.csproj
#	guiTests/guiTest.py
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/alphavss/Homepage.txt
#	thirdparty/alphavss/License.txt
#	thirdparty/alphavss/licensedata.json
2024-02-29 21:55:17 -08:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Jojo-1000 a0f9785f5c Check that the database exists before writing error logs.
Closes #5091

Fixes that any fatal error in test-filters created an INVALID! file in the current working dir.
2024-02-09 17:10:05 +01:00
gpatel-frandGitHub e2c77b5310 do a vacuum of bug report database (#5069) 2023-12-13 09:27:51 +01:00
gpatel-frandGitHub b08b59228c report all filesets in error in VerifyConsistency (#5052)
only reporting the first one could incite to deleting
it while many filesets can be impacted by a file error
2023-12-12 18:25:14 +01:00
gpatel-frandGitHub 0de4cb7968 allow full-remote-verification to target only list and index files (#5037)
The option is turned into an enum with values True, False, ListIndexes
True and False are for compatibility with the existing backups.
ListIndexes is equivalent to True but don't handle block volumes.
This is to make more workable a strategy of full remote testing
after each backup.
2023-12-10 00:24:07 +01:00
gpatel-frandGitHub ec9d49d684 Experiment/rebuildfaster (#5043)
* experimental faster db rebuild (#4955)

* try run update db query for every block file only when needed
add repair-force-block-use option
more appropriate naming for some routines
more tracing
more comments
2023-11-05 18:19:08 +01:00
Janandgpatel-fr 71eb03130a Improve result reporting for interrupted/failed backups (Fix #4829)
Direct import of PR#4978
Update web UI for new result reports.
For operations with fatal errors, write logs to same operation ID.
Test that Interrupted flag is correct in RunScriptTests.
Update backup log display for new result reporting.
Hide file statistics for fatal errors and change fatal icon.
2023-09-21 20:01:17 +02:00
Thomas Suckow 78197453a1 Merge remote-tracking branch master into feature/net5-avalonia
# Conflicts:
#	Duplicati CommandLine Only.sln
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/Library/AutoUpdater/UpdaterManager.cs
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/Utility.cs
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Service/app.config
#	Duplicati/UnitTest/BackendToolTests.cs
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	guiTests/guiTest.py
2022-11-11 21:40:08 -08:00
tariandeath d44496f6d4 Add semicolons 2022-02-27 13:29:58 -07:00
JEDandGitHub 662bad4526 Add newline 2022-02-27 00:13:37 -07:00
tariandeath 396119aa40 Add indexes to improve backup query performance. 2022-02-26 20:24:14 -07:00
Thomas Suckow d528703fe7 Merge branch 'master' into experiment/net5-split
# Conflicts:
#	.github/workflows/tests.yml
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/Library/Backend/AzureBlob/AzureBlobWrapper.cs
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/Mega/packages.config
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Tardigrade/Properties/AssemblyInfo.cs
#	Duplicati/Library/Backend/Tardigrade/packages.config
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Common/IO/SystemIOWindows.cs
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/packages.config
#	Duplicati/Library/Utility/Utility.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/packages.config
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
2022-02-22 19:04:01 -08:00
Thomas Suckow ecb0249259 dotnet no longer provides a method for a string to hashalgorithm
* Also fix up not diposing of hashers
2021-04-04 11:17:13 -07:00
Kenneth Skovhede 4f565a004f Merge branch 'netstandard' of https://github.com/mnaiman/duplicati into netstandard
# Conflicts:
#	Duplicati.sln
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SSHv2/packages.config
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Telegram/Duplicati.Library.Backend.Telegram.csproj
#	Duplicati/Library/Backend/Telegram/Properties/AssemblyInfo.cs
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
2021-04-01 15:08:42 +02:00
aureliandevel baf489e8a9 Fix column name typo 2021-03-20 13:26:53 +11:00
Kenneth SkovhedeandGitHub 90a8c34c06 Merge pull request #4398 from warwickmm/bugfix/repair_missing_blocklists
Fix bug in repairing multiple missing blocklist hashes
2021-03-06 14:23:20 +01:00
Kenneth Hsu bacc50513e Provide explicit column names when creating temp table.
If we do not use an AS clause, the name of the column in the result set
is unspecified and may change depending on the version of SQLite.

https://www.sqlite.org/c3ref/column_name.html

This fixes #4403.
2020-12-29 19:58:06 -08:00
Thomas Suckow acea2fe10f upstream
Update Telegram SharpAESCrypt dep
Update SystemIOWindows to split the difference

# Conflicts:
#	BuildTools/AutoUpdateBuilder/AutoUpdateBuilder.csproj
#	BuildTools/AutoUpdateBuilder/packages.config
#	BuildTools/DependencyFinder/DependencyFinder.csproj
#	BuildTools/DependencyFinder/packages.config
#	BuildTools/GnupgSigningTool/GnupgSigningTool.csproj
#	BuildTools/UpdateVersionStamp/UpdateVersionStamp.csproj
#	BuildTools/UpdateVersionStamp/app.config
#	Duplicati/CommandLine/BackendTester/Duplicati.CommandLine.BackendTester.csproj
#	Duplicati/CommandLine/BackendTester/app.config
#	Duplicati/CommandLine/BackendTool/Duplicati.CommandLine.BackendTool.csproj
#	Duplicati/CommandLine/BackendTool/app.config
#	Duplicati/CommandLine/ConfigurationImporter/Duplicati.CommandLine.ConfigurationImporter.csproj
#	Duplicati/CommandLine/Duplicati.CommandLine.csproj
#	Duplicati/CommandLine/RecoveryTool/Duplicati.CommandLine.RecoveryTool.csproj
#	Duplicati/CommandLine/RecoveryTool/packages.config
#	Duplicati/CommandLine/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/app.config
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/packages.config
#	Duplicati/Library/AutoUpdater/Duplicati.Library.AutoUpdater.csproj
#	Duplicati/Library/AutoUpdater/app.config
#	Duplicati/Library/AutoUpdater/packages.config
#	Duplicati/Library/Backend/AlternativeFTP/Duplicati.Library.Backend.AlternativeFTP.csproj
#	Duplicati/Library/Backend/AlternativeFTP/app.config
#	Duplicati/Library/Backend/AlternativeFTP/packages.config
#	Duplicati/Library/Backend/AzureBlob/Duplicati.Library.Backend.AzureBlob.csproj
#	Duplicati/Library/Backend/AzureBlob/packages.config
#	Duplicati/Library/Backend/Backblaze/Duplicati.Library.Backend.Backblaze.csproj
#	Duplicati/Library/Backend/Backblaze/packages.config
#	Duplicati/Library/Backend/Box/Duplicati.Library.Backend.Box.csproj
#	Duplicati/Library/Backend/Box/packages.config
#	Duplicati/Library/Backend/CloudFiles/Duplicati.Library.Backend.CloudFiles.csproj
#	Duplicati/Library/Backend/CloudFiles/app.config
#	Duplicati/Library/Backend/Dropbox/Duplicati.Library.Backend.Dropbox.csproj
#	Duplicati/Library/Backend/Dropbox/packages.config
#	Duplicati/Library/Backend/FTP/Duplicati.Library.Backend.FTP.csproj
#	Duplicati/Library/Backend/FTP/app.config
#	Duplicati/Library/Backend/File/Duplicati.Library.Backend.File.csproj
#	Duplicati/Library/Backend/File/app.config
#	Duplicati/Library/Backend/GoogleServices/Duplicati.Library.Backend.GoogleServices.csproj
#	Duplicati/Library/Backend/GoogleServices/packages.config
#	Duplicati/Library/Backend/HubiC/Duplicati.Library.Backend.HubiC.csproj
#	Duplicati/Library/Backend/Jottacloud/Duplicati.Library.Backend.Jottacloud.csproj
#	Duplicati/Library/Backend/Mega/Duplicati.Library.Backend.Mega.csproj
#	Duplicati/Library/Backend/Mega/packages.config
#	Duplicati/Library/Backend/OAuthHelper/Duplicati.Library.OAuthHelper.csproj
#	Duplicati/Library/Backend/OAuthHelper/packages.config
#	Duplicati/Library/Backend/OneDrive/Duplicati.Library.Backend.OneDrive.csproj
#	Duplicati/Library/Backend/OneDrive/packages.config
#	Duplicati/Library/Backend/OpenStack/Duplicati.Library.Backend.OpenStack.csproj
#	Duplicati/Library/Backend/OpenStack/packages.config
#	Duplicati/Library/Backend/Rclone/Duplicati.Library.Backend.Rclone.csproj
#	Duplicati/Library/Backend/Rclone/packages.config
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/S3/app.config
#	Duplicati/Library/Backend/S3/packages.config
#	Duplicati/Library/Backend/SSHv2/Duplicati.Library.Backend.SSHv2.csproj
#	Duplicati/Library/Backend/SSHv2/packages.config
#	Duplicati/Library/Backend/SharePoint/Duplicati.Library.Backend.SharePoint.csproj
#	Duplicati/Library/Backend/SharePoint/app.config
#	Duplicati/Library/Backend/Sia/Duplicati.Library.Backend.Sia.csproj
#	Duplicati/Library/Backend/Sia/packages.config
#	Duplicati/Library/Backend/TahoeLAFS/Duplicati.Library.Backend.TahoeLAFS.csproj
#	Duplicati/Library/Backend/TahoeLAFS/packages.config
#	Duplicati/Library/Backend/Tardigrade/Duplicati.Library.Backend.Tardigrade.csproj
#	Duplicati/Library/Backend/Tardigrade/packages.config
#	Duplicati/Library/Backend/WEBDAV/Duplicati.Library.Backend.WEBDAV.csproj
#	Duplicati/Library/Backend/WEBDAV/app.config
#	Duplicati/Library/Common/Duplicati.Library.Common.csproj
#	Duplicati/Library/Common/IO/SystemIOWindows.cs
#	Duplicati/Library/Common/app.config
#	Duplicati/Library/Common/packages.config
#	Duplicati/Library/Compression/Duplicati.Library.Compression.csproj
#	Duplicati/Library/Compression/app.config
#	Duplicati/Library/Compression/packages.config
#	Duplicati/Library/DynamicLoader/Duplicati.Library.DynamicLoader.csproj
#	Duplicati/Library/DynamicLoader/app.config
#	Duplicati/Library/Encryption/Duplicati.Library.Encryption.csproj
#	Duplicati/Library/Encryption/packages.config
#	Duplicati/Library/Interface/Duplicati.Library.Interface.csproj
#	Duplicati/Library/Localization/Duplicati.Library.Localization.csproj
#	Duplicati/Library/Localization/packages.config
#	Duplicati/Library/Logging/Duplicati.Library.Logging.csproj
#	Duplicati/Library/Logging/app.config
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/app.config
#	Duplicati/Library/Main/packages.config
#	Duplicati/Library/Modules/Builtin/Duplicati.Library.Modules.Builtin.csproj
#	Duplicati/Library/Modules/Builtin/packages.config
#	Duplicati/Library/SQLiteHelper/Duplicati.Library.SQLiteHelper.csproj
#	Duplicati/Library/Snapshots/Duplicati.Library.Snapshots.csproj
#	Duplicati/Library/Snapshots/app.config
#	Duplicati/Library/Snapshots/packages.config
#	Duplicati/Library/UsageReporter/Duplicati.Library.UsageReporter.csproj
#	Duplicati/Library/UsageReporter/packages.config
#	Duplicati/Library/Utility/Duplicati.Library.Utility.csproj
#	Duplicati/Library/Utility/app.config
#	Duplicati/Library/Utility/packages.config
#	Duplicati/License/Duplicati.License.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/Duplicati.Server.Serialization.csproj
#	Duplicati/Server/Duplicati.Server.Serialization/packages.config
#	Duplicati/Server/Duplicati.Server.csproj
#	Duplicati/Server/app.config
#	Duplicati/Server/packages.config
#	Duplicati/Service/Duplicati.Service.csproj
#	Duplicati/Service/app.config
#	Duplicati/Tools/Duplicati.Tools.csproj
#	Duplicati/UnitTest/Duplicati.UnitTest.csproj
#	Duplicati/UnitTest/RestoreHandlerTests.cs
#	Duplicati/UnitTest/app.config
#	Duplicati/UnitTest/packages.config
#	Duplicati/WindowsService/App.config
#	Duplicati/WindowsService/WindowsService.csproj
#	Tools/SQLiteTool/App.config
#	Tools/SQLiteTool/SQLiteTool.csproj
#	thirdparty/TlsTest/TlsTest.csproj
#	thirdparty/UnixSupport/UnixSupport.csproj
#	thirdparty/mozroots/mozroots.csproj
2020-12-29 09:59:42 -08:00
Thomas Suckow 624a5d698b Add a catch to prevent the database being locked when migrations fail. 2020-12-28 12:18:58 -08:00
Kenneth Hsu 534b71e409 Fix bug with repairing multiple missing blocklist hashes.
If there is more than one missing blocklist hash, we must reset the
offset index before copying hashes from the next blocklist into the
buffer.

This fixes #4397.
2020-12-26 21:37:52 -08:00
Kenneth SkovhedeandGitHub 8002d3d225 Merge pull request #4376 from warwickmm/fix_indentation
Fix incorrect or misleading indentations
2020-12-19 14:43:31 +01:00
Kenneth Hsu 5951047ff1 Fix incorrect or misleading indentations.
This also adds braces to clarify scope where beneficial.
2020-11-21 13:58:16 -08:00
Kenneth Hsu 20e599586a Fix typo in variable name. 2020-11-15 12:23:46 -08:00
Dean Ferreyra 5f972c2437 Make FilterEntry of type Simple for @ again
Switch back to using `FilterType.Simple` for `@` filters to fix the
restore picker again.

In LocalDatabase.cs, replace code that was explicitly looking for the
presence of wildcard characters to instead trigger off of
`FilterType.Wildcard`.  This fixes the "find" command.

Change FilterEntry.ToString() to prefix Simple filters with `@` so
that if simple filters with wildcard characters make the round-trip
through a string, their `FilterType.Simple` type is preserved.
2020-09-01 23:04:03 -07:00
Honoré Vasconcelos 71494826f9 Improve query performence used in GetFilesAndSourceBlocksFast
Tree changes in query
* Move inner selection S to a temporary table
* Reorder join field in outer query
* Remove intersection with File view in outer selection
File view is already intersected with in selection S

With the above-mentioned changes, on a specific testset, query took 9s instead of 6min.
2020-08-26 10:10:54 +01:00
Daniel Gehriger d143f8b3f9 Improved speed of AppendFilesFromPreviousSetWithPredicate, making USN backups usable when there are large file sets 2020-08-18 11:43:23 +02:00
Daniel Gehriger 82c54dde31 Fixing AppendFilesFromPreviousSetWithPredicate 2020-08-17 23:01:20 +02:00
Daniel Gehriger c5bef5c2e0 Improved Improved speed of AppendFilesFromPreviousSetWithPredicate, making USN backups usable when there are large file sets 2020-08-17 18:06:38 +02:00
Daniel Gehriger 8e8c758c65 Improved speed of AppendFilesFromPreviousSetWithPredicate, making USN backups usable when there are large file sets 2020-08-17 17:27:45 +02:00
Max c882953fdd upstream 2020-05-31 13:50:13 +02:00
Kenneth Skovhede 9eef0a92f7 Updated the schema to include DB version 11 changes. 2020-05-10 21:30:15 +02:00
Kenneth Hsu 6638af10e7 Add indices to Block table to improve query performance.
The index on Block.Size improves the performance of the query that
checks that the user has not altered the block size of the backup
configuration.

The index on Block.Hash and Block.VolumeID improves the performance of
the query that obtains the blocklists.

There are anecdotes that these indices can drastically improve
performance (queries that used to take hours completed in seconds with
the index):

https://forum.duplicati.com/t/repairing-a-big-database/8174/4
https://github.com/duplicati/duplicati/issues/3884#issuecomment-612296016

This fixes issue #3884.
2020-04-11 11:47:40 -07:00
Kenneth SkovhedeandGitHub e45b8c4b2e Merge pull request #4049 from seantempleton/filesetdifference
Fix unexpected difference in fileset when using Stop Now
2020-03-25 07:11:04 +01:00
Kenneth Hsu 13e1f0dac6 Remove unused Database members. 2020-03-14 16:45:08 -07:00
Sean Templeton d5e064ae45 Fix a few more issues with stop now
When a backup is interrupted it will now show as a partial backup in the list of backups on the restore files page.

Temporary filelist volumes are protected from deletion in several places. All but the RestoreHandler
only protect the most recent filelist. The RestoreHandler protects all temporary filelist volumes as the user
might be restoring from one that would be deleted.

Prevent the CoCoL BackendRequests channel is retired exception from appearing when the backup is stopped by checking
if the channel is retired before trying to add the FlushRequest to the channel. Any exceptions from the BackendUploader
task will still be thrown.
2020-03-01 21:19:07 -06:00
Sean Templeton 19010e6314 Fix unexpected difference in fileset when using Stop Now
The exception could occur due to the existing delete for FilesetEntry
deleting based on volume ids, which would only match a filelist.
The bsIdsSubQuery would return BlocksetIds that should
be deleted and would delete from the FileLookup table.

FilesetEntry would then have a higher count of rows
than what would be returned by the LEFT OUTER JOINS on all of the
other tables in VerifyConsistency() since more rows had been deleted
from FileLookup.

To fix this when rows are deleted in FileLookup, use the associated metadata id
to delete from FilesetEntry and other tables based on the blockset id for the
metadata id.

Some existing delete statements were moved, none were deleted.
The existing FileLookup delete had the match on metadata id removed as
a blockset id does not equal a metadata id.
2020-03-01 18:16:33 -06:00
Sean Templeton da08be84a6 Convert VolumeID to long instead of int 2020-02-29 16:43:10 -06:00
Sean Templeton 446120ae3e A synthetic filelist could not be uploaded due to the logic in UploadSyntheticFilelist. The code passing in the fileset id to the class would get the id based on if the volume was in the Temporary or Uploading states. If the volume was in the Uploading state, present on the remote backend, and of the right size it would get changed to Uploaded.
One of the first things UploadSyntheticFilelist does is throw an exception if the volume is not in the Uploaded state. Or if it is in the Uploaded state, the next check would prevent a synthetic filelist from being created if the volume was not in the Uploading or Temporary states.

Removed the check for the volume not being in the Uploaded state as the warning in the second check handles that case.

Removed the using statement as it would dispose of the FilesetVolumeWriter after passing it to the UploadChannel. The UploadChannel could not use it since it had been disposed.

Fixed the GetRemoteVolumeFromIDAsync taking a fileset id and matching it up against a volume id. Now it will use fileset id to get the volume that matches the fileset.

Change the StopNow disruption test to not use a synthetic file list. The test was written with it not working and therefore fails a check on the number of filesets after stopping a backup.

Bonus fix for a possible null exception in Dispose for VolumeWriterBase.
2020-02-23 16:50:36 -06:00
Kenneth Hsu 3551238000 Remove redundant calls to String.Format. 2020-01-25 17:07:02 -08:00
Kenneth Hsu 35ea890f96 Merge remote-tracking branch 'upstream/master' into refactor/fileset_removal 2020-01-21 20:23:49 -08:00
Sean Templeton 3b6306b556 Fix exceptions caused by Stop Now
The LocalDatabase class was deleting from the BlocklistHash table based on the volume id of blocks. The deletes for the Blockset and BlocksetEntry tables were using a union query which included the same query the BlocklistHash delete was using. Changed the BlocklistHash delete query to use the union as well to ensure that all three tables would use the same query to delete the necessary rows. This fixes the "Detected non-empty blocksets with no associated blocks!" error.

Moved a write call in the DataBlockProcessor to send a volume for upload after setting blockvolume to null. This prevents a null reference exception because otherwise it would be sent to the SpillCollector and it tries to add a block, but the volume has already been closed.

Added the ? operator to the RegistryUtility class to prevent a null reference exception while debugging. It was annoying catching it in the debugger.

Fixes #4037
2020-01-05 21:43:56 -06:00
Kenneth Hsu fdcef72a7c Merge remote-tracking branch 'upstream/master' into refactor/fileset_removal 2020-01-01 13:58:59 -08:00
Max 41a5591532 upstream 2019-12-25 16:34:56 +01:00
Kenneth Hsu 610fa4ffea Generate backup version numbers manually instead of in query.
The ROW_NUMBER window function is not available in SQLite versions prior
to 3.25.  In some cases, I think we do fall back to the system SQLite
libraries, and many distributions do not have versions that support the
window functions.  To be safe, we will generate the backup versions
manually.
2019-12-15 13:28:44 -08:00