Some of the database migrations I'm planning to perform are very slow if using SQL only
or entirely impossible using the standard SQLite toolchain. I added a simple interface
with "before SQL" and "after SQL" hooks so that a database upgrade can perform custom
code operations as well.
# Conflicts:
# Duplicati/Library/Main/Controller.cs
# Duplicati/Library/Main/Database/ExtensionMethods.cs
# Duplicati/Library/Main/Database/LocalBackupDatabase.cs
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
# Duplicati/Library/Main/Operation/BackupHandler.cs
# Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
# Duplicati/Library/Main/ResultClasses.cs
First attempt at merging all changes from 2016-10-08 to now :/
A simple backup that specifies --deleted-files causes a Fatal error.
Shortened repro steps:
(See Github comment for full repro steps)
>duplicati.commandline.exe backup file://C:\data\local\testchangedfiles\destination "C:\data\local\testchangedfiles\source\\" --encryption-module="none" --no-encryption="true" --dbpath="testchangedfiles.sqlite" --compression-module="zip" --keep-versions=-1 --no-auto-compact="true"
>del source\file6.txt
>duplicati.commandline.exe backup file://C:\data\local\testchangedfiles\destination "C:\data\local\testchangedfiles\source\\" --changed-files="C:\data\local\testchangedfiles\source\file1.txt;C:\data\local\testchangedfiles\source\file2.txt" --deleted-files="C:\data\local\testchangedfiles\source\file6.txt" --encryption-module="none" --no-encryption="true" --dbpath="testchangedfiles.sqlite" --compression-module="zip" --keep-versions=-1 --no-auto-compact="true"
Results in:
Fatal error => Input string was not in a correct format.
System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber
at System.Number.ParseInt64
at System.String.System.IConvertible.ToInt64
at System.Convert.ToInt64
at System.Data.SQLite.SQLiteStatement.BindParameter
at System.Data.SQLite.SQLiteStatement.BindParameters
at System.Data.SQLite.SQLiteCommand.BuildNextCommand
at System.Data.SQLite.SQLiteCommand.GetStatement
at System.Data.SQLite.SQLiteDataReader.NextResult
at System.Data.SQLite.SQLiteDataReader..ctor
at System.Data.SQLite.SQLiteCommand.ExecuteReader
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery
at Duplicati.Library.Main.Database.LocalBackupDatabase.AppendFilesFromPreviousSet
at Duplicati.Library.Main.Database.LocalBackupDatabase.AppendFilesFromPreviousSet
at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation
at Duplicati.Library.Main.Operation.BackupHandler.Run
at Duplicati.Library.Main.Controller.RunAction[T]
at Duplicati.Library.Main.Controller.Backup
at Duplicati.CommandLine.Commands.Backup
at Duplicati.CommandLine.Program.RunCommandLine
>Exit code: 100
With this patch, backup completes successfully.
Without this guard, it would be possible to extend the stream during processing and send wrong blocks forward.
This is likely a fix for issue #1400 and #2402
- Don't use Vacuum() just before PurgeLogData (which performs vacuum() already)
- Removed unncessary poorly performning query (after optimizing it I found out it's unused)
- Correctly log calls for ExecuteScalarInt64
Kenneth Skovhede