This should fix a number of cases where the backup returns a consistency error because the deletion and the backend check happens before the backend is in sync.
Previously, empty files were associasted with empty blocks. These blocks were added to volumes which could result in empty, but valid volumes being garbage collected and valid file entries deleted.
This change changes the logic so that empty blocks are not stored at all.
# Conflicts:
# Duplicati/Library/Main/Controller.cs
# Duplicati/Library/Main/Database/ExtensionMethods.cs
# Duplicati/Library/Main/Database/LocalBackupDatabase.cs
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
# Duplicati/Library/Main/Operation/BackupHandler.cs
# Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
# Duplicati/Library/Main/ResultClasses.cs
First attempt at merging all changes from 2016-10-08 to now :/
A simple backup that specifies --deleted-files causes a Fatal error.
Shortened repro steps:
(See Github comment for full repro steps)
>duplicati.commandline.exe backup file://C:\data\local\testchangedfiles\destination "C:\data\local\testchangedfiles\source\\" --encryption-module="none" --no-encryption="true" --dbpath="testchangedfiles.sqlite" --compression-module="zip" --keep-versions=-1 --no-auto-compact="true"
>del source\file6.txt
>duplicati.commandline.exe backup file://C:\data\local\testchangedfiles\destination "C:\data\local\testchangedfiles\source\\" --changed-files="C:\data\local\testchangedfiles\source\file1.txt;C:\data\local\testchangedfiles\source\file2.txt" --deleted-files="C:\data\local\testchangedfiles\source\file6.txt" --encryption-module="none" --no-encryption="true" --dbpath="testchangedfiles.sqlite" --compression-module="zip" --keep-versions=-1 --no-auto-compact="true"
Results in:
Fatal error => Input string was not in a correct format.
System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber
at System.Number.ParseInt64
at System.String.System.IConvertible.ToInt64
at System.Convert.ToInt64
at System.Data.SQLite.SQLiteStatement.BindParameter
at System.Data.SQLite.SQLiteStatement.BindParameters
at System.Data.SQLite.SQLiteCommand.BuildNextCommand
at System.Data.SQLite.SQLiteCommand.GetStatement
at System.Data.SQLite.SQLiteDataReader.NextResult
at System.Data.SQLite.SQLiteDataReader..ctor
at System.Data.SQLite.SQLiteCommand.ExecuteReader
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery
at Duplicati.Library.Main.Database.LocalBackupDatabase.AppendFilesFromPreviousSet
at Duplicati.Library.Main.Database.LocalBackupDatabase.AppendFilesFromPreviousSet
at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation
at Duplicati.Library.Main.Operation.BackupHandler.Run
at Duplicati.Library.Main.Controller.RunAction[T]
at Duplicati.Library.Main.Controller.Backup
at Duplicati.CommandLine.Commands.Backup
at Duplicati.CommandLine.Program.RunCommandLine
>Exit code: 100
With this patch, backup completes successfully.
- Don't use Vacuum() just before PurgeLogData (which performs vacuum() already)
- Removed unncessary poorly performning query (after optimizing it I found out it's unused)
- Correctly log calls for ExecuteScalarInt64
Previously, the output of the first replace was never used and therefore the absolute Windows paths were not restored
correctly.
The two patterns (i.e. ?: and \\) are disjoint so rewriting the previous result in case a match is found
for the second one is OK.
Kenneth Skovhede