A simple backup that specifies --deleted-files causes a Fatal error.
Shortened repro steps:
(See Github comment for full repro steps)
>duplicati.commandline.exe backup file://C:\data\local\testchangedfiles\destination "C:\data\local\testchangedfiles\source\\" --encryption-module="none" --no-encryption="true" --dbpath="testchangedfiles.sqlite" --compression-module="zip" --keep-versions=-1 --no-auto-compact="true"
>del source\file6.txt
>duplicati.commandline.exe backup file://C:\data\local\testchangedfiles\destination "C:\data\local\testchangedfiles\source\\" --changed-files="C:\data\local\testchangedfiles\source\file1.txt;C:\data\local\testchangedfiles\source\file2.txt" --deleted-files="C:\data\local\testchangedfiles\source\file6.txt" --encryption-module="none" --no-encryption="true" --dbpath="testchangedfiles.sqlite" --compression-module="zip" --keep-versions=-1 --no-auto-compact="true"
Results in:
Fatal error => Input string was not in a correct format.
System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber
at System.Number.ParseInt64
at System.String.System.IConvertible.ToInt64
at System.Convert.ToInt64
at System.Data.SQLite.SQLiteStatement.BindParameter
at System.Data.SQLite.SQLiteStatement.BindParameters
at System.Data.SQLite.SQLiteCommand.BuildNextCommand
at System.Data.SQLite.SQLiteCommand.GetStatement
at System.Data.SQLite.SQLiteDataReader.NextResult
at System.Data.SQLite.SQLiteDataReader..ctor
at System.Data.SQLite.SQLiteCommand.ExecuteReader
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery
at Duplicati.Library.Main.Database.LocalBackupDatabase.AppendFilesFromPreviousSet
at Duplicati.Library.Main.Database.LocalBackupDatabase.AppendFilesFromPreviousSet
at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation
at Duplicati.Library.Main.Operation.BackupHandler.Run
at Duplicati.Library.Main.Controller.RunAction[T]
at Duplicati.Library.Main.Controller.Backup
at Duplicati.CommandLine.Commands.Backup
at Duplicati.CommandLine.Program.RunCommandLine
>Exit code: 100
With this patch, backup completes successfully.
- Don't use Vacuum() just before PurgeLogData (which performs vacuum() already)
- Removed unncessary poorly performning query (after optimizing it I found out it's unused)
- Correctly log calls for ExecuteScalarInt64
Now we only look at the timestamp from the previous fileset, instead of for the last time the file was seen.
This should improve lookup times, and thus the running time for backups with many files.
This change means that the following sequence of operations will react differently:
- backup
- delete file
- backup
- undelete file
- backup
Prior to this commit, the undeleted file would be seen with the original timestamp and thus can avoid being scanned for changes.
After this commit, the file will be treated as a new file, even if it has the previous timestamp and will thus be scanned for changes.
An additional switch has been introduced, --check-filetime-only that improves the lookup further by not looking at the metadata or filesize, and thus makes a much lighter call to the database, where fewer tables need to be touched.
- Fixed SQL to create temp table for lookup of last backup's file
entries
- Removed temp table by writing directly to internal lookup struct.
- Direct lookup of single file entries from DB
On a smaller dataset, the performance is now only a little more than twice as slow with a low memory footprint compared to the default with all memory caches enabled.
Implemented better output for Restore.
Fixed a problem with errors during dispose, and a problem with selecting filesets that do not exist.
Fixed a bug with restore if a file existed with some correct data, a copy would be restored with partial data.
Added check to try to avoid excessive restores of the same files if a file in another version already exists.
Kenneth Skovhede