This PR adds support for locking files if the backend supports it.
To activate locking, set the option `--file-lock-duration=30D` and the backup will lock the files.
If the database is rebuilt with the intention of continuing the backups, use the option `--repair-refresh-lock-info` which will update lock information in the database after recreating the database.
The locking works by asking the backend to lock files after a backup has completed.
The implementation keeps track of which files are currently assigned a lock and prevents attempting to delete the files that are currently locked.
Note that the bucket should not have a default lock policy as Duplicati needs to finish the backup before the locking is applied.
In this initial version, Azure Blob Storage, B2, S3 and iDrive are supported with locking.
The CLI is updated to allow setting locks on a specific version. The backend tool is updated to allow setting locks on specific files.
The index on Block.Size improves the performance of the query that
checks that the user has not altered the block size of the backup
configuration.
The index on Block.Hash and Block.VolumeID improves the performance of
the query that obtains the blocklists.
There are anecdotes that these indices can drastically improve
performance (queries that used to take hours completed in seconds with
the index):
https://forum.duplicati.com/t/repairing-a-big-database/8174/4https://github.com/duplicati/duplicati/issues/3884#issuecomment-612296016
This fixes issue #3884.
codacy uses sqlint so formatting looks to be inportant or an error is found.
Format the SQL using sqlint or a tool that uses sqlint such as https://www.onlinegdb.com/online_sqlite_editor and run Beautify.
- add backup state to DB table 'fileset' (job database upgrade to version 10)
- modify the Restore page dropdown to display if a backup is "partial"
- modify retention logic to remove partial backups only when the next recent full-backup has been removed
- Fixed SQL to create temp table for lookup of last backup's file
entries
- Removed temp table by writing directly to internal lookup struct.
- Direct lookup of single file entries from DB
On a smaller dataset, the performance is now only a little more than twice as slow with a low memory footprint compared to the default with all memory caches enabled.
Kenneth Skovhede