* Added test to verify that compression modules will mask duplicate entries, if present
* Because there are likely many, otherwise valid, dindex zip archives with duplicated blocklists out there, the log message is now verbose.
To make sure we no longer create archives with duplicated entries, the `--unittest-mode` option makes this a warning that is checked in most tests.
* Added guard in `IndexVolumeCreator` to prevent creating zip files with duplicated block lists.
* Added debug check for duplicates
* Fixed the duplicate test
Rewrote the backup and restore to work within a transaction as that speeds things up a lot.
Rewrote the backup procedure to generate the file list from the database, instead of on-the-fly as that is more efficient.
Re-arranged the database to use less storage on large sets of unchanged files.
Still needs a work over in terms of error handling, progress reporting, warning output, refactoring etc.
Also misses the volume reuse/reclaim algorithms.
But it passes the unit tests now, with and without a local database.
git-svn-id: https://duplicati.googlecode.com/svn/sandboxes/Kenneth/ForestHash@1526 59da171f-624f-0410-aa54-27559c288bec
