haeaeae/duplicati
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8,782 Commits 140 Branches 416 Tags
b73e94de7ee24c5261e14f29bcceecb39b519aa8
Commit Graph
530 Commits
Author SHA1 Message Date
Dan Stahr 27c9e15875 Improved database performance and logging 
- Don't use Vacuum() just before PurgeLogData (which performs vacuum() already)
 - Removed unncessary poorly performning query (after optimizing it I found out it's unused)
 - Correctly log calls for ExecuteScalarInt64
 2017-03-23 15:22:58 +00:00
Dan Stahr af14151d91 Fixed an error in renaming absolute Windows paths. 
Previously, the output of the first replace was never used and therefore the absolute Windows paths were not restored
correctly.

The two patterns (i.e. ?: and \\) are disjoint so rewriting the previous result in case a match is found
for the second one is OK.
 2017-03-10 22:01:24 +00:00
Kenneth Skovhede 5919309da9 Added additional checks to attempt to gather information from the remote destination, if the local database is somehow missing the size or hash 2017-02-27 00:43:06 +01:00
Kenneth Skovhede 9fd0121142 Removing the code that updates the logdata, as it has no effect, since any updated error messages will be destroyed later, should they contain any path-like strings. 
This should help with #2298
 2017-02-02 22:23:54 +01:00
Kenneth Skovhede 7f4c9383c3 Merge branch 'feature/set_tempdir_for_sqlite' 2017-01-25 21:16:45 +01:00
Kenneth Skovhede 0ad55a1ef2 Merge branch 'master' into feature/improved_consistency_check 2017-01-19 14:06:42 +01:00
Kenneth Skovhede d08b7750f0 Removed the unnecessary distinct clause from the recreate query. 
Added a comment about the potentially unnecessary extra join in the statements that fills in block hashes.
 2017-01-19 10:53:32 +01:00
Kenneth Skovhede 305d8bc7fe Added an extra condition on the join for the recreate issue. 
This is a less experimental rewrite of the query that fixes the exact problem.
The speedups from this query fix are also substantial, going from 2.5 hours to 11 minutes.

This is likely a fix to these issues:
#1699, #2048, #2140, #2178

This could help with issues:
#1391, #2050
 2017-01-19 09:55:36 +01:00
Kenneth Skovhede 3b4cd3c9c2 Experiment to set the SQLite temp folder, both through the environment variable and the deprecated pragma statement 2017-01-15 23:09:47 +01:00
Kenneth Skovhede be46960a6b Added debugging code that dumps the temporary tables as permanent tables if the error from #1699, #2048, #2140, or #2178 2017-01-15 23:08:21 +01:00
Kenneth Skovhede 97ba7ef9a5 Fixed issue with restoring files to their original location. 
This fixes #2246
 2017-01-14 12:46:59 +01:00
Kenneth Skovhede f3570507e3 Merge branch 'master' into feature/improved_consistency_check 2017-01-12 09:31:39 +01:00
Kenneth Skovhede f5a1bcd32c Reverted the SQL query to use the Timestamp sorting instead of explicit fileset. 
This fixes #2244
 2017-01-11 23:48:41 +01:00
Kenneth Skovhede d3a3d8c85f Fixed a mistake in the debugging code for issue 2140 2017-01-11 13:51:56 +01:00
Kenneth Skovhede a029890409 Implemented better default error messsages 2017-01-09 11:35:38 +01:00
Kenneth Skovhede 9a849a2b58 Improved support for cross-os restores. 
This fixes #1302
This fixes #1983
 2017-01-06 23:01:54 +01:00
Kenneth Skovhede 8571475938 Merge branch 'feature/debug_listing_for_1400' 2017-01-06 10:03:03 +01:00
Kenneth Skovhede 684bd867ed Merge branch 'feature/debug_listing_for_2140' 2017-01-05 22:16:32 +01:00
Kenneth Skovhede 1b4cb3b5df Merge branch 'feature/implement_remote_file_recovery' 2017-01-05 22:16:25 +01:00
Kenneth Skovhede 179290d521 Added more extensive listing to further pinpoint the source of the error in #1400 2017-01-05 22:15:59 +01:00
Kenneth Skovhede fd0d0d5529 Added tracing output to track down issue #1699, #2140 and #2178 2017-01-05 21:41:21 +01:00
Kenneth Skovhede 1108d51d7f Implemented a new new list-broken-files and purge-broken-files command for the commandline to assist with handling remote file loss. 2017-01-05 16:46:01 +01:00
Kenneth Skovhede 76aae5de7b Changed the logic for getting the last-modified timestamp from the database. 
Now we only look at the timestamp from the previous fileset, instead of for the last time the file was seen.
This should improve lookup times, and thus the running time for backups with many files.

This change means that the following sequence of operations will react differently:

- backup
- delete file
- backup
- undelete file
- backup

Prior to this commit, the undeleted file would be seen with the original timestamp and thus can avoid being scanned for changes.
After this commit, the file will be treated as a new file, even if it has the previous timestamp and will thus be scanned for changes.

An additional switch has been introduced, --check-filetime-only that improves the lookup further by not looking at the metadata or filesize, and thus makes a much lighter call to the database, where fewer tables need to be touched.
 2017-01-05 10:03:32 +01:00
Kenneth Skovhede 3b85a77673 Minor style fixes 2017-01-05 09:57:06 +01:00
Kenneth Skovhede 4bf4871427 Added commandline operation to purge select files from filesets. 
This fixes #1009.
This paves the way for #1337.
 2016-12-29 22:41:13 +01:00
Kenneth Skovhede 9a227ada6b Reworked the cleanup process to allow a dangling filelist, such that this dangling entry can be used as the basis for a synthetic filelist. 
Also added an option to disable synthetic filelists.
This fixes #2038.
This fixes #2101
 2016-12-24 11:42:26 +01:00
Kenneth Skovhede 3182a227b9 Added additional debug info for tracing the cause of issue #1400 2016-12-23 17:04:12 +01:00
Kenneth Skovhede d8972b0f2b Removed unused memory cache code 2016-12-16 17:30:39 +01:00
Kenneth Skovhede ace2f92856 Rewrote some of the pretty printing routine to make a better log output. 
Cleaned up the output by removing some of the non-relevant items.
 2016-12-02 11:54:20 +01:00
Kenneth Skovhede 32423d78b5 Added EndTime value in a few places. 
This fixes #2127
 2016-12-02 00:08:09 +01:00
tianqiaoxiangzhe b05c5b4efd defaultvalue is not used 
defaultvalue is not used
 2016-12-01 09:52:30 +08:00
Kenneth Skovhede c1c8c9a1aa Merge branch 'master' into concurrent_processing 
Some changes were required for this merge,
now the StreamBlockSplitter process will take in any stream, store the blocks and return the blockset ID.
This makes it compatible with the changes for metadata handling where they support more than a single block of metadata.

# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRepairDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
 2016-10-18 13:27:36 +02:00
Kenneth Skovhede f281eb9ee2 Merge branch 'master' into feature/improved_consistency_check 
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
 2016-10-18 11:09:38 +02:00
Kenneth Skovhede 3a034c65ff Removed some extra dispose code 2016-10-17 15:32:30 +02:00
Kenneth Skovhede 309bf1648d Merge branch 'master' into feature/fix_for_different_blockhash_and_filehash_plus_multiblocks_metadata 
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/UnitTest/BorderTests.cs
 2016-10-17 13:28:27 +02:00
Kenneth Skovhede 126f12bf0a Merge branch 'master' into concurrent_processing 2016-09-29 13:48:30 +02:00
Kenneth Skovhede d381c99ca5 Updated serialization of the ResultClasses to provide nicer output. 
This fixes #1912
 2016-09-28 20:20:16 +02:00
Kenneth Skovhede 94c5e10480 Merge branch 'master' into concurrent_processing 
# Conflicts:
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/RemoteVolumeEntry.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/Library/Main/Operation/RepairHandler.cs
#	Duplicati/Library/Main/Operation/RestoreControlFilesHandler.cs
 2016-09-16 10:33:07 +02:00
Kenneth Skovhede db8c025497 Merge branch 'master' into concurrent_processing 
# Conflicts:
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
 2016-09-15 11:44:39 +02:00
Kenneth Skovhede 0c68daff6c Fixed all whitespace to be 4 spaces instead of tabs 2016-09-15 11:39:27 +02:00
Kenneth Skovhede 490776ed73 Implemented a database flag to mark a database as partially recreated. 
Fixed issue with error messages when attempting to recreate a partial database for restore as described in #1695.
 2016-09-13 21:55:15 +02:00
Kenneth Skovhede f0c0fc3a2e Merge branch 'master' into concurrent_processing 
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	thirdparty/CoCoL/CoCoL.dll
 2016-08-08 15:03:42 +02:00
Kenneth Skovhede 0f2549aa8f Merge pull request #1750 from FootStark/hotfix/LookupLastModified 
Suggested fix for lookup of file's last backup entry
 2016-05-09 23:03:34 +02:00
FootStark 568cc60856 Fixed lookup of a file's last entry during backup 
- Fixed SQL to create temp table for lookup of last backup's file
entries
- Removed temp table by writing directly to internal lookup struct.
- Direct lookup of single file entries from DB
 2016-05-06 00:14:03 +02:00
Kenneth Skovhede a350d942e6 Fixed a case where a specially crafted broken index file could prevent the database from being recreated correctly. 
This fix also optimizes recreates where data is missing in index files and block files are downloaded.
 2016-05-02 20:30:38 +02:00
Kenneth Skovhede 02423b5a46 Added additional logic for getting the backup path prefixes on multi-drive backups in Windows. 
This fixes #1672.
This fixes #1527.
 2016-04-19 10:04:22 +02:00
Kenneth Skovhede 742eab0913 Fixed multiple cases where connections and command instances were not disposed. 
This should fix all cases of "File is locked" on windows.
 2016-04-06 20:40:34 +02:00
Kenneth Skovhede 66eb681541 Fixed issues that arose when using multi-threaded execution 2016-04-05 23:29:14 +02:00
Kenneth Skovhede 32d8c18257 Fixed repair consistency check 2016-04-05 14:00:12 +02:00
Kenneth Skovhede bb2964345a Improved the consistency check to also find entries with no blocksetentries 2016-04-04 22:47:16 +02:00