Commit Graph
141 Commits
Author SHA1 Message Date
BlueBlock 54da323a1a fix codacy curly brace issue 2019-08-16 00:37:52 -04:00
BlueBlock 08e2c18185 add handling of a partial backup
- add backup state to DB table 'fileset' (job database upgrade to version 10)
- modify the Restore page dropdown to display if a backup is "partial"
- modify retention logic to remove partial backups only when the next recent full-backup has been removed
2019-08-05 20:14:05 -04:00
Kenneth Hsu a7779225b9 Run PRAGMA optimize when closing SQLite connection.
It's recommended that PRAGMA optimize be run just before closing each
database connection "to achieve the best long-term query performance
without the need to do a detailed engineering analysis of the
application schema and SQL".  Some users have been running this manually
and experiencing significant improvements in performance:

https://sqlite.org/pragma.html#pragma_optimize
https://forum.duplicati.com/t/verifying-backend-data/348/65
https://forum.duplicati.com/t/verifying-backend-data-forever/5324/32

This addresses issue #3745.
2019-04-22 21:16:29 -07:00
Kenneth Skovhede c01ab02712 Added support for "splitting" a root path.
This is a potential fix for #3637
2019-02-04 17:32:56 +01:00
Rune HenriksenandGitHub f5f353fe19 Merge branch 'master' into feature/fix_path_storage2 2019-01-25 23:37:57 +01:00
Kenneth SkovhedeandGitHub ac606d887f Merge pull request #3510 from LacunaSoftware/improve-backup-logs-ui
Improve UI for backup logs
2018-12-11 10:28:05 +01:00
Mikael MelloandGitHub 172b0284ab localDatabase: Remove trailing exclamation mark on exception text 2018-12-03 08:13:58 -02:00
Mikael Mello 1c94decf6c Serialize backup results in JSON format 2018-11-14 08:47:01 -02:00
Kenneth Skovhede 03722fcba1 Merge branch 'master' into feature/fix_path_storage2
# Conflicts:
#	Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
2018-10-23 11:33:35 +02:00
Kenneth Skovhede 68d2f10475 Merge branch 'master' of github.com:duplicati/duplicati 2018-10-17 14:37:52 +02:00
Kenneth Skovhede 2826dbe6f0 Improved error message for faults in database filelists 2018-10-17 14:37:23 +02:00
Kenneth Hsu 07c1d85b4a Remove empty statements. 2018-10-10 21:22:31 -07:00
Kenneth Hsu f4f676a1de Remove unnecessary variable assignment.
These variables are assigned values that are never read by any
subsequent instruction.
2018-09-28 20:13:45 -07:00
Kenneth Skovhede ca1872ce57 Added an option to disable filelist consistency checks as it was reported to slow backups with a large number of filesets.
This addresses #3194
2018-08-14 15:21:54 +02:00
Kenneth Skovhede cbc18970d5 Re-applied the path-storage fix as the merge failed somehow 2018-06-14 10:12:24 +02:00
Kenneth Skovhede b534c5649e Fixed a shared buffer being re-used for multiple results. 2018-06-06 08:37:09 +02:00
Kenneth Hsu 20f2f37806 Remove unnecessary calls to String.Format. 2018-06-02 12:02:55 -07:00
Kenneth Hsu f12858e6a5 Mark fields that shouldn't be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2018-05-25 10:04:52 -07:00
Kenneth Hsu 0b016cea2c Replace string concatenation with StringBuilder.
When appending to a string in a loop, using a StringBuilder should in
general yield better performance.
2018-05-19 15:36:00 -07:00
Kenneth Hsu 9c6bccf478 Use braces to improve readability.
Previously, the indentation of the following code block gave the
impression that it was part of the foreach loop when in fact it would
only be executed once.
2018-05-13 14:24:13 -07:00
Daniel Gehriger 7bf0a25912 Revert "Merge remote-tracking branch 'upstream/feature/fix_path_storage' into feature/usn"
This reverts commit 10d6b4c31c, reversing
changes made to 1af7f9e0ca.
2018-04-23 23:02:52 +02:00
Daniel Gehriger 10d6b4c31c Merge remote-tracking branch 'upstream/feature/fix_path_storage' into feature/usn 2018-04-23 22:57:32 +02:00
Kenneth Skovhede a8a32ea115 Added refactoring to improve the storage requirements for the local database, and speed up various queries.
This fixes #1283
2018-04-23 22:31:47 +02:00
Daniel Gehriger 1af7f9e0ca Merge remote-tracking branch 'upstream/master' into feature/usn 2018-04-23 22:28:27 +02:00
Daniel Gehriger 2c1a0ddb4e Implemented backup based on changes recorded in NTFS USN journal 2018-04-19 22:59:28 +02:00
Kenneth Skovhede cdd3c7b63c Merge branch 'master' into concurrent_processing 2018-04-12 15:10:03 +02:00
Kenneth Skovhede 79f7b22e0f Merge branch 'master' into concurrent_processing
Changed a few things on the way, removing the log process

# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/FilelistProcessor.cs
#	Duplicati/Library/Main/Operation/PurgeBrokenFilesHandler.cs
#	Duplicati/Library/Main/Operation/TestFilterHandler.cs
#	Duplicati/Library/Main/Options.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/UnitTest/BasicSetupHelper.cs
2018-04-11 23:02:47 +02:00
Tyler Gill baeb7e6a7c Added a new FilterType which is used by FilterEntry to represent a filter that should affect all files in one of the built-in filter groups.
This allows filter groups to be intermingled with other filters using the '{groupName}' syntax proposed by kenkendk.
It also means that when the filter that caused a file to be ignored is reported, it can report the name of the group, rather than a large and unreadable list of wildcard and regex patterns.
This is still missing several things (better documentation, removing the old --default-filters parameter, updating the UI), but should be another step in the right direction.
There might also might easily be other side effects that I haven't considered here that will need to be teased out and fixed.
2018-03-21 14:53:45 -06:00
Kenneth Skovhede 842fd96543 Implemented a new logging system that is more transparent and allows a more granular way of picking log messages.
Added ID's to each log message and each exception to allow later introduction of a Knowledgebase service that explains each error in more detail.
2018-03-15 09:12:34 +01:00
JonMikelV 4f0cf589ba Update duplicate error message to better distinguish failure point when error is thrown 2018-01-21 02:46:17 -06:00
Kenneth Skovhede 75bf6857c1 Merge branch 'master' into concurrent_processing
# Conflicts:
#	.travis.yml
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/Library/Main/Volumes/VolumeWriterBase.cs
#	Duplicati/Library/Utility/TempFile.cs
2018-01-20 15:02:41 +01:00
Kenneth Hsu 1c7391399d Remove XML documentation for unmatched parameters. 2017-12-09 18:30:01 -08:00
Kenneth Hsu 5fb1958ceb Remove redundant calls to ToString. 2017-11-26 15:44:34 -08:00
Kenneth Skovhede 49a8bd12f6 Merge branch 'master' into concurrent_processing 2017-09-18 14:39:26 +02:00
Kenneth Skovhede d57c789f4f Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Options.cs
2017-09-18 11:56:35 +02:00
Kenneth Skovhede 4a6bf3deba Added "DeleteGraceTime" to all files deleted from the backend mananger.
This should fix a number of cases where the backup returns a consistency error because the deletion and the backend check happens before the backend is in sync.
2017-09-18 11:53:50 +02:00
Dan Stahr 8a116fbf8d Fixed verification query 2017-08-12 19:14:11 +01:00
Dan Stahr 4d98960e75 Fixed empty block handling.
Previously, empty files were associasted with empty blocks. These blocks were added to volumes which could result in empty, but valid volumes being garbage collected and valid file entries deleted.

This change changes the logic so that empty blocks are not stored at all.
2017-08-12 14:14:23 +01:00
Dan Stahr 7c9b2fcd83 Fixed missed vacuum reference. 2017-08-04 12:26:15 +01:00
Kenneth Skovhede d18431b385 Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/Library/Main/ResultClasses.cs

First attempt at merging all changes from 2016-10-08 to now :/
2017-05-30 00:35:31 +02:00
Dan Stahr 27c9e15875 Improved database performance and logging
- Don't use Vacuum() just before PurgeLogData (which performs vacuum() already)
 - Removed unncessary poorly performning query (after optimizing it I found out it's unused)
 - Correctly log calls for ExecuteScalarInt64
2017-03-23 15:22:58 +00:00
Kenneth Skovhede 7f4c9383c3 Merge branch 'feature/set_tempdir_for_sqlite' 2017-01-25 21:16:45 +01:00
Kenneth Skovhede 3b4cd3c9c2 Experiment to set the SQLite temp folder, both through the environment variable and the deprecated pragma statement 2017-01-15 23:09:47 +01:00
Kenneth Skovhede f3570507e3 Merge branch 'master' into feature/improved_consistency_check 2017-01-12 09:31:39 +01:00
Kenneth Skovhede a029890409 Implemented better default error messsages 2017-01-09 11:35:38 +01:00
Kenneth Skovhede 3b85a77673 Minor style fixes 2017-01-05 09:57:06 +01:00
Kenneth Skovhede 9a227ada6b Reworked the cleanup process to allow a dangling filelist, such that this dangling entry can be used as the basis for a synthetic filelist.
Also added an option to disable synthetic filelists.
This fixes #2038.
This fixes #2101
2016-12-24 11:42:26 +01:00
Kenneth Skovhede ace2f92856 Rewrote some of the pretty printing routine to make a better log output.
Cleaned up the output by removing some of the non-relevant items.
2016-12-02 11:54:20 +01:00
Kenneth Skovhede 32423d78b5 Added EndTime value in a few places.
This fixes #2127
2016-12-02 00:08:09 +01:00
Kenneth Skovhede c1c8c9a1aa Merge branch 'master' into concurrent_processing
Some changes were required for this merge,
now the StreamBlockSplitter process will take in any stream, store the blocks and return the blockset ID.
This makes it compatible with the changes for metadata handling where they support more than a single block of metadata.

# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRepairDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
2016-10-18 13:27:36 +02:00