Kenneth Hsu
9ce1e807c7
Remove unused variables and fields.
2018-09-29 10:41:27 -07:00
Kenneth Hsu
f4f676a1de
Remove unnecessary variable assignment.
...
These variables are assigned values that are never read by any
subsequent instruction.
2018-09-28 20:13:45 -07:00
Kenneth Skovhede
809ba9a856
Changed the default query to be v==2, as multiple users reported this to be fastest.
...
The previous default query is now v==4.
Added an additional v==3, and a `LIMIT 1` clause to v==4 as suggested here:
https://forum.duplicati.com/t/release-2-0-3-10-canary-2018-08-30/4497/25
2018-09-05 10:12:28 +02:00
Kenneth Skovhede
ca1872ce57
Added an option to disable filelist consistency checks as it was reported to slow backups with a large number of filesets.
...
This addresses #3194
2018-08-14 15:21:54 +02:00
Kenneth Skovhede
d0026d8028
Reworked the file-path query, and enabled experimentally changing it by setting the environment variable TEST_QUERY_VERSION=1 or TEST_QUERY_VERSION=2.
...
Added check to the unittests to catch cases where it scans too many files.
2018-08-13 13:23:59 +02:00
Kenneth Skovhede
4d1bea86a6
Attempt at making a leaner version of getting the previous file's information
2018-08-11 14:39:03 +02:00
Kenneth Skovhede
5507496352
Fixed the query to correctly extract metadata for a file, using the Metadataset indirection
2018-08-11 14:37:23 +02:00
Kenneth Skovhede
52cb9fb1f9
Removed the deprecated --store-metadata option.
...
Rewrote the logic for pre-filtering files based on timestamps/metadata to be more verbose and well documented.
Fixed an issue where `--check-filetime-only` also checked the file size, which was not present due to the reduced query used to grab only the lastmodified information.
Fixed a problem where the check for existing metadata was wrong and produced a warning.
This fixes #3204
This handles some of #3194
2018-08-11 11:48:19 +02:00
Kenneth Skovhede and GitHub
64201ae706
Merge pull request #3314 from mnaiman/log_sql_variables
...
Log sql variables
2018-08-07 23:32:27 +02:00
Kenneth Skovhede
3831962c82
Fixed an infinite recursion issue
2018-08-07 13:35:30 +02:00
Kenneth Skovhede
e9d7c47602
Disabled logging block and file database access to reduce overhead in these queries.
...
Added an option to explicitly re-enable logging of the database queries (`--profile-all-database-queries`)
2018-08-07 09:40:43 +02:00
Max123456
99bb089b8b
log variables even in release
2018-07-06 19:44:05 +02:00
Max123456
af5e2119c0
typo
2018-07-06 19:41:25 +02:00
Kenneth Skovhede and GitHub
bbe8358ccb
Merge pull request #3225 from tygill/fix/improve-repair
...
Adds a check for BlocksetIDs that don't have any corresponding BlocksetEntrys to the broken fileset check.
2018-06-14 09:20:59 +02:00
Kenneth Skovhede
4b3389f4b6
Fix the missing block references prior to looking for missing block info
2018-06-12 08:29:55 +02:00
Kenneth Skovhede
1f32047042
Don't remove the extra index files immediately, but wait for next compact
2018-06-11 11:20:12 +02:00
Kenneth Skovhede
ec363f88a0
Fixed not deleting the non-index volumes
2018-06-11 10:34:32 +02:00
Kenneth Skovhede
bce1db123a
Merge branch 'master' into feature/speedup_backup_index_files
...
# Conflicts:
# Duplicati/Library/Utility/Utility.cs
2018-06-08 10:31:27 +02:00
Kenneth Skovhede
b534c5649e
Fixed a shared buffer being re-used for multiple results.
2018-06-06 08:37:09 +02:00
Kenneth Skovhede
d042e70ca2
Changed the returned IEnumerable values to be fully created, which avoids issues with locks and transactions being prematurely released.
...
This might help with #3202
2018-06-04 11:56:54 +02:00
Kenneth Hsu
20f2f37806
Remove unnecessary calls to String.Format.
2018-06-02 12:02:55 -07:00
Kenneth Hsu
f12858e6a5
Mark fields that shouldn't be reassigned as readonly.
...
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2018-05-25 10:04:52 -07:00
Tyler Gill
8acc9ba5c9
Blocksets with a length of 0 are not broken if the block they reference by hash is not listed.
2018-05-24 15:22:05 -06:00
Kenneth Skovhede and GitHub
fd26732e1f
Merge pull request #3228 from warwickmm/use_string_join
...
Simplify construction of query parameter string
2018-05-23 11:11:02 +02:00
Kenneth Skovhede and GitHub
19851c88f0
Merge pull request #3227 from warwickmm/use_string_builder
...
Replace string concatenation with StringBuilder
2018-05-23 11:10:36 +02:00
Tyler Gill
bbdf4c2747
Add more broken file detection:
...
Blocksets with either blockset entires or blocklist entries that refer to invalid / unreachable blocks are now treated as broken as well.
2018-05-21 10:16:51 -06:00
Kenneth Hsu
2384e6eb30
Simplify construction of query parameter string.
2018-05-19 16:12:12 -07:00
Kenneth Hsu
0b016cea2c
Replace string concatenation with StringBuilder.
...
When appending to a string in a loop, using a StringBuilder should in
general yield better performance.
2018-05-19 15:36:00 -07:00
Tyler Gill
b5931b2771
Adds a check for BlocksetIDs that don't have any corresponding BlocksetEntrys to the broken fileset check.
...
Bug #3210 (and several other possibly related ones) had cases where a recreated database would have these invalid blockset IDs.
It seems like they were caused by filesets referring to a blockset by hash, but not having any index or block file actually containing the block for that hash.
Also fixes countMissingInformation query - it should join with missingBlocklistVolumes, rather than missingBlocklistEntries, as it is selecting the VolumeID.
2018-05-17 08:11:13 -06:00
verhoek
30cd41585b
Corrected spelling function name.
2018-05-16 20:08:40 +02:00
Kenneth Skovhede
2c44cce136
Merge branch 'feature/remove-console-writeline'
2018-05-15 12:12:07 +02:00
Kenneth Skovhede
ee64517f7d
Removed calls to Console.WriteLine as we may run somewhere without a console.
2018-05-15 11:29:08 +02:00
Kenneth Hsu
9c6bccf478
Use braces to improve readability.
...
Previously, the indentation of the following code block gave the
impression that it was part of the foreach loop when in fact it would
only be executed once.
2018-05-13 14:24:13 -07:00
Kenneth Skovhede
57367890b9
Updated the query that fixes recreates where some remote dblock files are found through the dindex files, but these are not present.
2018-05-11 11:34:48 +02:00
Kenneth Skovhede
632b4ff0e2
Added support for cases where the recreate finds missing block files that are not needed.
2018-05-10 00:57:51 +02:00
Daniel Gehriger
8236cae390
USN change journal based backup now correctly reporting source size
2018-05-02 11:24:24 +02:00
Daniel Gehriger
7bf0a25912
Revert "Merge remote-tracking branch 'upstream/feature/fix_path_storage' into feature/usn"
...
This reverts commit 10d6b4c31c1af7f9e0ca
2018-04-23 23:02:52 +02:00
Daniel Gehriger
10d6b4c31c
Merge remote-tracking branch 'upstream/feature/fix_path_storage' into feature/usn
2018-04-23 22:57:32 +02:00
Kenneth Skovhede
a8a32ea115
Added refactoring to improve the storage requirements for the local database, and speed up various queries.
...
This fixes #1283
2018-04-23 22:31:47 +02:00
Daniel Gehriger
1af7f9e0ca
Merge remote-tracking branch 'upstream/master' into feature/usn
2018-04-23 22:28:27 +02:00
Daniel Gehriger
2c1a0ddb4e
Implemented backup based on changes recorded in NTFS USN journal
2018-04-19 22:59:28 +02:00
Kenneth Skovhede
cdd3c7b63c
Merge branch 'master' into concurrent_processing
2018-04-12 15:10:03 +02:00
Kenneth Skovhede
79f7b22e0f
Merge branch 'master' into concurrent_processing
...
Changed a few things on the way, removing the log process
# Conflicts:
# Duplicati/Library/Main/Controller.cs
# Duplicati/Library/Main/Database/ExtensionMethods.cs
# Duplicati/Library/Main/Database/LocalBackupDatabase.cs
# Duplicati/Library/Main/Duplicati.Library.Main.csproj
# Duplicati/Library/Main/Operation/BackupHandler.cs
# Duplicati/Library/Main/Operation/FilelistProcessor.cs
# Duplicati/Library/Main/Operation/PurgeBrokenFilesHandler.cs
# Duplicati/Library/Main/Operation/TestFilterHandler.cs
# Duplicati/Library/Main/Options.cs
# Duplicati/Library/Main/ResultClasses.cs
# Duplicati/UnitTest/BasicSetupHelper.cs
2018-04-11 23:02:47 +02:00
Tyler Gill
a3a98c644a
Merge branch 'master' of https://github.com/duplicati/duplicati into feature/filter-groups
2018-04-10 10:48:46 -06:00
Kenneth Hsu
cb0a3ed2a1
Make constructors for abstract classes protected.
...
While this does not affect any behavior, it more accurately describes
the accessibility. Since abstract classes can only be instantiated by
an instance of a derived type, the constructors should at most have
protected access.
2018-04-07 18:04:41 -07:00
Tyler Gill
baeb7e6a7c
Added a new FilterType which is used by FilterEntry to represent a filter that should affect all files in one of the built-in filter groups.
...
This allows filter groups to be intermingled with other filters using the '{groupName}' syntax proposed by kenkendk.
It also means that when the filter that caused a file to be ignored is reported, it can report the name of the group, rather than a large and unreadable list of wildcard and regex patterns.
This is still missing several things (better documentation, removing the old --default-filters parameter, updating the UI), but should be another step in the right direction.
There might also might easily be other side effects that I haven't considered here that will need to be teased out and fixed.
2018-03-21 14:53:45 -06:00
Kenneth Skovhede
e124329f1e
Merge branch 'feature/improved-logging'
2018-03-16 08:38:14 +01:00
Kenneth Skovhede
842fd96543
Implemented a new logging system that is more transparent and allows a more granular way of picking log messages.
...
Added ID's to each log message and each exception to allow later introduction of a Knowledgebase service that explains each error in more detail.
2018-03-15 09:12:34 +01:00
Kenneth Hsu
58ab443eb4
Fix generic class code references in XML documentation.
2018-03-07 18:34:26 -08:00
Kenneth Skovhede
9093603be1
Added support for disabling the file scanner during backups.
...
This fixes #2963
2018-02-11 20:57:38 +01:00
Kenneth Hsu