Commit Graph
65 Commits
Author SHA1 Message Date
Tyler Gill e14be62c2b Make the temp file the default and only bevavior - fileset volume no longer uses memory stream 2019-11-01 19:27:49 -06:00
Tyler Gill bc0686794a Cache to disk 2019-10-30 17:30:47 -06:00
Kenneth SkovhedeandGitHub 99d0cb3c57 Merge pull request #3953 from warwickmm/make_fields_readonly
Mark fields that don't need to be reassigned as readonly
2019-10-21 08:36:17 +02:00
Kenneth Hsu 2aa76881a5 Mark fields that don't need to be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2019-10-19 13:59:22 -07:00
Kenneth Hsu afab61a05b Use pattern matching to simplify casts. 2019-10-19 10:15:38 -07:00
Kenneth SkovhedeandGitHub 579f9911d5 Merge pull request #3938 from warwickmm/bugfix/wrong_dblock_in_dindex
Ensure that dindex references correct dblock file after renames
2019-10-19 14:19:11 +02:00
Kenneth Hsu 614ccaf79d Convert duplicated static methods into instance method. 2019-10-04 21:37:33 -07:00
Kenneth Hsu 8d7c6c6e21 Ensure that filelist is always written to dlist files.
This fixes a regression introduced around revisions e0899a2a91 ("add
method to add the filelist file") and 3b5af1cf01 ("add method to add
filelist file"), where the filelist was not written by all usages of the
FilesetVolumeWriter.  Previously, one would have to invoke
AddFilelistFile with each usage.  Now, we simply do so when the
FilesetVolumeWriter is closed.

We also modified the UploadSyntheticFilelist so that the usage of the
FilesetVolumeWriter is contained in a using statement, which will ensure
that it is disposed.

This fixes issue #3924.
2019-09-27 20:35:06 -07:00
BlueBlock e0899a2a91 add method to add the filelist file
this also fixes "control-files" being added to dlist
2019-09-10 13:09:25 -04:00
BlueBlock 6c05c19ca3 add methods to handle reading the fileset file 2019-09-10 13:07:57 -04:00
BlueBlock 075f9faa92 add method to create fileset file 2019-09-10 13:06:55 -04:00
BlueBlock a230e3b228 Revert "spelling fix"
This reverts commit 8bb4bfaeb4.
2019-09-09 18:26:01 -04:00
BlueBlock 8bb4bfaeb4 spelling fix 2019-09-09 18:25:29 -04:00
BlueBlock 74e4bcd8de add FilesetData object 2019-09-09 17:42:51 -04:00
Kenneth Hsu a1d8637c3a Fix typo in comment. 2019-03-11 21:37:42 -07:00
Kenneth Hsu e57c714ee8 Dispose of manually enumerated enumerator.
The IEnumerator<T> interface implements IDisposable, so the enumerator
should be disposed of after its use.
2019-03-11 21:15:00 -07:00
Kenneth Hsu daf0311bf8 Allow removal of VolumeWriterBase temp files.
This reverts a change made in revision 31ac40083a ("Prevent temp files
in VolumeWriterBase from being deleted prematurely" in pull request #2637).
It's unclear what issue was being worked around by preventing the
removal of the temporary files (based on comments in pull request #2637,
it appears to be related to tests that failed only on Travis).  However,
this workaround prevented the removal of temporary files created during
compact operations, which caused several users' temporary directories to
be filled up with files that should have been removed.

This concerns issue #3652.
2019-02-16 19:53:06 -08:00
Kenneth Hsu c27ea24826 Ensure that FilesetVolumeWriter removes temp file upon disposal.
This ensures the removal of the temporary file containing the
filelist.json that remained regardless if files needed to be uploaded or
not.

This concerns issue #3652.
2019-02-16 19:53:06 -08:00
Kenneth Hsu 9e0b031fee Remove unused members. 2018-10-15 21:20:30 -07:00
Kenneth Hsu 731a8b1644 Remove unused empty methods. 2018-10-10 21:20:40 -07:00
Kenneth Hsu d72128ec83 Remove unused method parameters. 2018-10-06 15:09:43 -07:00
Kenneth Skovhede bce1db123a Merge branch 'master' into feature/speedup_backup_index_files
# Conflicts:
#	Duplicati/Library/Utility/Utility.cs
2018-06-08 10:31:27 +02:00
Kenneth Hsu 20f2f37806 Remove unnecessary calls to String.Format. 2018-06-02 12:02:55 -07:00
Kenneth Hsu f12858e6a5 Mark fields that shouldn't be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2018-05-25 10:04:52 -07:00
Kenneth Skovhede 52987da027 Improved error message when reading invalid blocklists 2018-05-09 17:37:54 +02:00
Kenneth Skovhede b0110acab6 Added a check to make sure we do not write invalid blocklist files 2018-05-09 17:21:50 +02:00
Kenneth Skovhede 79f7b22e0f Merge branch 'master' into concurrent_processing
Changed a few things on the way, removing the log process

# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/FilelistProcessor.cs
#	Duplicati/Library/Main/Operation/PurgeBrokenFilesHandler.cs
#	Duplicati/Library/Main/Operation/TestFilterHandler.cs
#	Duplicati/Library/Main/Options.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/UnitTest/BasicSetupHelper.cs
2018-04-11 23:02:47 +02:00
Kenneth Hsu cb0a3ed2a1 Make constructors for abstract classes protected.
While this does not affect any behavior, it more accurately describes
the accessibility.  Since abstract classes can only be instantiated by
an instance of a derived type, the constructors should at most have
protected access.
2018-04-07 18:04:41 -07:00
Kenneth Skovhede 842fd96543 Implemented a new logging system that is more transparent and allows a more granular way of picking log messages.
Added ID's to each log message and each exception to allow later introduction of a Knowledgebase service that explains each error in more detail.
2018-03-15 09:12:34 +01:00
Kenneth Skovhede bf07f70318 Added method to automatically create folder for async upload.
This fixes #1384
2018-01-26 09:22:21 +01:00
Kenneth Skovhede 75bf6857c1 Merge branch 'master' into concurrent_processing
# Conflicts:
#	.travis.yml
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/Library/Main/Volumes/VolumeWriterBase.cs
#	Duplicati/Library/Utility/TempFile.cs
2018-01-20 15:02:41 +01:00
Kenneth Skovhede d9377bab7e Removed TempFileStram and fixed the two places that were using it, as they both had errors in the way it was used. 2018-01-18 14:25:25 +01:00
vnau 1c67118737 Type pattern conditions removed to compile with AppVeyor. 2017-12-25 23:51:10 +07:00
vnau 62a7901aca Compression modules refactored to work with streams instead of filenames;
Added unit test to check compression reversibility
2017-12-25 04:12:19 +07:00
Kenneth Hsu 604b6e4337 Dispose enumerators after manually iterating over elements.
Since the IEnumerator<T> interface implements IDisposable, we should
make sure to dispose of the enumerator when manually iterating over the
elements.
2017-10-07 12:27:11 -07:00
Kenneth Skovhede 0dd56f1650 Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Options.cs
2017-09-19 12:51:40 +02:00
Tyler Gill 83a1dcfb64 Replace all instances of InvariantCultureIgnoreCase with OrdinalIgnoreCase in string comparisons.
InvariantCulture is useful when comparing / sorting human language strings in a culturely correct way. It handles things like accented letters in a way that makes sense to humans (e.g., 'a' should be sorted next to 'á', rather than after 'z').
Ordinal looks just at the raw code points of the characters. As such, it is recommended for use in cases when comparing system strings (file paths, command line parameters, config settings, etc.). Since it doesn't need to use the culture specific sorting rules, this method can often be faster.

For more information, see https://stackoverflow.com/questions/492799/difference-between-invariantculture-and-ordinal-string-comparison (and other related questions)
2017-09-18 23:55:08 -06:00
Dan Stahr 31ac40083a Prevent temp files in VolumeWriterBase from being deleted prematurely.
The fix is hacky and the root cause should be investigated properly.
2017-08-13 11:58:39 +01:00
Kenneth Skovhede a029890409 Implemented better default error messsages 2017-01-09 11:35:38 +01:00
Kenneth Skovhede 309bf1648d Merge branch 'master' into feature/fix_for_different_blockhash_and_filehash_plus_multiblocks_metadata
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/UnitTest/BorderTests.cs
2016-10-17 13:28:27 +02:00
Kenneth Skovhede e91cfe33be Added support for the asynchronous upload folder option.
This fixes #1795
And handles some of #1914
2016-09-28 10:36:40 +02:00
Kenneth Skovhede 0c68daff6c Fixed all whitespace to be 4 spaces instead of tabs 2016-09-15 11:39:27 +02:00
Kenneth Skovhede 9688ca4438 Added app-version to the manifest info to aid in tracking down problems later 2016-04-13 12:40:24 +02:00
Kenneth Skovhede d101f91c94 Fixed support for metadata larger than blocksize. Now supports one full blocklist of metadata, appx 300mb with default settings.
Added correct handling of metadata hashes, such that a metadata blockset is now handled the same was as a regular file blockset.
The query that creates the filesets does not support more than a single blocklist of metadata, hence the limitiation.
2016-04-04 18:11:48 +02:00
Kenneth Skovhede e40bd7f02a Proposed version, not complete with handling large metadata chunks 2016-04-03 23:10:47 +02:00
Kenneth Skovhede bf07813ca8 Real fix for the problem with duplicate fileset timestamp on really short backups 2016-03-20 12:38:36 +01:00
Kenneth Skovhede ade9db9dca Utility fixes 2016-03-17 21:43:09 +01:00
Kenneth Skovhede 2dc2fb4197 Improved the BlocklistEnumerable with yield statements instead of a manual implementation 2016-03-17 21:42:03 +01:00
Kenneth Skovhede 9ae7ea233a Possible fix for PR #1456 and issue #1449 2015-09-08 16:41:16 +02:00
Kenneth Skovhede 5bc424437a Added options to do partial repairs of a temporary local database, to allow faster restore/searching with no local database 2015-04-08 21:01:36 +02:00