Commit Graph
72 Commits
Author SHA1 Message Date
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Jon Schewe e5f57e8f66 Remove unused LOGTAG field 2020-07-11 18:12:59 -05:00
Jon Schewe 2747f78c40 Remove unneeded catch statement
Using the finally block to close out the writer allows the exception to
show up in the log.
2020-07-11 17:33:38 -05:00
Jon Schewe 36336ac6f1 Rethrow exception after logging
This ensures that the backup properly fails.
2020-07-10 06:55:41 -05:00
Jon Schewe e41c0d8bf9 Catch compression errors and report them
When the file list is over 4GB one needs to have the zip64 option
enabled. This commit catches the exception and reports it as an error so
that the user knows what the problem is.

Fixes duplicati/duplicati#/4206
2020-07-08 07:55:25 -05:00
Sean Templeton 446120ae3e A synthetic filelist could not be uploaded due to the logic in UploadSyntheticFilelist. The code passing in the fileset id to the class would get the id based on if the volume was in the Temporary or Uploading states. If the volume was in the Uploading state, present on the remote backend, and of the right size it would get changed to Uploaded.
One of the first things UploadSyntheticFilelist does is throw an exception if the volume is not in the Uploaded state. Or if it is in the Uploaded state, the next check would prevent a synthetic filelist from being created if the volume was not in the Uploading or Temporary states.

Removed the check for the volume not being in the Uploaded state as the warning in the second check handles that case.

Removed the using statement as it would dispose of the FilesetVolumeWriter after passing it to the UploadChannel. The UploadChannel could not use it since it had been disposed.

Fixed the GetRemoteVolumeFromIDAsync taking a fileset id and matching it up against a volume id. Now it will use fileset id to get the volume that matches the fileset.

Change the StopNow disruption test to not use a synthetic file list. The test was written with it not working and therefore fails a check on the number of filesets after stopping a backup.

Bonus fix for a possible null exception in Dispose for VolumeWriterBase.
2020-02-23 16:50:36 -06:00
Kenneth Hsu d7e3ac9b10 Move CreateFilesetFile method to FilesetVolumeWriter class.
This only concerns filesets, so it does not belong in the base class.
2020-01-25 18:53:15 -08:00
Tyler Gill e14be62c2b Make the temp file the default and only bevavior - fileset volume no longer uses memory stream 2019-11-01 19:27:49 -06:00
Tyler Gill bc0686794a Cache to disk 2019-10-30 17:30:47 -06:00
Kenneth SkovhedeandGitHub 99d0cb3c57 Merge pull request #3953 from warwickmm/make_fields_readonly
Mark fields that don't need to be reassigned as readonly
2019-10-21 08:36:17 +02:00
Kenneth Hsu 2aa76881a5 Mark fields that don't need to be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2019-10-19 13:59:22 -07:00
Kenneth Hsu afab61a05b Use pattern matching to simplify casts. 2019-10-19 10:15:38 -07:00
Kenneth SkovhedeandGitHub 579f9911d5 Merge pull request #3938 from warwickmm/bugfix/wrong_dblock_in_dindex
Ensure that dindex references correct dblock file after renames
2019-10-19 14:19:11 +02:00
Kenneth Hsu 614ccaf79d Convert duplicated static methods into instance method. 2019-10-04 21:37:33 -07:00
Kenneth Hsu 8d7c6c6e21 Ensure that filelist is always written to dlist files.
This fixes a regression introduced around revisions e0899a2a91 ("add
method to add the filelist file") and 3b5af1cf01 ("add method to add
filelist file"), where the filelist was not written by all usages of the
FilesetVolumeWriter.  Previously, one would have to invoke
AddFilelistFile with each usage.  Now, we simply do so when the
FilesetVolumeWriter is closed.

We also modified the UploadSyntheticFilelist so that the usage of the
FilesetVolumeWriter is contained in a using statement, which will ensure
that it is disposed.

This fixes issue #3924.
2019-09-27 20:35:06 -07:00
BlueBlock e0899a2a91 add method to add the filelist file
this also fixes "control-files" being added to dlist
2019-09-10 13:09:25 -04:00
BlueBlock 6c05c19ca3 add methods to handle reading the fileset file 2019-09-10 13:07:57 -04:00
BlueBlock 075f9faa92 add method to create fileset file 2019-09-10 13:06:55 -04:00
BlueBlock a230e3b228 Revert "spelling fix"
This reverts commit 8bb4bfaeb4.
2019-09-09 18:26:01 -04:00
BlueBlock 8bb4bfaeb4 spelling fix 2019-09-09 18:25:29 -04:00
BlueBlock 74e4bcd8de add FilesetData object 2019-09-09 17:42:51 -04:00
Kenneth Hsu a1d8637c3a Fix typo in comment. 2019-03-11 21:37:42 -07:00
Kenneth Hsu e57c714ee8 Dispose of manually enumerated enumerator.
The IEnumerator<T> interface implements IDisposable, so the enumerator
should be disposed of after its use.
2019-03-11 21:15:00 -07:00
Kenneth Hsu daf0311bf8 Allow removal of VolumeWriterBase temp files.
This reverts a change made in revision 31ac40083a ("Prevent temp files
in VolumeWriterBase from being deleted prematurely" in pull request #2637).
It's unclear what issue was being worked around by preventing the
removal of the temporary files (based on comments in pull request #2637,
it appears to be related to tests that failed only on Travis).  However,
this workaround prevented the removal of temporary files created during
compact operations, which caused several users' temporary directories to
be filled up with files that should have been removed.

This concerns issue #3652.
2019-02-16 19:53:06 -08:00
Kenneth Hsu c27ea24826 Ensure that FilesetVolumeWriter removes temp file upon disposal.
This ensures the removal of the temporary file containing the
filelist.json that remained regardless if files needed to be uploaded or
not.

This concerns issue #3652.
2019-02-16 19:53:06 -08:00
Kenneth Hsu 9e0b031fee Remove unused members. 2018-10-15 21:20:30 -07:00
Kenneth Hsu 731a8b1644 Remove unused empty methods. 2018-10-10 21:20:40 -07:00
Kenneth Hsu d72128ec83 Remove unused method parameters. 2018-10-06 15:09:43 -07:00
Kenneth Skovhede bce1db123a Merge branch 'master' into feature/speedup_backup_index_files
# Conflicts:
#	Duplicati/Library/Utility/Utility.cs
2018-06-08 10:31:27 +02:00
Kenneth Hsu 20f2f37806 Remove unnecessary calls to String.Format. 2018-06-02 12:02:55 -07:00
Kenneth Hsu f12858e6a5 Mark fields that shouldn't be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2018-05-25 10:04:52 -07:00
Kenneth Skovhede 52987da027 Improved error message when reading invalid blocklists 2018-05-09 17:37:54 +02:00
Kenneth Skovhede b0110acab6 Added a check to make sure we do not write invalid blocklist files 2018-05-09 17:21:50 +02:00
Kenneth Skovhede 79f7b22e0f Merge branch 'master' into concurrent_processing
Changed a few things on the way, removing the log process

# Conflicts:
#	Duplicati/Library/Main/Controller.cs
#	Duplicati/Library/Main/Database/ExtensionMethods.cs
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Duplicati.Library.Main.csproj
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/Operation/FilelistProcessor.cs
#	Duplicati/Library/Main/Operation/PurgeBrokenFilesHandler.cs
#	Duplicati/Library/Main/Operation/TestFilterHandler.cs
#	Duplicati/Library/Main/Options.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/UnitTest/BasicSetupHelper.cs
2018-04-11 23:02:47 +02:00
Kenneth Hsu cb0a3ed2a1 Make constructors for abstract classes protected.
While this does not affect any behavior, it more accurately describes
the accessibility.  Since abstract classes can only be instantiated by
an instance of a derived type, the constructors should at most have
protected access.
2018-04-07 18:04:41 -07:00
Kenneth Skovhede 842fd96543 Implemented a new logging system that is more transparent and allows a more granular way of picking log messages.
Added ID's to each log message and each exception to allow later introduction of a Knowledgebase service that explains each error in more detail.
2018-03-15 09:12:34 +01:00
Kenneth Skovhede bf07f70318 Added method to automatically create folder for async upload.
This fixes #1384
2018-01-26 09:22:21 +01:00
Kenneth Skovhede 75bf6857c1 Merge branch 'master' into concurrent_processing
# Conflicts:
#	.travis.yml
#	Duplicati/Library/Main/Database/LocalBackupDatabase.cs
#	Duplicati/Library/Main/Operation/BackupHandler.cs
#	Duplicati/Library/Main/ResultClasses.cs
#	Duplicati/Library/Main/Volumes/VolumeWriterBase.cs
#	Duplicati/Library/Utility/TempFile.cs
2018-01-20 15:02:41 +01:00
Kenneth Skovhede d9377bab7e Removed TempFileStram and fixed the two places that were using it, as they both had errors in the way it was used. 2018-01-18 14:25:25 +01:00
vnau 1c67118737 Type pattern conditions removed to compile with AppVeyor. 2017-12-25 23:51:10 +07:00
vnau 62a7901aca Compression modules refactored to work with streams instead of filenames;
Added unit test to check compression reversibility
2017-12-25 04:12:19 +07:00
Kenneth Hsu 604b6e4337 Dispose enumerators after manually iterating over elements.
Since the IEnumerator<T> interface implements IDisposable, we should
make sure to dispose of the enumerator when manually iterating over the
elements.
2017-10-07 12:27:11 -07:00
Kenneth Skovhede 0dd56f1650 Merge branch 'master' into concurrent_processing
# Conflicts:
#	Duplicati/Library/Main/Options.cs
2017-09-19 12:51:40 +02:00
Tyler Gill 83a1dcfb64 Replace all instances of InvariantCultureIgnoreCase with OrdinalIgnoreCase in string comparisons.
InvariantCulture is useful when comparing / sorting human language strings in a culturely correct way. It handles things like accented letters in a way that makes sense to humans (e.g., 'a' should be sorted next to 'á', rather than after 'z').
Ordinal looks just at the raw code points of the characters. As such, it is recommended for use in cases when comparing system strings (file paths, command line parameters, config settings, etc.). Since it doesn't need to use the culture specific sorting rules, this method can often be faster.

For more information, see https://stackoverflow.com/questions/492799/difference-between-invariantculture-and-ordinal-string-comparison (and other related questions)
2017-09-18 23:55:08 -06:00
Dan Stahr 31ac40083a Prevent temp files in VolumeWriterBase from being deleted prematurely.
The fix is hacky and the root cause should be investigated properly.
2017-08-13 11:58:39 +01:00
Kenneth Skovhede a029890409 Implemented better default error messsages 2017-01-09 11:35:38 +01:00
Kenneth Skovhede 309bf1648d Merge branch 'master' into feature/fix_for_different_blockhash_and_filehash_plus_multiblocks_metadata
# Conflicts:
#	Duplicati/Library/Main/Database/LocalDatabase.cs
#	Duplicati/Library/Main/Database/LocalRecreateDatabase.cs
#	Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs
#	Duplicati/UnitTest/BorderTests.cs
2016-10-17 13:28:27 +02:00
Kenneth Skovhede e91cfe33be Added support for the asynchronous upload folder option.
This fixes #1795
And handles some of #1914
2016-09-28 10:36:40 +02:00
Kenneth Skovhede 0c68daff6c Fixed all whitespace to be 4 spaces instead of tabs 2016-09-15 11:39:27 +02:00
Kenneth Skovhede 9688ca4438 Added app-version to the manifest info to aid in tracking down problems later 2016-04-13 12:40:24 +02:00