haeaeae/duplicati
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,149 Commits 140 Branches 416 Tags
992a0fd90cd9ca93647d052d55c05fdb110ac1ce
Commit Graph
10149 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Sean Templeton 8be3203aa2 Fix index files not being created for full dblock files 
As part of the parallel upload changes a bug slipped in where a check was
being made if the index volume writer exists before creating said index
volume writer. Fixed by changing the test to check if the temporary index
volume exists and then creating the writer for the index file.
 2019-03-30 19:55:36 -05:00
Rune HenriksenandGitHub fc5249d304 Merge pull request #3699 from warwickmm/rename_async_property 
Fix typo in async property name
 2019-03-29 17:04:44 +01:00
Kenneth Hsu da4dabe838 Fix typo in async property name. 2019-03-28 20:53:55 -07:00
Kenneth Skovhede a48cc01eca Version bump to v2.0.4.16-2.0.4.16_canary_2019-03-28 
You can download this build from:

Binaries: https://updates.duplicati.com/canary/duplicati-2.0.4.16_canary_2019-03-28.zip

Signature file: https://updates.duplicati.com/canary/duplicati-2.0.4.16_canary_2019-03-28.zip.sig

ASCII signature file: https://updates.duplicati.com/canary/duplicati-2.0.4.16_canary_2019-03-28.zip.sig.asc

MD5: d39e182d7da8393d92ab9e0eb7953c90

SHA1: 3b506d77549b32ec6a13d819efbcb63988b6c822

SHA256: 7d09e1922b41944568ff503e00d567e3f52fc744b0c6990295ba584aa522b7d1
v2.0.4.16-2.0.4.16_canary_2019-03-28 		2019-03-28 10:14:18 +01:00
warwickmmandGitHub 5b54c26571 Merge pull request #3691 from nescafe2002/master 
Implement Jottacloud multithreaded downloading support
 2019-03-26 18:59:59 -07:00
Kenneth SkovhedeandGitHub 762cc8e555 Merge pull request #3695 from warwickmm/unwrap_reflection_exceptions 
Unwrap TargetInvocationExceptions while preserving stack trace
 2019-03-26 09:33:48 +01:00
Joris d1161f9872 Codacy recommendation 2019-03-25 10:47:18 +01:00
Joris 5bffd23879 Commented on chunk size limiting 2019-03-25 10:46:58 +01:00
Joris 6440bf7c90 Fixed: The file X was downloaded and had size 0 but the size was expected to be N 2019-03-22 09:35:09 +01:00
Kenneth Hsu 81b9e75a29 Preserve stack trace while unwrapping TargetInvocationExceptions. 
If we simply throw the inner exception, the stack trace is lost.  If we
throw a new exception with the originating exception as the inner
exception, the user is not provided with any meaningful information.
 2019-03-21 19:26:14 -07:00
Joris c93e0eed4d Processed code review findings 2019-03-21 10:14:57 +01:00
Joris eeab3ced58 Removed unnecessary GetResponse calls 2019-03-20 09:12:19 +01:00
JorisandGitHub 38968d03eb Removing Codacy recommendation 2019-03-19 21:44:22 +01:00
JorisandGitHub 719360a899 Merge branch 'master' into master 2019-03-19 21:40:46 +01:00
Joris c46e07b1e5 Implemented Jottacloud multithreaded downloading support 2019-03-19 19:32:00 +01:00
Kenneth SkovhedeandGitHub 57b28182cb Merge pull request #3684 from seantempleton/parallelupload 
Implement parallel uploads to a single backend
 2019-03-19 08:41:33 +01:00
Sean Templeton 1f388f4d03 Fix MegaBackend null reference exception 
null was being passed to UploadAsync instead of an instance of an IProgress class.
MegaApiClient does not check if IProgress is null before using it.
 2019-03-17 22:14:00 -05:00
Sean Templeton b02b19e683 Change Put method name to PutAsync 2019-03-17 18:28:45 -05:00
Kenneth SkovhedeandGitHub 38dfdffb90 Merge pull request #3681 from verhoek/epic/keyring/1-refactor-server-program 
Epic/keyring/1 refactor server program
 2019-03-17 21:33:40 +01:00
HendrikandGitHub 9816cfa0c0 Merge pull request #3687 from warwickmm/dispose_enumerator 
Dispose of manually iterated enumerator
 2019-03-14 20:06:03 +01:00
Sean Templeton 19c1c908df Check upload throttle speed every 2 seconds 
The upload throttle speed was being checked on average once every
5 milliseconds.

Changed a few variable names.
 2019-03-13 21:51:06 -05:00
Sean Templeton cc3e94fece Fix code review comments and other minor changes 
- Rename indexVolume to indexVolumeWriter in DataBlockProcessor to reduce
confusion.

- When flushing the remaining uploads, check the tasks for exceptions in
completion order rather than waiting for them all at once. If there is an
exception all other uploads are cancelled anyway.

- Remove setting the operation progress as Backup_WaitForUpload in the
BackendUploader since the BackupHandler sets it.

- Use an exception filter in BackendUploader.

- Use Options property MaxUploadPrSecond instead of getting the raw value.
 2019-03-13 21:51:06 -05:00
Sean Templeton 8bd4b825c8 Fix several Codacy issues 2019-03-13 21:51:05 -05:00
Sean Templeton 61f511c087 Properly catch exceptions from an upload and cancel all other uploads 
If a backend throws an exception while uploading, cancel other uploads and
throw an exception from the BackendUploader. Exceptions from cancelled
uploads are caught and not thrown so as to not spam the user with useless
information.
 2019-03-13 21:51:02 -05:00
Kenneth Hsu a1d8637c3a Fix typo in comment. 2019-03-11 21:37:42 -07:00
Kenneth Hsu e57c714ee8 Dispose of manually enumerated enumerator. 
The IEnumerator<T> interface implements IDisposable, so the enumerator
should be disposed of after its use.
 2019-03-11 21:15:00 -07:00
Sean Templeton ac08aa2fb1 Dynamically throttle and track the progress of concurrent uploads 
Changed AddBackendEvent() in ResultClasses to allow the option to not update
the backend progress. This allows the  BackendUploader to log the act of starting
an upload without changing the backend progress. The backend progress is now
handled by a new class, FileProgressThrottler, which will determine the transfer
rate of all uploads currently in progress.

This class also handles throttling the upload streams. The algorithm will try
to decrease or increase the streams throttle rates as appropriate by changing
the value based on the over/under amount or based on a percentage to prevent
throttling one stream directly to zero.
 2019-03-05 21:25:35 -06:00
Sean Templeton 12b15c09c8 Add asynchronous-concurrent-upload-limit option 
This new option limits the number of concurrent uploads allowed when
performing asynchronous uploads. The default value is 4.
 2019-03-05 21:25:35 -06:00
Sean Templeton c05f304fd7 Update S3, Sharepoint, Tahoe, and WEBDAV backends for async Put 2019-03-05 21:25:35 -06:00
Sean Templeton 13f83a22a1 Update Mega, OneDrive, and Rclone backends for async Put 2019-03-05 21:25:29 -06:00
Sean Templeton c3472823de Update HubiC and OpenStack backends for async Put 2019-03-05 21:08:26 -06:00
Sean Templeton 7843f837a4 Update Google Cloud and Drive backends for async Put 2019-03-05 21:08:25 -06:00
Sean Templeton 9a42eca1d6 Update FTP and Jottacloud backends for async Put 2019-03-05 21:08:25 -06:00
Sean Templeton 4ca067a8f2 Update Dropbox and File backends for async Put 2019-03-05 21:08:25 -06:00
Sean Templeton 7f09c7af27 Update Box and Rackspace Cloud Files backends for async Put 2019-03-05 21:08:25 -06:00
Sean Templeton 5db326bcdb Update Azure Blob and Backblaze B2 backends for async Put 2019-03-05 21:08:24 -06:00
Sean Templeton b1c6e9798d Update Amazon Cloud Drive backend for async Put 2019-03-05 21:08:14 -06:00
verhoek 9a39276ca9 Remove unwanted files. 2019-03-04 09:43:39 +01:00
Sean Templeton 662627a98a Update AlternativeFtp Backend for async Put 
Removed System.Net.FtpClient and added the FluentFTP Nuget package which
is the new name of System.Net.FtpClient.
 2019-03-03 21:43:13 -06:00
Sean Templeton 0c122ff550 Change IBackend and IStreamingBackend to return Task for Put() 2019-03-03 21:43:10 -06:00
Sean Templeton 3de04da39e Run each upload on a separate thread 
Added a small Worker class, private to the BackendUploader, to track the
upload tasks and the backend for each task.
 2019-03-03 21:43:00 -06:00
Sean Templeton 1029d99e10 Simplify BackendUploader and remove unnecessary code 2019-03-03 21:42:58 -06:00
Sean Templeton 05c3f4808c Move encrypting, hashing, and creating index volumes out of BackendUploader 
Encrypting, hashing, and creating index volumes are now done
in the DataBlockProcessor and the SpillCollectorProcess. This allows
uploads to always be transferring data and not have to stop to create
a file, encrypt data, etc.
 2019-03-03 21:41:09 -06:00
Sean Templeton 4f4529a88a Move upload related code from BackendHandler to BackendUploader 
As the BackendUploader is in charge of uploads the code has been moved
there to centralize the responsibility and prepare for parallel uploads.
 2019-03-03 21:35:09 -06:00
verhoek b8875f0559 Removed duplicate string. 2019-03-03 18:40:30 +01:00
verhoek a62c2f3db4 Small fixes. 2019-02-28 20:34:19 +01:00
verhoek 59d3ceb91a null propagation. 2019-02-28 17:56:32 +01:00
verhoek 26f42a3ea8 Extracted worker thread and webserver methods. 2019-02-28 17:51:52 +01:00
Kenneth SkovhedeandGitHub 356ca81d8b Merge pull request #3662 from warwickmm/remove_temporary_files 
Remove temporary files from backup and compact operations
 2019-02-26 14:22:18 +01:00
Rune HenriksenandGitHub 9d3fb26989 Merge pull request #3677 from FroggieFrog/update-known-locations 
added new known locations of S3
 2019-02-25 23:45:47 +01:00