Sean Templeton
b02b19e683
Change Put method name to PutAsync
2019-03-17 18:28:45 -05:00
Kenneth Skovhede and GitHub
38dfdffb90
Merge pull request #3681 from verhoek/epic/keyring/1-refactor-server-program
...
Epic/keyring/1 refactor server program
2019-03-17 21:33:40 +01:00
Hendrik and GitHub
9816cfa0c0
Merge pull request #3687 from warwickmm/dispose_enumerator
...
Dispose of manually iterated enumerator
2019-03-14 20:06:03 +01:00
Sean Templeton
19c1c908df
Check upload throttle speed every 2 seconds
...
The upload throttle speed was being checked on average once every
5 milliseconds.
Changed a few variable names.
2019-03-13 21:51:06 -05:00
Sean Templeton
cc3e94fece
Fix code review comments and other minor changes
...
- Rename indexVolume to indexVolumeWriter in DataBlockProcessor to reduce
confusion.
- When flushing the remaining uploads, check the tasks for exceptions in
completion order rather than waiting for them all at once. If there is an
exception all other uploads are cancelled anyway.
- Remove setting the operation progress as Backup_WaitForUpload in the
BackendUploader since the BackupHandler sets it.
- Use an exception filter in BackendUploader.
- Use Options property MaxUploadPrSecond instead of getting the raw value.
2019-03-13 21:51:06 -05:00
Sean Templeton
8bd4b825c8
Fix several Codacy issues
2019-03-13 21:51:05 -05:00
Sean Templeton
61f511c087
Properly catch exceptions from an upload and cancel all other uploads
...
If a backend throws an exception while uploading, cancel other uploads and
throw an exception from the BackendUploader. Exceptions from cancelled
uploads are caught and not thrown so as to not spam the user with useless
information.
2019-03-13 21:51:02 -05:00
Kenneth Hsu
a1d8637c3a
Fix typo in comment.
2019-03-11 21:37:42 -07:00
Kenneth Hsu
e57c714ee8
Dispose of manually enumerated enumerator.
...
The IEnumerator<T> interface implements IDisposable, so the enumerator
should be disposed of after its use.
2019-03-11 21:15:00 -07:00
Sean Templeton
ac08aa2fb1
Dynamically throttle and track the progress of concurrent uploads
...
Changed AddBackendEvent() in ResultClasses to allow the option to not update
the backend progress. This allows the BackendUploader to log the act of starting
an upload without changing the backend progress. The backend progress is now
handled by a new class, FileProgressThrottler, which will determine the transfer
rate of all uploads currently in progress.
This class also handles throttling the upload streams. The algorithm will try
to decrease or increase the streams throttle rates as appropriate by changing
the value based on the over/under amount or based on a percentage to prevent
throttling one stream directly to zero.
2019-03-05 21:25:35 -06:00
Sean Templeton
12b15c09c8
Add asynchronous-concurrent-upload-limit option
...
This new option limits the number of concurrent uploads allowed when
performing asynchronous uploads. The default value is 4.
2019-03-05 21:25:35 -06:00
Sean Templeton
c05f304fd7
Update S3, Sharepoint, Tahoe, and WEBDAV backends for async Put
2019-03-05 21:25:35 -06:00
Sean Templeton
13f83a22a1
Update Mega, OneDrive, and Rclone backends for async Put
2019-03-05 21:25:29 -06:00
Sean Templeton
c3472823de
Update HubiC and OpenStack backends for async Put
2019-03-05 21:08:26 -06:00
Sean Templeton
7843f837a4
Update Google Cloud and Drive backends for async Put
2019-03-05 21:08:25 -06:00
Sean Templeton
9a42eca1d6
Update FTP and Jottacloud backends for async Put
2019-03-05 21:08:25 -06:00
Sean Templeton
4ca067a8f2
Update Dropbox and File backends for async Put
2019-03-05 21:08:25 -06:00
Sean Templeton
7f09c7af27
Update Box and Rackspace Cloud Files backends for async Put
2019-03-05 21:08:25 -06:00
Sean Templeton
5db326bcdb
Update Azure Blob and Backblaze B2 backends for async Put
2019-03-05 21:08:24 -06:00
Sean Templeton
b1c6e9798d
Update Amazon Cloud Drive backend for async Put
2019-03-05 21:08:14 -06:00
verhoek
9a39276ca9
Remove unwanted files.
2019-03-04 09:43:39 +01:00
Sean Templeton
662627a98a
Update AlternativeFtp Backend for async Put
...
Removed System.Net.FtpClient and added the FluentFTP Nuget package which
is the new name of System.Net.FtpClient.
2019-03-03 21:43:13 -06:00
Sean Templeton
0c122ff550
Change IBackend and IStreamingBackend to return Task for Put()
2019-03-03 21:43:10 -06:00
Sean Templeton
3de04da39e
Run each upload on a separate thread
...
Added a small Worker class, private to the BackendUploader, to track the
upload tasks and the backend for each task.
2019-03-03 21:43:00 -06:00
Sean Templeton
1029d99e10
Simplify BackendUploader and remove unnecessary code
2019-03-03 21:42:58 -06:00
Sean Templeton
05c3f4808c
Move encrypting, hashing, and creating index volumes out of BackendUploader
...
Encrypting, hashing, and creating index volumes are now done
in the DataBlockProcessor and the SpillCollectorProcess. This allows
uploads to always be transferring data and not have to stop to create
a file, encrypt data, etc.
2019-03-03 21:41:09 -06:00
Sean Templeton
4f4529a88a
Move upload related code from BackendHandler to BackendUploader
...
As the BackendUploader is in charge of uploads the code has been moved
there to centralize the responsibility and prepare for parallel uploads.
2019-03-03 21:35:09 -06:00
verhoek
b8875f0559
Removed duplicate string.
2019-03-03 18:40:30 +01:00
verhoek
a62c2f3db4
Small fixes.
2019-02-28 20:34:19 +01:00
verhoek
59d3ceb91a
null propagation.
2019-02-28 17:56:32 +01:00
verhoek
26f42a3ea8
Extracted worker thread and webserver methods.
2019-02-28 17:51:52 +01:00
Kenneth Skovhede and GitHub
356ca81d8b
Merge pull request #3662 from warwickmm/remove_temporary_files
...
Remove temporary files from backup and compact operations
2019-02-26 14:22:18 +01:00
Rune Henriksen and GitHub
9d3fb26989
Merge pull request #3677 from FroggieFrog/update-known-locations
...
added new known locations of S3
2019-02-25 23:45:47 +01:00
Rune Henriksen and GitHub
27ab74143e
Merge pull request #3676 from FroggieFrog/issue-3665
...
fix #3665 added "Wasabi Hot Storage (EU Central)"
2019-02-25 23:41:42 +01:00
FroggieFrog
496b923079
added new known locations of S3
2019-02-25 16:24:28 +01:00
FroggieFrog
97894bc655
fix #3665 added "Wasabi Hot Storage (EU Central)"
2019-02-25 16:13:57 +01:00
verhoek
e1e29962e3
Moved livecontrol to own method.
2019-02-25 12:34:32 +01:00
verhoek
f7ccb75e13
Minor refactoring.
2019-02-25 11:35:43 +01:00
verhoek
0bea4e0289
Extract purge temp files timer logic to method.
2019-02-25 11:35:43 +01:00
verhoek
57c9805848
Extract method.
2019-02-25 11:35:43 +01:00
verhoek
8330d7bdb9
Call GenerateWebserverPasswordTrayIcon from one place only.
2019-02-25 11:35:43 +01:00
verhoek
b57f679899
Extracted method.
2019-02-25 11:35:43 +01:00
verhoek
2249de7dfa
Promote vars to member instances.
2019-02-25 11:35:43 +01:00
verhoek
b699783182
Extracted function.
2019-02-25 11:35:43 +01:00
verhoek
4cff586da3
Consolidate logic of settings from database or not.
2019-02-25 11:35:43 +01:00
verhoek
d4de82b070
Renamed instance.
2019-02-25 11:35:43 +01:00
verhoek
d9bba6ccc3
Conver static consts to lowercase. Extract logging config to private function.
2019-02-25 11:35:43 +01:00
verhoek
38ddcf765f
Refactored parameterfile option.
2019-02-25 11:35:43 +01:00
verhoek
c352b9f117
Make alternative help commands more compact.
2019-02-25 11:35:43 +01:00
verhoek
baad3e2c95
Simplify show help.
2019-02-25 11:35:43 +01:00
Sean Templeton