The only code in the finally block was to set the result of the
FlushRequest if the completed task was not faulted. This could be done
without the try-finally.
The FlushBackend method uses Task.WhenAny to await the first completion
of the uploader task or the flush request. Since the uploader was
setting the status of the flush request in the finally block, the flush
request would complete first when an exception was thrown, causing the
uploader's exception to be unobserved. To resolve this, we check that
the uploader's status is not faulted before setting the flush request
status.
This addresses issue #3673.
As part of the parallel upload changes a bug slipped in where a check was
being made if the index volume writer exists before creating said index
volume writer. Fixed by changing the test to check if the temporary index
volume exists and then creating the writer for the index file.
If we simply throw the inner exception, the stack trace is lost. If we
throw a new exception with the originating exception as the inner
exception, the user is not provided with any meaningful information.
- Rename indexVolume to indexVolumeWriter in DataBlockProcessor to reduce
confusion.
- When flushing the remaining uploads, check the tasks for exceptions in
completion order rather than waiting for them all at once. If there is an
exception all other uploads are cancelled anyway.
- Remove setting the operation progress as Backup_WaitForUpload in the
BackendUploader since the BackupHandler sets it.
- Use an exception filter in BackendUploader.
- Use Options property MaxUploadPrSecond instead of getting the raw value.
If a backend throws an exception while uploading, cancel other uploads and
throw an exception from the BackendUploader. Exceptions from cancelled
uploads are caught and not thrown so as to not spam the user with useless
information.
Changed AddBackendEvent() in ResultClasses to allow the option to not update
the backend progress. This allows the BackendUploader to log the act of starting
an upload without changing the backend progress. The backend progress is now
handled by a new class, FileProgressThrottler, which will determine the transfer
rate of all uploads currently in progress.
This class also handles throttling the upload streams. The algorithm will try
to decrease or increase the streams throttle rates as appropriate by changing
the value based on the over/under amount or based on a percentage to prevent
throttling one stream directly to zero.
Encrypting, hashing, and creating index volumes are now done
in the DataBlockProcessor and the SpillCollectorProcess. This allows
uploads to always be transferring data and not have to stop to create
a file, encrypt data, etc.
Kenneth Skovhede