Commit Graph
72 Commits
Author SHA1 Message Date
Kenneth Hsu e964072690 Target .NET Framework 4.7.1.
This updates all projects to target .NET Framework 4.7.1.  The
TencentCOS and Tardigrade backends depend on .NET Standard 2.0.  When a
.NET Framework prior to 4.7.1 is targeted, the system cannot be sure
that all the dependencies exist, so it copies all dependent assemblies
to the output directory.  This causes many assemblies from the System
namespace to become bundled in the release.

https://stackoverflow.com/a/48875007

We had previously attempted to make individual projects target 4.7.1
(see pull request #4242), but this can cause compatibility issues when
4.6.2 projects depend on 4.7.1. projects.

This will require Mono 5.10.0 or greater (previously, we required 5.0.0
or greater).

https://www.mono-project.com/docs/about-mono/releases/5.10.0/#class-libraries

This fixes issue #4234.
2020-07-26 19:46:01 -07:00
Kenneth Skovhede 2f1f5bb7e0 The purpose of this update is to provide better error messages from servers that return error messages, but are expected to return JSON data.
In the current setup, the response stream is parsed by the JSON decoder, which throws an exception due to reading a non-json formatted error message.
The reported exception simply states that the first character on the first line was not valid json, which is true, but does not help diagnose the problem.

In many cases this is a human readable message, such as "over quota", "permission denied", etc.
This update adds a stream that keeps the first 2kb of the response in an internal buffer, which can then be printed in case the JSON parsing fails.
Most of the added code is just simple redirects for calling the base stream in the peeking stream.
2020-05-10 14:14:35 +02:00
Kenneth SkovhedeandGitHub 6a8912c473 Merge pull request #4145 from tygill/feature/ms-graph-non-httpclient
Allow MS Graph backends (e.g., OneDrive v2) to use OAuthHelper instead of OAuthHttpClient
2020-03-25 07:09:44 +01:00
Tyler Gill 1ae534e45e Add ConfigureAwait(false) to JSONWebHelper.cs 2020-03-23 20:22:22 -06:00
Tyler Gill e742205f65 Add --use-http-client=false parameter to backends based on Microsoft Graph APIs (OneDrive v2, SharePoint v2, etc.) to make them use the OAuthHelper and HttpWebRequest infra used by most / all other backends for HTTP requests. This lives in parallel to the original implementation that uses the OAuthHttpClient class, but is intended to see if this prevents the memory leaks and problems seen in some versions of Mono when using the OneDrive v2 backend. 2020-03-23 14:10:40 -06:00
Kenneth Hsu 21d0247e6d Throw TimeoutException when an HTTP timeout occurs.
The HttpClient.GetAsync method throws an OperationCanceledException when
the timeout is exceeded.  In order to provide a more informative
exception, we will detect this case and throw a TimeoutException
instead.

This is related to issue #3772.
2020-03-05 12:00:23 -08:00
Kenneth Hsu 83ac03843d Fix spelling errors in comments.
In doing so, we also normalized some line endings.
2019-12-14 09:52:55 -08:00
Tyler Gill 7012c49b8f Support BufferRequests in HttpClient based backends 2019-10-22 15:55:11 -06:00
Kenneth SkovhedeandGitHub 99d0cb3c57 Merge pull request #3953 from warwickmm/make_fields_readonly
Mark fields that don't need to be reassigned as readonly
2019-10-21 08:36:17 +02:00
Kenneth Hsu 2aa76881a5 Mark fields that don't need to be reassigned as readonly.
This makes it explicit at compile-time that these fields should not be
reassigned outside the constructor.
2019-10-19 13:59:22 -07:00
Kenneth Hsu afab61a05b Use pattern matching to simplify casts. 2019-10-19 10:15:38 -07:00
Kenneth Hsu 31a89f142a Add workaround for bug in Mono HttpClient.
There is a regression in Mono 6.0 where an HttpClient with a BaseAddress
cannot make requests to URLs that begin with '/'.

This addresses issue #3852.
2019-09-08 19:34:12 -07:00
BlueBlock 815e30b63b update csproj toolverion and nuget packages 2019-07-31 13:20:03 -04:00
BlueBlock 08a1e9998a initial upgrade to framework 4.6.2
- no code changes except those noted below
- projects upgrade to 4.6.2
- wixinstaller project upgraded automatically by VisualStudio
- wixinstaller updated to require 4.6.2
- Library.Encryption changed to Standard2.0 so accommodate update to SharpAesCrypt
2019-07-26 09:18:16 -04:00
Kenneth Hsu 51e0298597 Remove unused variable that caused compiler warning. 2019-07-07 10:15:13 -07:00
Kenneth Hsu a30f578071 Rename method parameter. 2019-05-26 15:14:42 -07:00
Kenneth Hsu 6b04625cea Add XML comment for method parameter. 2019-05-26 15:09:24 -07:00
Kenneth Hsu fe5e17a3fa Throw TimeoutException upon HTTP timeout.
The HttpClient.SendAsync method throws an OperationCanceledException
when the timeout is exceeded.  In order to provide a more informative
exception, we will detect this case and throw a TimeoutException
instead.  This will also allow the BackendUploader to differentiate
between cancellations requested by the user and those generated by
an HTTP timeout.

Previously, a ThreadAbortException or OperationCanceledException would
be rethrown in the BackendUploader.DoWithRetry method.  However, the
modifications made in revision 61f511c087
from pull request #3684 appear to have let this case slip through.

See the following links for more information:

https://stackoverflow.com/questions/10547895/how-can-i-tell-when-httpclient-has-timed-out
https://thomaslevesque.com/2018/02/25/better-timeout-handling-with-httpclient

This addresses issue #3772.
2019-05-26 14:57:05 -07:00
Kenneth Hsu 988c445233 Provide cancellation token to async methods. 2019-05-26 14:55:31 -07:00
Kenneth Hsu ffaaa55f2f Use AsyncHttpRequest to handle async HttpWebRequests.
For some reason, when using HttpWebRequest.GetRequestStreamAsync, the
Amazon Cloud Drive and Box backends would fail when attempting to upload
larger amounts of data (these are the only backends that use the
PostMultipartAsync method).

This fixes issue #3756.
2019-05-04 16:38:36 -07:00
Sean Templeton 8bd4b825c8 Fix several Codacy issues 2019-03-13 21:51:05 -05:00
Sean Templeton c3472823de Update HubiC and OpenStack backends for async Put 2019-03-05 21:08:26 -06:00
Sean Templeton 5db326bcdb Update Azure Blob and Backblaze B2 backends for async Put 2019-03-05 21:08:24 -06:00
Sean Templeton b1c6e9798d Update Amazon Cloud Drive backend for async Put 2019-03-05 21:08:14 -06:00
Kenneth Hsu 9e0b031fee Remove unused members. 2018-10-15 21:20:30 -07:00
Kenneth Hsu 18aae7a5c4 Simplify method resolution by removing default parameters.
The previous implementations resulted in an overload being hidden by one
without a default parameter, making it unclear which method was being
called.  For example, consider the following signatures:

public MultipartItem(string content, string contenttype = null, string name = null, string filename = null)
public MultipartItem(string contenttype, string name = null, string filename = null)

From https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/named-and-optional-arguments,

> If two candidates are judged to be equally good, preference goes to a
> candidate that does not have optional parameters for which arguments
> were omitted in the call. This is a consequence of a general preference
> in overload resolution for candidates that have fewer parameters.
2018-09-20 19:30:49 -07:00
Kenneth Hsu 54c4f9224d Allow tasks to complete on any thread when context is irrelevant.
When the code following the await can be executed on any thread, it's
recommended to use ConfigureAwait(false) to avoid unnecessary context
switching and potential deadlocks.
2018-06-21 19:28:27 -07:00
verhoek 3fb7175931 Merge remote-tracking branch 'origin/master' into feature/tempdir
# Conflicts:
#	Duplicati/Server/Runner.cs (copied readonly prop)
2018-06-07 16:37:40 +02:00
Tyler Gill 5be38df560 Fix typo in OAuthHttpClient 2018-06-04 14:59:58 -06:00
verhoek e80f30f8a9 Added multiple context setting keys to callcontextsettings, exploiting
reflection. Contextsettings don't destroy each other anymore.
2018-05-26 21:56:15 +02:00
verhoek 55e349f410 Renamed variable to indicate meaning properly. 2018-05-09 18:57:04 +02:00
Tyler Gill f93f9dcc5a Add new utility class, OAuthHttpClient.
This class extends HttpClient and uses an OAuthHttpMessageHandler under the covers to automatically authenticate requests.
Additionally, it automatically respects the global HttpContextSettings for overall timeout, read/write timeout, and SSL certificate validation.
(BufferRequests isn't currently handled, as HttpClient doesn't seem to easily expose that flag, and as of .NET 4.5, seems to not buffer by default.)
2018-03-23 15:07:51 -06:00
Tyler Gill 97a93a071f Make m_oauth readonly in OAuthHttpMessageHandler. 2018-03-21 11:23:50 -06:00
Tyler Gill 4578a5bd91 Merge branch 'master' of https://github.com/duplicati/duplicati into feature/ms-graph-backends 2018-03-20 14:24:48 -06:00
Tyler Gill 20fc793dee Add OAuthHttpMessageHandler helper class to OAuthHelper library.
This class implements the abstract HttpMessageHandler class (mostly via the default HttpClientHandler implementation).
This abstract class is used by System.Net.Http.HttpClient as the underlying HTTP mechanism, and in this particular case is used to automatically add an authorization header to each request.
It also provides a method for marking a request as one that should not be authenticated.
2018-03-20 14:00:52 -06:00
Tyler Gill f9a8b30b2f Update JSONWebHelper.PostAndGetJSONData to remove parameters that aren't actually being used (setup and setupreq).
These parameters weren't being used in the method body, or being passed in by any callers (it looks like a copy / paste error when a previous method was converted into this one)
These are replaced with a new optional parameter (method) which allows the caller to request a custom HTTP method to be used.
2018-03-20 13:56:05 -06:00
Kenneth Skovhede 842fd96543 Implemented a new logging system that is more transparent and allows a more granular way of picking log messages.
Added ID's to each log message and each exception to allow later introduction of a Knowledgebase service that explains each error in more detail.
2018-03-15 09:12:34 +01:00
Kenneth Hsu 5fb1958ceb Remove redundant calls to ToString. 2017-11-26 15:44:34 -08:00
Kenneth Hsu 3d51e9153b Avoid signing assemblies.
Using strong-named assemblies can cause difficulties with the GNU LGPL
license, which allows for one to recombine or relink their application
with modified versions of the code.  While one solution is to share the
private key so that people can sign the assemblies themselves, this
would break the trust that is expected from signed assemblies.  For now,
the easiest fix is to simply not sign the assemblies.  Note that by
doing so, we prevent the code from being referenced from other signed
assemblies.

This also fixes an issue introduced in revision ba94d36a80 ("Added
auto-update for WindowsService and Service."), where the WindowsService
project (signed) referenced the AutoUpdater project (not signed).

We also removed instances of <SignAssembly>false</SignAssembly> to be
consistent with newly created .csproj files that do not contain the
SignAssembly element.

This was motivated by the discussion in issue #2814.
2017-10-15 22:00:23 -07:00
Kenneth Skovhede a06f38442a Updated all Newtonsoft.Json packages 2017-09-23 00:25:50 +02:00
Kenneth Skovhede 19a040970f Implemented call-context aware settings for HTTP options, temp folder settings, buffers sizes and oauth settings.
This makes it more smooth to run multiple operations in parellel, but it should be possible to launch an operation in a separate app-domain
2017-09-04 11:05:16 +02:00
Kenneth Skovhede 046b522a8d Updated NewtonSoft.Json 2017-05-17 11:08:51 +02:00
Kenneth Skovhede a7b46b622c Spelling fix in documentation 2017-03-03 20:50:11 +01:00
Kenneth Skovhede a029890409 Implemented better default error messsages 2017-01-09 11:35:38 +01:00
Kenneth Skovhede 52eb8e8478 Disabled MSBuild for much faster builds on MacOS 2016-12-29 23:12:41 +01:00
Kenneth Skovhede 0c68daff6c Fixed all whitespace to be 4 spaces instead of tabs 2016-09-15 11:39:27 +02:00
Kenneth Skovhede 5a03b573cd Extended OAuthHelper to work better with the Dropbox OAuth implementation 2016-09-08 15:03:16 +02:00
Kenneth Skovhede bde606ae60 Updated packages 2016-06-27 15:30:25 +02:00
Kenneth Skovhede 3005594b38 Merge branch 'master' into nuget_setup
# Conflicts:
#	Duplicati/GUI/Duplicati.GUI.TrayIcon/Duplicati.GUI.TrayIcon.csproj
#	Duplicati/Library/Backend/S3/Duplicati.Library.Backend.S3.csproj
#	Duplicati/Library/Backend/S3/S3Wrapper.cs
#	thirdparty/CoCoL/CoCoL.dll
#	thirdparty/SSH.NET/Renci.SshNet.dll
2016-06-27 15:25:52 +02:00
Kenneth Skovhede 5f7e63ae78 Fixed the timeout issue 2016-04-16 17:51:44 +02:00