Commit Graph
3525 Commits
Author SHA1 Message Date
Kenneth Hsu 71edf641a5 Avoid null reference exception when building Sia error messages.
If the resource does not provide an error response, the Response
property will be null.

This fixes #4193.
2020-09-27 09:44:50 -07:00
Kenneth SkovhedeandGitHub df3fa30841 Merge pull request #4314 from warwickmm/add_storadera_s3_provider
Add support for Storadera S3 provider
2020-09-23 11:11:48 +02:00
Kenneth SkovhedeandGitHub a3ff9f6a0d Merge pull request #4307 from warwickmm/remove_misleading_comment
Remove misleading comment
2020-09-23 11:11:22 +02:00
Kenneth SkovhedeandGitHub a5cea90867 Merge pull request #4293 from warwickmm/fix_export_encoding
Avoid decoding URL parameters when removing passwords
2020-09-23 11:10:02 +02:00
Dean Ferreyra edb2819d6f Disable "legacy path handling" for executables
This is a defensive measure in case legacy path handling has been
enabled at the system level.  Duplicati requires that legacy path
handling be _disabled_ to function properly.

Issue #4295 appears to be a result of legacy path handling being
enabled globally.
2020-09-18 19:58:18 -07:00
Kenneth Hsu 7c59b3d531 Add Storadera S3 provider.
This fixes #4313.
2020-09-10 16:44:31 -07:00
Kenneth Hsu dd902ce7b2 Remove misleading comment.
The retention policy logic was changed, and this comment was leftover
from the old implementation.

https://forum.duplicati.com/t/new-retention-policy-deletes-old-backups-in-a-smart-way/2195/34?u=ts678

This fixes #3306.
2020-09-03 19:48:47 -07:00
Dean Ferreyra 5f972c2437 Make FilterEntry of type Simple for @ again
Switch back to using `FilterType.Simple` for `@` filters to fix the
restore picker again.

In LocalDatabase.cs, replace code that was explicitly looking for the
presence of wildcard characters to instead trigger off of
`FilterType.Wildcard`.  This fixes the "find" command.

Change FilterEntry.ToString() to prefix Simple filters with `@` so
that if simple filters with wildcard characters make the round-trip
through a string, their `FilterType.Simple` type is preserved.
2020-09-01 23:04:03 -07:00
Dean Ferreyra fd76fb32ef Fix FilterEntry for @ entries to be of type Regexp 2020-08-31 21:25:18 -07:00
Dean Ferreyra aa50b1dfbf Fix typo in comment 2020-08-30 20:04:07 -07:00
Dean Ferreyra 5f94cc19a7 Fix restore GUI handling of folders with * or ?
Add new filter syntax, prefixing with an "at sign"; i.e., `@...`, that
disables glob expansion, treating `*` and `?` wildcard characters as
literal characters.

In the GUI, change the restore file picker to prefix paths with `@`
when expanding a node to avoid mistaking literal `*` or `?` characters
for wildcards characters and triggering the following error:
```
Filter for list-folder-contents must be a path prefix with no wildcards Parameter name: filter
```

This fixes #4300.
2020-08-30 17:07:04 -07:00
Honoré Vasconcelos 71494826f9 Improve query performence used in GetFilesAndSourceBlocksFast
Tree changes in query
* Move inner selection S to a temporary table
* Reorder join field in outer query
* Remove intersection with File view in outer selection
File view is already intersected with in selection S

With the above-mentioned changes, on a specific testset, query took 9s instead of 6min.
2020-08-26 10:10:54 +01:00
Kenneth Hsu f28550f4b5 Use overload instead of optional parameters.
From Codacy:

The overloading mechanism should be used in place of optional parameters for several reasons:

* Optional parameter values are baked into the method call site code,
  thus, if a default value has been changed, all referencing assemblies
  need to be rebuilt, otherwise the original values will be used.
* The Common Language Specification (CLS) allows compilers to ignore
  default parameter values, and thus require the caller to explicitly
  specify the values. For example, if you want to consume a method with
  default argument from another .NET compatible language (for instance
  C++/CLI), you will have to provide all arguments. When using method
  overloads, you could achieve similar behavior as default arguments.
* Optional parameters prevent muddying the definition of the function
  contract. Here is a simple example: if there are two optional
  parameters, when one is defined, is the second one still optional or
  mandatory?
2020-08-24 07:51:57 -07:00
Kenneth Hsu d4d7874aaf Avoid decoding URL parameters when removing passwords.
In revision c41f2c6a60 ("moved all sanitization logic into backup class
and moved from regex to using internal Uri library"), we cleaned up the
removal of passwords from an exported backup configuration.  However,
the use of Uri.QueryParameters resulted in exporting decoded parameter
values, which violated some assumptions made by the decode_uri function
in AppUtils.js.  This caused usernames in the JSON to contain '@'
instead of '%40', which led to incorrect decomposition of the target URL
into its components in the UI.

This concerns issue #3619.
2020-08-24 07:41:00 -07:00
warwickmmandGitHub bcaedebecd Merge pull request #4287 from dgehri/feature/improve_usn_2
Improve USN performance.
2020-08-22 09:28:05 -07:00
warwickmmandGitHub 83318877a1 Merge pull request #4285 from dferreyra/source_path_with_unc_prefix_3
Handle Windows source paths starting with "\\?\"
2020-08-21 07:38:12 -07:00
Daniel Gehriger 31bb24d605 USN: moving exclusion filter earlier in the enumeration process to improve performance 2020-08-18 16:07:01 +02:00
Daniel Gehriger bb1853085d USN: only considering events relevant to backup 2020-08-18 12:21:59 +02:00
Daniel Gehriger b6752bead3 USN: excluding non-relevant events 2020-08-18 11:43:52 +02:00
Daniel Gehriger d143f8b3f9 Improved speed of AppendFilesFromPreviousSetWithPredicate, making USN backups usable when there are large file sets 2020-08-18 11:43:23 +02:00
Daniel Gehriger 420c23cf05 USN: corrected exclusion criteria 2020-08-17 23:03:51 +02:00
Daniel Gehriger 82c54dde31 Fixing AppendFilesFromPreviousSetWithPredicate 2020-08-17 23:01:20 +02:00
Daniel Gehriger c5bef5c2e0 Improved Improved speed of AppendFilesFromPreviousSetWithPredicate, making USN backups usable when there are large file sets 2020-08-17 18:06:38 +02:00
Daniel Gehriger 8e8c758c65 Improved speed of AppendFilesFromPreviousSetWithPredicate, making USN backups usable when there are large file sets 2020-08-17 17:27:45 +02:00
Daniel Gehriger 7d2c6befb0 Using USN journal in CountFilesHandler 2020-08-17 15:08:15 +02:00
Daniel Gehriger 245046e8e5 USN: adding verbose message 2020-08-17 14:41:41 +02:00
Daniel Gehriger 0caf724982 USN journal: removing create & delete sequences between two backups from list of files 2020-08-17 14:41:23 +02:00
Dean Ferreyra b8eecba62c Handle Windows source paths starting with "\\?\"
Under Windows, strip paths of any `\\?\` prefix to prevent Duplicati
from storing paths prefixed with `\\?\`.

This fixes an inconsistency in the stored data where, if the user
provided a prefixed path to back up (e.g., `\\?\C:\Temp`), Duplicati
would store an entry with the prefixed path (e.g., `\\?\C:\Temp`)
along with entries with paths stripped of their prefix (e.g.,
`C:\Temp\file.txt`).  By only storing non-prefixed paths, this also
avoids an error reported by the GUI when it would encounter a prefixed
path:
```
Filter for list-folder-contents must be a path prefix with no wildcards Parameter name: filter
```
2020-08-15 20:09:54 -07:00
Dean Ferreyra ecd62e8cea Fix snapshots broken by #4256
Add comment in `WindowsSnapshot.ConvertToSnapshotPath()` warning how
replacing some manual path construction code with Path.Combine() alone
can lead to invalid snapshot paths.
2020-08-08 15:32:54 -07:00
Dean Ferreyra 1198c10d43 Fix snapshots broken by #4256
Revert the change made to `WindowsSnapshot.ConvertToSnapshotPath()`
by pull request #4256.

The change in #4256 replaced some existing manual path construction
code with a call to `SystemIO.IO_WIN.PathCombine()`, but calling
`SystemIO.IO_WIN.PathCombine()` this way did not always result in
valid VSS paths.  For example, trying to back up a root volume like
"C:\" would result in the following error:
```
The filename, directory name, or volume label syntax is incorrect.
```
2020-08-07 21:51:11 -07:00
Kenneth SkovhedeandGitHub 648d199427 Merge pull request #4277 from dgehri/feature/USN-deleted-files-fix
Enabled support for \$Extend\$Deleted special MTF folder when using USN
2020-08-07 12:07:44 +02:00
Kenneth SkovhedeandGitHub ec1975b5a7 Merge pull request #4269 from warwickmm/target_net_471
Target .NET Framework 4.7.1
2020-08-07 10:50:35 +02:00
Kenneth SkovhedeandGitHub 04dd510be0 Merge pull request #4261 from warwickmm/avoid_deleting_unrelated_files
Avoid deleting unrelated files when removing backup configuration
2020-08-07 10:33:31 +02:00
Daniel Gehriger fcf44d595d Restoring original scheduling of CountFilesHandler 2020-08-06 18:14:07 +02:00
Daniel Gehriger 279d8d2d2c Fixed usn reason comparison (was too severe) 2020-08-06 17:58:19 +02:00
Daniel Gehriger 51ce673831 Removed debug assert 2020-08-06 16:56:52 +02:00
Daniel Gehriger b25d4b47ca Enabled support for \$Extend\$Deleted special MTF folder when using USN 2020-08-06 16:37:48 +02:00
Dean Ferreyra 2e777b2711 Change tests to work with long paths in Windows
Change SystemIOWindows.PathGetFullPath() to convert forward
slashes to backslashes.

Change Duplicati.UnitTest.BasicSetupHelper.ZipFileExtractToDirectory()
to rely on slash fix-ups.
2020-07-30 05:08:18 -07:00
Dean Ferreyra 73e31a79a1 Change tests to work with long paths in Windows
Even in .NET 4.6.2
`System.IO.Compression.ZipFile.ExtractToDirectory()` cannot handle
long paths.  Replace call to
`System.IO.Compression.ZipFile.ExtractToDirectory()` with an
equivalent that extracts files to a temporary location and uses I/O
functions that support long paths to move them to their final
location.

In CommandLineOperationsTests.cs, use ISystemIO functions to handle
potentially long paths.

Add a fix to RecoveryTool for long paths that was missed by #4258.

This fixes #3863.
2020-07-28 05:51:53 -07:00
Kenneth Hsu e964072690 Target .NET Framework 4.7.1.
This updates all projects to target .NET Framework 4.7.1.  The
TencentCOS and Tardigrade backends depend on .NET Standard 2.0.  When a
.NET Framework prior to 4.7.1 is targeted, the system cannot be sure
that all the dependencies exist, so it copies all dependent assemblies
to the output directory.  This causes many assemblies from the System
namespace to become bundled in the release.

https://stackoverflow.com/a/48875007

We had previously attempted to make individual projects target 4.7.1
(see pull request #4242), but this can cause compatibility issues when
4.6.2 projects depend on 4.7.1. projects.

This will require Mono 5.10.0 or greater (previously, we required 5.0.0
or greater).

https://www.mono-project.com/docs/about-mono/releases/5.10.0/#class-libraries

This fixes issue #4234.
2020-07-26 19:46:01 -07:00
warwickmmandGitHub e7daffcb31 Merge pull request #4256 from dferreyra/support_problematic_windows_paths
Support problematic Windows paths that end with spaces or periods.
2020-07-25 17:48:50 -07:00
TopperDEL fde7e84afc fix: Resovle codacy-issue 2020-07-22 14:31:03 +02:00
TopperDEL c451724520 build: Switch to uplink.NET v2.3.4 fixing one download-issue 2020-07-22 12:40:37 +02:00
TopperDEL 6a8a029c70 cleanup: Reduce one line 2020-07-22 12:40:21 +02:00
Kenneth Hsu fea55b9981 Avoid removing files unrelated to backup being deleted.
This fixes issues #3845 and #4244.
2020-07-21 12:59:28 -07:00
Dean Ferreyra d154702020 Support problematic Windows paths
Fix pull request string comparisons in SystemIOWindows.cs to use
StringComparison.Ordinal.

Fix pull request to use SystemIO.IO_WIN.PathCombine() in
WindowsSnapshot.cs.
2020-07-19 16:24:51 -07:00
TopperDEL 17c639cb26 fix: Build error 2020-07-19 11:14:33 +02:00
TopperDEL 6d5e8bfc77 feat: Add new Tardigrade-Satellite-Adresses 2020-07-19 11:00:10 +02:00
TopperDEL badcb7dbfc fix: Correc failing build - referencing correct uplink.NET-nuget 2020-07-19 10:58:36 +02:00
Dean Ferreyra 6acbbf2136 Support problematic Windows paths
Allow historically problematic Windows paths (i.e., files or
directories that end in a dot or a space) to be backed up with
Duplicati in Windows.

This fixes #3839.
2020-07-17 21:50:29 -07:00