Commit Graph
8782 Commits
Author SHA1 Message Date
warwickmmandGitHub ba48f14746 Merge pull request #4311 from dferreyra/symlink_unit_tests
Add symlink policy tests and file and directory comparison methods.
2020-09-13 10:09:23 -07:00
Dean Ferreyra a3c27a2ba2 Add Assert.Fail to catch unexpected symlink policies 2020-09-13 08:12:23 -07:00
Dean Ferreyra a060b85039 Add travis_wait to "Border" tests to avoid timeout 2020-09-13 07:45:30 -07:00
Dean Ferreyra 641a590d89 For "Ignore" policy, also ensure a file doesn't exist 2020-09-12 20:33:39 -07:00
Dean Ferreyra e3b8a12ec1 Unit test improvements per review comments 2020-09-12 17:03:48 -07:00
Dean Ferreyra eafe7320e9 Address review comments 2020-09-12 14:45:55 -07:00
Dean Ferreyra 3366071d04 Add back missing stream reset to AssertFilesAreEqual 2020-09-12 14:29:04 -07:00
Dean Ferreyra 0367418925 Fix typo in log and assert messages 2020-09-12 10:52:01 -07:00
Dean Ferreyra 75b688c999 Restore use of Utility.CompareStreams in AssertFilesAreEqual
For performance reasons, use `Utility.CompareStreams` in
AssertFilesAreEqual.
2020-09-12 08:08:52 -07:00
Dean Ferreyra 2c56634899 Add tolerance to file time comparisons 2020-09-11 23:25:03 -07:00
Dean Ferreyra f4195e4eb0 Address review comments from @warwickmm
Replace `VerifyDir` and `CompareFiles` with
`AssertDirectoryTreesAreEquivalent` and `AssertFilesAreEqual`.

Move `WriteFile` to `TestUtils`.

Add "Symlink" category to start.sh.
2020-09-11 20:06:53 -07:00
Kenneth Hsu 7c59b3d531 Add Storadera S3 provider.
This fixes #4313.
2020-09-10 16:44:31 -07:00
Dean Ferreyra f56f64e541 Merge branch 'master' into symlink_unit_tests 2020-09-09 20:03:32 -07:00
warwickmmandGitHub fcc44eacaf Merge pull request #4279 from warwickmm/add_local_test_categories
Add missing test categories to local test runner script
2020-09-09 18:05:29 -07:00
Dean Ferreyra b3e01e500c Add SymLink test category to .travis.yml 2020-09-08 07:38:40 -07:00
Dean Ferreyra be6ac941ea Add symlink policy tests
This adds a basic symlink policy test that performs backups using each
symlink policy.

If you don't have the required privilege in Windows, you can get an
exception trying to create a symbolic link.  If an exception is thrown
trying to create a symbolic link, the exception is trapped and the
test is marked as "Ignored"; e.g.,
```
1) Ignored : Duplicati.UnitTest.SymLinkTests.SymLinkPolicy
Client could not create a symbolic link.  Error reported: (1314) A required privilege is not held by the client. | Read: [C:\td\backup-data\target] | Write: [C:\td\backup-data\symlink]
```
2020-09-07 22:50:21 -07:00
warwickmmandGitHub 174edf422e Merge pull request #4302 from dferreyra/literal_wildcard_characters
Fix restore GUI handling of folders with wildcard characters.

This does so by introducing a new syntax for verbatim path specifications.
Similar to C#, expressions that are prefixed by '@' are treated as verbatim 
string literals.
2020-09-07 19:31:41 -07:00
Dean Ferreyra 93601d0a82 Address review comments from @warwickmm 2020-09-07 18:56:59 -07:00
Dean Ferreyra 78555241c1 Address review comments from @warwickmm 2020-09-07 14:45:08 -07:00
Dean Ferreyra 536ce69668 Address review comments from @warwickmm 2020-09-06 21:30:46 -07:00
Dean Ferreyra 27da5a4dc7 Include Windows in tests for directory names with wildcards 2020-09-06 15:00:48 -07:00
Kenneth Hsu 7d13d9ed81 Add tests for directory names with wildcards. 2020-09-05 15:05:11 -07:00
Dean Ferreyra 5436d16c20 Fix restore GUI for filters with wildcard characters
In RestoreController.js, remove the branch on whether the path
contains wildcard characters and instead use code that works for all
cases.
2020-09-04 16:54:49 -07:00
Dean Ferreyra 5c2fb96f5f Support globbing for "@" filters in "find" and "restore"
Change some comments.
2020-09-04 09:54:33 -07:00
Dean Ferreyra 3a66668e6a Support globbing for "@" filters in "find" and "restore"
Remove unneeded calls to `ToList()`.
2020-09-04 07:25:31 -07:00
Dean Ferreyra f6e79da78d Support globbing for "@" filters in "find" and "restore"
For `@` filters, support normal behavior of "find" and "restore" where
they automatically prefix or suffix some file name arguments with `*`.
2020-09-04 02:01:37 -07:00
Dean Ferreyra 071f7ed7b0 Fix restore GUI for filters with wildcard characters
Fix bug in line to detect literal wildcard characters.
2020-09-03 23:55:27 -07:00
Dean Ferreyra ff91c47f68 Fix restore GUI for filters with wildcard characters
Change RestoreController.js to handle the case where the selected
restore paths contain literal wildcard characters.  For files, prefix
with `@`.  For directories, convert to regular expression filters so
that we can preserve the literal wildcard characters and also support
the equivalent of a globbing `*` suffix.
2020-09-03 22:59:24 -07:00
Kenneth Hsu dd902ce7b2 Remove misleading comment.
The retention policy logic was changed, and this comment was leftover
from the old implementation.

https://forum.duplicati.com/t/new-retention-policy-deletes-old-backups-in-a-smart-way/2195/34?u=ts678

This fixes #3306.
2020-09-03 19:48:47 -07:00
Dean Ferreyra b8501dfd77 Add additional handling for @ filters
Change "find" and "restore" commands to not prefix `@` file filters
with `*` and to not suffix `@` folder filters with `*` since wildcards
cannot be used with `@` filters.
2020-09-01 23:56:00 -07:00
Dean Ferreyra 5f972c2437 Make FilterEntry of type Simple for @ again
Switch back to using `FilterType.Simple` for `@` filters to fix the
restore picker again.

In LocalDatabase.cs, replace code that was explicitly looking for the
presence of wildcard characters to instead trigger off of
`FilterType.Wildcard`.  This fixes the "find" command.

Change FilterEntry.ToString() to prefix Simple filters with `@` so
that if simple filters with wildcard characters make the round-trip
through a string, their `FilterType.Simple` type is preserved.
2020-09-01 23:04:03 -07:00
Dean Ferreyra fd76fb32ef Fix FilterEntry for @ entries to be of type Regexp 2020-08-31 21:25:18 -07:00
Kenneth Hsu 4008736494 Fix typo that resulted in hidden SSH fingerprint error.
This fixes #4301.
2020-08-31 19:52:17 -07:00
Dean Ferreyra aa50b1dfbf Fix typo in comment 2020-08-30 20:04:07 -07:00
Dean Ferreyra 5f94cc19a7 Fix restore GUI handling of folders with * or ?
Add new filter syntax, prefixing with an "at sign"; i.e., `@...`, that
disables glob expansion, treating `*` and `?` wildcard characters as
literal characters.

In the GUI, change the restore file picker to prefix paths with `@`
when expanding a node to avoid mistaking literal `*` or `?` characters
for wildcards characters and triggering the following error:
```
Filter for list-folder-contents must be a path prefix with no wildcards Parameter name: filter
```

This fixes #4300.
2020-08-30 17:07:04 -07:00
Kenneth Hsu aa2e92ea3a Fix appearance of loading icon in file restore tree.
This fixes #4294.
2020-08-29 10:07:59 -07:00
warwickmmandGitHub 40597aaec2 Merge pull request #4297 from pcmind/fixGetFilesAndSourceBlocksFast
Improve query performance used in GetFilesAndSourceBlocksFast
2020-08-27 09:17:26 -07:00
Honoré Vasconcelos 71494826f9 Improve query performence used in GetFilesAndSourceBlocksFast
Tree changes in query
* Move inner selection S to a temporary table
* Reorder join field in outer query
* Remove intersection with File view in outer selection
File view is already intersected with in selection S

With the above-mentioned changes, on a specific testset, query took 9s instead of 6min.
2020-08-26 10:10:54 +01:00
Kenneth Hsu f28550f4b5 Use overload instead of optional parameters.
From Codacy:

The overloading mechanism should be used in place of optional parameters for several reasons:

* Optional parameter values are baked into the method call site code,
  thus, if a default value has been changed, all referencing assemblies
  need to be rebuilt, otherwise the original values will be used.
* The Common Language Specification (CLS) allows compilers to ignore
  default parameter values, and thus require the caller to explicitly
  specify the values. For example, if you want to consume a method with
  default argument from another .NET compatible language (for instance
  C++/CLI), you will have to provide all arguments. When using method
  overloads, you could achieve similar behavior as default arguments.
* Optional parameters prevent muddying the definition of the function
  contract. Here is a simple example: if there are two optional
  parameters, when one is defined, is the second one still optional or
  mandatory?
2020-08-24 07:51:57 -07:00
Kenneth Hsu c15cdcf26a Remove unused local variable. 2020-08-24 07:49:14 -07:00
Kenneth Hsu 5915fdfba9 Add test for exporting backup configuration to JSON.
This tests that we escape '@' with '%40', as the import code in
escape_uri in AppUtils.js cannot handle '@'.

This concerns issue #3619.
2020-08-24 07:41:00 -07:00
Kenneth Hsu e59e9a42b8 Extract removal of passwords to method.
This makes it easier to create a simple test for the behavior.
2020-08-24 07:41:00 -07:00
Kenneth Hsu 9dd98457f1 Clean up code. 2020-08-24 07:41:00 -07:00
Kenneth Hsu acba6351dc Extract local function to method. 2020-08-24 07:41:00 -07:00
Kenneth Hsu d4d7874aaf Avoid decoding URL parameters when removing passwords.
In revision c41f2c6a60 ("moved all sanitization logic into backup class
and moved from regex to using internal Uri library"), we cleaned up the
removal of passwords from an exported backup configuration.  However,
the use of Uri.QueryParameters resulted in exporting decoded parameter
values, which violated some assumptions made by the decode_uri function
in AppUtils.js.  This caused usernames in the JSON to contain '@'
instead of '%40', which led to incorrect decomposition of the target URL
into its components in the UI.

This concerns issue #3619.
2020-08-24 07:41:00 -07:00
Kenneth Hsu 8ae5b81e16 Remove script used for previous Travis configuration.
As of pull request #4005, our Travis builds no longer use this script.
2020-08-22 16:33:35 -07:00
Kenneth Hsu 201bb53b29 Add mono-complete to Docker image.
After targeting .NET 4.7.1, some users started encountering errors like:

Could not find method '.ctor' due to a type load error: Could not load
file or assembly 'netstandard, Version=2.0.0.0, Culture=neutral,
PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies.
assembly:/opt/duplicati/SharpAESCrypt.exe type:SharpAESCrypt
member:(null)

The TencentCOS and Tardigrade backends are built on .NET Standard 2.0,
and it appears that the dependencies can be awoken even if these
backends are not used.

This fixes #4289.
2020-08-22 15:24:25 -07:00
warwickmmandGitHub bcaedebecd Merge pull request #4287 from dgehri/feature/improve_usn_2
Improve USN performance.
2020-08-22 09:28:05 -07:00
warwickmmandGitHub 83318877a1 Merge pull request #4285 from dferreyra/source_path_with_unc_prefix_3
Handle Windows source paths starting with "\\?\"
2020-08-21 07:38:12 -07:00
Daniel Gehriger 31bb24d605 USN: moving exclusion filter earlier in the enumeration process to improve performance 2020-08-18 16:07:01 +02:00