Commit Graph
10149 Commits
Author SHA1 Message Date
Jojo-1000 6a85544fa5 Check blocklist consistency in recreate to prevent broken filesets.
Compact used broken LocalDatabase.GetBlocklists without a hash check, so some backups may contain invalid index files.
2023-12-12 21:28:15 +01:00
Jojo-1000 cf8f5e64cd Fix LocalDatabase.GetBlocklists for some cases.
Closes #3202

Also add test to ensure that repair works as intended.
2023-12-12 21:28:05 +01:00
gpatel-frandGitHub b08b59228c report all filesets in error in VerifyConsistency (#5052)
only reporting the first one could incite to deleting
it while many filesets can be impacted by a file error
2023-12-12 18:25:14 +01:00
gpatel-frandGitHub d20e692d20 enhance Python restore tool (#5051)
add command line parameters
parameterize memory cache and use bigger cache by default
optional use of aescrypt to make decryption faster by caching the results
tentative attributes support for Linux (use with sudo)
2023-12-12 00:35:01 +01:00
gpatel-frandGitHub 79b87566dd set default log level to Dryrun if dryrun option is set (#5059) 2023-12-11 14:42:45 +01:00
gpatel-frandGitHub 11d6364b4a remove agsXmpp, replace it with Sharp.Xmpp (#5046)
also add a missing space in default body (fixes a small bug under Windows)
2023-12-10 00:26:26 +01:00
JanandGitHub ccfa4e204e Fix restore search error in RestoreController (#4998)
* Fix restore search error in RestoreController.

TypeError: Cannot read Property 'Path' of undefined.
Add a break to for loop, thanks @tx0h (#4997).

* Remove empty '\' entries from restore picker.

The paths are split by dirsep, but directories end with it, which results in an empty part at the end.
2023-12-10 00:25:47 +01:00
gpatel-frandGitHub 0de4cb7968 allow full-remote-verification to target only list and index files (#5037)
The option is turned into an enum with values True, False, ListIndexes
True and False are for compatibility with the existing backups.
ListIndexes is equivalent to True but don't handle block volumes.
This is to make more workable a strategy of full remote testing
after each backup.
2023-12-10 00:24:07 +01:00
gpatel-frandGitHub af4fd5af05 change error reporting for full backup remote verification (#5004)
Before the change, it could happen that when the postbackup test
was finding problems in the backend, the job would be reported
as having completed successfully, the found problems being displayed
only in the complete log. After the change, an error is logged
that cause the job to be failed.
To repro, the following procedure can be used in a test backup:
change a hash for a block in the block table database and run a backup having set
--full-remote-verification and --full-block-verification to true and
--backup-test-percentage to 100.
2023-12-10 00:23:16 +01:00
gpatel-frandGitHub 3a9239ccff Merge pull request #4962 from Jojo-1000/fix-filter-ui
Fix filter UI
2023-12-10 00:22:31 +01:00
Jojo-1000 f7e435b1dd Improve file backend list speed.
Directly list FileInfo instead of lookups by filename. This greatly speeds up backup verify on exFAT.

Closes #5061
2023-11-19 20:04:37 +01:00
Jojo-1000 a45ce610df Fix URI parsing for IPv6 addresses and add tests.
Closes #5057
2023-11-10 13:53:10 +01:00
gpatel-frandGitHub ec9d49d684 Experiment/rebuildfaster (#5043)
* experimental faster db rebuild (#4955)

* try run update db query for every block file only when needed
add repair-force-block-use option
more appropriate naming for some routines
more tracing
more comments
2023-11-05 18:19:08 +01:00
gpatel-frandGitHub 43dea88b4f enable to perform tests without checking file list (#5054)
following the option --disable-filelist-consistency-checks
experimental
2023-11-04 14:25:59 +01:00
Jojo-1000 71cf250450 Fix missing index-block link in interrupted backup.
See issue #5023.
The index-block link should be created before the upload is started, otherwise an interrupted backup could cause an index file without a link.
2023-10-08 22:34:15 +02:00
Jojo-1000 2a2dd92ba4 Fix index block link for extra index files in repair.
The link was created before the volume was added to the database, resulting in ID -1.
2023-10-08 22:34:15 +02:00
Jojo-1000 33aab2aec2 Add test for repair with extra index files. 2023-10-08 22:34:09 +02:00
Jojo-1000 b4d0ed2cd9 Add categories and reproduction of issue #5038. 2023-10-08 22:25:44 +02:00
Jojo-1000 ea6c418b7b Add test case to reproduce part of #5023. 2023-10-08 22:25:42 +02:00
Jojo-1000 4c5543bcfe Add test for interrupted compact operation. 2023-10-08 22:24:39 +02:00
gpatel-frandGitHub 706c64a899 restore warning/error result handling changed in PR4978 (#5040) 2023-10-07 09:33:15 +02:00
JanandGitHub f6440a8e70 Update stale action to v8 (#5036) 2023-09-27 23:44:40 +02:00
gpatel-fr e9611fca53 update appveyor image to vs 2019 2023-09-26 10:16:10 +02:00
taz-ilandGitHub 10d24d8615 Specify the name of the backup when reporting failures (#4839)
Fixes duplicati/duplicati#4828
2023-09-25 14:35:31 +02:00
JanandGitHub 3a0667dc64 Add option to disable chunked encoding for AWS S3. (#4999)
Some S3 providers do not support MultiChunkedEncoding (#4994).
2023-09-25 02:02:55 +02:00
JanandGitHub a493b37a97 Fix hidden username and password options when changing backends. (#4973)
Remove username and password from target options if the backend does not show the input fields.

The username and password options are stored in different variables than the other backend options. Therefore, they are only visible if the backend supports them.

Closes #3082
2023-09-24 16:19:20 +02:00
gpatel-frandGitHub 4012e7f608 add console log to the Alternative FTP backend for debugging purposes (#5034) 2023-09-24 10:52:57 +02:00
taz-ilandgpatel-fr f791066b7a FluentFTP version update - Fix handling of missing base folder (#4946)
* Improve errors specificity by:
a. Avoiding unnecesary wrapping in AggregateException - Using Library.Utility.Await instead of Wait / Result
b. Using messages from InnerException during TestConnection

* Fix detection of missing base directory:
Depend on error code while trying to SetWorkingDirectory, rather than very wide try/catch and a hardcoded error string (Different FTP servers return different error strings. E.g. hardcoded here was "Directory not found." while Filezilla returns "Can't open file or directory").

* Fix indentation

Intentionally seperated from the previous commit for easier reviewing (otherwise Github text diff display is confused by the removal of the large try / catch blocks)

* Fix creation of missing base directory:
(previously CreateClient would fail since it would try to SetWorkingDirectory to the not-yet-existing directory)
2023-09-24 09:28:19 +02:00
taz-ilandgpatel-fr dbd8cdb86a Update FluentFTP to latest version (46.0.2) (#4935)
* Upgrade FluentFTP to v42 in an attempt to resolve TLS session issues:
duplicati/duplicati#4831

Note v40 introduced breaking changes, specifically seperating between sync and async clients.

Using a different approach this time, single async client.

* Bump FluentFTP to latest (42.0.0 --> 46.0.2)
2023-09-24 09:28:19 +02:00
gpatel-frandGitHub b19f6f3743 update restsharp to 106.10.1 (#5029)
to make Dependabot happy - have to update minio as well
2023-09-23 14:57:27 +02:00
gpatel-frandGitHub d1fa327ce6 Fixissue5028 (#5031)
* update thirdparty info to remove obsolete codeplex references

* fix bug in negated in expression (thanks @gorosgobe and Bloomberg)
2023-09-22 12:34:06 +02:00
gpatel-frandGitHub 63c14533e0 update thirdparty info to remove obsolete codeplex references (#5030) 2023-09-22 10:07:52 +02:00
JanandGitHub 91eace3ab2 Include exception type and message in job log. (#4976)
If an exception is provided for a log message, display the type and message (excluding stack trace) even if withExceptionDetails=false. This helps to diagnose bugs if the live log was not saved.

Closes #4843
2023-09-21 23:13:15 +02:00
Janandgpatel-fr b894a197a9 Fix missing file error caused by interrupted compact.
Before the dindex files are deleted on the remote, update the state in the database to Deleting. Previously, only dblock files were handled this way.
Patch by warwickmm
Closes #4129
2023-09-21 22:56:52 +02:00
gpatel-frandGitHub b219bf1271 add returncode 3 to help.txt (#5009) 2023-09-21 20:16:47 +02:00
Janandgpatel-fr 71eb03130a Improve result reporting for interrupted/failed backups (Fix #4829)
Direct import of PR#4978
Update web UI for new result reports.
For operations with fatal errors, write logs to same operation ID.
Test that Interrupted flag is correct in RunScriptTests.
Update backup log display for new result reporting.
Hide file statistics for fatal errors and change fatal icon.
2023-09-21 20:01:17 +02:00
gpatel-frandGitHub 44ce6d746f update serbian translation thanks to Zoran Tasić (@sagitarion) (#5021) 2023-08-31 11:25:04 +02:00
gpatel-frandGitHub 91f1fc6a60 update microsoft.rest.clientruntime to 2.3.24 (#5020) 2023-08-31 10:54:50 +02:00
gpatel-frandGitHub 3fd118ac44 remove obsolete tool TlsTest (#5019) 2023-08-31 01:10:44 +02:00
gpatel-frandGitHub f985af4a5a removes obsolete backend HubiC - fixes #4836 (#5018) 2023-08-30 13:45:55 +02:00
JanandGitHub 32d163ef93 Add manual link to footer and remove dead facebook link (#4983)
* Add manual link to footer and remove dead facebook link.

* Fix vertical padding for manual link.

Change to 8px like the other footer sections from 5px.
2023-08-29 08:20:09 +02:00
gpatel-frandGitHub 84d9918041 add stale issues management (#5013) 2023-08-28 17:11:33 +02:00
gpatel-frandGitHub 729a84fdc1 update mail libraries (#5014)
known side effect: the mail server user password will no longer be displayed in the verbose log
2023-08-28 11:50:37 +02:00
Jojo-1000 aa365f7f34 Update --quota-size option to assign a backup size.
The assigned quota size is checked in addition to the reported backend quota (if available). It can be used to limit the backup size on the remote. Warnings are created according to the same quota warning threshold.
Previously, this option had no function.
2023-07-16 19:49:54 +02:00
Jojo-1000 0754a5cd72 Ignore File backend quota if total size is zero.
Mono sometimes reports the total size of unknown file systems as zero. Because it should be obvious when the size is actually zero, ignore the quota in this case.
2023-07-16 19:24:37 +02:00
Jojo-1000 8af1dbb6fd Add --quota-disable option to disable reported backend quota.
If set, the backend is treated as if it does not report a quota at all. No warnings and errors are created and no quota values are set in the backup log.
This option is intended to be used if a backend reports a wrong quota, so it needs to be disabled.
2023-07-16 19:16:11 +02:00
Jojo-1000 cff1049347 Add unit tests for recovery tool restore paths.
Test that paths are generated correctly from a different source OS.
Also test that empty files are restored by recovery tool.
2023-07-11 22:48:28 +02:00
Jojo-1000 6b5623651c Fix recovery tool restore paths on different OS.
On Linux, paths were created with \ instead of / separators if backup was originally from Windows (#4884).
Also fix bug that created one unnecessary level of directories in the restore path.
2023-07-11 22:37:08 +02:00
Jojo-1000 0f5b4b9707 Fix restore of empty files with recovery tool.
Empty files do not have data blocks stored, so don't try to get a block for empty files.
2023-07-10 16:44:24 +02:00
Jojo-1000 b9d4be60ca Do not set advanced options with empty string to true.
There are advanced options such as force-local that expect an empty string in some cases. While saving, these were replaced with 'true' even though there was an equals sign.
Now only options without an equals sign are set to true.

Closes #4757
2023-06-23 16:38:54 +02:00