Commit Graph
3818 Commits
Author SHA1 Message Date
Kenneth SkovhedeandGitHub ffacd37e44 Merge branch 'master' into fix-ipv6-uri 2024-03-04 13:41:10 +01:00
Kenneth SkovhedeandGitHub 550b7d985d Merge pull request #5062 from Jojo-1000/improve-list-speed
Improve file backend list speed
2024-03-04 13:39:38 +01:00
Kenneth SkovhedeandGitHub 07da5662ff Merge pull request #5105 from Jojo-1000/fix-backend-requests-retired
Catch RetiredException in BackupHandler to avoid StopNow race condition
2024-03-04 13:33:58 +01:00
Kenneth SkovhedeandGitHub 7a50c52fbc Merge pull request #5101 from gpatel-fr/updatesshnet202301
update sshnet to 2023.0.1
2024-03-04 13:29:07 +01:00
Kenneth SkovhedeandGitHub cbb2b6f295 Merge pull request #5096 from Jojo-1000/fix-invalid-file
Check that the database exists before writing error logs
2024-03-04 13:27:20 +01:00
Kenneth Skovhede d16d702975 More fixes for stray using directives 2024-02-28 20:42:33 +01:00
Kenneth Skovhede 1f3b00d7ff Fixed a using directive that was caught in the license region 2024-02-28 15:57:06 +01:00
Kenneth Skovhede 40dd3c6816 Updated all license mentions to MIT 2024-02-28 15:45:30 +01:00
Jojo-1000 929e61f6b4 Catch RetiredException in BackupHandler to avoid StopNow race condition. 2024-02-22 18:48:44 +01:00
gpatel-fr f7def9355b update sshnet to 2023.0.1 2024-02-16 11:39:45 +01:00
Jojo-1000 a0f9785f5c Check that the database exists before writing error logs.
Closes #5091

Fixes that any fatal error in test-filters created an INVALID! file in the current working dir.
2024-02-09 17:10:05 +01:00
gpatel-frandGitHub 396f7dfebe don't report backup as failed if all tests problems are 'extra' (#5074)
these problems don't block database rebuilding so it's not
a good thing to fail the backup.
Also fix erroneous message for ListAndIndexes
2023-12-23 14:25:28 +01:00
gpatel-frandGitHub e2c77b5310 do a vacuum of bug report database (#5069) 2023-12-13 09:27:51 +01:00
gpatel-frandGitHub b08b59228c report all filesets in error in VerifyConsistency (#5052)
only reporting the first one could incite to deleting
it while many filesets can be impacted by a file error
2023-12-12 18:25:14 +01:00
gpatel-frandGitHub 79b87566dd set default log level to Dryrun if dryrun option is set (#5059) 2023-12-11 14:42:45 +01:00
gpatel-frandGitHub 11d6364b4a remove agsXmpp, replace it with Sharp.Xmpp (#5046)
also add a missing space in default body (fixes a small bug under Windows)
2023-12-10 00:26:26 +01:00
gpatel-frandGitHub 0de4cb7968 allow full-remote-verification to target only list and index files (#5037)
The option is turned into an enum with values True, False, ListIndexes
True and False are for compatibility with the existing backups.
ListIndexes is equivalent to True but don't handle block volumes.
This is to make more workable a strategy of full remote testing
after each backup.
2023-12-10 00:24:07 +01:00
gpatel-frandGitHub af4fd5af05 change error reporting for full backup remote verification (#5004)
Before the change, it could happen that when the postbackup test
was finding problems in the backend, the job would be reported
as having completed successfully, the found problems being displayed
only in the complete log. After the change, an error is logged
that cause the job to be failed.
To repro, the following procedure can be used in a test backup:
change a hash for a block in the block table database and run a backup having set
--full-remote-verification and --full-block-verification to true and
--backup-test-percentage to 100.
2023-12-10 00:23:16 +01:00
Jojo-1000 f7e435b1dd Improve file backend list speed.
Directly list FileInfo instead of lookups by filename. This greatly speeds up backup verify on exFAT.

Closes #5061
2023-11-19 20:04:37 +01:00
Jojo-1000 a45ce610df Fix URI parsing for IPv6 addresses and add tests.
Closes #5057
2023-11-10 13:53:10 +01:00
gpatel-frandGitHub ec9d49d684 Experiment/rebuildfaster (#5043)
* experimental faster db rebuild (#4955)

* try run update db query for every block file only when needed
add repair-force-block-use option
more appropriate naming for some routines
more tracing
more comments
2023-11-05 18:19:08 +01:00
gpatel-frandGitHub 43dea88b4f enable to perform tests without checking file list (#5054)
following the option --disable-filelist-consistency-checks
2023-11-04 14:25:59 +01:00
JanandGitHub 3a0667dc64 Add option to disable chunked encoding for AWS S3. (#4999)
Some S3 providers do not support MultiChunkedEncoding (#4994).
2023-09-25 02:02:55 +02:00
gpatel-frandGitHub 4012e7f608 add console log to the Alternative FTP backend for debugging purposes (#5034) 2023-09-24 10:52:57 +02:00
taz-ilandgpatel-fr f791066b7a FluentFTP version update - Fix handling of missing base folder (#4946)
* Improve errors specificity by:
a. Avoiding unnecesary wrapping in AggregateException - Using Library.Utility.Await instead of Wait / Result
b. Using messages from InnerException during TestConnection

* Fix detection of missing base directory:
Depend on error code while trying to SetWorkingDirectory, rather than very wide try/catch and a hardcoded error string (Different FTP servers return different error strings. E.g. hardcoded here was "Directory not found." while Filezilla returns "Can't open file or directory").

* Fix indentation

Intentionally seperated from the previous commit for easier reviewing (otherwise Github text diff display is confused by the removal of the large try / catch blocks)

* Fix creation of missing base directory:
(previously CreateClient would fail since it would try to SetWorkingDirectory to the not-yet-existing directory)
2023-09-24 09:28:19 +02:00
taz-ilandgpatel-fr dbd8cdb86a Update FluentFTP to latest version (46.0.2) (#4935)
* Upgrade FluentFTP to v42 in an attempt to resolve TLS session issues:
duplicati/duplicati#4831

Note v40 introduced breaking changes, specifically seperating between sync and async clients.

Using a different approach this time, single async client.

* Bump FluentFTP to latest (42.0.0 --> 46.0.2)
2023-09-24 09:28:19 +02:00
gpatel-frandGitHub b19f6f3743 update restsharp to 106.10.1 (#5029)
to make Dependabot happy - have to update minio as well
2023-09-23 14:57:27 +02:00
JanandGitHub 91eace3ab2 Include exception type and message in job log. (#4976)
If an exception is provided for a log message, display the type and message (excluding stack trace) even if withExceptionDetails=false. This helps to diagnose bugs if the live log was not saved.

Closes #4843
2023-09-21 23:13:15 +02:00
Janandgpatel-fr b894a197a9 Fix missing file error caused by interrupted compact.
Before the dindex files are deleted on the remote, update the state in the database to Deleting. Previously, only dblock files were handled this way.
Patch by warwickmm
Closes #4129
2023-09-21 22:56:52 +02:00
Janandgpatel-fr 71eb03130a Improve result reporting for interrupted/failed backups (Fix #4829)
Direct import of PR#4978
Update web UI for new result reports.
For operations with fatal errors, write logs to same operation ID.
Test that Interrupted flag is correct in RunScriptTests.
Update backup log display for new result reporting.
Hide file statistics for fatal errors and change fatal icon.
2023-09-21 20:01:17 +02:00
gpatel-frandGitHub 91f1fc6a60 update microsoft.rest.clientruntime to 2.3.24 (#5020) 2023-08-31 10:54:50 +02:00
gpatel-frandGitHub f985af4a5a removes obsolete backend HubiC - fixes #4836 (#5018) 2023-08-30 13:45:55 +02:00
gpatel-frandGitHub 729a84fdc1 update mail libraries (#5014)
known side effect: the mail server user password will no longer be displayed in the verbose log
2023-08-28 11:50:37 +02:00
Andrew RuthvenandAndrew Ruthven dc609e297d Add Catalyst Cloud as an OpenStack target
I successfully have backups running to Catalyst Cloud and I'm able to restore.
2023-05-12 16:02:00 +12:00
adithajayandGitHub 16c13f4fef Update S3Backend.cs
Added endpoints for S3 compatible storage from Scaleway. I feel adding this provider is important as they provide 75GB of free storage and many users would utilize that.
2023-04-12 02:31:15 +05:30
gpatel-fr cf17f488d1 update Newtonsoft lib to 13.02 2023-03-24 23:17:11 +01:00
gpatel-frandGitHub 692fb32141 Merge pull request #4907 from gpatel-fr/sqlitecustom
Add custom sqlite pragmas
2023-03-24 21:06:37 +01:00
gpatel-frandGitHub 4a67d57400 Merge pull request #4905 from daniel0m0baker/master
Adding Impossible Cloud US endpoint
2023-03-24 21:05:51 +01:00
gpatel-frandGitHub 1bfc02e9ac Merge pull request #4901 from gpatel-fr/updatesslnet
bump SSH.NET to 20.0.2
2023-03-24 21:03:13 +01:00
gpatel-frandGitHub 5f61d34d2c Merge pull request #4897 from gpatel-fr/updatemega
Update Mega library to 1.10.3
2023-03-24 21:02:32 +01:00
gpatel-frandGitHub 4897a096c2 Merge pull request #4849 from vilaureu/master
fix #4716 by falling back to System hasher
2023-03-24 21:01:27 +01:00
gpatel-frandGitHub 4a6a5a52dc Merge pull request #4815 from dgasaway/master
Add a few more compressed, audio, and video exclusions
2023-03-24 21:00:19 +01:00
gpatel-fr fbbc78619e add custom sqlite pragmas
through environment variable CUSTOMSQLITEOPTIONS_DUPLICATI
2023-03-23 20:25:18 +01:00
Daniel BakerandGitHub a8053ce702 Update S3Backend.cs
Added Impossible Cloud US endpoint.
2023-03-15 16:27:58 +01:00
gpatel-fr 1b93d70656 bump SSL.NET to 20.0.2 2023-03-06 11:08:18 +01:00
gpatel-fr c2509b62d8 Update Mega library to 1.10.3 2023-02-28 00:28:39 +01:00
vilaureu 71971df893 fix #4716 by falling back to System hasher
When only having OpenSSL 3 installed, FasterHashing.FasterHash.Create()
currently throws an EntryPointNotFoundException.
Mitigate this by falling back to HashAlgorithm.Create in the error case.
2022-11-19 11:33:33 +01:00
David Gileadi 353850748b Merge branch 'master' into cocoa-mac 2022-11-02 09:11:32 -07:00
dgasawayandGitHub 0ba4da94a6 Add a few more compressed, audio, and video exclusions
Supporting information:
https://wiki.xiph.org/MIME_Types_and_File_Extensions
https://en.wikipedia.org/wiki/Opus_(audio_format)
https://en.wikipedia.org/wiki/Zstd
https://en.wikipedia.org/wiki/MPEG-2#Filename_extensions
2022-09-06 14:52:51 -07:00
David Gileadi 88480f2ee2 Get actual exception when logging errors 2022-08-04 09:37:38 -07:00