Commit Graph
4937 Commits
Author SHA1 Message Date
Kenneth Skovhede c1d89e32e9 Guard against name clashes for Windows folder 2025-01-27 13:09:35 +01:00
Kenneth Skovhede 805d3b1c31 Added special handling of Linux to avoid storing data in the root of the filesystem 2025-01-27 11:51:03 +01:00
Kenneth Skovhede c05b00ae0b Implemented support for avoiding storing data in the Windows folder 2025-01-27 11:49:17 +01:00
Kenneth Skovhede 97d62163c5 Renamed methods for database path to better distinguish what they do 2025-01-27 11:46:24 +01:00
Kenneth Skovhede bee1f141ea Added UI warning for unencrypted database 2025-01-27 08:52:06 +01:00
Kenneth SkovhedeandGitHub 6a8edd75dc Added additional folders to MacOS. (#5899)
This adds the common "Movies" and "Public" folders to the list of special folders for MacOS (and Linux, if applicable)
2025-01-25 12:42:16 +01:00
Kenneth SkovhedeandGitHub b6981d8668 This fixes errors with the Minio backend not re-throwing exceptions (#5897)
* This fixes errors with the Minio backend not re-throwing exceptions

* Throwing the correct exception ...
2025-01-22 23:29:27 +01:00
Kenneth SkovhedeandGitHub 6c493186a7 Changed MissingFileDetected message to Warning (#5893)
* Changed `MissingFileDetected` message to `Warning`

* Updated tests to match errors/warning counts
2025-01-22 21:18:33 +01:00
Kenneth SkovhedeandGitHub 997891525a Updated license-updater to update csproj files (#5898)
* Updated license-updater to update csproj files

* License updates
2025-01-22 21:04:19 +01:00
Marcelo C. cee1b8428d Centralizing the check on Helper function. 2025-01-21 12:45:53 -03:00
Kenneth Skovhede 04205d5084 Added support for CORS 2025-01-18 11:03:52 +01:00
Marcelo C. 2e502ea7be Help Command unification
All CLI tools and executables now accept a unified set of help command variants:

"help", "/help","--help", "usage", "/usage", "--usage", "/h", "-h"

Affected executables:

Duplicati.Agent
Duplicati.CommandLine
Duplicati.CommandLine.AutoUpdater
Duplicati.CommandLine.BackendTester
Duplicati.CommandLine.BackendTool
Duplicati.CommandLine.ConfigurationImporter
Duplicati.CommandLine.SecretTool
Duplicati.CommandLine.Snapshots
Duplicati.GUI.TrayIcon
Duplicati.Server
Duplicati.Service
Duplicati.WindowsService

Note that SharpAesCrypt command is not included in this because the parameter parsing is part of the SharpAesCrypt code base.
2025-01-17 14:00:14 -03:00
Carl Johnsen fe7cb13fde Fixed issue with stream not being open in the correct mode during restore 2025-01-17 12:27:05 +01:00
Carl Johnsen 0861bc0a73 Added FileOpenReadWrite function 2025-01-17 12:26:03 +01:00
Kenneth SkovhedeandGitHub 045c6b9147 Feature/adjust pcloud exception (#5883)
* Fixed pCloud exception type

Changed pCloud exception to `FileMissingException` (was `FileNotFoundException`)

* Fixed a label in tests
2025-01-16 22:21:16 +01:00
marceloduplicatiandGitHub 08f231dd74 Fixes to timeouts on WebDav (#5878)
* Removing timeout from Webdav

The timeout will be controlled externally by the TimeoutObservingStream and or the cancellation token.

Timeout for List set to default of 10 minutes

Since filesystems may take quite sometime to list large number of files, the timeout was set to a high value.

Previous timeout of 30 seconds was triggering in a case reported by user.

Timeout for CreateFolder and DeleteAsync was kept to default of 30 seconds.

Minor formmating/refactoring was done.

* Removing timeout from Webdav

The timeout will be controlled externally by the TimeoutObservingStream and or the cancellation token.

Timeout for List set to default of 10 minutes

Since filesystems may take quite sometime to list large number of files, the timeout was set to a high value.

Previous timeout of 30 seconds was triggering in a case reported by user.

Timeout for CreateFolder and DeleteAsync was kept to default of 30 seconds.

Minor formating/refactoring was done.

* Fixes to Integrated Authentication

This is a fix to the webdav when using integrated authentication, the hint for the underlying issue was found on an old [thread] (https://github.com/dotnet/wcf/issues/2923). By setting up a vm environment with Windows 2022 Server+IIS/Webdav it was possible to reproduce the [issue](https://github.com/duplicati/duplicati/issues/5872) and fix it.

Change to validation script is to avoid the requirement of filling the username when integrated authentication is true.
2025-01-16 16:29:40 +01:00
marceloduplicatiandGitHub 5e477e7e87 Migrate from HttpWebClient to HttpClient with async/await + Refactoring (#5875)
Modernize HTTP client implementation across backend services by:
- Replace legacy HttpWebClient with modern HttpClient
- Implement proper async/await patterns throughout the codebase
- Add cancellation token support for improved request handling
- Refactor related code for better maintainability

This change is part of our broader initiative to modernize the codebase
and improve its resilience through proper asynchronous operations.
2025-01-16 16:29:16 +01:00
Kenneth SkovhedeandGitHub edd69bfd59 Improve cancel robustness 2 (#5882)
* Attempt to avoid comitting transaction without any queries being made

* Guard internal logic
2025-01-16 16:01:54 +01:00
Kenneth SkovhedeandGitHub 58bf21306a Improve S3 logic for permission tests (#5879) 2025-01-16 14:28:20 +01:00
Kenneth SkovhedeandGitHub 52a73c3418 Fixed issue 5845 (#5880)
* Fixed issue 5845

This commit introduces the test that makes the error.

* Fixed query cleaning up duplicates, in the case that where a block has multiple valid source candidates.

* Reduced number of replicas for the test
2025-01-16 14:27:15 +01:00
Kenneth SkovhedeandGitHub 15856fd46f Fixed 2 compile warnings (#5873) 2025-01-14 19:42:50 +01:00
Kenneth Skovhede fb1d15fb9d Avoid optimize after vacuum 2025-01-14 19:42:03 +01:00
Kenneth SkovhedeandGitHub 6bbd64a655 Fixed nullability warnings in Utility.cs (#5861)
* Fixed nullability warnings in `Utility.cs`

* Fixed a number of nullability issues and warnings

* More nullable fixes

* Fixed the overwrite issue

* Fixed additional nullability issues

* Removed additional nullability issues

* More nullability fixes and added some xmldoc

* Added more xmldocs and removed an unused argument

* Removed more warnings
2025-01-14 14:03:48 +01:00
marceloduplicatiandGitHub 10b3068d7a Merge pull request #5865 from marceloduplicati/feature/cifs-fixes
CIFS Fixes and improvements
2025-01-10 16:15:11 -03:00
Marcelo C. cffba06749 Add configurable read and write buffer sizes for SMB operations
Adds support for user-configurable read and write buffer sizes while enforcing protocol-negotiated maximums. When not specified, operations default to using the maximum sizes negotiated during the SMB session setup.
2025-01-10 13:53:51 -03:00
Marcelo C. 3ce67e529a Remove no-reuse-connection Option
Tests revealed that by not using a single connection throttling issues appeared when connecting to Windows shares (possibly due to multiple authentications in a short period of time)
2025-01-10 12:10:03 -03:00
Marcelo C. a1263cd48c Path normalization
Path separators are converted to \ which is Windows & Linux compatible, overwriting the Path.Combine separator (which would be host dependant)
2025-01-10 12:05:40 -03:00
Kenneth SkovhedeandGitHub 9c808e07b3 Correctly formatting size strings as Kikibytes. This fixes #3685 (#5778)
Note: This may cause issues for scripts that parse the string as opposed to using the numeric values.
2025-01-10 09:25:40 +01:00
Kenneth SkovhedeandGitHub d64c7e721f Handle duplicated missing blocks (#5858)
* Implemented test for #5845

* Removed unittests not executing cleanup.

* Cleanup commit

* Fixed the query that causes the constraint violation
2025-01-10 09:24:54 +01:00
Kenneth SkovhedeandGitHub 382ba172b3 Fixed duplicate entries in index files (#5856)
* Added test to verify that compression modules will mask duplicate entries, if present

* Because there are likely many, otherwise valid, dindex zip archives with duplicated blocklists out there, the log message is now verbose.

To make sure we no longer create archives with duplicated entries, the `--unittest-mode` option makes this a warning that is checked in most tests.

* Added guard in `IndexVolumeCreator` to prevent creating zip files with duplicated block lists.

* Added debug check for duplicates

* Fixed the duplicate test
2025-01-10 09:24:31 +01:00
Marcelo C. f598b478f9 Add 'no-reuse-connection' option for SMB/CIFS operations
The 'no-reuse-connection' option controls whether SMB/CIFS connections are reused:

When enabled, creates a new connection and authentication for each file operation
When not specified, defaults to reusing the same connection across operations
2025-01-08 22:25:34 -03:00
Kenneth SkovhedeandGitHub 9464caf622 Feature/update license 2025 (#5851)
* Fixed some minor whitespace issues

* Updated all copyright to 2025
2025-01-07 09:40:39 +01:00
Carl Johnsen 46b05b94ef Limit file name conflict attempts to 100 and log an error if exceeded 2025-01-06 16:01:53 +01:00
Carl Johnsen 6f263ad874 Added a note about potential memory consumption of restores FileProcessor 2025-01-06 16:00:11 +01:00
Carl Johnsen 284b14b72f Merge branch 'master' into bugfix/restore-overwrite 2025-01-06 14:43:03 +01:00
marceloduplicatiandGitHub 99ee234640 Primary functionality implemented for CIFS (#5780)
* Primary functionality implemented for CIFS

This will add CIFS support to Duplicati.

One of the advantages is that the resource does not need to be mounted as the backend connects and authenticates directly to the resource via TCP/or Netbios over TCP.

The authentication supported is domain/username/password. At present integrated Windows authentication is not supported.

This implementation relies on SMBLibrary library (https://github.com/TalAloni/SMBLibrary).

* Fixes to XML documentation

* Implementing single CIFS connection + Fix on PutAsync

Single CIFS connection to avoid re-authentication on every operation.
Fix to PutAsync which only because apparent after file sizes were larger than _smb2Client.MaxWriteSize

* Adding CI tests for CIFS

The CI test for CIFS is self contained, requiring no environment variables. Also updated README.
2025-01-06 14:22:23 +01:00
Kenneth SkovhedeandGitHub 2a757dea0f Quota check disabled fix (#5843)
The QuotaCheck is done in two places, and only one place was checking if the quota check was supposed to be disabled.
This adds the check in the other place.
2025-01-06 14:21:53 +01:00
Carl Johnsen 88846251b1 Restoring a symlink that already exists triggers an overwrite of the file, which should count towards number of files restored. 2025-01-06 08:14:32 +01:00
Carl Johnsen 1bea0f3696 Restored empty files should count towards the restoration file count. 2025-01-05 12:46:08 +01:00
Carl Johnsen 4cd565d592 Added handling of already restored timestamp-retargeted files 2025-01-05 12:09:38 +01:00
Carl Johnsen b759b4564d Added docstrings to new functions 2025-01-05 12:09:04 +01:00
Carl Johnsen eb64320a49 Changed the reporting of restored files to match the legacy restore's reporting. 2025-01-05 11:51:52 +01:00
Carl Johnsen 637bfcefc8 Forgot to check whether the file exists 2025-01-05 11:25:19 +01:00
Carl Johnsen eba477c271 Added handling of the 'overwrite' option in the new restore flow 2025-01-05 10:44:08 +01:00
Carl Johnsen f76bf90e68 Added additional dry run logging 2025-01-05 10:43:23 +01:00
Carl Johnsen 8e92754efd VerifyLocalBlocks now keeps track of internally verified blocks for logging 2025-01-05 10:43:01 +01:00
Carl Johnsen be99f03bac VerifyLocalBlocks wasn't respecting dry run properly 2025-01-05 10:41:33 +01:00
Carl Johnsen a14d29da93 Modified VerifyTargetBlocks to also return the blocks that were verified. 2025-01-05 10:40:14 +01:00
Carl Johnsen a7507fb765 Simplified variable assignment 2025-01-05 10:35:50 +01:00
Carl Johnsen 83d1617984 Fixed the IO calls to also work on Windows 2025-01-03 23:27:35 +01:00