Commit Graph
3112 Commits
Author SHA1 Message Date
Carl Johnsen 48180770e3 Removed unnecessary project reference to builtin modules 2026-01-08 14:35:46 +01:00
Carl Johnsen 50f18dca84 Moved ResultSerialization from builtin modules to a shared library 2026-01-08 14:34:11 +01:00
Carl Johnsen 8519e3f591 Promoted LightWeightBackendManager to the main Duplicati library 2026-01-08 14:26:26 +01:00
Kenneth Skovhede cf42426d1b Store password with TrayIcon
When using the TrayIcon with no hosted server (i.e., connecting to a running server), it is now possible to save the password and url in the secret provider.

There is a checkbox that allows saving the settings, and if checked, will save the settings to the secret provider (using the OS default if none is specifically provided).

This fixes #6379
2025-12-07 22:26:37 +01:00
Kenneth SkovhedeandGitHub 179fd4da47 Merge pull request #6653 from duplicati/feature/secret-provider-write-support
Initial implementation of write-support for SecretProviders
2025-12-05 15:23:32 +01:00
Kenneth Skovhede 7322c46861 Pre-execution check for RemoveRemoteVolumes
This PR adds a check before entering the RemoveRemoteVolumes that checks if there are dangling FilesetEntry records when calling the function.

This makes it possible to figure out if the problem is created by the RemoveRemoteVolumes code, or an already existing problem in the database.

Also added a few tests that check that the condition is correctly detected.
2025-12-03 11:34:30 +01:00
David Kartchner 5113d0e989 added an option to supply a new passphrase for reencryption with RecoveryTool recompress command 2025-12-02 02:24:06 -07:00
Kenneth Skovhede 695d190b91 Updated to allow IsSupported to be async 2025-12-01 15:29:24 +01:00
Kenneth Skovhede 8ac2273f21 Added OS markers to avoid accessing providers from unsupported OS.
Updated Pass provider to probe for pass via path searching.

Changed `IsSupported` to a method because it could take a while to evaluate
2025-12-01 13:21:44 +01:00
Kenneth Skovhede 6d9826df06 Initial implementation of write-support for SecretProviders
This updates the MacOS secret provider to use PInvoke for a more stable API than the secrets tool that was used before.
2025-11-30 23:01:57 +01:00
Kenneth Skovhede ef649f3926 Updated reporting to include more details
This PR updates the reports sent to include a more detailed OS version string, similar to what is used by the usagereporter and systeminfo call.

It also reports parts of the destination hostname, if using S3 and the host is a known public cloud. This is done to avoid leaking internal hostnames or actual bucket names, but still provide some way to identify what storage service is used.
2025-11-28 12:04:47 +01:00
Kenneth Skovhede e1b9015a85 Added support for exclusion of files based on xattrs
This PR adds detection of exclusion attributes on files and folders. If a file or folder has an exclusion extended attribute, the file or folder is excluded from the backup.

The option `--disable-backup-exclusion-xattr` can be used to revert to the previous behavior where the xattrs were not checked.

This has the biggest impact on MacOS where it will not perform like other backup software and avoid files that are marked as excluded from backup.

This fixes #6393
2025-11-21 10:59:28 +01:00
Kenneth Skovhede 3b695cf065 Added support for Photos on MacOS
This PR adds detection of the MacOS Photos folder, and intercepts reads and replaces them with PhotoKit calls.
With this, it is possible to make backups of all MacOS Photos, even if they are not stored locally.

The previous versions would just make a backup of the on-disk structure, which was not guaranteed to contain all photos, but instead has various indexing for finding photos, and may contain some original photos.

The option `--photos-handling` controls how Duplicati now deals with the Photos folder. The options are:
- `LibraryOnly`: Same as before, just treat it as a folder
- `PhotosOnly`: Ignore the folder contents and just back up the actual photos
- `PhotosAndLibrary` (default): Make a backup of the photos and the library on-disk. This may cause images to be stored twice, but de-duplication will usually limit the storage increase.

The option `--photos-library-path` can be used to point to the on-disk Photo library that should be handled, in case the auto-detection does not pick it up. If this does not point to a valid Photoslibrary, or the path is not being backed up, no special handling will be done.

Note that the restore is not restoring into Photos itself, but instead restores into a sub-folder in the Photolibrary that is called `dup_backup`. To get the photos out after a restore, one needs to right-click the Photolibrary folder, and choose "Show package contents" and then the `dup_backup` folder is revealed.

This is done to keep all photos in the same folder, but avoid messing with the structure of the on-disk Photolibrary.

A future update could allow restoring back into Photos, and metadata is captured for each image to eventually allow this.

This fixes #6381
2025-11-17 17:12:10 +01:00
Kenneth SkovhedeandGitHub 2632ba8164 Merge pull request #6625 from duplicati/bugfix/restore-deadlock-timer-better-start
Bugfix/Better starting time for restore deadlock detection.
2025-11-15 11:04:25 +01:00
Kenneth Skovhede 43d38efdd3 Fixed remaining warnings 2025-11-14 16:09:05 +01:00
Kenneth Skovhede 46428936c4 Various updates after the upgrade 2025-11-14 15:45:19 +01:00
Kenneth Skovhede 1ea74a3c63 Updated to .NET10 2025-11-14 15:05:39 +01:00
Carl Johnsen d48afd7496 Check the database for volume sizes reported in remotevolume 2025-11-10 14:56:20 +01:00
Carl Johnsen 77f3000e4f Reduced the timing of a block to the lifetime of the block request as it roundtrips from/to the BlockManager 2025-11-10 13:28:59 +01:00
Carl Johnsen 57e40a6290 Initial deadlock detection threshold is now set to 1 minute per 10 MB of volume size. 2025-11-10 12:54:33 +01:00
Kenneth Skovhede 012aeed7de Add dynamic streaming toggle
This PR adds a dynamic property so a backend can signal if it supports streaming, based on the settings.

This is currently used for the File backend, so that toggling `--use-move-for-put` will disable streaming on the backend instead of relying on the `--disable-streaming-transfers` flag.
2025-11-03 12:48:37 +01:00
Carl Johnsen 96921bc5fb RestoreHandler now sets up the DeadLock timer alongside the restore process network 2025-10-28 14:44:13 +01:00
Carl Johnsen dafb38443c FileProcessor id (used for debugging) started at 1, rather than 0. 2025-10-28 14:43:35 +01:00
Carl Johnsen 91009297c1 Fixed typo 2025-10-28 14:42:58 +01:00
Carl Johnsen 435792c8fa BlockManager now uses times reported by the DeadlockTimer as deadlock detection threshold instead of a hardcoded 5 minutes 2025-10-28 14:42:43 +01:00
Carl Johnsen 4e5d015e0c Added the deadlock timer task 2025-10-28 14:41:58 +01:00
Carl Johnsen ccfcffcc1e Added internal timers to keep track of maximum processing times of the restore processes 2025-10-28 14:41:39 +01:00
Kenneth Skovhede 777bc06842 Report remote filename on hash errors
This PR changes the reported filename to use the remote filename instead of the random temporary filename that has little value to the user as the file is deleted immediately after the error is raised.
2025-10-27 11:37:10 +01:00
Kenneth Skovhede 8224d84f57 Reverted to use the shorter name --asynchronous-upload-limit and then added an alias for --asynchronous-concurrent-upload-limit. 2025-10-16 13:25:34 +02:00
Kenneth Skovhede 55c19c8752 Removed more references to the unused option 2025-10-16 13:21:40 +02:00
Kenneth Skovhede d3b15f136e Removed unused option
The option `asynchronous-upload-limit` was not used, and has been removed.
2025-10-14 21:30:02 +02:00
Kenneth SkovhedeandGitHub 9507b57f1c Merge pull request #6534 from carljohnsen/performance/restore-lock-reduce
Performance/restore lock reduce
2025-10-03 11:46:00 +02:00
Carl Johnsen 91fc698314 The block manager only adds a block to the cache, if it's needed at a later point. 2025-10-03 09:50:08 +02:00
Carl Johnsen 2aa2103f33 Changed BlockManager to only use the cache if the max size is > 0 2025-10-03 09:26:38 +02:00
Carl Johnsen aafe192766 Added docstring 2025-10-03 09:26:08 +02:00
Carl Johnsen a1e1afce28 Changed the default volume cache hint to be volumesize * 100 2025-10-03 08:50:35 +02:00
Carl Johnsen 842cc00979 Added profiling stopwatch for DataBlock instantiation in volume decompressor 2025-10-03 08:32:29 +02:00
Carl Johnsen dff4b52066 Moved instantiation of the block wrapper to the volume decompressor 2025-10-03 08:32:20 +02:00
Carl Johnsen 8f056d5db9 Added profiling stopwatch for volume wrapper in the volume decryptor 2025-10-03 08:25:56 +02:00
Carl Johnsen cd4e23efc4 Moved instantiation of the volume wrapper to the volume decryptor 2025-10-03 08:25:46 +02:00
Carl Johnsen 2ad6e088ac Added another profiling stopwatch to blockmanager 2025-10-03 08:21:42 +02:00
Carl Johnsen 0c1643568e Volume cache now counts the size of the volumes, not just the count of the volumes 2025-10-03 08:11:04 +02:00
Carl Johnsen 48e5a57654 Adapted volume cache eviction to only evict on a download response 2025-10-03 07:51:24 +02:00
Carl Johnsen 42cba24bd9 Removed old decompress in flight code 2025-10-03 07:50:48 +02:00
Carl Johnsen 1f0a4617a6 The reader has to be disposed before disposing the file 2025-10-03 07:50:19 +02:00
Carl Johnsen 34cfd47ceb Added docstrings 2025-10-03 07:50:03 +02:00
Carl Johnsen 1f44b5e01b Removed old ack profiling from the block manager 2025-10-03 07:49:06 +02:00
Carl Johnsen d03259d687 Redacted the ack channel in the restore process network in favor of a referenced counted volume wrapper that'll dispose the reader and tempfile once safe. 2025-10-02 21:40:40 +02:00
Carl Johnsen 03db41e392 FileProcessor had an incorrect call to logging 2025-10-02 21:36:07 +02:00
Carl Johnsen 4f2bfe1fa0 Reduced very verbose logging on errors in BlockManager 2025-10-02 21:35:35 +02:00