This PR extends the handling of excluded file attributes on Windows to include additional values that are present in Windows 10+11, but are not yet part of the .NET file attributes.
Specifically, this handles the `FILE_ATTRIBUTE_RECALL_ON_OPEN` and `FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS` attributes which are used to signal files are not available locally and will be fetched on access.
On Windows, a new option `--exclude-non-local-files` is added which will simply update `--exclude-files-attributes` with `Offline,RecallOnOpen,RecallOnDataAccess` such that setting the option will exclude offline files from the backup, without requiring that the user is aware of the attribute names or effects.
For all platforms, this PR also updates the parsing of the `--exclude-files-attributes` so it is now a flags-type enum and will validate the input as well as show the possible options in help and dropdowns.
This PR adds a flag to the test method so we can choose to test for read-only access (needed for restore) or read-write (needed for backups).
Additionally, this PR fixes a logic bug in that the BackendSourceProvider would not actually test the connection, but just check the placeholder element returned (which is always returned). With this PR the backend's ListAsync command is tested fully.
Also renamed the methods on ISourceProvider to have the `...Async` prefix.
This replaces #6939
This PR changes so the ReleaseBuilder uses the regular secret reading code instead of a replicated version.
The TrayIcon's attachment code now explicitly maps up the streams if possible.
Updated the Duplicati backend to support direct backups where the storage server returns keys to an AWS compatible storage and lets the client communicate directly with the storage.
The gateway mode is stil supported.
This PR partially re-introduces the logic attempting to get the free space for a specific path.
The code before this would incorrectly always return the free space of `/` on non-Windows, instead of the actual mounted path's free space.
This PR adds support for generating a self-signed CA and then using that CA to generate TLS certificates.
A new tool `duplicati-configure` / `Duplicati.CommandLine.ConfigureTool.exe` is added to manage the certificates. The tool saves the configuration in the database and is meant to run with elevated privileges for the initial CA installation.
The option `--configure-https` can be added to server/trayicon, and if the process has permissions, this will automatically install the CA certificate.
If a CA is configured, the server will automatically issue, renew and use a TLS certificate.
With this setup, the TLS is opt-in, but once the flow is well tested, we can switch it to opt-out.
This PR fixes a reported issue with strings that are less than 3 characters.
This is unlikely, but happens if the code tries to get the scheme of a path, like `/a` or `/`.
The updated logic returns the scheme if one exists, and it is less than 15 characters (to avoid leaking sensitive information).
This PR adds checks for free temporary space when starting the server and when restoring.
On systems that have limited space in the temp folder a warning will now be shown.
This is intended to capture issues on some Docker systems where /tmp is mounted in memory instead of being disk backed.
It will also detect the issue on other systems that are space constrained.
This PR now checks for the SSL certificate and the UI will offer to pin the certificate if this is not a trusted certificate.
Also, the sourceproviders and restoredestinationproviders now actually invoke the `Test` method when being tested.
- Added Cloudflare to list of known S3 provider domains.
- Removed AWS specific naming on the help texts.
- Fixed an edge case where calculating how many files to test with an empty list would crash
- Fixed a bug with getting a safe display url for very short urls
This PR updates the reports sent to include a more detailed OS version string, similar to what is used by the usagereporter and systeminfo call.
It also reports parts of the destination hostname, if using S3 and the host is a known public cloud. This is done to avoid leaking internal hostnames or actual bucket names, but still provide some way to identify what storage service is used.
This PR adds detection of exclusion attributes on files and folders. If a file or folder has an exclusion extended attribute, the file or folder is excluded from the backup.
The option `--disable-backup-exclusion-xattr` can be used to revert to the previous behavior where the xattrs were not checked.
This has the biggest impact on MacOS where it will not perform like other backup software and avoid files that are marked as excluded from backup.
This fixes#6393
This PR adds a dynamic property so a backend can signal if it supports streaming, based on the settings.
This is currently used for the File backend, so that toggling `--use-move-for-put` will disable streaming on the backend instead of relying on the `--disable-streaming-transfers` flag.
This fixes an issue with formatting for invariant values that was caused by incrorrect function overload selection.
To avoid future issues, the two similar functions have been renamed to clarify what they are working for.
This fixes pre-calculating the SHA256 hash for AWS SDK requests to S3.
This fixes an issue where a throttled stream would not be throttled correctly, because the AWS library would read the throttled stream twice (once for hashing, and once for transmitting).
To ensure compatibility with Wasabi S3, the hash is also lowercased.
This fixes the encoding of hostnames with `.` characters in them. Without this fix, the hostname would be encoded making it impossible to use regular DNS hostnames.
The URLs passed to the custom URL parser were not decoding the path part, but passing it directly. This caused problems if the input was correctly URL encoded.
The updated code now correctly decodes the hostname+path parts, and encodes them again.
Kenneth Skovhede