This PR adds support for generating a self-signed CA and then using that CA to generate TLS certificates.
A new tool `duplicati-configure` / `Duplicati.CommandLine.ConfigureTool.exe` is added to manage the certificates. The tool saves the configuration in the database and is meant to run with elevated privileges for the initial CA installation.
The option `--configure-https` can be added to server/trayicon, and if the process has permissions, this will automatically install the CA certificate.
If a CA is configured, the server will automatically issue, renew and use a TLS certificate.
With this setup, the TLS is opt-in, but once the flow is well tested, we can switch it to opt-out.
This PR fixes a reported issue with strings that are less than 3 characters.
This is unlikely, but happens if the code tries to get the scheme of a path, like `/a` or `/`.
The updated logic returns the scheme if one exists, and it is less than 15 characters (to avoid leaking sensitive information).
This PR adds checks for free temporary space when starting the server and when restoring.
On systems that have limited space in the temp folder a warning will now be shown.
This is intended to capture issues on some Docker systems where /tmp is mounted in memory instead of being disk backed.
It will also detect the issue on other systems that are space constrained.
This PR now checks for the SSL certificate and the UI will offer to pin the certificate if this is not a trusted certificate.
Also, the sourceproviders and restoredestinationproviders now actually invoke the `Test` method when being tested.
- Added Cloudflare to list of known S3 provider domains.
- Removed AWS specific naming on the help texts.
- Fixed an edge case where calculating how many files to test with an empty list would crash
- Fixed a bug with getting a safe display url for very short urls
This PR updates the reports sent to include a more detailed OS version string, similar to what is used by the usagereporter and systeminfo call.
It also reports parts of the destination hostname, if using S3 and the host is a known public cloud. This is done to avoid leaking internal hostnames or actual bucket names, but still provide some way to identify what storage service is used.
This PR adds detection of exclusion attributes on files and folders. If a file or folder has an exclusion extended attribute, the file or folder is excluded from the backup.
The option `--disable-backup-exclusion-xattr` can be used to revert to the previous behavior where the xattrs were not checked.
This has the biggest impact on MacOS where it will not perform like other backup software and avoid files that are marked as excluded from backup.
This fixes#6393
This PR adds a dynamic property so a backend can signal if it supports streaming, based on the settings.
This is currently used for the File backend, so that toggling `--use-move-for-put` will disable streaming on the backend instead of relying on the `--disable-streaming-transfers` flag.
This fixes an issue with formatting for invariant values that was caused by incrorrect function overload selection.
To avoid future issues, the two similar functions have been renamed to clarify what they are working for.
This fixes pre-calculating the SHA256 hash for AWS SDK requests to S3.
This fixes an issue where a throttled stream would not be throttled correctly, because the AWS library would read the throttled stream twice (once for hashing, and once for transmitting).
To ensure compatibility with Wasabi S3, the hash is also lowercased.
This fixes the encoding of hostnames with `.` characters in them. Without this fix, the hostname would be encoded making it impossible to use regular DNS hostnames.
The URLs passed to the custom URL parser were not decoding the path part, but passing it directly. This caused problems if the input was correctly URL encoded.
The updated code now correctly decodes the hostname+path parts, and encodes them again.
This PR removes the `OAuthContextSettings` class, and makes the `--oauth-url` option available on each backend that uses it, so it can be configured as part of the destination.
To assemble everything related to configuring OAuth, the code was moved to the `AuthIdOptionsHelper`, such that the usage can be shared between implementations.
This PR also adds the option to set the default OAuth url from the environment variable `DUPLICATI_OAUTH_SERVICE`.
The server can then be set in the following locations (most important last):
- Environment variable
- Server-wide advanced settings
- Backup job advanced settings (or commandline)
- Destination url
This PR adds the ability to subscribe to messages over a websocket.
To prevent polling data, the server can now push messages through the websocket, so the client can instantly update when new data is available.
The change is backwards compatible, still serving the status updates over the socket. If the client is providing the authentication token, it is auto-subscried to the legacy status message.
If the client is using the new authentication message, it needs to subscribe to get the status updates (and any other services it needs).
The event system has been extended to support new, and more accurate, events. This is the first step towards removing the general status update and the long-poll mechanism.
This also re-introduces the blocking marker, so the client can know if the operation is halted due to too many pending transfers.
Some endpoints have been moved to service implementations, to allow serving the exact same data from both endpoints and websocket.
This PR also updates the way the progress is handled, so that all transfers are returned to the client, and the transfer speeds for each transfer is calculated based on a small sample buffer, so the values are more accurate even if the program is paused during transfers.
This PR moves the check from the FTP backend into shared code, so all backends will now check for read/write permissions when using the "Test" button.
For situations where write permissions are not required (like restore or verify) there is a flag on the v2 API that indicates if the connection was established, so the UI can show a message suggesting that it is possible to proceed, if the current UI operation does not require write permissions.
This fixes#2473
If th user has the SeBackupPrivilege, set the default snapshot policy to `Auto` and otherwise use `Off` (the current default).
This ensures that if the process is running with the correct permissions, there will be fewer warnings.
This only changes the default value on Windows.
This fixes#2833
This adds nullability to FilterExpression and Options.
It also updates almost all properties in Options to use a common format that enforces uniform parsing of the values.
A few options have been updates to have the default values specified as a constant, to avoid replicating the value in two places.
This PR adds the option to suppress warnings by their log id. Warning IDs that are supplied to `--suppress-warnings` will be converted to information messages before being logged.
If a specific warning is disabled, such as `CompressionReadErrorFallback`, this will then no longer count as a warning for the job, and the log file will see the warning as an information message.
This PR also adds two simpler filter options that makes it possible to filter log messages by supplying the log IDs. This can already be achieved with log filters, but the ID filter is a bit simpler to apply, as you only need to know the ID.
Finally, this PR also adds common logging for errors in the categories:
- Permission denied, id = `PermissionDenied`
- File locked, id = `FileLocked`
- Path not found, id = `PathNotFound`
- Path too long, id = `PathTooLong`
These new log ids makes it simpler to ignore warnings about locked or inaccesible files.
Kenneth Skovhede