When all files are deleted from the remote destination and the user runs repair
(which recreates the local database), the recreate fails because the destination
is empty. Previously the partial/empty database that had just been created was
left on disk, which blocked every subsequent operation: a fresh backup could not
run, and even retrying the repair failed because the recreate refuses to run when
the database file already exists.
RecreateDatabaseHandler.RunAsync now removes the database it created when the
recreate fails because the remote destination is empty (UserInformationException
with help id EmptyRemoteLocation / EmptyRemoteLocationWithPrefix), so the next run
starts cleanly. The database connection is disposed before the cleanup runs, so the
file is not locked, and the cleanup is best-effort and never masks the original error.
The cleanup is deliberately scoped to the empty-destination case. Other recreate
failures - in particular a database with missing blocks / broken filelists (help id
DatabaseIsBrokenConsiderPurge) - intentionally keep the recreated database so the
user can run list-broken-files / purge-broken-files against it.
Adds an Issue6205 regression test.
Fixes#6205
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
During a backup, LogExceptionHelper.LogCommonWarning is the shared helper used
to log "excluding path" warnings from many enumeration/metadata steps. For known,
expected problems (permission denied, locked file, path not found, path too long)
it passed the exception to WriteWarningMessage, so the file log, console and
report modules rendered the full stack trace (via LogEntry.AsString(true)) for
every such path — bloating the logs with no diagnostic value.
The known-category branches now log without attaching the exception, so no stack
trace is stored; the self-describing warning message (e.g. "Excluding path due to
permission denied: <path>") remains. The fallback branch keeps the exception so
genuinely unexpected failures are still fully debuggable.
Adds Issue6426 tests asserting known-problem warnings carry no exception while
unknown errors keep it.
Fixes#6426
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The legacy ngax restore progress showed a "Cancel" button inside the
`wait-area` (restore.html) in addition to the existing stop control in the
top navigation bar. Restore cancellation already works via the top control
(StateController.stopDialog, which posts /task/{id}/stop or /abort); the extra
button called a separate /task/{id}/stop that was not handled by the restore
flow, so it was redundant and confusing.
Remove the redundant button and its dedicated handler:
- restore.html: drop `allow-cancel="true"` from the wait-area.
- waitarea.html: remove the Cancel button block.
- waitArea.js: remove the `allowCancel` binding, `cancelTask()`, and the now
unused `AppService` dependency.
Legacy ngax UI only; the new ngclient UI does not have this button. No backend,
API, cancellation, or restore behavior is changed.
Fixes#4453
When running `test-filters --parameters-file=...`, the parameters file's
--source entries were dropped and its --target was injected as the (only)
positional path, so filter testing ran against the backend URL instead of the
real source paths. The parameters-file handling only appended --source for the
`backup` command and always treated the first positional argument as a target.
test-filters/test-filter take source paths as positional arguments and have no
target, so the handling now appends --source for those commands (as it does for
backup) and does not inject --target (reported at verbose level, consistent with
how a --source on a non-backup command is already handled).
Adds an Issue4812 regression test driving Program.RunCommandLine.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This PR updates the logic for metadata update after an operation has completed, such that the data displayed in the UI is more consistent with the remote storage situation.
This PR adds a new locking service that keeps track of which local databases are currently in use by the server. If a database is in use due to a running task, immediate calls will now be rejected.
Prior to this change, immediate calls could interfere with the running task, causing a backup to fail abruptly if the user attempted to check the logs or restore.
TempFile.GenerateUniqueName (DEBUG only) builds each name from the caller, a
second-precision timestamp, and the first 8 characters of a Guid, then adds it
to a static tracking dictionary. When many temp files are created within the
same second by the same caller (e.g. BorderTests.Run10kTzstdCompressionAsync,
which writes ~10k blocks), the 8-char Guid prefix can collide, so the generated
name repeats: Dictionary.Add throws "An item with the same key has already been
added", and two TempFiles would otherwise share the same path. Release builds
are unaffected (full Guid, no dictionary).
Append a process-wide atomic counter to the generated name so it is always
unique regardless of timestamp granularity or Guid-prefix collisions. Expose
GenerateUniqueName as internal (Library.Utility already grants InternalsVisibleTo
to the unit-test project) and add a concurrency stress test asserting the
generated names never throw and are all unique.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bumps the CI actions to their latest majors to clear the recurring "Node.js 20
is deprecated" warnings (the pinned actions bundled Node 20, which GitHub is
retiring on runners):
- actions/checkout v4 -> v7
- actions/setup-dotnet v4 -> v5
- actions/upload-artifact v4 -> v7
- actions/setup-node v4 -> v6
- actions/stale v9 -> v10
atomicjar/testcontainers-cloud-setup-action is already on the latest major (v1).
Only the action version tags change. All bumped majors run on Node.js 24 and
require Actions Runner >= v2.327.1, which the GitHub-hosted runners used here
always satisfy. checkout v7's fork-PR checkout block only applies to
pull_request_target/workflow_run (not used here), and setup-node's auto-cache
only triggers with a packageManager field in package.json (not present), so
behavior is unchanged. Validated with actionlint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A backup can have two notions of its local database path: the stored
Backup.DBPath field and a --dbpath advanced option in its settings. The runner
computes an effective path that lets --dbpath override Backup.DBPath, so most
operations honor it. But the "Show log", "Show remote log" and "Delete database"
endpoints read Backup.DBPath directly, so when --dbpath differs they open/delete
the wrong database - "Show log" then fails with "no such table: LogData".
Add a shared Runner.GetEffectiveDBPath(IBackup) helper (same precedence as
Runner.ApplyOptions) and use it in ExecuteGetLog, ExecuteGetRemotelog and
ExecuteDeleteDb so all operations agree on the database file. Adds a unit test
for the precedence.
The database move/update endpoints are intentionally left unchanged, as they
manage the DBPath field itself.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This PR is a security hardning change and will break some setups.
The hardning is done to ensure the datafolder is not compromised and particularly that it is not maliciously seeded.
The changes now **requires** that the datafolder is locked down with exact permissions, or Duplicati will refuse to use it. This is a change to prior functionality where the folder would simply be locked down if it was not already locked.
Previously, a file named `insecure-permissions.txt` could be placed in the folder to instruct Duplicati not to set permissions. With this change that file is no longer supported.
The only way to opt-out of the permission check is to supply `--allow-insecure-datafolder` or the environment variable `DUPLICATI__ALLOW_INSECURE_DATAFOLDER`. Once either of those are set, Duplicati will no longer check permissions and accept folders with other permissions.
The setting can be applied to `preload.json`, but the file needs to be in a trusted location, which is either the directory with the binaries or the file pointed to by `DUPLICATI_PRELOAD_SETTINGS`. The setting cannot be placed in a `preload.json` file that is in an insecure folder as the folder is not read if the permissions are not correct.
The previous paths `/usr/local/share/Duplicati/preload.json` and `C:\ProgramData\Duplicati\preload.json` are no longer supported as they cannot be guaranteed to be locked down.
A `preload.json` in the datafolder is still supported, provided the folder passes the permission check (or the `--allow-insecure-datafolder` is passed to the executable).
For most users, this should not cause any problems as Duplicati has been locking down the folders, so they should be correctly locked down already.
The configuretool has been updated with a `secure-datafolder` command that can be used to force the correct permissions on a folder, in case the folder did not have the correct permissions.
For some Docker setups it may not be possible to set the permissions and the check will fail. These setup will need to apply `DUPLICATI__ALLOW_INSECURE_DATAFOLDER=true` in the image to run without the protections.
The `find` and `list` commands share the same handler. A bare filename is
rewritten to a wildcard filter (e.g. `*/report.txt`), and the handler only
searches every version for a Simple filter or when --all-versions is set, so a
wildcard filter searches only the newest version. The all-versions fallback in
Commands.List only triggers when the newest pass returns zero files, so a file
present only in an older version was silently missed whenever the newest version
still contained a same-named file (and full-path finds behaved differently from
bare-name finds).
Give `find` its own entry point that defaults to all-versions (respecting an
explicit --all-versions/--version/--time), so `find <name>` reliably searches
every backup version. `list` keeps its documented newest-only default. Update the
help text and add a regression test (Issue2287) that a file present only in an
older version is found by `find` but not by `list`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Log the "transaction already completed/disposed" cases as warnings instead of
verbose messages, so they are captured in tests and the underlying issue can be
fixed (per review feedback).
- Replace the fragile message-string matching in IsInactiveTransactionException
with an exception-type check. Microsoft.Data.Sqlite throws InvalidOperationException
("This SqliteTransaction has completed; it is no longer usable.") for an
already-completed transaction; matching the type avoids breaking under
localization or matching unintended messages (verified locally via the Issue5827
test, which catches System.InvalidOperationException).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The new restore engine uses two process-wide static TaskCompletionSources as
synchronization barriers in Restore/FileProcessor.cs: file_processor_continue
(the folder-metadata rendezvous) and priority_files_completed. Their counters
are reset at the start of each restore, but the TaskCompletionSources were only
created once at field initialization and were never reset.
After the first restore in a process completes the folder-metadata rendezvous,
its TaskCompletionSource stays completed, so on the second and later restores in
the same process the rendezvous "await" returns immediately and no longer waits
for all file content to be restored before folder metadata (including Windows
ACLs) is applied. This lost ordering guarantee (introduced by #6079) causes
intermittent folder metadata/permission restore failures, observed as the flaky
Issue6068FileAndFolderAttributesAndPermissions unit test on the Windows CI runner.
Reset both TaskCompletionSources in RestoreHandler before the FileLister/
FileProcessor tasks are started (so there is no race), alongside the existing
counter reset. file_processor_continue is the demonstrated root cause;
priority_files_completed has the identical never-reset pattern and is reset
defensively.
Verified locally on .NET 10: before the reset, file_processor_continue is already
completed on the 2nd+ restore in a process; the Issue6068 suite passes repeatedly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This addresses a logic issue in the previous check for TLS certs introduced with the option to ignore CRL failures. Since the call would end up calling `.Verify()` on the certificate, even toggling for soft-fail CRL issues, the certificate would be rejected.
With the new logic we manually verify the chain. This is only done if the user has requested specific certificates should be validated as we otherwise rely on the OS cert validation.
The macOS DiskImage unit tests intermittently failed in CI with:
System.InvalidOperationException : Process diskutil with arguments
eject /dev/rdiskN exited with code 1: Volume failed to eject
ReAttach() and CleanupDisk() called `diskutil eject` directly while a
volume could still be busy right after writing, so the eject transiently
failed (e.g. Test_GPT_FAT32_FAT32_FAT32_Async). This is test-infrastructure
only; no product code is affected.
Add a shared EjectDisk() helper that flushes buffers (sync), force-unmounts
all volumes first, then ejects with exponential backoff (reusing
Utility.GetRetryDelay), falling back to a forced `hdiutil detach` of the
image-backed device. Also make Unmount() force-unmount and return on success.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This ensures all in-flight backend requests are completed before the compact handler continues.
Prior to this, particular situations could cause the compact to throw an exception before actually removing files.
