* fix(scanner): never purge files under unreachable library roots An unreachable root is not a removed root. When one root of a multi-root library dies (unmounted share, dead drive) while another root still has files, the whole-library empty-root guard does not fire — the surviving root produced files — so the scan marks everything under the dead root missing_since (desired: hides it from browse/playback) and then, with the default scanner.empty_trash_after_scan=true + 24h file_removal_grace, the next scan after the grace hard-deletes every row under the dead root. A week-long drive outage silently destroys the root's entire catalog state: probe data, intro/credits markers, file hashes. Worse, membership reconciliation immediately purges media_items whose only files lived on the dead root, cascading user collections (library_collection_items has ON DELETE CASCADE) and deleting cached artwork. This change makes "temporarily offline" survivable: - Probe each configured root at scan start (os.Stat + IsDir + ReadDir, factored into the new internal/rootcheck package and shared with the admin mount-check endpoint). Unreachable roots are skipped by the walk but their scopes still reconcile, so files are still marked missing. - The trash sweep (DeleteMissingByFolder) now excludes rows whose path sits under an unreachable root, using the same exact-path + escaped prefix-LIKE matching as ListIDsOutsideRoots (a sibling root that merely shares a string prefix is never protected). With all roots reachable the emitted SQL is unchanged. - Membership removal still happens — browse/home hide items via media_item_libraries, so removal is what keeps a dead-root-only title out of the catalog — but the orphan media_items purge exempts items whose files sit under an unreachable root. Their metadata, artwork, and collection links survive; when the root returns, the upsert clears missing_since and syncPresentLibraryState re-inserts the membership, restoring the item with zero re-probing or re-matching. - The folder surfaces scan_warning_code='dead_root' with a message naming the unreachable roots; a fully healthy scan or a successful mount check clears it, mirroring empty_root. The admin UI shows a badge and banner. - Deliberate deletion is untouched: removing a path from the library config still purges via ListIDsOutsideRoots, files under reachable roots keep the exact 24h-grace purge, the empty-root guard and the autoscan dead-mount guard are unchanged. The audiobook/podcast/ebook reconcile paths share the same folder-wide sweep and orphan purge, so they get the same guard. Covered by tests: an end-to-end two-root scan (root dies -> rows survive a zero-grace sweep and warning is set; root returns -> rows resurrect with their original ids and the warning clears; deleting a file under a reachable root still purges), repo-level sweep-protection and sibling-prefix tests, orphan-purge exemption, and rootcheck unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scanner): probe uncompacted roots and take dead-root path on full outage Review follow-ups: (1) probe every configured path instead of the compacted traversal roots, so a nested child mount that dies under a reachable parent is still protected from the sweep; (2) when every configured root is unreachable, bypass the empty-root confirm flow (without consuming the one-time cleanup allowance), mark files missing, and raise dead_root instead of empty_root; (3) dead_root warning banner no longer shows empty-root confirm-deletion guidance as its fallback hint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(scanner): simplify dead-root protection plumbing - extract pathscope.CoverageClauses as the single builder for the exact-path + escaped prefix-LIKE root predicate; scanner's rootCoverageClauses delegates to it and catalog's excludeOrphansUnderProtectedPrefixes reuses it instead of hand-rolling the same clause loop - extract Scanner.sweepMissingAndReconcile to replace the identical trash-sweep + membership-reconcile + S3-image-cleanup block that was triplicated across the audiobook, ebook, and podcast scans (callers keep their flavor-specific log lines so messages stay constant) - add unreachableConfiguredRoots helper for the repeated probeUnreachableRoots(ctx, folder.ID, cleanScanRoots(folder.Paths)) expression in scanPaths and ScanFile - drop the unread Path field from rootcheck.Result - move the dead/empty-root warning text constants in AdminLibraries.tsx out of the middle of the import block Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): close dead-root protection gaps found in review Remediates the confirmed findings from the deep review of this PR: - Scoped audiobook scans (autoscan file events, subtree scans) ran the folder-wide sweep while probing only the scoped clone's Paths, so a healthy-subtree event could hard-delete a dead sibling root's rows. sweepMissingAndReconcile now reloads the folder's configured roots from the DB and probes them uncompacted, which also protects nested child mounts in the audiobook/ebook/podcast reconcilers. - A lost mount that leaves an empty, stat-able mountpoint probed as reachable and kept the historical purge timeline. A reachable root that is a literally empty directory while cataloged rows remain under it is now treated as suspect: rows are only marked missing, the sweep and orphan purge exempt it, dead_root is raised, and the mount-check endpoint reports it (additive suspect_empty field) instead of clearing the warning. Arming the one-time empty-cleanup allowance completes the deletion, including in the mixed case where other roots are healthy. Roots that still have directory entries keep the historical grace-then-purge path. - Confirmed empty cleanup (allow_empty_cleanup_once) no longer force-deletes rows under probe-dead roots: an outage is not a confirmation, so a dead sibling root's catalog survives a confirmed cleanout of a reachable empty root. - Root probes are now bounded (rootcheck.ProbeWithTimeout, 5s): a hung network mount degrades into the protected unreachable path with a probe_timeout error code instead of stalling every scan of the folder indefinitely. - Documented the cross-library limitation of the orphan-purge exemption next to the query it applies to. All behavior is pinned by new DB-backed tests (suspect-empty protection + confirmed completion, confirmed-cleanup dead-root survival, scoped/nested-root sweep protection, suspect-root query, probe timeout). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): address dead-root review findings --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
203 lines
5.2 KiB
Go
203 lines
5.2 KiB
Go
package rootcheck
|
|
|
|
import (
|
|
"context"
|
|
"os"
|
|
"path/filepath"
|
|
"sync/atomic"
|
|
"testing"
|
|
"time"
|
|
)
|
|
|
|
func TestProbeReachableDirectory(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
res := Probe(t.TempDir())
|
|
if !res.Reachable {
|
|
t.Fatalf("Probe(temp dir) = %+v, want reachable", res)
|
|
}
|
|
if res.ErrorCode != "" || res.ErrorMessage != "" {
|
|
t.Fatalf("Probe(temp dir) error fields = %q/%q, want empty", res.ErrorCode, res.ErrorMessage)
|
|
}
|
|
}
|
|
|
|
func TestProbeMissingPath(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
res := Probe(filepath.Join(t.TempDir(), "does-not-exist"))
|
|
if res.Reachable {
|
|
t.Fatal("Probe(missing path) reported reachable")
|
|
}
|
|
if res.ErrorCode != ErrCodeNotFound {
|
|
t.Fatalf("ErrorCode = %q, want %q", res.ErrorCode, ErrCodeNotFound)
|
|
}
|
|
}
|
|
|
|
func TestProbeRegularFile(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
path := filepath.Join(t.TempDir(), "file.txt")
|
|
if err := os.WriteFile(path, []byte("x"), 0o644); err != nil {
|
|
t.Fatalf("write file: %v", err)
|
|
}
|
|
|
|
res := Probe(path)
|
|
if res.Reachable {
|
|
t.Fatal("Probe(regular file) reported reachable")
|
|
}
|
|
if res.ErrorCode != ErrCodeNotDirectory {
|
|
t.Fatalf("ErrorCode = %q, want %q", res.ErrorCode, ErrCodeNotDirectory)
|
|
}
|
|
}
|
|
|
|
func TestProbeUnreadableDirectory(t *testing.T) {
|
|
t.Parallel()
|
|
if os.Getuid() == 0 {
|
|
t.Skip("running as root; permission bits are not enforced")
|
|
}
|
|
|
|
path := filepath.Join(t.TempDir(), "locked")
|
|
if err := os.Mkdir(path, 0o000); err != nil {
|
|
t.Fatalf("mkdir: %v", err)
|
|
}
|
|
t.Cleanup(func() { _ = os.Chmod(path, 0o755) })
|
|
|
|
res := Probe(path)
|
|
if res.Reachable {
|
|
t.Fatal("Probe(unreadable dir) reported reachable")
|
|
}
|
|
if res.ErrorCode != ErrCodePermissionDenied {
|
|
t.Fatalf("ErrorCode = %q, want %q", res.ErrorCode, ErrCodePermissionDenied)
|
|
}
|
|
}
|
|
|
|
func TestProbeWithTimeoutReachableDirectory(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
res := ProbeWithTimeout(context.Background(), t.TempDir(), DefaultProbeTimeout)
|
|
if !res.Reachable {
|
|
t.Fatalf("ProbeWithTimeout(temp dir) = %+v, want reachable", res)
|
|
}
|
|
}
|
|
|
|
func TestProbeBoundedTimesOutOnHungProbe(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
release := make(chan struct{})
|
|
t.Cleanup(func() { close(release) })
|
|
res := probeBounded(context.Background(), 10*time.Millisecond, func() Result {
|
|
<-release // simulate a stat/readdir blocked on a hung mount
|
|
return Result{Reachable: true}
|
|
})
|
|
if res.Reachable {
|
|
t.Fatal("hung probe reported reachable")
|
|
}
|
|
if res.ErrorCode != ErrCodeTimeout {
|
|
t.Fatalf("ErrorCode = %q, want %q", res.ErrorCode, ErrCodeTimeout)
|
|
}
|
|
}
|
|
|
|
func TestProbeBoundedHonorsContextCancellation(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
ctx, cancel := context.WithCancel(context.Background())
|
|
cancel()
|
|
release := make(chan struct{})
|
|
t.Cleanup(func() { close(release) })
|
|
res := probeBounded(ctx, time.Minute, func() Result {
|
|
<-release
|
|
return Result{Reachable: true}
|
|
})
|
|
if res.Reachable || res.ErrorCode != ErrCodeTimeout {
|
|
t.Fatalf("canceled probe = %+v, want unreachable/%s", res, ErrCodeTimeout)
|
|
}
|
|
}
|
|
|
|
func TestProbeBoundedReturnsFastResult(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
res := probeBounded(context.Background(), time.Minute, func() Result {
|
|
return Result{Reachable: false, ErrorCode: ErrCodeNotFound, ErrorMessage: "Path does not exist"}
|
|
})
|
|
if res.Reachable || res.ErrorCode != ErrCodeNotFound {
|
|
t.Fatalf("probeBounded passthrough = %+v, want the probe's own result", res)
|
|
}
|
|
}
|
|
|
|
func TestProbeCoordinatorCoalescesBlockedPath(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
var calls atomic.Int32
|
|
release := make(chan struct{})
|
|
t.Cleanup(func() { close(release) })
|
|
coordinator := newProbeCoordinator()
|
|
probe := func(string) Result {
|
|
calls.Add(1)
|
|
<-release
|
|
return Result{Reachable: true}
|
|
}
|
|
|
|
results := make(chan Result, 2)
|
|
for range 2 {
|
|
go func() {
|
|
results <- coordinator.probe(context.Background(), "/hung", 20*time.Millisecond, probe)
|
|
}()
|
|
}
|
|
for range 2 {
|
|
if result := <-results; result.Reachable || result.ErrorCode != ErrCodeTimeout {
|
|
t.Fatalf("coalesced blocked probe = %+v, want timeout", result)
|
|
}
|
|
}
|
|
if got := calls.Load(); got != 1 {
|
|
t.Fatalf("underlying probe calls = %d, want 1", got)
|
|
}
|
|
}
|
|
|
|
func TestProbeManyPreservesOrderAndBoundsConcurrency(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
var active atomic.Int32
|
|
var peak atomic.Int32
|
|
probe := func(_ context.Context, path string, _ time.Duration) Result {
|
|
current := active.Add(1)
|
|
for {
|
|
observed := peak.Load()
|
|
if current <= observed || peak.CompareAndSwap(observed, current) {
|
|
break
|
|
}
|
|
}
|
|
time.Sleep(10 * time.Millisecond)
|
|
active.Add(-1)
|
|
return Result{Reachable: true, ErrorMessage: path}
|
|
}
|
|
|
|
paths := []string{"one", "two", "three", "four"}
|
|
results := probeMany(context.Background(), paths, time.Second, 2, probe)
|
|
if got := peak.Load(); got > 2 {
|
|
t.Fatalf("peak concurrent probes = %d, want at most 2", got)
|
|
}
|
|
for i, result := range results {
|
|
if result.ErrorMessage != paths[i] {
|
|
t.Fatalf("result[%d] = %q, want %q", i, result.ErrorMessage, paths[i])
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestProbeReportsEmptyDirectory(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
dir := t.TempDir()
|
|
res := Probe(dir)
|
|
if !res.Reachable || !res.Empty {
|
|
t.Fatalf("Probe(empty dir) = %+v, want reachable and empty", res)
|
|
}
|
|
|
|
if err := os.Mkdir(filepath.Join(dir, "sub"), 0o755); err != nil {
|
|
t.Fatalf("mkdir: %v", err)
|
|
}
|
|
res = Probe(dir)
|
|
if !res.Reachable || res.Empty {
|
|
t.Fatalf("Probe(non-empty dir) = %+v, want reachable and not empty", res)
|
|
}
|
|
}
|