20 Commits

Include renames

Author	SHA1	Message	Date
3 people	42602b7896	feat(policy): access groups + embedded OPA policy engine with decision audit log (#282) ... * docs(policy): add OPA policy engine design spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(deps): add OPA v1.18.2 SDK for the policy engine Pulls github.com/open-policy-agent/opa v1.18.2 (policy engine core for the upcoming internal/policy subsystem) and the transitive upgrades go mod tidy applied (otel 1.44, grpc 1.81.1, prometheus/common 0.67.5). Full build verified. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add OPA engine core, vendor scope policy, and parity suite New internal/policy package (dead code — nothing wires into request paths yet): prepared-query Engine with 25ms eval timeout and fail-closed decode, typed PDP.ResolveViewerScope, go:embed vendor bundle, capabilities lockdown for future admin-authored Rego, and vendor scope.rego reproducing access.Resolver.Resolve (library intersection, disabled-library handling, quality/rating ceilings) with a narrowing-only silo_custom.scope.override extension hook. Parity proven by 1368 dual-execution subtests against the real access.Resolver, including the nil-vs-empty AllowedLibraryIDs battery and quality/rating variation; rank tables are test-pinned to internal/access. Rego unit tests run via opa/v1/tester inside go test. Bench: ~106µs/op per scope decision incl. input marshaling. Also restores the OPA requirement to go.mod (the earlier deps commit ran go mod tidy before any import existed, so tidy dropped it). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed, corrected (quality.allowed raw-file-rank divergence), and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy document store, foundation schema, and compile-check policy_foundation migration: policy_documents (one enabled doc per domain via partial unique index — two enabled docs would define override twice and conflict at eval), immutable policy_document_versions, single-row policy_generation counter, and the partitioned policy_decisions log table (daily range partitions, no FK, denial partial index). PolicyStore: transactional version numbering (FOR UPDATE), activation that verifies compiled_ok and bumps the generation in the same tx, enable/disable with typed ErrDomainAlreadyEnabled, and a delete guard for documents with an active version. CompileCheck sandboxes admin Rego: locked capabilities (no http.send/net.*/opa.runtime), enforced silo_custom.<domain> package path, vendor+stub layering, 2s budget, structured row/col errors. Engine gains NewEngineWithCustom / NewEngineFromStore with WARN-and-skip for invalid custom rows. DB-backed tests verified against a migrated Postgres (concurrent version numbering, atomic generation bumps, activation guards). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (domain constants extracted). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy System lifecycle with hot reload and cross-node invalidation policy.System owns one long-lived Engine and reloads it in place when policy documents change: EventPolicyChanged on the existing ChannelAdmin bus (new cache event constant) plus a 60s generation-poll fallback for Redis-less deployments, with a generation-consistent snapshot read. Vendor compile failure is startup-fatal; store/custom failures degrade to vendor-only and the poll loop heals them; runtime reload failures keep the last known-good engine. NotifyChanged gives the future admin handlers synchronous local reload + cross-node publish. Wiring: constructed in integrated/api modes only, PolicySystem field on api.Dependencies (unused by routes yet), policy.eval_timeout_ms setting (hot-reloaded via configWatcher.OnChange; default 25ms). Verified by a full server boot smoke and DB-backed convergence tests (event + poll paths, degraded boot, last-known-good). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add async decision logging with sampling, retention, and query repo DecisionLogger batch-inserts each node's policy decisions straight to the partitioned policy_decisions table via a non-blocking buffered channel (drop-and-count on overflow — logging never adds latency to or fails a decision). Scope decisions sample 1-in-N (default 50, setting policy.decision_log_scope_sample_rate); denials and eval errors always log; input/result JSON samples only at policy.decision_log_verbosity= verbose. Cursor-paginated DecisionRepository backs the upcoming admin log viewer. Retention via partman (daily partitions) and a PolicyDecisionLogCleanupTask honoring policy.decision_log_retention_days (default 14). PDP emits entries per evaluation; the System owns the logger lifecycle and settings hot-reload. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (removed an unused, unsynchronized PDP setter). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): add admin policy management API and capability endpoint /api/v1/policy/capability (authenticated feature detection) plus the acting-admin /api/v1/admin/policy surface: vendor Rego viewer, document CRUD with the one-enabled-per-domain conflict mapped to 409, immutable version creation (compile-checked; failed versions persist as audit history with structured row/col errors and can never activate), activate/rollback with synchronous reload + cross-node invalidation via System.NotifyChanged, stateless validate, throwaway-bundle simulate (never touches the live engine, never logs decisions), and cursor-paginated decision-log queries. Routes mount only when the policy system is wired, keeping proxy/transcode modes untouched. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (seeded the FK'd test user; replaced an unchecked fmt.Sscanf with strconv). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add /admin/policy workspace with Rego editor, simulate, and decision log New Policy admin page (System nav group): documents list with one-enabled-per-domain conflict handling, CodeMirror 6 Rego editor (hand-rolled StreamLanguage mode) with server compile issues rendered as inline lint diagnostics, explicit Save-version vs Activate flow with confirm, read-only vendor module viewer, simulate panel with seeded example inputs, version history with rollback, and a cursor-paginated decision-log browser. Capability-gated via /policy/capability. Adds the three decision-log settings to Log Retention. First code-editor dependency in web/ (@uiw/react-codemirror + @codemirror/*), decided in the design spec. Implementation drafted by Codex (GPT-5.5) via codex exec; verified here (lint, format:check, tsc --noEmit, vitest policy suites). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for viewer scope resolution policy.ViewerResolver implements the ViewerResolver interface backed by PDP.ResolveViewerScope and replaces access.Resolver at all five construction sites: router viewer middleware, notifications scopes, the reconciler, jellycompat's scope filter, and the ABS resolver (which now accepts a pre-built resolver, preserving its PIN-at-login semantics). PIN/profile-token verification and disabled-library loading are extracted into shared exported helpers used by both implementations, so the legacy resolver stays compiled as the parity reference with identical behavior. The adapter lives in internal/policy (which already depends on internal/access transitively) — direct typed PDP calls, no new import cycle. Sites without a policy system (proxy modes, bare test routers) keep the legacy resolver until the cleanup phase. Verified: full test suite green (jellycompat TestBeginWebOperation* and one playback GPU test are pre-existing failures, confirmed identical on main), 1368-case parity suite, dedicated ViewerResolver parity/PIN/ nil-vs-empty/fail-closed tests, and a full server boot smoke. Implementation drafted by Codex (GPT-5.5) via codex exec; a first-pass reflection-based adapter was rejected and reworked into the typed in-policy adapter; reviewed line-by-line and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for acting-admin and permission gates vendor/permission.rego reproduces the acting-admin rule (admin role + primary-profile-or-none), HasEffectivePermission semantics for marker_edit, and the metadata-curation rule including the subtle admin-past-refused-bypass case that requires the explicitly ASSIGNED permission. Policy-backed middleware in policy_gates.go keeps all Go-side lookups (declared-profile primary check, item->library resolution, the 404-on-unknown-item path) and preserves the legacy status/body taxonomy exactly — proven by dual-execution middleware tests that run every scenario through both implementations and assert byte-equal responses. Permission decisions always log (allowed flag populated); simulate and the capability endpoint gain the permission domain automatically via the domain registry. Router swaps behind single constructor choice points with the legacy gates retained for policy-less wiring. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for download and playback admission decisions vendor/action.rego decides download eligibility (downloads enabled + user allowed), download-transcode eligibility (transcode enabled + user allowed + artifacts available), and playback admission (stream/transcode counts vs limits, zero = unlimited), with a tightening-only silo_custom.action override that can also clamp a quality ceiling (never widen — merged via quality.min). Go keeps everything stateful: config loading, preset-ladder enumeration, and live session counting. Downloads consult an optional ActionDecider (nil = legacy logic) mapped back to the existing sentinel errors and capability response. Playback gains a minimal AdmissionDecider hook at the exact point of the legacy limit comparison: counts snapshot under the session mutex, PDP evaluated OUTSIDE the lock, then revalidated under lock before insert (retry on count drift) — no admission ever decided on stale counts and no eval under the mutex. Deny reasons map to the legacy ErrTooManyStreams / ErrTooManyTranscodes sentinels, pinned by tests. Parity: combination tables driven against the real PresetsFor / ensureTranscodeAllowed / SessionLimits math; full suite green (known pre-existing jellycompat flakes only). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (locking design verified line-by-line) and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): satisfy tsc -b strict return typing in the Rego stream tokenizer The production build (tsc -b) rejects assigning CodeMirror's string | void next() result to string | undefined; tsc --noEmit did not catch it. Restructured the string-literal loop. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): clearer error when a decision is undefined for partial input Vendor policies index required input fields directly, so a hand-written simulate payload missing fields yields an undefined decision. Surface that as 'decision X is undefined for this input (missing required input fields?)' instead of 'empty result' — found while exercising the simulate API against a live server. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(web): set changeOrigin automatically when the API proxy target is remote Remote dev backends sit behind vhost-routing proxies that reject a localhost Host header; local targets keep the existing pass-through behavior. Enables pointing the Vite dev server at a hosted backend via VITE_API_PROXY_TARGET in web/.env.local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): redesign the policy workspace around the decision pipeline The first-pass UI was structurally generic: a five-column document table squeezed beside the editor, three equal-weight action buttons with hidden preconditions, raw version IDs, and jargon copy — nothing taught the model. The page now teaches it: - A pipeline strip states the mental model up front: Silo decides the baseline -> your overrides narrow it -> every decision is logged. Tabs renamed to Overrides / Baseline / Decision Log (ids stay stable for bookmarked URLs). - The document table becomes one card per domain (Library visibility / Admin & permissions / Downloads & playback) with plain-language descriptions, example rules, status pills (Live vN / Draft / Disabled), inline creation, and the enable kill-switch in place. - Selecting an override drills into a full-width editor with a visible lifecycle rail (Draft -> Validated -> Saved -> Live) and one contextual primary action per step; the unedited live source shows no actions until edited. Version comments appear only at the save step. - Simulate is reframed as 'Test before going live' with a human verdict chip (Allowed / Denied — reason / ceiling summary) above the raw JSON; internal generation counters no longer surface. - History uses 'Make live' with plain go-live copy; authors read 'User N'; the baseline tab explains that upgrades never touch overrides. Hand-written redesign (no Codex); verified via vitest, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): present the policy baseline as readable rules, not raw Rego The Baseline tab dumped five Rego modules into read-only editors. It now leads with what the rules actually do: one card per domain with plain-language statements of the shipped behavior and a note on what an override may change, plus content-rating and playback-quality tier ladders parsed live from the lib module sources (so the tiers shown are the ones the server enforces, not a hardcoded copy). The Rego source stays one click away behind a per-module accordion and remains the stated source of truth; unrecognized modules fall back to source-only. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(policy): add access-groups design addendum Groups with permission toggles become the everyday admin surface; the Rego editor is demoted behind policy.editor_enabled (default off). Restriction-only composition: group grants are an upper bound, per-user settings tighten further — same rule as the existing account/profile merge, one layer up. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): add access groups — group defaults with restriction-only composition New access_groups table + users.access_group_id (one group per user, NULL = today's behavior). Group grants are an upper bound composed with the user's own settings by strictest-wins rules — library intersection, MinQuality, AND'd booleans, strictest positive stream/transcode limits, permission-mask intersection, and a requests toggle gating CreateRequest. The merge happens in Go (access.ApplyGroupPolicy / EffectivePolicyForUser) before policy inputs are built, so vendor Rego, the parity suites, and the decision log are untouched; every enforcement surface (viewer scope in both resolvers, permission gates, downloads, playback admission, requests) consumes the effective policy and fails closed on provider errors. Changing a group's quality ceiling bumps its members' access_policy_revision, mirroring the per-user rule. Additive admin API: /admin/access-groups CRUD with member counts; PUT /admin/users/{id} + user DTOs gain access_group_id. Also demotes the Rego editor: policy.editor_enabled (default off, hot-reloaded) drives the capability endpoint's editor_available and 403-gates editor endpoints while the engine and decision logging keep running. Design: docs/superpowers/specs/2026-07-02-access-groups-design.md. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (composition core + fail-closed call-site audit) and verified here. DB-backed group-store tests pending local Postgres recovery. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add Access Groups admin page and gate the policy editor New /admin/access-groups: a card grid summarizing each group (member count + key restrictions), drilling into an editor that reuses the same LibraryAccessSelector and quality presets as the user editor, with toggles for downloads/transcoded-downloads/requests, concurrent-stream and transcode limits, and a permissions mask (all-assignable by default, narrowable to specific permissions). Delete warns how many members fall back to the built-in defaults. Copy states the composition rule up front: a group grants the most a member can do; their own restrictions still apply on top. The user editor gains a Group picker and read-only row; the Policy nav entry is now hidden unless the capability reports the editor enabled. Plumbing (types, hooks, user-editor picker, nav gating) drafted by Codex (GPT-5.5); the Groups page hand-built. Verified: 25 tests across the touched suites, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): seed a Default Group and auto-assign newly created users Adds access_groups.is_default with a partial unique index (one default at most — the profiles is_primary pattern) and seeds a permissive 'Default Group' whose ceiling is a no-op, so assignment never changes anyone's effective access until an admin edits it. The seed is guarded against pre-existing defaults and name collisions; the Down migration only removes the row if it is still untouched. Assignment happens at the single INSERT INTO users choke point (UserRepository.Create): when no explicit group is given, access_group_id is filled by a scalar subquery on the default flag — NULL when no default exists. Every creation path (setup, signup, invites, OAuth, admin create) is covered by construction. Setting a new default via the API atomically clears the previous one in the same transaction. Deleting or unsetting the default is legal: new users then start with no group, which is pre-feature behavior. Implementation drafted by Codex (GPT-5.5); migration guards and the choke-point subquery reviewed line-by-line here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): surface the default access group Cards show a Default badge; the group editor gains a 'Default for new users' toggle (with copy noting existing users are never moved); the delete dialog warns when removing the default that new accounts will start with no group. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): ship the Default Group with house-rule ceilings Seed values per product decision: 5 concurrent streams, 5 transcodes, transcoded downloads off, and a permission mask of marker_edit only (metadata curation excluded). Plain downloads and requests stay on. The Down guard matches the new values so it still only removes an untouched seed row. Only newly created users are affected; existing users are never assigned. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): retire per-user defaults — the Default Group is the sole default policy Removes both legacy 'user defaults' mechanisms now that the seeded Default Group owns new-user policy: - users.max_streams / max_transcodes column defaults drop from 6/2 to 0 (= unrestricted at the user layer), so group ceilings apply to new signups/invites/OAuth users instead of fighting stale per-user numbers. Existing rows keep their stored values — nobody is silently uncapped on upgrade. - The dead defaults.max_playback_quality / defaults.max_profiles settings validation goes away with its only writer (the User Defaults dialog, removed on the web side). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): replace the User Defaults dialog with group-governed creation The Users page's 'User Defaults' dialog (defaults.* server settings) duplicated what access groups now do properly, and its values were only ever form prefill — no backend path applied them. The button now links to Access Groups, and the create-user form seeds unrestricted user-layer values (0 streams/transcodes, any quality, downloads allowed) so the member's group governs; per-user fields remain for tightening individual users. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): migrate existing non-admin users into the Default Group Existing users join the seeded Default Group on upgrade so one policy source governs the whole instance. Their per-user limits still holding the retired 6/2 column defaults are normalized to 0 in the same statement so the group's ceilings actually apply; deliberately customized values are preserved. Admin accounts stay ungrouped — scope/action decisions are role-blind, so grouping an admin would cap the server owner on upgrade. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): keep admins out of the Default Group and treat group moves as policy changes New-user creation now mirrors the migration's admin exclusion: the default access group is only auto-assigned to non-admin roles, so a fresh server owner no longer inherits the starter group's transcode denial and stream caps. Changing a user's access group now bumps access_policy_revision (the group carries permissions, quality, and limits, exactly like the per-user fields that already bump it) and triggers admin session revocation when the group actually changes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): enforce marker_edit through the PDP on marker write routes The Rego permission policy owned marker_edit but no Go caller ever consulted it: PUT/DELETE /markers went through a handler-local check that short-circuited admins and read only the user's own permissions, so group permission masks and custom policy overrides were ignored. Marker writes are now gated by router middleware like the other permission surfaces: a PDP-backed RequireMarkerEdit that evaluates the group-merged effective permissions (plus the legacy variant for proxy/test wiring without a policy system). The handler-local check and its user loader are gone. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert device/quality policy facts and honor the quality ceiling The download_transcode action check hard-coded an empty device ID and never asserted the requested quality, and no caller consumed ActionDecision.QualityCeiling — custom download policies keyed on those inputs were silently ineffective. Resolve now threads the request's device ID and requested quality into the action input, and a returned quality ceiling downscales the prepared transcode target (the ceiling applies to what is served, matching the serve-time rule in serveDownloadBytes). FileQuality and the content-rating pair stay intentionally empty for downloads — documented on downloadActionInput: those ceilings are enforced against the served artifact by the scope-derived access filter, and asserting the source's quality would wrongly deny capped transcodes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(access): align the default-group seed assertions with the migration The DB test still asserted the earlier no-op seed (transcode allowed, unlimited streams/transcodes, null permissions); the shipped migration seeds transcode denied, 5/5 limits, and marker_edit-only permissions, so the test failed on any database with the migration applied. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): lock the Rego sandbox by builtin purity and bound compile work Exclude every nondeterministic builtin from the admin sandbox instead of denylisting names, so OPA upgrades cannot silently expose impure builtins while pure helpers like net.cidr_contains stay usable. Apply the same capabilities to the runtime engine, cap concurrent compile checks, and reject oversized sources before they reach the uncancelable compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): require literal booleans in vendor override and input checks Bare object.get truthiness treated any non-false value as satisfied, so a malformed override 'allowed' value could fail to tighten a base grant and hand-crafted simulate input could flip flag predicates. Compare against literal true so anything else denies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): surface decision log cleanup failures to the task manager CleanupDecisionLogsOnce now returns the first error alongside the deleted count so a broken partition manager or DB outage marks the scheduled task failed instead of reporting 100% success while policy_decisions grows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): log admission decider errors before failing closed A policy-evaluation failure was silently mapped to the too-many-streams denial, making an engine outage indistinguishable from a real limit hit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): nil-guard the downloads user and restore the ABS legacy resolver effectiveDownloadUser dereferenced policy state before its nil-user check, and the ABS handler lost viewer-scoped filtering entirely when the policy system was unavailable because no legacy access.NewResolver fallback was wired like the other resolver paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): address admin policy review feedback - invalidate the version query by version_number, the key usePolicyVersion actually caches under - keep the goPrevious cursor-stack updater pure (Strict Mode double-invoke) - make version history rows keyboard-selectable like the document list - clamp download_transcode_allowed when downloads are disabled so groups cannot save a contradictory record Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): cap policy endpoint request bodies at 1 MiB The policy write endpoints (create document/version, set enabled, validate, simulate) decoded JSON bodies without a size limit, so an oversized payload buffered fully in memory before CompileCheck's 256 KiB source cap could reject it. Route all five through a shared decodePolicyRequest helper that wraps the body in http.MaxBytesReader and returns 413 with the repo's standard too_large error shape. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(access): forbid deleting or demoting the default access group Deleting the default group (or unsetting its is_default flag) left the server with no default: new non-admin users were then created ungrouped with max_streams/max_transcodes of 0 — unlimited — because the legacy per-user column defaults were retired in favor of the group's ceilings. The store now rejects both operations with ErrDefaultGroupRequired (mapped to 409); promoting another group remains the supported way to move the default, and atomically clears the previous one. The admin UI disables the delete button and the default toggle on the default group and explains the promote-another-group flow. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(web): keep unsaved policy drafts when a newer version activates elsewhere The editor state was keyed on the active version's id/sha, so a background refetch after another admin (or another tab) activated a version remounted the editor and silently discarded the dirty draft. PolicyEditorPanel now pins the seed it is editing against and only adopts an incoming seed when nothing can be lost: the editor is clean, the draft already equals the incoming source (the same-admin activate flow), or the selection moved to a different document. Otherwise the pinned editor stays mounted and an inline notice offers an explicit "Load live version" action. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(policy): fail reloads on invalid custom sources and surface degraded/apply state A stored custom source that stops compiling used to be silently skipped on reload: the bundle widened to vendor-only for that domain while the generation reported fully applied. Reload is now strict — a bad enabled source fails the reload and the last known-good engine keeps serving. Boot keeps its vendor fallback for availability, but skips are recorded on the engine and exposed (with store-outage reasons) through System.DegradedState and additive degraded fields on GET /policy/capability. Activate/SetEnabled re-run CompileCheck instead of trusting the stored compiled_ok flag. Mutation endpoints also no longer conflate persistence with live apply: activation/enable responses carry additive applied/failed_step/ loaded_generation fields and return 202 when the store change persisted but the local reload failed. Addresses review findings C1, C2, and the degraded-signal gap (6.1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): type deny reasons across the contract and enforce profile_verified Deny handling used to branch on exact free-text reason strings in three Go consumers, and playback reported ANY unrecognized reason — including custom override free text and engine failures — as a stream-limit error. Decisions now carry a stable reason_code (custom overrides always get custom_denial); downloads, the metadata-curation gate, and playback admission switch on codes, with a new ErrPlaybackNotAllowed -> 403 playback_not_allowed mapping for non-limit denials. Rego tests pin every vendor code. The scope contract's tighten-only profile_verified output was also emitted but never consumed; a policy revocation now surfaces as ErrProfileUnverified (403 profile_unverified) instead of silently proceeding. Addresses review findings 6.2 and C4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): close the dual-library disabled-scope bypass in direct item authorization EnsureAccessible, EnsureAccessibleIDs, and FilterAccessibleContentIDs gated library access with allow/deny predicates over a single joined media_item_libraries row, so an item linked to BOTH a passing library and a disabled one satisfied the disabled check via the passing row — a direct-ID bypass of disabled-library scope on the detail, media-file, playback, and download paths. All library access predicates now share one helper (libraryAccessConditions) emitting independent EXISTS / NOT EXISTS subqueries, the semantics GetByIDsWithAccess already used, including the orphan-item membership guard for disabled-only scopes. SQL-shape tests pin every builder and a DB-gated regression test covers the dual-library item end to end. Addresses review finding C3 (plus the same shape in buildFilterAccessibleContentIDsSQL, which the review did not flag). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): serialize quota check and row creation under a per-user advisory lock The concurrent-download quota was check-then-insert with nothing serializing the pair: parallel creates could all observe free quota before any row existed, bypassing the cap and stacking artifact encode jobs. All four check->insert spans (ephemeral original, artifact-backed, series batch, managed batch) now run inside Repository.WithUserQuotaLock — a pg_advisory_xact_lock keyed by user, so the serialization holds across nodes. The artifact path keeps the limiter-before-Ensure ordering (a rejected request must not leave an encode job behind) by holding the lock across Ensure. Managed-entry replacement stays quota-exempt and lock-free. A DB-gated barrier test races 8 creates against a cap of 1. Addresses review finding C5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert served quality at create time for original and remux downloads Direct-original and remux downloads serve the source resolution unchanged, but create-time policy checks left file_quality empty — an over-ceiling source registered a row serveDownloadBytes could never satisfy. Resolve now runs a final download action check with FileQuality populated on those two paths (capped transcodes keep the ceiling-on-artifact behavior), a custom override ceiling below the served resolution denies, and quality_ceiling_exceeded maps to ErrQualityUnavailable. The ActionInput contract now documents exactly when file_quality and the rating facts are supplied so custom policy authors are not misled. Addresses review finding C6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): guard activation against slow overrides and make eval timeouts observable A custom scope override that exceeds the 25ms eval budget compiled fine, activated fine, and then converted to 500s on every authenticated request — server-wide lockout authored in the admin editor. Activation and enable now run GuardEvalCost: the candidate source is evaluated on a throwaway engine against a canned representative input under the live budget, and a source that cannot complete is rejected 422 with ErrPolicySlowEval before it goes live. Runtime timeouts keep failing closed but now carry a distinct ErrPolicyEvalTimeout sentinel, an Error log, and a per-engine counter exposed as eval_timeouts on GET /policy/capability so intermittent near-budget policies are attributable. Addresses review finding C7. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt remediation files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-05 17:38:19 -04:00
3 people	9cae868a27	feat(downloads): offline sync for mobile — downloads v2 (#258) ... * feat(downloads): offline sync for mobile (downloads v2) Replace internal/download with a unified internal/downloads package and add fully-offline download + watch-sync support for mobile clients, across five independently-shippable phases: - Phase 0: reshape the downloads table and the /downloads contract to be device- and format-aware; add GET /downloads/capability; extend DownloadConfig (default-off keys); update the web download hooks/components in lockstep. This is the one approved pre-lock exception to the additive-only /api/v1 rule (the web app is the only consumer and is updated together). - Phase 1: managed device-library entries (create/list/PATCH/delete/serve), keyed on the X-Silo-Device-Id header. - Phase 2: offline playback manifest plus artwork/subtitle proxy endpoints that strip every presigned URL (inline thumbhashes + authenticated proxies). - Phase 3: prepare-to-file (remux + transcode-to-single-file) as a durable, leased artifact queue with startup recovery, hosted on the task manager; playback.PrepareFile emits one +faststart MP4. Adds the admin transcode toggle and per-artifact LRU cleanup. - Phase 4: offline progress reconciliation -- a clamped event_at LWW key plus a server-assigned synced_seq cursor on watch_progress; an optional clamped updated_at on POST /sync/progress and an opaque ?since= cursor on GET /progress (additive; existing callers unaffected). Security & reliability invariants, each with an acceptance test: 1. Server-owned sync ordering: ?since= delta delivery is driven only by the server-assigned synced_seq; the client clock is bounded (event_at, clamped to now+skew) and used only for last-write-wins on the caller's own profile. 2. Full profile+device authorization on every managed endpoint, with a per-profile content/library access re-check before serving any bytes/assets. 3. Durable artifact recovery: a transactionally-claimed (FOR UPDATE SKIP LOCKED), lease-heartbeat, attempt-counted queue with a startup sweep, so no crash strands a download in preparing and concurrent workers never double-encode. Migrations are timestamped Goose files: reshape downloads (device/format); download_artifacts (durable queue); watch_progress event_at/synced_seq. DB-backed acceptance tests skip without SILO_TEST_DATABASE_URL and run in CI; the invariant-1 progress test also runs against the real SQLite backend locally. Client repos (silo-android, silo-apple) consume the reshaped /downloads/* contract and the updated_at/?since= progress fields and require coordinated follow-up. Implements the maintainer-approved v1 capability proposal for offline sync (downloads v2). AI-use disclosure: implemented by Claude (Claude Code) from the approved design doc under docs/superpowers/specs, with human review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(downloads): series & season downloads + client-pull monitoring Build season downloads and a "monitor a series" capability on top of the downloads v2 (offline sync for mobile) work. Season downloads: - POST /downloads accepts season_number (with series:true) to download one season. CreateSeries/CreateSeason share one body via a listEpisodes closure and register managed entries under a shared batch_id (original-only). Episode files are resolved in a single batched query. Series monitoring (auto-download), client-driven: - New device-scoped download_subscriptions table with a Sonarr-style mode (all | future | latest_season | specific_seasons), a client-enforced delete_watched flag, and a max_storage_bytes cap. The server never deletes on-device files; retention and the hard cap are the client's, the server only soft-gates registration. - The client calls POST /downloads/subscriptions/sync on open / background refresh; the server registers the in-scope, not-yet-downloaded episodes (idempotent via the managed-entry unique index) and the device pulls them on its own schedule. No background worker and no dependency on the notifications subsystem. latest_season follows new seasons (>= subscribe-time season); future excludes the back catalog via air date. - Subscription CRUD + sync are profile+device authorized (device id from the X-Silo-Device-Id header only) with a per-request content-access re-check. The capability endpoint advertises season_download / series_monitoring / monitoring_modes. Also lands the downloads-v2 work already present in the tree: durable artifact (remux/transcode) preparation and offline watch-progress reconciliation, plus the design-spec updates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WIP: epitaxy pre-switch from feat/downloads-v2-offline-sync * test(downloads): fix deterministic ID collision in reconcile test Artifact IDs are time-sortable, so two artifacts created in the same moment share their first 8 chars; combined with a captured timestamp the two preparing-download IDs collided on downloads_pkey. Use the full artifact ID, which is unique per row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): support sqlite userdb backend for managed downloads With the sqlite userdb backend, profiles live only in per-user SQLite stores and public.user_profiles stays empty, so user_devices' profile FK made every managed create/subscription/offline-sync request fail with an FK violation. Drop the FK (shared Postgres tables must not FK profile tables — same rule as notifications) and replace the lost cascade with an app-level purge on profile deletion, wired through ProfileHandler for both backends. DB-backed regression tests cover the no-Postgres-profile-row path and the purge cascade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): dispatch encode kick asynchronously triggerDrain invoked the kick inline, and the kick (taskmanager RunTask) executes the encode task on the caller's goroutine — so a POST /api/v1/downloads with a bitrate quality blocked the HTTP request on the entire queue drain, ffmpeg encodes included, delaying the 202 by minutes on an idle queue. Dispatch the kick on a goroutine; the task manager already serializes concurrent runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): enforce per-user quota on the encode pipeline Two gaps let a user bypass MaxConcurrentPerUser entirely for prepared downloads: artifact-backed rows are created in 'preparing' (never 'queued'/'downloading'), which CountActiveByUser didn't count, and createArtifactDownload enqueued the encode job before limiter.Check, so even a 429-rejected request left a job the worker would transcode. Count 'preparing' as active and check the limiter before Ensure; managed replacements stay quota-exempt since they don't add a row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): protect ephemeral artifact links from LRU eviction HasActiveLink only counted managed (device_id IS NOT NULL) rows, so under a byte budget Cleanup could delete an artifact still referenced by a ready-but-unfetched ephemeral web download — permanently 404ing a row the API kept listing as ready (the artifact row is gone, so recovery can't re-queue it). Any non-terminal link now protects the artifact; only artifacts whose links are all cancelled/failed/revoked are evictable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): batch manifests skip bad entries instead of failing whole batch One deleted or access-filtered episode made GET /downloads/batches/{id}/manifests 404 for the entire season, so a client could no longer fetch manifests for the still-valid entries. Report unbuildable entries in a skipped[] array (revoked | not_found | error) alongside the delivered manifests, mirroring the create path's skip idiom. Also cut the batch cost: the shared series detail is resolved once per batch instead of once per episode, and buildSubtitles reuses the already-loaded media file instead of re-querying it per manifest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(migrations): wrap DO block in StatementBegin/End markers Under NO TRANSACTION goose splits statements on semicolons, so the dollar-quoted DO block failed every fresh install with 'unterminated dollar-quoted string' (SQLSTATE 42601). Already-applied databases are unaffected. Same fix is being applied to main; identical content merges cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): allow season 0 (Specials) in season downloads season_number was a plain int dispatched with '> 0', so requesting the Specials season was indistinguishable from omitting the field and silently broadened to a full-series download. Dispatch on pointer presence, treat 0 as the Specials season, and reject negatives with 400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): capability quality_presets is never JSON null PresetsFor returned a nil slice when downloads are disabled or the user lacks the permission, and Capability's []string{} initialization was immediately overwritten by it — so GET /downloads/capability serialized "quality_presets": null where the contract documents an array. Normalize at the source so every caller inherits the guarantee. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): subscription sync correctness + batched registration Three subscription fixes: - A paused subscription no longer syncs: PATCHing scope (or pausing and changing scope in one request) registered episodes for a monitor the user had just stopped, inconsistently with SyncSubscriptions' guard. - SubModeFuture compares calendar days (UTC): air_date is date-only, so the strict instant comparison permanently excluded episodes airing the same day the user subscribed; episodes with no air date now fall back to their ingest time instead of never registering. - Registration is one batched fetch (GetManagedEntriesByKeys) plus one batched INSERT ... ON CONFLICT DO NOTHING RETURNING (CreateManagedEntriesBatch) instead of a SELECT+INSERT per episode — a 300-episode series cost ~600 sequential round trips per request and every no-op sync re-walked the full set. RETURNING yields exactly the new rows, so the sync response's 'registered' count now honestly reports 0 in the steady state instead of the full in-scope count on every app open. The now-unused InsertManagedEntryIfAbsent is removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(userstore): stamp triggers own the event_at LWW key MarkProgressBatch (jellycompat series mark-played) advanced updated_at but never event_at, and both stamp triggers only defaulted event_at when NULL — so a queued offline event with a client time between the row's old event_at and the mark could win SetProgressIfNewer and resurrect a stale resume position that then re-synced to every device. Make the triggers authoritative instead of adding a tenth hand-written SET clause: whenever an UPDATE changes updated_at without explicitly changing event_at, the trigger advances the LWW key; writes that do set event_at (offline sync's clamped client event time) keep their value. Postgres gets a CREATE OR REPLACE migration; SQLite gets a v12 userdb migration that drops and reinstalls the trigger bodies (CREATE TRIGGER IF NOT EXISTS never replaces). Conformance tests cover both batch paths, the preserved-client-time invariant, and the v11→v12 upgrade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): lifecycle hygiene — squash migrations, dead status, stale-row sweeps Migrations: fold the 20260621 corrective migration back into the base Downloads V2 migrations (its columns/constraints already exist there) and fix the reshape Down, which re-added the narrow status CHECK without collapsing managed-lifecycle rows first — rollback aborted on any DB with preparing/ready/revoked rows; validated against a live row. Branch databases that applied the corrective migration need its version row removed: DELETE FROM goose_db_version WHERE version_id = 20260621020459. Code: drop the dead 'registered' status (nothing ever wrote it; the lifecycle is preparing -> ready; 'revoked' stays reserved for the planned admin revoke flow) along with unused KindDirect and ErrInvalidFormat. Sweeps: Cleanup now runs an age-based hygiene pass independent of the byte budget — cold terminally-failed artifacts (with .part leftovers), orphaned ready artifacts no download row references, and ephemeral web rows older than their convenience-record lifetime (also unpinning their artifacts and bounding GET /downloads growth). The byte budget remains the disk quota per the limits & restrictions design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): sync API doc with v2 fixes; HEAD on file route; Android handoff Document the contract changes from the review fixes: batch-manifest skipped[] shape, honest subscription 'registered' semantics, season 0 = Specials, always-array quality_presets, bytes_sent actual behavior, ephemeral 7-day retention, header-pairing requirement, progress-delta deletion caveat, and the ready/failed push event schema (new §9.4). Add an Android client handoff section (§11) mirroring the Apple one, register HEAD on /downloads/{id}/file for download stacks that probe before ranged GETs, and add season_number to the web create-request type. Flag the /direct-download session-token-in-URL tradeoff; a short-lived download-scoped URL is a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: consolidate download/progress helpers, prune dead code, gate sweeps Behavior-preserving consolidation from the Downloads V2 review: - appendVideoFilterArgs: one home for the burn-in/hwaccel -vf selection, shared by the HLS builder and the single-file prepare builder (the drift pattern that already bit tone-mapping once). - userstore.ResolveProgressState: one home for the min-resume/watched threshold rule, replacing five identical copies across both store backends and the offline-sync ingest. - Download file selection ranks resolutions via access.CompareQuality (adds 4320p, agrees with playback) instead of a private switch. - writeSubtitle uses the shared subtitles.SubtitleContentType mapping. - config.DefaultTranscodeDir replaces three '/tmp/silo-transcode' literals. - Read-side quality/revision defaulting helpers removed: insertArgs plus the NOT NULL/CHECK schema already guarantee the invariant. - Dead code removed: Repository.ListByUser, SubscriptionRepository. ListActiveBySeries, and the stale auto-register-worker comments (the design is client-pull; no worker exists). - Redundant left-prefix indexes dropped from the base migrations (their unique indexes serve the same prefixes). - recover()'s disk-presence sweep and the stale-row hygiene sweep run on startup then hourly instead of every 30s tick (both are O(cache size)). - gofmt/prettier fixes for pre-existing drift in handlers/playback.go and pages/Profiles.tsx. Deferred (noted for follow-ups): quality-ladder preset table collides with the drafted download limits & restrictions design, which specifies its own ladder helper; Download-literal construction consolidation and the managed-identity value object remain open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): draft download limits & restrictions design Design input for the follow-up v1 capability proposal (quality ceiling, batch size cap, per-user quantity/bandwidth overrides). Committed with downloads v2 because the remediation work explicitly defers the quality ladder refactor and revocation wiring to this spec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(progress): reject malformed updated_at; clamp negative progress inputs Review findings on #258: - A malformed (non-RFC3339) updated_at in POST /sync/progress previously parsed to the zero time, which clampEventAt treated as "now" — letting a stale offline event win LWW as a fresh server-time write. The item is now rejected with a per-item error instead. - ResolveProgressState now clamps negative position/duration before classification so no backend can persist negative progress through UpdateProgress/SetProgress. - The online-write event_at invariant test is table-driven over both SetProgress and UpdateProgress, which share the same contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): close review gaps — permission gates, file-access recheck, artifact-true manifests Review findings on #258: - UpdateSubscription now applies the same feature/DownloadAllowed gate as CreateSubscription and SyncSubscriptions; a PATCH could previously re-activate or widen a monitor and register managed rows after an admin disabled downloads or revoked the user. - Serving download bytes (managed and ephemeral) and /direct-download now mirror playback's per-file authorization via catalog.FileAllowedByAccess: library scope and the profile's max playback quality are re-checked at serve time, with artifact-backed rows checked against the artifact's resolution (a 720p transcode of a 4K source stays servable under a 1080p ceiling). - Offline manifests for remux/transcode entries now describe the prepared artifact (container, codecs, resolution, single selected audio track) instead of the catalog source file the client never receives. - ArtifactRepository.Requeue reports ErrNotFound when the row was concurrently swept; ArtifactManager.Ensure recreates the job in that case instead of linking downloads to a dead artifact id. - "No downloadable episodes" is a sentinel (mapped to 404 no_downloadable_episodes) rather than a bare error that surfaced as 500. - Subscription season_numbers are bounds-checked (0–9999) before the int32 narrowing in the repo could silently wrap them. - HandlePatchDownload reuses requireManaged instead of hand-rolling the same managed-identity checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-01 22:05:36 -04:00
3 people	2486345679	Reduce metadata image cache R2 churn (#249) ... * Reduce metadata image cache R2 churn * Preserve image cache failure cooldowns --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com>	2026-07-01 17:37:06 -04:00
Quick	6ca427096b	Add catalog search provider support	2026-06-25 16:20:14 -04:00
3 people	14ffc91dfb	[codex] Expand provider image cache queue (#176) ... * feat(metadata): expand provider image cache queue * fix(metadata): harden provider image cache queue Addresses bug-review feedback from Codex/CodeRabbit on the metadata image cache pipeline. All findings validated against the code before fixing; false positives (rows/connection deadlock, PhotoSourcePath merge coupling) were confirmed non-issues and left unchanged. - Honor metadata.cache_images for the background processor. The cache_metadata_images task was registered whenever S3 was configured, so merely enabling object storage downloaded the entire provider-artwork catalog even with caching disabled. Add ImageCacheProcessor.SetEnabled, gate RunOnce/RunUntilIdle on it, and wire it (with hot reload) from cfg.Metadata.CacheImages in main.go. - Guard terminal job updates with lease ownership. EnqueueBatch can repurpose a running row with a new source; MarkSucceeded/MarkFailed keyed on id alone let a stale worker finalize the replacement job and drop the new artwork. Thread locked_by through and add status='running' AND locked_by=$n guards. - Avoid uploading stale jobs onto the live artwork key. Verify the target still references the job's source (CurrentTargetSourcePath) before CacheImage, so a job whose source an admin/refresh already replaced cannot overwrite the deterministic storage object. - COALESCE nullable external IDs in EnqueueExistingProviderArtwork. A NULL tmdb_id/tvdb_id/imdb_id on any candidate failed the scan and aborted the whole cache run; matches the existing item_repo pattern. - Stop re-downloading the catalog every 30 days. Discovery now skips targets whose *_path is already a cached relative path, making the cached row the durable dedup marker instead of the prunable job row. - Decouple catalog sweeps from queue draining. RunOnce no longer runs discovery per batch; RunUntilIdle sweeps only when the queue drains and throttles full sweeps to every 15m, so idle installs stop full-scanning every entity table each minute. - Requeue claimed-but-unstarted jobs on cancellation. Acquire the semaphore before spawning workers and RequeueClaimed any jobs not yet started, instead of leaving them locked until the 15m lease expires. - Skip the backoff sleep after the final upload attempt in putObjectWithRetry (saves ~1.5s on permanent failures). - Add the s3/file/local/upload/generated exclusion to the seasons and episodes backfill in migration 20260617184537 for consistency with the later migration (the bad backfill was inert downstream, but the asymmetry is removed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-18 10:07:58 -04:00
4 people	c4cbcddeae	feat(manga): manga library type — series grouping, reading loop, AniList/MangaDex metadata + status badge (#138) ... * docs: design spec for manga library type (host sub-project) Forks the ebooks library type into a 'manga' type: series detected from the folder tree as a first-class type='manga' item, .cbz/.cbr chapters stay readable ebook items linked via a new manga_chapters table, browse shows series cards, enrichment targets the series item at content level 'manga'. Hands off to a follow-on plugin spec for the manga metadata source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for manga library type (host) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(scanner): manga filename index/volume parser * feat(scanner): manga series-name-from-folder detection Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(plan): align manga DB/scanner tasks to scanner pure-planner pattern (no test-DB) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(scanner): manga parser corpus regression Add TestParseMangaIndexCorpus — 36 real-world scanlation filenames covering bare chapter, decimal chapter, v/vol-prefix volume, and c/ch-prefix chapter patterns; asserts <5% miss rate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(db): manga_chapters link table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(scanner): manga_chapters repository + pure chapter-write mapping Adds mangaChapterWrite (pure, unit-tested), upsertMangaChapter, and listMangaChapters following the ebook/audiobook thin-SQL pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(scanner): recognize manga library type Add isMangaLibraryType helper (unexported, matching the style of isEbookLibraryType / isAudiobookLibraryType) with a corresponding TestIsMangaLibraryType unit test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(api): manga library content level Map library type "manga" to content level ["manga"] in metadataContentLevelsForLibraryType so that seedDefaultChain seeds a manga-level metadata provider chain when a manga library is created. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(scanner): route manga libraries to a manga scan path Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(scanner): group manga chapters under a manga series item Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): give manga series item a library membership so it browses Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(catalog): browse manga libraries as series Accept "manga" as a valid media_scope so a manga library browses only its type='manga' series items; the per-chapter type='ebook' items are naturally excluded because MediaScopeItemTypes("manga") expands to {"manga"}. Add the manga default library sections (scoped to media_scope='manga') so the library feed shows series cards. Refresh the two media_scope validation error messages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(catalog): manga series detail lists chapters For a type='manga' item, attach its chapters to the detail response via a new MangaDetailExtension. fetchMangaChapters joins manga_chapters to media_items on the chapter content ID, scopes to the series, and orders by chapter_index (NULLS LAST) then sort_title — matching the scanner's chapter ordering. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga detail types + library browse scoping Add MangaChapter/MangaDetailExtension TS types mirroring the host catalog structs, wire manga? onto ItemDetail, and admit "manga" as a QueryDefinition.media_scope. Scope manga libraries to media_scope=manga in browse (host expands it to type=manga series items) while reusing the ebook sort universe via getLibrarySortRelevanceScope. Add isMangaLibraryType. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga series detail with volume-grouped chapter list Add MangaContent detail view: a DetailHero series header plus a chapter list grouped by volume. groupMangaChapters (pure, unit-tested) buckets chapters by their volume token, orders chapters within a group by chapter_index (nulls last) and orders groups by their minimum index; loose (volume-less) chapters collapse into a trailing "Chapters" group. Each chapter links to the existing ebook reader by content_id alone (file_id is optional — the reader resolves the file server-side), reusing buildMediaPlayHref. Admit "manga" into ItemDetail.type and wire the detail switch. Continue-reading is deferred (needs per-chapter progress fan-out / a last-read timestamp not in the current payload). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): handle manga in playable-type + collection filter-scope unions Adding "manga" to the shared ItemDetail["type"] and QueryDefinition["media_scope"] unions leaked into consumers with narrower local types, breaking the production tsc build. Fixes: - mediaNavigation: admit "manga" into PlayableMediaType. Manga series are not directly playable (you open the detail page and read a chapter, itself an ebook item), so buildMediaPlayHref falls through to the item href for them, like series/season. - FilterRuleEditor: add "manga" to FilterRuleMediaScope and relabel "watched" -> "Read" for manga as well as ebook (manga is read). - CollectionGuidedRulesEditor: add "manga" to GuidedFormState.mediaScope, a "Manga" media-type option, ebook-like "Read Status" labels, and map manga -> ebook sort-relevance scope (manga has no dedicated sort scope). - CatalogFilterBar (cascading leak surfaced after the above): add a "Manga" scope option and map manga -> ebook sort-relevance scope in both scope handlers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): offer manga as a library type in the create dialog Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(scanner): strip scene-release junk from manga series names Add cleanMangaSeriesName which repeatedly strips trailing parenthetical groups (year, year-range, Digital, release-group tags) then trims any dangling dash, so folder names like "404 Demons (Digital) (Oak)" resolve to "404 Demons". Wire it into mangaSeriesFromPath so both the series title and the mangaSeriesGroupKey identity key use the cleaned value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): flat volume/chapter manga list; nest only multi-chapter volumes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): parse manga index after stripping series-name prefix Numbers inside a series title (e.g. "404 Demons", "365 Days to the Wedding") were wrongly grabbed as the chapter number because parseMangaIndex matched the first bare number in the full filename. mangaIndexForFile now strips the series-name prefix before delegating to parseMangaIndex, so only the number that follows the title is used. reconcileMangaFile in manga_scan.go is updated to call mangaIndexForFile instead of parseMangaIndex directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): stop missing-file reconcile from deleting manga series items Manga series items are file-less virtual parents; the shared ReconcileFolderMembership swept them every scan because they have no media_file. Exclude type='manga' from file-presence membership reconciliation, and add a manga-scan step that deletes only series with zero remaining chapters. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ebooks): exclude manga chapters from individual ebook enrichment Manga chapters are type='ebook' parts of a series; the ebook enrichment sweep was searching each one against book sources (Gutenberg/Anna's/etc.) and failing in a pointless storm. Exclude items with a manga_chapters link; series-level enrichment is handled separately. * docs: design spec for manga metadata plugin + series enrichment (sub-project 2) New silo-plugin-manga-metadata (AniList, high-confidence matching) + a host MangaEnricher for type='manga' series; default-enabled metadata source for manga libraries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for manga metadata plugin + series enrichment Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(db): manga_enrichment_state table Mirrors ebook_enrichment_state: dedicated failure counter for the manga enrichment sweep so it does not contend with media_items.refresh_failures. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(manga): series enricher (claims type='manga', resolves manga chain) * feat(manga): sync_manga_metadata task + enricher wiring * feat(catalog): expose manga chapter/volume counts in browse Add manga_chapter_count and manga_volume_count to browse cards so the frontend can render a Vols N / Ch N chip on manga series. The counts come from two index-backed correlated subqueries over manga_chapters in the browse SELECT (mangaCountColumns), scanned positionally before added_at and nilled out for non-manga rows. Threaded through models.MediaItem and exposed on the itemListResponse JSON card. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sections): scope manga home recent sections to type=manga series A manga library mixes type='manga' series with type='ebook' chapters, so the auto-generated home 'Recently Added/Released in <Library>' rows surfaced the junk chapter filenames. Add GeneratedHomeLibraryRecentConfigScoped which emits the modern QueryDefinition shape (library_ids + media_scope) so a manga library's generated home rows filter to type='manga' only. Other library types are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(catalog): exclude manga chapters from browse/section/search surfaces Manga CHAPTER items (type='ebook' rows linked into a type='manga' series via manga_chapters) were leaking into catalog browse, section resolution, and search as standalone items showing junk filenames. They are internal sub-units of the series and only the series should appear. There is no single shared item-listing chokepoint: browse, the query/preview executor, and search each build their own WHERE. Add a shared, index-backed anti-join predicate (manga_chapters.chapter_content_id is the PK) via mangaChapterExclusionWhere and wire it into all three builders. By-id fetch paths that legitimately resolve chapters (ebook reader, continue-reading, series detail chapter list) use separate queries and are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scanner): use #NN as the manga volume for Vol.YYYY #NN releases mangaVolYearIssue early-return was returning the year token (e.g. "Vol.2003") as the volume label, which the frontend couldn't prettify to "Volume N". Now returns "v<issue>" (e.g. "v04") so the existing frontend regex ^v?(\d+)$ renders it as "Volume 4" correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): manga count chip on posters Add an optional manga_chapter_count / manga_volume_count to the browse item type and render a top-right "Vols N" / "Ch N" chip on ItemCard, strictly gated on type==='manga'. The label prefers "Vols" when the volume count dominates, "Ch" otherwise; the chip is hidden when the chapter count is missing or non-positive. No other card type renders it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): manga reader back returns to series (no loop) The ebook reader's back action defaulted to the chapter's own item detail (/item/<chapter>), whose back returned to the reader — an infinite loop for manga chapters. The reader now honors an explicit backTo search param when present, navigating there instead. Absent for normal ebooks, so their back behavior is unchanged. Only manga chapter rows pass backTo, keeping the fix manga-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga chapter row actions (read/mark-read/download) Each manga chapter/volume row now offers Read (the existing reader link, now carrying a backTo to the series), Mark-read (the shared watched-state mutation per chapter content_id), and Download (lazily fetches the chapter's file versions on demand and opens the shared DownloadVersionPicker, gated on user.download_allowed). The volume-unit / loose-chapter / section structure from buildMangaList is unchanged. Scoped to MangaContent only; EbookContent is untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): validate reader backTo param is a safe in-app relative path Prevents open-redirect / javascript:-URI XSS from a crafted ?backTo= URL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(catalog): include per-chapter read state in manga detail Manga chapters are ebook items, so a chapter is "read" when the viewer's ebook_reader_progress row crosses the finished threshold. fetchMangaChapters now LEFT JOINs that table scoped to the AccessFilter's user_id/profile_id and exposes a per-chapter Read bool on MangaChapter, threaded through buildMangaExtension. The detail payload previously carried no read state, so the row toggle always started unread. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): manga rows reflect read state on load MangaChapter now carries an optional read flag from the detail payload, and MangaRow seeds its mark-read toggle from chapter.read instead of always starting unread. The optimistic toggle + shared watched mutation are unchanged; only the initial value is seeded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sections): exclude manga chapters from recently-added/released/random + other library-listing sections Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sections): manga recently-added/released cards show the latest volume's cover * fix(manga): keep enrichment honest about no-match vs enriched, batch 50->200 - sweep stats now separate enriched / no_match / failed: a stamped no-match was counted (and logged) as an enrichment, which masked a collapse of the real match rate during the backfill - batch size 50 -> 200 (SILO_MANGA_ENRICH_BATCH overrides): with the plugin serving GetMetadata from its search cache an item costs one rate-limited AniList request, so a sweep still fits the 5-minute task interval Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): size enrich batch to the 5-minute interval at AniList's real budget 140 items x ~2.1s/request fits the interval; an overlong sweep makes the task manager drop the next trigger and the effective rate falls below the AniList budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): manga count chip data missing from library browse manga_chapter_count/manga_volume_count were only added to BrowseRepository, but /library/{id}?tab=library flows through previewQuerySource -> QueryExecutor.PreviewPage, which selects qualifiedListItemColumns and scans with scanItems - so manga cards never carried the counts and the Vols/Ch poster chip stayed hidden. Append mangaCountColumns to the preview-page SELECT and scan them via a new scanItemsWithMangaCounts (nil for non-manga rows, mirroring scanBrowseItems). Extract listItemScanDests so the three scan variants share one destination list instead of duplicating the 48-column scan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): manga chip reads 'X Volumes · X Chapters', menu verbs say Read - chip: show distinct-volume and loose-chapter counts side by side instead of the single 'Vols N'/'Ch N' heuristic; mangaCountColumns now counts DISTINCT volume tokens (rows sharing a volume are one volume) and only un-volumed rows as chapters - watched-state labels: type='manga' fell through to the video default, so the card dot menu and detail page said 'Mark Watched' - manga now uses the ebook reading verbs (Mark Read / Mark Unread, 'Marked as read' toast) - format MangaContent.test.tsx (pre-existing prettier miss) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): backdrop enrichment - banner hero art + backdrop-only backfill - cache remote backdrops like posters (cacheRemoteImages generalizes the poster-only path; failures keep the provider URL, which still renders) - claim arm for enriched items missing a backdrop: fetched by stored provider ID (search skipped - no rate spend, no re-match risk) and only the backdrop is written; stamping after the attempt keeps banner-less series from being re-claimed every sweep - backfill = one-time SQL clearing last_refreshed for poster-set/ backdrop-empty manga Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): reading-loop UX - continue CTA, next chapter, series-aware cards, file details Fixes the four high-priority findings from the manga UX review plus a file-inspector request: - H1: series hero gets a Continue / Start Reading / Read Again CTA targeting the first unread chapter (firstUnreadChapter over the ordered list), plus an overflow menu (View Details, admin Refresh Metadata) - H2: the reader resolves its owning manga series (chapter detail now carries series_id/series_title) and offers next-chapter navigation: a header next button and an end-of-book floating CTA at >=99.5% progress; back defaults to the series even without a backTo param - H3: chapter rows show a persistent read check + muted title, and the mark-read mutation carries series_id so the series detail cache invalidates (read states no longer revert on revisit) - H4: continue-reading cards for manga chapters present the series: sections payload resolves chapter->series linkage, the card heading/image link to the series, and meta lines launch the reader - View Details: manga series menus (card dot menu + detail overflow) open a file inspector showing folder paths and per-chapter file names/sizes via GET /catalog/items/{id}/manga-files; paths are stripped for viewers without file-path visibility (item-versions policy) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): UX mediums - richer detail page, smarter list, manga sort scope Second batch from the manga UX review (M1-M7): - M1: multi-chapter volume sections are collapsible (fully read sections start collapsed) with sticky headers, and long series get a 'Jump to <next unread>' anchor above the list - M2: the series hero shows the author line (HeroCrewLine learns Author credits with person links; DetailHero now renders crewLine and genre chips independently) and Volumes/Chapters badges - M3: browse-card count chip abbreviates to '12 Vol - 3 Ch' so it fits narrow cards without occluding covers - M4: manga gets its own sort scope: Duration/Bitrate (meaningless for file-less series rows) disappear, reading labels (Date Read / Reads) apply, Author stays - M5: global search labels manga results 'Manga' instead of the raw type - M6: chapters carry the viewer's reading fraction; part-read rows show an inline progress bar + percent - M7: chapter rows show the extracted cover thumbnail (presigned poster_url on the chapters payload) instead of a generic icon Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): UX lows - volume token dedupe, comic reader chrome, empty-state hint - buildMangaList buckets volumes by canonical numeric token so mixed release naming (v01 + 1) yields one Volume 1 instead of duplicates - cbz/cbr readers start with the side panel closed and hide prose-only chrome (reading ruler, TTS, typography/font controls, hyphenation, writing mode) while keeping comic-relevant settings (theme, brightness, margin, right-to-left, spread, flow) - manga empty state mentions chapters appear after the library scan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): publication status badge via new SDK status field - vendor the unpublished plugin SDK (adds MetadataItem.status) under internal/compat/ with a relative go.mod replace, following the zishang520-webtransport-go convention; swap to the published module before the upstream PR - map plugin status into MetadataResult.ShowStatus, persist it during manga enrichment, and show it as the hero status badge (show_status was already on the detail payload and MetadataBadges) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(manga): generalize backdrop pass to secondary fields (backdrop + status) The backdrop-only claim arm becomes a secondary-fields pass: enriched items missing a backdrop and/or publication status are claimed, fetched by stored provider ID, and only the missing secondary fields are written. Lets the new status field backfill across the already-enriched library instead of applying only to future enrichments. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): merge ShowStatus through MergeMetadata/MergeGlobalMetadata The new MetadataResult.ShowStatus never reached the accumulated result the manga enricher persists from - the field-by-field merges didn't know it, so the status backfill pass obtained nothing. Regression-tested on both paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): keep scanner identity IDs out of the metadata flow filterMangaProviderIDs passed the scanner's manga_series identity row through, so the search-skip-when-already-matched guard saw provider IDs on every item and never searched: unmatched items went straight to a by-ID fetch with no usable ID and were stamped as terminal no-match without a single provider request (and the MangaDex fallback was never consulted). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: gitignore docker-compose.override.yml (local deployment override) The override unpublishes the bundled redis/postgres host ports (ports: !override []). It is a per-deployment, local-only file: ignoring it keeps a rebase from main and git clean -fd from disturbing it, and keeps it out of any PR. Its accidental absence once exposed Redis to the internet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): code-review fixes — no-match guard, sort comparator, volume-count consistency - enrichWithProviders: set accumulator.HasMetadata after a provider result merges (MergeMetadata doesn't propagate it). Without this, a confident match carrying only genres/authors/status/year but no cover and no overview failed the no-match check and was discarded + terminally stamped. - byChapterIndex: both un-indexed chapters yield POSITIVE_INFINITY, so the subtraction was Infinity-Infinity=NaN (Array.sort treats NaN as 0, leaving order undefined). Compare explicitly for a stable order. - MangaContent volume/chapter badges: derive counts from the rendered buildMangaList entries (which canonicalize v01 ≡ 1) instead of raw distinct volume tokens, so the badge can no longer say '2 Volumes' over one row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(manga): clarify the enrichment claim's secondary arm is admin-reset-only The secondary arm (poster present, backdrop/status missing) requires last_refreshed IS NULL, so it is only reachable when an operator resets last_refreshed to backfill a newly-added field — not an automatic periodic re-check (which would re-fetch banner-less series every sweep). Documents the intent so it does not read as dead code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(manga): collapse continue-reading chapters per series; batch provider-id lookup - Continue Reading now collapses multiple in-progress chapters of the same manga into one card (most recently read kept), mirroring the episode→series collapse. The reading section resolves chapter→series linkage into itemMeta (applyMangaChapterSeriesMeta) and runs the shared collapseContinueWatchingSeriesCandidates, which the reading path previously skipped. - claimBatch resolves provider IDs for the whole batch in one query via the new ProviderIDRepository.GetByContentIDs (content_id = ANY), replacing the per-item GetByContentID N+1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): manga publication-status chip on browse cards + more legible chips - Color-coded publication status pill (Ongoing/Completed/Hiatus/Cancelled/ Upcoming) in the manga card's top-left corner, mirroring the vol/chapter count chip top-right. Strictly manga-gated; show_status was already on the browse payload. - New .glass-chip (78% surface vs glass-subtle's 40%) for the manga count + status pills so the labels stay legible over busy cover art. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(manga): depend on published silo-plugin-sdk v0.7.0 Replace the vendored internal/compat/silo-plugin-sdk copy with a normal dependency on the published SDK module at v0.7.0, which adds MetadataItem.status (publication/airing status) consumed by the manga status badge at internal/metadata/plugin_provider.go. - go.mod: pin v0.7.0, drop the local-path replace directive - remove the vendored internal/compat/silo-plugin-sdk tree - Dockerfile: drop the vendored-SDK COPY - strip the manga design docs/plans from docs/superpowers (internal) Requires Silo-Server/silo-plugin-sdk#4 merged and tagged v0.7.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(manga): exclude chapters from the matcher's unmatched-item lister Manga chapters are type='ebook' items that stay status='pending' by design - provider metadata lives on the type='manga' series item. The scan-final RetryUnmatchedItemsByFolderAndPathPrefix listed all of them and ran a rate-limited ebook-plugin search per chapter: 31,564 chapters x ~1s = 8h46m appended to a 2-minute manga library scan (observed live), every one a guaranteed no-match. Earlier runs never survived to completion, so the library's last_scanned_at stayed NULL forever. Add the same manga_chapters NOT EXISTS guard the ebook enricher's claim query already uses. Verified live: the same library now scans in 27s with retried_items=0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): never probe-repair ebook/comic files (ebook+manga detail-page killer) NeedsCriticalProbeRepair was always true for BaseType 'ebook' files (epub, pdf, cbz, cbr — incl. manga chapters): buildEbookMediaFile leaves ProbeUpdatedAt nil and they have no audio/video, so probeEnsurer.Ensure spawned ffprobe per file on every detail/watch load and never converged (ffprobe errors on zip/rar, result never persisted). Short-circuit probe-repair for ebook base type — they're read directly and never use the transcode/playback probe pipeline. SHARED fix: benefits both the ebooks and manga library types. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf+fix(ebooks): parallelize detail extension + preserve finished read-state - buildEbookExtension ran its 3 related-content queries (series, also-by-author, similar) sequentially; run them concurrently like buildAudiobookExtension so ebook detail latency is the slowest query, not their sum. - PGEbookReaderProgressStore.Upsert did an unconditional SET progress=EXCLUDED; a routine autosave (e.g. reopening a finished book) could drop it below the 0.9 finished threshold and silently un-mark it read (and clear the manga chapter checkmark, which rides on the same row). Guard: once finished, progress only moves on an explicit unread (row delete); below threshold it tracks freely. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(manga): batch chapter presign, index volume counts, quiet scan log - fetchMangaChapters presigned each chapter poster individually; a long-running series has hundreds of chapters. Batch them in one PresignImageURLs call, and add the missing rows.Err() check (was silently returning partial lists). - The browse manga count chip's count(DISTINCT volume) subquery wasn't covered by manga_chapters_series (series_content_id, chapter_index); add idx_manga_chapters_series_volume (series_content_id, volume) so both count subqueries are index-only. - Downgrade the per-chapter "manga scan: indexed" log from Info to Debug (one line per .cbz; the 500-file progress log already covers operator visibility). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(manga): address PR #138 code-review findings Folds PR #142 into the manga branch (already done via fast-forward) and remediates the issues surfaced in the #138 code review. Correctness: - Preserve the scanner's manga_series identity anchor through enrichment. ReplaceByContentID's DELETE was unconditional, so the first successful enrichment wiped the manga_series provider-id row the scanner relies on for idempotency, causing duplicate series + metadata loss on the next scan. excludedProviderIDs now also means "not deleted", and the DELETE preserves those rows. (internal/catalog/provider_id_repo.go) - Fall back to the series cover when the latest chapter has no poster. Poster columns default to '' (not NULL), so the manga series-card poster override blanked cards via a plain COALESCE; wrap operands in NULLIF. (internal/sections/fetcher.go) - Keep backTo a real query param on reader links when libraryId is absent. It was string-concatenated with '&', producing a malformed URL on deep-links; route it through the query helper instead. (web/src/lib/mediaNavigation.ts, EbookReader.tsx, MangaContent.tsx) Quality: - Hide manga chapters from favorites/watchlist browse, matching the exclusion enforced on every other listing surface. (internal/catalog/favorites_browse.go) - Centralize the manga chapter exclusion predicate into a single exported catalog.MangaChapterExclusionWhere, removing four duplicated copies. (catalog, sections, ebooks) - Skip the two manga count subqueries on browse scopes that cannot contain manga (non-manga type filters), substituting NULL placeholders. (internal/catalog/browse.go) - Normalize provider publication status (AniList/MangaDex/SDK variants) into a stable label set so show_status carries one manga value-domain. (internal/manga/enrichment.go) Adds unit tests for the poster NULLIF contract, browse gating, and status normalization. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: regenerate go.sum after rebase onto main Drops stale silo-plugin-sdk v0.6.0 and other leftover hashes from the intermediate rebased states; go.mod is now on the published v0.7.0 tag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scanner): adapt manga scan to ebookFileShouldSkip 3-value signature main changed ebookFileShouldSkip to also return the existing content ID; the manga scan path only needs the unchanged flag, so discard the new return. Resolves a silent semantic conflict from the rebase onto main. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Silo Server Developer <warmasterx555@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 20:13:10 -04:00
QuickandClaude Fable 5	1e3780d4fb	fix(notifications): address code review findings ... - pin the four new sensitive setting keys (SMTP password, Discord secret/bot token, VAPID keypair) in the encryption audit test so a future drop from SensitiveSettingKeys fails CI - bound account-channel digest drains strictly before the stamped digest time so consecutive digest windows partition rows exactly, instead of recapping rows created at or after the previous stamp - keep the events websocket open when an event-frame snapshot fails, matching the writeSnapshotFrame degrade-gracefully contract - rename the seed task to Seed Content Availability to match its episode+movie seeding behavior - carry poster_source_path into realtime dispatch rows per the DeliveryRow contract Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-12 10:32:21 -04:00
QuickandClaude Fable 5	b091f0c6c1	feat(notifications): in-app inbox, realtime, webhooks, web push + shared SMTP core ... Implements the notification system foundation and all v1 delivery channels that need no external infrastructure (specs 00/01/04/05 in docs/superpowers/plans/notifications/): Foundation (spec 01): - episode_availability seeding + per-library seed markers: "newly available" means newly released to this server, so back-catalog imports and first scans never flood (verified on dev: 1.13M episodes seeded silently) - release_events -> profile_series_interest fanout worker with settling delay, per-series burst caps, FOR UPDATE SKIP LOCKED multi-node claims, and a guarded last-notified cursor - interest index maintained via a userstore provider decorator so every favorites/watchlist/progress mutation path (REST, jellycompat, imports, playback) feeds it; progress writes only recompute on state transitions - durable per-profile inbox + read state, forward-sync cursor API, websocket channel with short-lived single-use handshake tickets - web UI: sidebar badge, inbox page, toasts, per-profile preferences - startup/daily tasks: availability seeding, interest rebuild, retention Outbound webhooks (spec 04): - Discord embeds (text-only per the v1 privacy contract) and generic JSON signed Stripe-style with per-webhook secrets - HTTPS-only + private-destination guard enforced at registration and at connect time (DNS-rebinding mitigation); URLs/secrets encrypted at rest - durable per-target outbox enqueued in the fanout transaction, lease-based claims, 24h exponential retry, 3x-consecutive-4xx auto-disable with an in-app notice (loop-guarded) Web push (spec 05): - VAPID keypair self-provisioned at startup (single atomic JSON setting, private half encrypted at rest) — no third-party accounts needed - payloads E2E-encrypted (RFC 8291); 404/410 treated as unsubscribe - service worker + subscribe flow in Settings -> Notifications Shared SMTP core (internal/mail): - feature-agnostic mail.Sender over live email.* settings, STARTTLS or implicit TLS, encrypted password, admin Email settings page with synchronous test send; no consumer yet by design (digest is v1.5) APNs/FCM (specs 02/03) are deferred to v2; the capability endpoint reports them unavailable so clients render truthfully. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 14:55:46 -04:00
4 people	8b70357703	feat(ebooks): first-class ebook libraries, scanner, and reader (#124) ... * docs: define ebook architecture matching audiobooks * docs: plan ebook audiobook-parity implementation * feat: add ebook scanner parser foundation * fix: harden ebook scanner foundation * fix: handle ebook isbn labels * fix: guard ebook subtree scans * feat: scan ebook libraries in core * fix: preserve ebook scan people credits * fix: refresh ebook scan metadata safely * feat: persist ebook series membership * test: cover ebook series persistence decisions * fix: address ebook scanner PR review * docs: clarify ebook foundation PR scope * feat: add ebook metadata enricher * fix: harden ebook poster cache * feat: wire ebook metadata sync task * feat: expose ebook library metadata setup * feat: add ebook catalog scope support * feat: add ebook detail view * feat: label ebook file versions by format * feat: use file-size copy for downloads * feat: use file language in download dialog * test: cover ebook detail authors and downloads * fix: drop narrator credits from ebook scanner merges * fix: align ebook collection filters with book media * fix: drop asin provider ids from ebook enrichment * fix: force ebook people refresh for stale narrators * chore: omit ebook planning docs from branch * feat: add ebook detail related content * feat: add ebook reader file entrypoint * feat: render ebooks with foliate reader * feat: persist ebook reader progress * feat: add ebook reader controls * feat: extract ebook pdf metadata * feat: favor scanner isbn during ebook enrichment * feat: extract fbz ebook metadata * feat: count cbz ebook pages * feat: show ebook file page counts * feat: show ebook download summaries * feat: switch ebook reader files * feat: prefer epub for ebook read action * feat: surface ebook reader progress * feat: sync ebook reader progress cache * feat: hide ebook read action for unsupported files * feat: filter ebook reader file selector * fix: serve fbz ebook archives with reader mime type * fix: detect fbz ebooks from compound filename * fix: authorize fbz ebooks from compound filename * fix: scope ebook catalog facets * fix: reject narrator queries for ebooks * fix: build ebook recommendation text from authors * fix: include ebooks in embedding eligibility * fix: include ebooks in recommendation media mix * fix: include ebooks in recently added recommendations * feat: include ebook progress in recommendation signals * feat: include ebooks in continue watching sections * feat: include ebooks in catalog progress metrics * fix: read ebook isbn from epub metadata * fix: filter ebook asin provider aliases * fix: fall back from unsupported ebook reader files * fix: sort ebook catalogs by reader progress * fix: filter ebook catalogs by reader progress * fix: include ebooks in last watched catalog filters * feat: reflect ebook reader progress in item user state * feat: share ebook progress state across item surfaces * feat: report ebook scan progress * fix: include ebook activity in recommendations * fix: expose ebook reader progress on item detail * fix: support ebook subtree scans * fix: honor profile header for ebook item progress * fix: add ebook library default sections * fix: route ebook continue cards to reader * fix: hide watched toggle for ebooks * fix: route ebook watch tonight cards to reader * fix: route ebook hero actions to reader * fix: detect archive ebook reader formats by filename * feat: cache embedded ebook covers during scan * fix: encode ebook hero reader links * fix: persist non-epub ebook reader progress * fix: scope narrator catalog badges to audiobooks * fix: merge ebook reader progress during item repair * fix: label ebook progress filters as read * fix: show ebook related rails as book covers * fix: remove txt ebook reader support * fix: reject txt ebook reader files * fix: label ebook advanced filters as read * fix: label ebook personalized sorts as read * fix: remove plain text reader loader path * test: cover ebook unread catalog rules * fix: preserve ebook reader library context * fix: link ebook genres with library scope * fix: encode related rail item links * fix: encode catalog card item links * fix: encode hero and continue item links * fix: encode watch tonight item links * fix: encode recommendation and search item links * test: cover ebook scan format set * fix: label ebook search results clearly * fix: make global search prompt media neutral * fix: encode catalog read API ids * fix: encode item API ids * fix: include ebook reader vendor in docker build * fix: make ebook reader build clean * fix: clean ebook embedded descriptions * docs: plan ebook reader shell parity * feat: add ebook reader shell controls * fix: widen ebook scrolled reader flow * fix: remove scrolled reader content width cap * docs: plan ebook reader full parity * feat: persist ebook reader config * feat: add ebook annotations and bookmarks * feat: add ebook reader tools and aids * feat: add ebook advanced reader settings * fix: keep ebook reader panel in viewport * fix: use foliate sizing units for ebook scroll flow * fix: keep ebook settings controls readable * fix: simplify ebook reader settings controls * feat(ebooks): extract local covers during scan (#98) * feat(ebooks): extract local covers during scan * fix(ebooks): read nullable poster paths during cover scan * fix(catalog): coalesce nullable media artwork fields * fix(ebooks): group sibling formats by book identity * fix(ebooks): tolerate legacy ebook metadata encodings * fix(ebooks): decode PDF hex metadata strings * fix(ebooks): harden local cover extraction and format grouping Address review findings on the local cover scan: - Restrict generic sidecar covers (cover.jpg, folder.png, ...) to single-book directories, always accept images named after the book file, and apply exactly one cover per reconcile with sidecar taking precedence over the embedded cover. - Replace the read-then-write poster update with an atomic conditional UPDATE (ItemRepository.SetLocalPoster) so provider/admin artwork is never clobbered by concurrent writers, and refresh locally owned posters when the extracted cover bytes change (thumbhash compare). - Preserve UTF-8 PDF Info strings (including a UTF-8 BOM) instead of forcing everything through Windows-1252; the cp1252 fallback now only applies to non-UTF-8 bytes. - Select EPUB covers by manifest media-type with properties="cover-image" outranking the EPUB2 meta name="cover" id, so XHTML cover pages no longer shadow the real image. - Order CBZ pages naturally (2.jpg before 10.jpg, ch2/ before ch10/) when picking the cover page, via a single O(n) min-scan. - Bump the ebook content group key scheme to version 2 and reprocess rows written under older versions so pre-existing libraries gain sibling-format grouping instead of accumulating duplicates. - Group different formats only (a same-format sibling with colliding sparse metadata stays a separate item) and stop a joining sibling's embedded metadata from overwriting a provider-matched item. - Decode any IANA-labelled OPF/FB2 XML charset (windows-1251, koi8-r, shift_jis, ...) via x/net/html/charset, and wire the charset reader into FB2 parsing which previously had none. - Strip the full .fb2.zip double extension from filename-derived titles and group keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * feat(ebooks): add reader profiles and ruler (#99) * feat(ebooks): extract local covers during scan * fix(ebooks): read nullable poster paths during cover scan * fix(catalog): coalesce nullable media artwork fields * fix(ebooks): group sibling formats by book identity * fix(ebooks): tolerate legacy ebook metadata encodings * fix(ebooks): decode PDF hex metadata strings * feat(ebooks): add reader profiles and ruler * fix(ebooks): address reader ruler and profile review findings - skip renderer setStyles/render when computed styles and attributes are unchanged, so ruler position updates no longer re-style the book view - drag the ruler via a local draft that commits on release, with the surface rect cached at pointer-down - migrate font values persisted before the generic stacks (Inter, Georgia, Merriweather, legacy serif) so the font select never renders blank, with a Custom fallback option for unknown values - make the ruler band click-through and move dragging to a dedicated keyboard-accessible slider handle so links and text selection keep working under the band - share font stacks between options and profiles via READER_FONT_STACKS - surface the active reading profile, move presets to the top of the settings panel, and drop the redundant profile button aria-labels Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ebooks): resolve prefer-const lint error in readest document lib `pnpm run lint` failed on the branch because `direction` is never reassigned in getDirection; split the destructure so only the reassigned `writingMode` stays mutable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * Merge branch 'main' into work/ebooks-reader-base Brings the ebook integration branch up to date with main (audiobook library redesign, continue-watching rework and card affordances, quic-go bump, jellycompat fixes). Conflict resolutions favor main's generalized mechanisms and register ebooks with them: - media scope validation goes through IsValidMediaScope (now including "ebook" alongside main's "video" group scope), in Go and in the web filter/search types - continue-watching uses main's typed rails; reading-type sections pull resume points from ebook_reader_progress and the ebook library default section is wired to ContinueTypeConfig(ContinueTypeReading) - item_repo keeps main's derived select-list machinery (itemColumnExpr) and both poster accessors (GetPoster/SetLocalPoster for ebook covers, GetPosterPath for audiobook covers) - web cards/hero/watch-tonight adopt main's buildMediaPlayHref helpers, which now route ebooks to /reader/ebook and encode content ids; ebook affordances (BookOpen icon, Read verb, percent-read subtitle) carry over onto main's reworked components - LibraryForm ebook support ported into main's refactored useLibraryForm/libraryTypes modules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docker): copy foliate-js vendor into Dockerfile.dev frontend stage foliate-js is a file:vendor/foliate-js dependency, so pnpm install needs the vendor directory before the lockfile install layer. The production Dockerfile already copies it; the dev image was missed, breaking make dev-deploy with ENOENT on /app/web/vendor/foliate-js. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ebooks): render Continue Reading sections as upright poster cards All-ebook continue sections previously fell through to the horizontal 16:9 wide card; include ebooks in the poster-variant check so book covers render in their natural 2:3 framing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): stop related-rail highlight ring clipping on detail pages Move the current-item ring onto the cover artwork with a themed ring-offset color (matching the sidebar profile highlight) and give the scroll container top headroom so the ring is not cut off by overflow-x-auto. Applies to both ebook and audiobook detail rails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): harden ebook scanning against data loss and bad metadata - Reconcile missing ebook files like video/audio, with real per-root walk failure tracking (failed/unmounted roots are excluded from deletion), symlinked-root support via the shared logical walker, and the empty-root cleanup allowance before any destructive reconciliation. - Create ebook items as 'pending' so enrichment can promote them to 'matched' (backfill migration included), and protect matched items from re-scan clobbering: title/year skipped, people/series fill-empty only. - PDF metadata: scan head + tail windows (non-linearized PDFs keep the Info dict at the end), require proper key delimiters, head values win. - Cap plain .fb2 reads like .fbz entries; drop .md as an ebook format. - gofmt internal/scanner/audiobook.go (pre-existing drift). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ebooks): make enrichment failures non-terminal with dedicated backoff state - Provider errors now record a failure (capped retries) instead of stamping last_refreshed, which permanently excluded items after transient outages. - Unconfigured metadata chains and the scan-window membership race skip the item without stamping or burning a retry. - Failure tracking moves to a new ebook_enrichment_state table, decoupling it from media_items.refresh_failures (shared with metadata refresh debt). - Preserve non-author people credits when persisting enrichment results. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): gate ebook progress on hidden history and centralize threshold - Apply user_history_hidden_items gating (video semantics) to the ebook watched/in-progress filters, progress sort plan, and Continue Reading. - Continue Reading pages past dismissed items via the shared collector and dedupes items across pages (also fixes the video path's latent exposure). - Centralize the 0.9 finished threshold as models.EbookFinishedProgressThreshold with a single SQL-interpolated mirror in catalog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(recommendations): correct watcher counting and wire ebook taste signals - itemWatchersQuery dedupes to distinct (watcher, item) rows so one binge-watcher can no longer satisfy minWatchers; the eligibility floor now counts distinct accounts rather than profiles. - Hidden-history gating on GetEbookReaderProgressForUser (signal reader). - Ebook reading produces canonical implicit taste signals (weighted like the equivalent movie progress ratio); ebooks join taste-seed candidates. - Stale GetRecentlyAddedItems doc comment corrected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): harden ebook reader endpoints and serve a Content-Security-Policy - Serve a CSP on all SPA HTML responses: blob/srcdoc book iframes inherit it, so script-src 'self' 'wasm-unsafe-eval' blocks script execution from malicious book content (sandbox alone is defeated by the WebKit allow-scripts requirement). Threat model documented on the constant. - X-Content-Type-Options: nosniff on frontend, jellycompat, and ebook file responses; MIME resolution can no longer fall through to octet-stream for an admitted ebook file. - Annotation PATCH: presence-aware field semantics (absent keeps, present sets/clears), invariant re-validation on the merged row, and an atomic SELECT ... FOR UPDATE read-merge-write. - Request size caps (413) on progress/config/annotation writes; Content-Disposition via mime.FormatMediaType; hidden-history gating in the shared ebook progress lister; FK-cascade indexes for reader tables. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): native read-state endpoints for ebooks - POST/DELETE /watched/{id} accepts ebook content IDs: mark read upserts progress 1.0 preserving the reader's file/location (or picks the preferred reader file for never-opened books); mark unread mirrors video unwatch semantics and deletes the progress row. - /history/remove accepts ebooks: hides via user_history_hidden_items without touching the reading position (hidden != unread; next reading activity resurfaces the book, mirroring video re-watch). - Access-filter checks match the video branch; shared logic lives in ebook_read_state.go. Sort metrics/user-state thresholds use the shared constant; profile-header fallback deduplicated. Clients: response is {type: "ebook", affected_count: 1, played: bool}; the existing watched SSE event fires. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): harden the ebook reader UI - Open-flow race: cancellation checked after every await with full stale-run teardown (no wrong-file progress saves, no leaked views/blob URLs); book.destroy() on cleanup. - Progress: monotonic stale-response guard; visibilitychange flush uses the refresh-capable client, pagehide uses keepalive; per-book cross-format progress documented as deliberate. - Settings: side effects out of the setState updater; local edits no longer clobbered by late server config; pending saves flushed on unmount/pagehide. - TTS: generation token so Stop actually stops (Chromium/Firefox synthetic events); Media Session uninstalled on unmount. - External book links: http(s) only, opened with noopener,noreferrer. - apiBlob 512 MiB guard with a user-facing error; fraction bookmarks navigable; search-result key collisions fixed; dead e-ink code removed; getLibrarySortRelevanceScope deduplicated; md format dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): mark read/unread affordances for ebooks - Item detail gets a Mark Read/Unread button; card menus drop the ebook gate and share type-aware labels/toasts (also dedupes audiobook wording). - Watched-state invalidation includes the reader progress query key so the Continue button and percent refresh after toggling. - Continue Reading dismiss copy for ebooks; dismissal path now URL-encodes item IDs (ebook content IDs can contain reserved characters). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: record the PR #124 review and hardening pass Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 08:18:35 -04:00
Quick	b7439884ec	fix(tasks): skip idle audiobook maintenance	2026-06-08 22:46:44 -04:00
zZebrahzandGitHub	96bae2f346	fix(library): clean orphaned provisional items (#76) ... * fix(library): clean orphaned provisional items * test(metadata): use item not found sentinel in fake repo * fix(library): preserve abs references during orphan cleanup * fix(library): avoid dropped abs cleanup tables * fix(library): harden provisional orphan cleanup * fix(library): preserve home dismissal series orphans * fix(library): Delete matched items created after the first orphan sweep	2026-06-07 21:32:40 -04:00
4 people	eb6024573e	feat(audiobooks): make audiobook libraries first-class catalog items (#73) ... * docs(audiobooks): design spec for plugin absorption Plan to absorb silo-plugin-audiobooks into silo-server as a first-party feature. Audiobooks land in silo's existing SPA; ABS clients connect directly. Hard constraints: reuse existing tables (media_items, media_files, user_watch_progress, user_playback_sessions, people, item_people, library_collections); only two new tables (abs_sessions, podcast_feeds) and at most one column add (media_libraries.kind); silo's main :8080 listener handles ABS Socket.io natively. Out of scope: audiobook requests flow, smart collections, share links, external recommender, custom metadata providers, separate audiobook SPA. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 1 (discovery + schema) First of six sub-plans for the absorption. Six tasks: a discovery audit that resolves the spec's Risk questions, four idempotent SQL migrations (abs_sessions, podcast_feeds, media_libraries.kind, audiobooks.enabled feature flag), and an empty-but-compiling internal/audiobooks package scaffolded into cmd/silo. Lands as a strict no-op for users (feature flag defaults to false). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): discovery findings for absorption sub-plan 1 Locks schema/code decisions for migrations 139-142 and downstream sub-plans. Resolves open Risk questions from the absorption design spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 139 add abs_sessions table Parallel of jellycompat_sessions for Audiobookshelf-compatible clients. Lets ABS mobile/desktop apps maintain a device-bound session that silo's audiobooks/abs handlers will validate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): match codebase conventions in migration 139 Lowercases type keywords in the abs_sessions CREATE TABLE body to match neighboring migrations, fixes the client_version column alignment, and replaces the misleading "parallel to jellycompat_sessions" header comment with a more accurate description of the table's role. Cosmetic only — the running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 140 add podcast_feeds table Side table on media_items for RSS-subscribed podcasts. Holds feed URL, ETag/Last-Modified for conditional fetches, last-refresh timestamp, and the per-feed refresh interval consumed by the upcoming podcastfeed.Refresher scheduled task. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): uppercase PRIMARY KEY in migration 140 Aligns with the codebase convention (type keywords lowercase, constraint keywords uppercase) established in migration 139's post-style-fix form. Cosmetic only — running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(audiobooks): migration 141 no-op for media_folders.type Sub-plan 1 originally reserved migration 141 to add a 'kind' column to media_libraries discriminating audiobook/podcast libraries. Discovery audit (sub-plan 1 Task 1) found that the actual table is media_folders and it already has a type text NOT NULL column with no CHECK constraint or enum, so 'audiobooks' and 'podcasts' can be added as future values without DDL. Landing this migration as a documented no-op preserves the version numbering audit trail and pins the decision in git history. The matching down migration is also a no-op. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 142 add audiobooks.enabled flag Server-settings row that gates the absorbed audiobooks feature. Defaults to 'false' so sub-plan 1 lands as a strict no-op; subsequent sub-plans branch on this flag and operators flip it to 'true' at cutover. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scaffold internal/audiobooks package Empty-but-compiling Service that reads the audiobooks.enabled feature flag from server_settings. Wired into cmd/silo so the package is referenced from the binary; no routes mounted, no scheduled tasks registered, no DB writes. Subsequent sub-plans hang scanner branches, ABS handlers, Socket.io, podcast refresher, and SPA pages off this Service. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): cosmetic cleanups in scaffolded package Two pre-emptive cleanups flagged by code review before sub-plan 2 copies the patterns: 1. Sort the internal/audiobooks import after internal/adminjob in cmd/silo/main.go (alphabetical). 2. Drop the redundant "audiobooks: " prefix from the Enabled() error wrap; matches how every other top-level service package (watchstate, scanqueue, metadata, etc.) formats errors. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 2 (scanner) Second of six sub-plans. 10 tasks: PersonKind constants for Author and Narrator, audio-extension recognizer, library-type helpers, a walkLogicalTree refactor (movieLibrary bool -> typed walkMode), chapter extraction via ffprobe, single-file and multi-file audiobook parsers, scanner write path producing media_items.type='audiobook', and a filesystem podcast parser (RSS deferred to sub-plan 5). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add Author and Narrator PersonKind constants Discovery audit confirmed item_people.kind is unconstrained smallint with values 1-6 in use. Reserve 7 = Author, 8 = Narrator for audiobook people-links written by the upcoming scanner branches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add audio-extension recognizer for scanner Mirrors the existing videoExtensions/SupportsVideoFile pair. Used by upcoming audiobook and podcast scanner branches to filter directory walks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): library-type recognizers for scanner dispatch isAudiobookLibraryType and isPodcastLibraryType match singular and plural forms case-insensitively, mirroring isMovieLibraryType. Used by upcoming scanner walk branches (Task 4) that filter audio files into audiobook and podcast libraries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(scanner): replace movieLibrary bool with typed walkMode Lets walkLogicalTree dispatch on multiple library shapes (video, movie, audiobook, podcast) without proliferating boolean flags. Behavior for existing video and movie libraries is unchanged; audiobook and podcast modes will be consumed by the upcoming audiobook.go and podcast.go parsers in later tasks of this sub-plan. walkModeFor() derives the mode from a media_folders.type string; unknown types default to walkModeVideo to preserve prior behavior for any caller still passing a raw type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose ffprobe format tags on ProbeData The audiobook scanner needs format-level tags (title, artist, album, date) for media_items metadata; ffprobe already parses them in ffprobeFormat.Tags but ProbeData previously discarded them. Add FormatTags map[string]string to ProbeData, populate it in convertProbeData via a new normalizeFormatTags helper that lowercases keys and trims values. Adds a fixture audiobook .m4b with embedded chapters (Intro/Outro) and format tags, and a test that verifies ProbeFile() returns both correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): parser for single-file audiobook folders parseAudiobookFolder reads tags + chapters via the existing ProbeFile (now that Task 5 exposes FormatTags on ProbeData) and produces a parsedAudiobook struct. Title falls back from "title" tag to "album"; author from "artist" -> "album_artist" -> "composer"; series from "album" -> "series" -> "mvnm" (Movement Name, used by some MP4 tools). Year parsed from "date" or "year" tags, tolerating ISO dates and parenthesized forms. Single-file case only; multi-file folders (one audio file per chapter) return a placeholder error and arrive in Task 7. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): multi-file audiobook folder support Folders containing N audio files (one per chapter/part) get one parsedAudiobookFile per file; each file's chapter list is synthesized as a single chapter with title = filename stem. Title/author/series/ year come from the first file's tags. Also drops the duplicate pickFirstNonEmpty helper added in Task 6 in favor of the existing firstNonEmpty already in probe.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scanner write path produces audiobook media_items ScanAudiobookFolder walks an audiobooks-typed media folder and treats each immediate subdirectory as one audiobook. For each parsed audiobook it upserts: - one media_items row with type='audiobook' - one media_files row per audio file (with chapters JSONB) - author/narrator links in item_people (kind=7, kind=8) Adds itemRepo and personRepo to the Scanner struct, wired from fileRepo.Pool() in NewScanner — no constructor signature change needed. ScanFolder dispatches to this path when folder.Type='audiobooks', bypassing the per-file movie/TV pipeline because audiobooks are folder-scoped entities. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): filesystem podcast scanner ScanPodcastFolder walks a podcasts-typed media folder, treating each subdirectory as a podcast show and each audio file inside as an episode. Writes media_items.type='podcast' + episodes rows + media_files rows. RSS-subscribed feeds (podcast_feeds table) arrive in sub-plan 5; this task covers filesystem-only ingestion. ScanFolder dispatches to this path when folder.Type='podcasts'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 5 (podcasts) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose audiobooks/podcasts library types in admin UI Adds 'Audiobooks' and 'Podcasts' options to the library-type dropdown in the admin libraries page so operators can flag a folder as an audiobook or podcast library. Extends contentLevelsForType() so the admin UI's downstream filtering treats those types correctly (audiobook -> ['audiobook'], podcasts -> ['podcast', 'podcast_episode']). Backend scanner branches for these types were already wired in sub-plan 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 139_abs_sessions to 147 for origin/main merge origin/main adds 139_media_requests at the same number our local audiobook branch had used for abs_sessions. Renumber ours to 147 to free up 139 for the upstream migration. The schema_versions row is updated in lockstep on the running database so the migrator sees the abs_sessions migration as already applied at its new version. Migrations 140-146 (podcast feeds, media_folders kind noop, audiobook feature flag, abs playback sessions, podcast episode guid, audiobook series, audiobook title cleanup) stay where they are — they don't collide with anything on origin/main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 140_podcast_feeds to 157 for origin/main merge origin/main added 140_user_permissions at the same version this branch had used for podcast_feeds. Renumber ours to 157 (next free above the collections-unify migration at 156) so 140 is free for the upstream migration. schema_versions on the running database is updated in lockstep so the migrator sees podcast_feeds as already applied at its new version. Same pattern as d59c1cb (renumber 139_abs_sessions to 147 for the prior main merge). Pending migrations after this rename: 132 (downloaded subtitles admin index, main), 140 (user_permissions, main), and 156 (unify_user_collections, this branch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 141_media_folders_kind_noop to 159 for origin/main merge Same shape as eb8f67d (the 140→157 renumber from the previous main merge). origin/main added 141_episode_title_sort_index at the same version this branch had used for media_folders_kind_noop. Renumber ours to 159 (next free above the audiobook_series truncate at 158) so 141 is open for the upstream migration. schema_versions on the running database is updated in lockstep so the migrator sees media_folders_kind_noop as already applied at its new version. Pending migrations on silo-prod after this rename: 141 (episode_title_sort_index, main) and any other newer ones from main that the branch hasn't picked up yet. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 142_audiobooks_feature_flag to 160 for origin/main merge Companion to 3c6f062's 141 renumber — origin/main also added 142_episode_catalog_entries (alongside 141_episode_title_sort_index) at a version this branch had used for the audiobooks feature flag. Renumber ours to 160 so 142 is open for the upstream migration; schema_versions on silo-prod is updated in lockstep so the migrator sees audiobooks_feature_flag as already applied at its new version. This was the only remaining collision (verified by checking for duplicate version prefixes across migrations/). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audiobooks): address foundation review comments * fix(audiobooks): tighten scanner identity handling * fix(audiobooks): propagate scanner cancellation * chore(audiobooks): adopt goose migration layout * docs(audiobooks): implementation plan sub-plan 3 (API + frontend MVP) Third of six sub-plans. 9 tasks: three REST endpoints (list/detail/ progress), TanStack Query hooks + types, three React pages (Library/Detail/Player), and navigation integration. Scoped to MVP — author/series indices, smart collections, share links, and other nice-to-haves from the spec are deferred. Streaming reuses silo's existing /api/v1/stream/{session_id}; no new transcode code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): list endpoint at GET /api/v1/audiobooks Paginated list of media_items with type='audiobook' scoped to the caller's accessible libraries via the existing access filter. Mirrors silo's existing list-style handlers for movies and series. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): detail endpoint at GET /api/v1/audiobooks/{id} Returns the media_items row, its media_files (with chapters JSONB), author/narrator extracted from item_people (kinds 7/8), and the caller's per-profile listening progress from user_watch_progress. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): progress endpoint at POST /api/v1/audiobooks/{id}/progress UPSERTs user_watch_progress for the caller's (user_id, profile_id, content_id). Body carries position_seconds; clients are expected to post every 5-10s during playback plus on pause/seek (matching silo's existing video progress cadence). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): frontend types and TanStack Query hooks TypeScript types match the JSON shapes from the new /api/v1/audiobooks endpoints (list, detail, progress). Three hooks: useAudiobookLibrary (list), useAudiobook (detail), and useReportAudiobookProgress (mutation that invalidates the detail query on success so progress updates reflect immediately). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): library grid page at /audiobooks Renders a paginated grid of audiobook cards using the useAudiobookLibrary hook. Each card links to /audiobooks/book/{id}. Cards show poster, title, and year; falls back to a "No cover" placeholder when the audiobook has no poster_url. Empty state hints to operators that they need to set a library's type to 'audiobooks'. Routes themselves are wired in Task 8 (navigation integration). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): detail page with chapter list Renders cover, title, author, narrator, year, and overview alongside a chapter list. Clicking a chapter opens an inline sticky AudiobookPlayer at that chapter's start. A "Resume" button restarts playback at the saved progress position if present. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): HTML5 audio player with chapter navigation Single-file audiobook playback for MVP. Multi-file queuing arrives in a follow-up. Streams via the existing /api/v1/direct-download GET endpoint. Position is reported to /api/v1/audiobooks/{id}/progress every 10s while playing plus on pause/seek/end. Skip-30s, playback rate select, chapter list panel. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): wire navigation and routes Adds an Audiobooks entry to the sidebar and registers the two new routes (/audiobooks for the library grid, /audiobooks/book/:id for detail). The player renders inline inside the detail page; no dedicated player route is required for MVP. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audiobooks): address native API review comments * feat(audiobooks): add ABS compatibility and polish * fix(audiobooks): stabilize ABS playback progress reporting * fix(audiobooks): clean up ABS branch review fixes * chore(audiobooks): adopt goose layout for ABS migrations * fix(audiobooks): align player seek bar props * feat(audiobooks): make libraries first-class catalog items * feat(admin): add server restart endpoint * fix(audiobooks): address review comment findings --------- Co-authored-by: RXWatcher <14085001+RXWatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-06-07 15:57:05 -04:00
3 people	0163df3683	[codex] Add IntroDB marker integration and dialogue-aware Chromaprint refinement (#57) ... * docs(markers): design + implementation plans for multi-source markers & TheIntroDB contribution * fix(markers): TheIntroDB read-path correctness (TVDB, real confidence, best candidate) Honor TVDB ids in /media lookups (previously dropped — anime/TheTVDB-first libraries got no markers), decode and use the real per-segment confidence and submission_count instead of a hardcoded 0.9, and pick the most-submitted / highest-confidence candidate when several are returned. Adds httptest coverage for the introdb client and provider. Phase 1 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): multi-source dispatch, per-provider config, per-segment provenance Add marker_provider_config (per-provider fetch enable/priority + contribute gates, contribution off by default) and a cached ProviderConfigStore. Add Registry.FetchMerged: query all fetch-enabled providers concurrently and keep the best candidate per segment (submission_count, then confidence, then fetch priority), stamping each winning marker with its provider/algorithm. Thread per-segment provenance through MarkerUpdatePayload and scanner.MarkerUpdate (additive SegmentProvenance overrides) so a merged result writes correct per-segment provider/confidence/algorithm; the legacy shared columns keep a summary. The lazy-playback path now uses FetchMerged. With only TheIntroDB enabled, behavior is unchanged. Phase 2 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): TheIntroDB submission client, contribution audit, service engine Add a markers.Submitter capability and implement it on the introdb provider (POST /v3/submit, GET /v3/user/stats; key required, usage-limit aware, applies the null start/end conventions). Add the marker_contributions audit table and a value-hash-keyed ContributionStore for idempotency. Add ContributionService: resolves enabled submitter providers, gates eligibility (never re-submit online-sourced markers; auto runs require contribute_auto_local + scanner-intro above the per-provider confidence threshold), checks idempotency, submits, and records. Wired in main.go; no trigger yet (admin API and task follow). Phase 3 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): admin marker editing, contribution, and provider config endpoints Add the RequireAdmin marker API: GET/PUT /admin/files/{id}/markers (read with provenance; manual upsert where a segment object sets and null clears), DELETE .../markers/{segment}, POST .../contribute and GET .../contributions, plus GET/PUT /admin/markers/providers[/{provider}] and a .../validate key-check returning user stats. Manual writes go through the priority-gated UpsertMarkers (source=manual) and notify live sessions; a new FileRepository.ClearMarkers nulls a segment's columns. Validation mirrors the contribution rules. Phase 4 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): daily auto-contribution task for local intro markers Add ContributeMarkersTask (daily 04:00, after local detection): when a provider has contribute_enabled + contribute_auto_local, page through episode files with a scanner intro marker at/above the provider's confidence threshold (new ContributionStore.CandidateLocalIntroFiles keyset query) and run them through ContributionService with Auto=true. No-op when no provider opts in; idempotent and resumable across runs. Phase 5 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(intromarkers): refine chromaprint starts with dialogue cues * feat(markers): finish marker management backend * feat(web): add marker editing UI * feat(markers): use plugin marker providers * fix(markers): address PR review feedback * feat(player): show marker labels on seek hover * fix(markers): type nullable marker mutation params * feat(markers): audit marker edits and add permission --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-06 22:29:26 -04:00
5 people	9f73ac6f1a	feat(realtime): improve web UI reactivity and admin visibility (#48) ... * fix(web): scope realtime user state events * feat(events): add canonical catalog event publishers * feat(events): publish canonical catalog events * refactor(web): centralize realtime events provider * feat(events): normalize user state event name * feat(web): patch item user state from realtime events * fix(web): refetch active catalog on realtime changes * fix(events): publish item changes during metadata enrichment * fix(web): improve dashboard and mutation reactivity * feat(admin): improve realtime session activity * feat(admin): refine playback admin surfaces * feat(admin): improve library task controls * fix(collections): position defaults progress below header * feat(library): surface matcher backlog * fix(admin): hide matcher backlog from server activity * chore(migrations): renumber branch migrations * feat(admin): show registered devices without overrides * feat(admin): improve scheduled task visibility * fix(realtime): tighten admin update handling * docs(admin): document library job id parsing * docs(library): explain mount check feedback timing * fix(library): guard metadata match queue handlers * fix(admin): avoid stale queued job cancellation * fix(settings): harden device registration and task timing * fix(jellycompat): fill large browse pages * perf(jellycompat): compress and batch list image work * feat(autoscan): pluggable scan-source autoscan category (Sonarr/Radarr) (#44) * docs: design spec for autoscan arr polling Periodic poller over autoscan-enabled Radarr/Sonarr instances (reusing request_integrations) that maps import paths to Silo media folders and enqueues targeted scans via the existing scantrigger + scanqueue. Lean single-service model: no cross-node fan-out guard or retry queue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan arr polling Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): settings and sources schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): core types Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): path rewrite helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): dedupe imported paths to parent folders Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): arr import-history client Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): settings + sources repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): redis scan-suppression seam Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): PollOnce poll cycle * feat(autoscan): poll task and wiring * feat(autoscan): admin API endpoints * feat(autoscan): admin API endpoints Adds ErrIntegrationNotFound sentinel (errors.Is) instead of string matching. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan types and hooks Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan admin tab * fix(autoscan): release suppression claim on enqueue failure; reconfigure trigger on interval change; skip source on key-resolution error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): update handler test for 3-arg NewAutoscanHandler * fix(autoscan): per-path suppression key, bounded poll window + overlap, boundary-safe rewrites, GREATEST cursor guard, async trigger, quiet unresolved-path skip, FK->404 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize Windows path separators, surface status errors, re-seed source editor on save Addresses minor code-review findings: Windows backslash paths now normalized before rewrite/dedupe; HandleStatus returns repository errors instead of 200; the per-source editor re-seeds from server data after a save. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan rewrite-sync from arr root folders Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan rewrite-sync Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): suffix-match rewrite suggester Add suggestRewrites / commonSuffixLen for Task 1 of the autoscan arr-polling feature. Pure function: matches arr root-folder paths to Silo media folder paths by longest common trailing segment count, adjusted for depth-delta so coincidental same-named segments at different structural levels don't inflate confidence. Categorises each arr root as Proposed, Ambiguous, Unmatched, or Covered by an existing PathRewrite rule. TDD: test file written first, verified failing, then implementation added. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): GetSource single-source lookup * feat(autoscan): arr root-folder client + Silo folder lister * feat(autoscan): Service.SuggestRewrites Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): rewrite-suggestions endpoint Add GET /autoscan/sources/{id}/rewrite-suggestions admin endpoint: extend the autoscanTriggerer interface with SuggestRewrites, wire SetRewriteResolvers in the router, and add handler + test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan rewrite-suggestions types and hook * feat(web): autoscan sync-rewrites preview * fix(autoscan): normalize covered-rule paths, dedup roots/folders, skip no-op suggestions Addresses final-review edge cases: coveredBy normalizes the existing rewrite's From (so a stored Windows/dup-slash rule still covers a root); duplicate arr roots and duplicate Silo folder paths are de-duplicated; an arr path that already equals its Silo path is not proposed as a no-op rewrite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): vitest 4 compatible fetch spy in recipes.test (unblocks build after vitest 4.1.0 bump) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): non-null suggestion slices + move Sync into rewrites card - suggestRewrites initializes Proposed/Unmatched/Ambiguous/Covered to empty slices so the JSON response is [] not null — fixes the 'Something went wrong' crash when every root is already covered (frontend mapped over null). - Move the sync button into the Path rewrites card beside 'Add rewrite' and rename it 'Sync rewrites'; guard the proposed map with ?? []. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): long root-folder timeout + sync spinner + collapse rewrites on load - Root-folder fetch for sync uses a 2-min timeout: Radarr/Sonarr compute unmappedFolders by scanning all roots, so a large library's /rootfolder takes 20-30s+ and tripped arrclient's 30s default (Sonarr 502'd at exactly 30s). - Spin the sync icon + show 'Syncing…' while the request is in flight. - Path rewrites card starts collapsed on page load. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): rescan on Sonarr/Radarr file renames History polling previously only tracked downloadFolderImported events. A rename in Sonarr/Radarr (episodeFileRenamed / movieFileRenamed) moves a file without an import event, leaving the library folder stale until the next full scan. Extend the history client to also surface renamed paths: both the new path and the old sourcePath, since a rename can move a file between folders and both parents may need rescanning. Delete events are still skipped — upgrade-deletes are covered by the paired import, and standalone deletes carry no file path in arr history. Renames the interface method ImportedPaths -> ChangedPaths to reflect the broader scope. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): synchronize trigger test with detached PollOnce goroutine HandleTrigger dispatches PollOnce on a detached goroutine and responds 202 immediately. The test read trig.called straight after the handler returned, racing the goroutine (usually 'PollOnce was not invoked') and reading the field without synchronization (a data race under -race). Signal completion through a channel the fake sends on when PollOnce runs; the test waits on it (bounded) before asserting. The channel send happens-before the receive, so the subsequent read of called is race-free. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan as a pluggable scan-source category Reframes autoscan from a Requests-coupled, arr-only feature into a standalone Autoscan category. Change-detection providers become out-of-process plugins via a new additive scan_source.v1 capability (client-pull, opaque marker); Sonarr/Radarr is the first provider. Host keeps a provider-agnostic resolve/suppress/enqueue engine; all arr-specific logic (and path rewrites) move into the plugin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for scan_source.v1 SDK capability First of the per-repo plans from the autoscan-plugin-architecture spec. Adds the additive scan_source.v1 capability to silo-plugin-sdk (proto + codegen + capability allowlist + runtime wiring), TDD per task, tagged as v0.5.0 so the host and arr-plugin plans can build against it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan host backend (part 1 of 2) Backend for the standalone Autoscan category: scan_source.v1 plugin plumbing (pluginhost client + plugins.Service resolver), generalized engine driven by a provider seam, autoscan_connections + autoscan_sources schema (decoupled from Requests), connection resolution (own or Requests-linked), admin API. Depends on silo-plugin-sdk v0.5.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plans for autoscan arr plugin + host UI arr plugin: new installable scan_source.v1 plugin (history imports+renames, rewrites, Silo-native paths), structured like silo-plugin-tmdb; ports the arr-specific logic from the closed PR #43. host UI (part 2 of 2): standalone Autoscan admin category (connections, sources, settings) extracted out of Requests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(autoscan): replace silo-plugin-sdk with local scan_source.v1 checkout Temporary dev replace so the host backend can build against the unreleased scan_source.v1 capability (silo-plugin-sdk PR #2). Finalize to v0.5.0 once the SDK is tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pluginhost): scan_source.v1 capability client wrapper Adds ScanSourceClient struct, the Client.ScanSource() accessor (mirrors ScheduledTask pattern), and a PollChanges method. Also introduces client_test.go with capability-gate tests for both scheduled_task.v1 and scan_source.v1 using a lazy gRPC ClientConn. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test+fix(pluginhost): cover capability-id gate, dedicated scan_source timeout Adds a "wrong id returns error" subtest to both capability-gate tests so the capability-ID component is exercised independently of the type. Introduces DefaultScanSourceTimeout (2m) for PollChanges, which polls an external arr API that can be slow, instead of the generic 10s control timeout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(plugins): expose scan_source.v1 client resolver Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(migrations): autoscan v2 schema (connections + sources, no requests FK) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): v2 types and repository Replace the request_integrations-coupled model with the decoupled v2 schema (autoscan_settings + autoscan_connections + autoscan_sources). Connection CRUD, source upsert/list/get, and AdvanceMarker/RecordError for opaque marker bookkeeping. ErrIntegrationNotFound becomes ErrNotFound. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): resolve connections (own credentials or Requests-linked) ConnectionResolver turns a stored Connection into concrete credentials, reading a soft-linked Requests integration's live base URL/key when RequestIntegrationID is set, then resolving the api-key ref to plaintext. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): scan-source provider seam over the plugin resolver ScanSourceProvider lets the engine poll changed paths without a live plugin; pluginProvider adapts plugins.Service.ScanSourceClient in production. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): generic engine drives sources via scan_source provider Rewrite PollOnce to iterate enabled sources, resolve each connection, poll the provider for changed paths, and run the salvaged resolve→suppress→enqueue loop (uniqueParentDirs, (folder,path) suppression key, RequestError quiet-skip, release-claims-on-enqueue-fail) verbatim. Store the opaque next marker via AdvanceMarker only after a successful enqueue; RecordError + keep marker on provider failure. Tests reworked onto a fakeProvider/fakeStore with an added opaque-marker-verbatim assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): drop conflicting connection CHECK, add connection resolver tests - migration 172: remove the autoscan_connections_source_present CHECK. It conflicted with request_integration_id ON DELETE SET NULL: deleting a Requests integration that a linked-only connection (base_url NULL) points at would null the FK and trip the CHECK, blocking the delete. The intended behavior is for the connection to survive as an orphaned 'needs attention' row. Creation-time validity is now enforced at the application layer. Verified on a throwaway DB: full chain applies and the delete-cascade leaves an orphaned (both-null) connection. - connection.go: TrimSpace the api key ref + resolved secret before the empty-string checks, matching requests.resolveAPIKey parity. - connection_test.go: fake-based tests for ConnectionResolver.Resolve (own creds, linked, linked-missing error, lookup error, trim/fallback). - repository.go: bound RecordError's stored last_error to 2048 chars. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): autoscan v2 admin endpoints Rewrite the autoscan admin HTTP handler against the v2 model: settings, connection CRUD, source update, manual trigger (detached PollOnce), and status. Connection/source responses omit api_key_ref and resolved keys (has_api_key flag only); unknown connection/source ids map to 404 via autoscan.ErrNotFound. Retire the host-side rewrite-suggestions endpoint (now lives in the arr plugin). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): wire v2 service, routes, retire rewrite-suggestions Export PollChangesClient/ScanSourceResolver from the autoscan provider so the api package can declare a structurally-conformant plugin adapter (Go has no return-type covariance, so the adapter must name the interface as its return type). Add api.BuildAutoscanService with the requests-integration lookup and plugin scan-source adapters, shared by the router (manual trigger) and the background poll task. Re-wire router routes to the v2 connections/sources/settings/trigger/status surface and drop the rewrite-suggestions route. Update cmd/silo to build the v2 poll task, seeding its interval from Settings.DefaultPollIntervalSeconds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): enforce connection requires own URL or a Requests link Migration 172 dropped the DB CHECK that required an autoscan connection to carry either its own base_url or a request_integration_id, delegating that invariant to the application layer — but the enforcement was never added, so HandleCreateConnection/HandleUpdateConnection accepted both-NULL orphans that ConnectionResolver.Resolve would hand a plugin as an empty base URL. Add a shared validateConnectionInput helper (whitespace-only request_integration_id counts as absent) and reject both-empty payloads with HTTP 400 on both the create and update paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): deliver resolved connection to plugin PollChanges now populates PollChangesRequest.Connection with the resolved {base_url, api_key} instead of dropping the conn param on the floor. Drops the stale doc comment claiming the connection was delivered out-of-band at upsert time -- that mechanism never existed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): auto-discover sources from installed scan_source plugins Auto-discovery seeds a disabled, connection-less source row per installed scan_source.v1 capability before an operator binds a connection, so connection_id is now nullable end to end: - migration 172: connection_id drops NOT NULL (still ON DELETE RESTRICT) - Source.ConnectionID becomes *string; repository scans/writes it as nullable and adds idempotent EnsureSource (INSERT ... ON CONFLICT DO NOTHING) - new ScanSourceLister seam + Service.DiscoverSources, called at the start of PollOnce (errors logged, non-fatal); production adapter enumerates ListEnabled -> ListCapabilities filtered to scan_source.v1 - PollOnce skips an enabled source with no connection bound, recording 'no connection bound' so the UI can surface it - HandleUpdateSource rejects enabling a source with no effective connection (400); source DTOs expose connection_id as nullable - BuildAutoscanService / NewService thread the installation store at both wiring sites (router + poll task) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): honor per-source poll interval PollOnce now skips an enabled source that ran too recently: the floor is source.PollIntervalSeconds when set, else settings.DefaultPollIntervalSeconds. The global poll task fires at the default cadence, so this makes the per-source interval a 'poll at most every N seconds' floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): reconcile spec + arr-plugin plan with credential-in-request + auto-discovery The credential-delivery mechanism changed during execution: the host now passes resolved {base_url, api_key} in PollChangesRequest.connection each poll (not plugin runtime config). Also records source auto-discovery, nullable connection_id, and the per-source interval floor decided at the final integration review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan v2 types and query hooks Replace v1 autoscan types and hooks with v2 DTOs matching the backend handler (autoscan.go): settings, connection (with has_api_key, no raw key), source (installation_id/capability_id/connection_id), status. Add connections CRUD hooks, useAutoscanStatus, update sources hook to v2 input shape. Retain deprecated shims for AutoscanPathRewrite, AutoscanRewriteSuggestions, and useAutoscanRewriteSuggestions so AdminRequests.tsx continues to compile until Task 6 removes that tab. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan connections panel (reuse or own) Card+Table listing connections with "Reused from Requests" / "Own" badges. Add/edit dialog with two modes: reuse a Sonarr/Radarr Requests integration or enter own name/URL/API-key credentials. Delete with alert-dialog confirm. Never renders key material — only has_api_key is sent by the backend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan sources panel Table of auto-discovered scan sources (one row per installed scan_source plugin capability). Operator can bind a connection via inline Select (auto-saved on change), set a per-source poll interval (saved on blur), and toggle enabled. Shows a "Needs connection" badge for unbound sources; attempting to enable without a connection lets the backend 400 surface via the existing toast in useUpdateAutoscanSource.onError. Status column shows last_run_at relative time or last_error with icon. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): standalone Autoscan admin page Tabs page (Sources | Connections | Settings) mirroring AdminRequests header/layout. Settings tab exposes global enable switch, default poll interval, and debounce — all auto-saved on blur or toggle. "Run now" button calls useTriggerAutoscan and toasts "Autoscan triggered" on 202. Route and sidebar nav are intentionally deferred to Task 5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): route and sidebar nav for Autoscan category Add /admin/autoscan route pointing to AdminAutoscan and a matching "Autoscan" item in the Content group of the admin sidebar (with RefreshCw icon), so the new standalone page is reachable from the nav. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(web): move Autoscan out of Requests into its own category Remove the Autoscan tab, AutoscanTab/AutoscanSourceEditor component definitions, and AutoscanSettingsFormState from AdminRequests.tsx. Delete the Task-1 compatibility stubs: AutoscanPathRewrite and AutoscanRewriteSuggestions types from api/types.ts, and the useAutoscanRewriteSuggestions no-op shim from useAutoscan.ts. The build confirms zero dangling references. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): allow unbinding a source connection (full-state source update) Change the source-update input struct's connection_id from string to *string so the UI can send null to unbind, a UUID to bind, or omit (null) to clear. Remove the fall-back-to-existing logic; the handler now sets the source's ConnectionID directly from the input. The enable-guard fires when the resulting connection is nil regardless of cause. Frontend sends the complete triple (connection_id, enabled, poll_interval_seconds) on every mutation site; selecting "— No connection —" sends null for a real unbind. Adds aria-label to connection Select and interval Input for accessibility. Backend tests cover bind, unbind, unbind while enabled → 400, and enable without connection → 400. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(migrations): backfill autoscan v1 settings+connections instead of dropping Migration 172 unconditionally DROPped the shipped v1 autoscan_settings/ autoscan_sources (migration 171), losing an upgraded operator's enable flag, poll cadence, debounce, and arr server list — autoscan came back OFF. Rewrite 172 up to be non-destructive of what can be carried: rename the v1 tables aside, create the v2 schema, backfill settings (poll minutes -> seconds) and seed a reusable LINKED connection per distinct v1 source integration, then drop the renamed v1 tables. v2 sources are keyed on a plugin (installation_id, capability_id) that did not exist in v1, so they are left to runtime discovery; path rewrites move to plugin config and are intentionally not carried. Verified against a throwaway DB: after 171 + v1 seed data, applying 172 yields enabled=true, default_poll_interval_seconds=300, debounce_seconds=30, and one autoscan_connections row linked to the v1 integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): preserve api key on metadata-only connection edit UpdateConnection unconditionally wrote api_key_ref = nullable(c.APIKeyRef), so a metadata-only edit (the UI omits the key when left blank — "leave blank to keep existing") NULLed the stored key and broke the next poll. Mirror requests' UpdateIntegration: api_key_ref = CASE WHEN $5 = '' THEN api_key_ref ELSE $5 END, passing the raw trimmed string so a blank incoming ref keeps the existing value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): skip orphaned sources + add source delete endpoint An enabled source whose scan_source plugin was uninstalled/disabled kept its autoscan_sources row, which errored every poll cycle, and there was no way to remove it. DiscoverSources now returns the set of currently-discovered (installation_id, capability_id) pairs; PollOnce skips any enabled source not in that set quietly (no RecordError), stopping the per-cycle error spam for orphans. A nil set (no lister / discovery failed) disables pruning so a transient discovery failure does not silence live sources. Adds DELETE /admin/autoscan/sources/{id} -> HandleDeleteSource -> repo.DeleteSource so an operator can clear orphans (unknown id -> 404). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reject reused connection when Requests integration is disabled RequestIntegrationLookup.Get returned a linked integration's base_url/api_key even when the integration was disabled or had a blank base_url (the v1 poll gate `WHERE ri.enabled = true` was dropped in v2). Now Get surfaces a disabled or unconfigured linked integration as an error, which the engine turns into a logged skip / RecordError instead of polling an unusable target. The gating is extracted into a pure checkRequestIntegrationUsable helper so it is unit-testable without a DB-backed repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reschedule poll task on settings change HandleUpdateSettings no longer rescheduled the poll task (the v1 triggerUpdater / UpdateTriggers wiring was dropped in v2), so a default_poll_interval_seconds change only applied after a restart. Re-add an optional triggerUpdater (taskmanager.UpdateTriggers) on AutoscanHandler, wired via SetTriggerUpdater from the router when a task manager is available. On a successful settings update the handler recomputes the interval trigger from default_poll_interval_seconds and calls UpdateTriggers("autoscan_poll", ...). The dependency is optional: a nil updater skips rescheduling so tests need no task manager, and a reschedule failure is non-fatal (the interval is persisted). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): disable enable toggle for unbound sources, add source delete + interval hint - Disable the Enable switch when a source has no effective bound connection (connection_id null and no pending edit selection), re-enabling once bound. - Add useDeleteAutoscanSource hook mirroring useDeleteAutoscanConnection pattern. - Add per-row delete button (Trash2 icon → AlertDialog confirm) to let operators remove orphaned/unwanted source rows. - Add interval floor helper text showing the global default poll interval so operators know values below it have no effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): consume source_paths from merged scan_source contract The merged plugin SDK renamed PollChangesResponse.changed_paths to source_paths and the plugin now returns RAW source-namespace paths. pluginProvider.PollChanges reads GetSourcePaths(); the host applies per-source path rewrites before resolving/enqueueing (separate commit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(migrations): add path_rewrites to autoscan_sources Add path_rewrites jsonb NOT NULL DEFAULT '[]' to the autoscan_sources CREATE in migration 172 (unreleased/branch-only, so amended in place). The host now owns per-source prefix rewrites. v1 path_rewrites cannot be backfilled (v2 sources key on a plugin installation/capability with no v1 mapping); documented that operators must re-enter rewrites post-upgrade. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-owned per-source path rewrites Rewrite ownership moved from the scan_source plugin to the host. The plugin returns raw source-namespace paths; the host now normalizes separators and applies the source's per-source prefix rewrites before dedupe/resolve/enqueue. - types: add PathRewrite{From,To} and Source.PathRewrites - rewrite: re-add applyRewrites/normalizeSeparators; apply the MOST-SPECIFIC (longest From) match, not first-match, so a broad rule can't shadow a nested one regardless of ordering - service.PollOnce: rewrite raw provider paths before resolveAndClaim - repository: marshal/unmarshal path_rewrites jsonb in UpsertSource and all source scans (EnsureSource discovery rows take the DB default []) - handlers: autoscanSourceInput/response + status DTO carry path_rewrites (full-state like connection_id); reject blank from/to with 400 - tests: rewrite unit tests, engine applies rewrites before enqueue, handler round-trips path_rewrites and 400s on a blank rewrite Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): discover installed scan_source plugins on sources-list view A scan_source plugin installed via the normal /admin/plugins flow must show up in the Autoscan component immediately, not only after a poll cycle (which runs only when autoscan is enabled). HandleListSources now runs discovery (seeding a disabled, connection-less source row per installed scan_source capability) before listing. Best-effort: discovery failure does not block listing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): per-source path rewrites editor + plugins-page install hint Add AutoscanPathRewrite type and path_rewrites fields to AutoscanSource/ AutoscanSourceInput. SourcesPanel gains an expandable rewrite editor per source row (from→to pairs, Add/Remove/Save) threaded into the full-state body so connection, interval, and rewrite changes always carry all fields. Adds a Plugins-page install hint in both the empty state and above the table for discoverability. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): host-owned path rewrites + install/discovery flow Reconcile the spec with the merged SDK decision (rewrites moved host-side; PollChangesResponse.source_paths carries raw provider paths). Document that scan-source plugins install via the normal /admin/plugins page and surface in Autoscan via discovery (run on poll cycles and on sources-list view). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: depend on merged silo-plugin-sdk via pseudo-version (drop local replace) PR #2 (scan_source.v1 + source_paths) is merged to silo-plugin-sdk main, so the host can resolve the canonical module at the merged commit (v0.4.1-0.20260603030807-807b07e785b2) instead of a local-path replace. The branch now builds off-machine (CI/Docker). Bump to a clean v0.5.0 once tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(migrations): single clean autoscan v2 migration (v1 never shipped) The v1 in-process autoscan (migration 171) was never released to origin/main, so no live system has v1 autoscan data to preserve. Collapse the v1-create + v2-rename/backfill/drop dance into one clean 171 that creates the v2 connections-based schema directly. Removes 172 entirely. The runner applies by version set-difference with no checksum validation, so the already-migrated test instance (171+172 recorded) skips both and is unaffected; fresh installs get the clean v2 schema in one step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): allow many sources per plugin + add-source enumeration Drop the one-source-per-(installation, capability) model. A single installed scan_source plugin capability can now back many sources, each bound to a different connection (e.g. one Sonarr plugin fronting four arr servers). - migration 171: remove the autoscan_sources UNIQUE(installation_id, capability_id) constraint; sources are operator-created, not auto-seeded. - repository: replace UpsertSource (relied on the unique conflict) with a plain CreateSource (fresh uuid) + a by-id UpdateSource; remove EnsureSource. - discovery: replace auto-seeding (DiscoverSources/RefreshDiscovered) with ListAvailableScanSources (the Add-source picker list, enriched with plugin id + display name) and an installedScanSources set used only for orphan-skip. - service: PollOnce stops seeding and instead fetches the installed-capability set for orphan detection; Store gains GetSource and drops EnsureSource. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): connection test endpoint (engine) Add Service.TestConnection / TestConnectionByID: resolve a connection (ad-hoc input or an existing stored connection) to concrete credentials and probe the arr GET /api/v3/system/status with a short timeout. A reachable/authorized target yields OK=true plus the reported version; an unreachable / 401 / non-200 target yields OK=false with a human-readable error (the probe failure is part of the result payload, never an error from the method itself). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-side rewrite suggester + admin API for new endpoints Port the path-rewrite suggester back host-side (it had moved into the plugin): suggestRewrites suffix-matches arr root folders against Silo media folders to propose path rewrites, reporting proposed / unmatched / ambiguous / covered. Service.SuggestRewrites resolves the source's bound connection, lists arr roots (GET /api/v3/rootfolder) and Silo folder paths, and runs the matcher; a source with no bound connection returns ErrNoConnection (400). Admin API (all admin-gated): - POST /admin/autoscan/sources create a source - GET /admin/autoscan/scan-source-plugins Add-source picker list - POST /admin/autoscan/connections/test probe a connection - GET /admin/autoscan/sources/{id}/rewrite-suggestions sync rewrites HandleListSources no longer auto-seeds; create validates the capability is currently installed and that enabling requires a connection. Wiring threads the arr root-folder/status client and the catalog folder lister through BuildAutoscanService; the lister now surfaces plugin id + display name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan hooks + types for sources, connection test, rewrites Add types and React Query hooks backing the autoscan admin UI batch: - AutoscanAvailableSource / useAvailableScanSources (scan-source plugins) - AutoscanSourceCreateInput / useCreateAutoscanSource (POST sources) - AutoscanConnectionTestResult / useTestAutoscanConnection (advisory test) - AutoscanRewriteSuggestions / useAutoscanRewriteSuggestions (on-demand) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): add-source dialog + sync-from-arr rewrites in SourcesPanel Add a "+ Add source" header action opening a dialog that creates a scan source from any installed scan-source plugin bound to an arr connection, so operators can add one source per connection (e.g. four arr instances). Empty state links to /admin/plugins when no plugins are installed. Add a "Sync from arr" button to each source's rewrite editor that fetches root-folder rewrite suggestions and renders a preview: checkbox-selectable Proposed rewrites plus collapsed Unmatched / Ambiguous / Already-mapped sections. "Apply selected" merges the checked rewrites (dedupe by `from`) and persists via the normal full-state source PUT. Sync is disabled until the source has a bound connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): test-connection button in autoscan ConnectionsPanel dialog Add an advisory "Test connection" button to the add/edit connection dialog. It probes the current dialog input — connection_id when editing, request_integration_id in reuse mode, or base_url/api_key_ref for own credentials — and renders the result inline: green "Connected (vX.Y)" on success, red error on failure. Never blocks save; stale results clear when credential fields change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan page polish + global enable toggle in header Surface a global Autoscan enable toggle and an enabled/disabled status badge next to the page title, alongside the existing "Run now" header action so primary controls are reachable without opening a tab. Remove the now-redundant enable switch from the Settings tab (it points at the header toggle instead). Tighten header layout for wrap on narrow widths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): right-align autoscan enable toggle + Run now in the page header Drop the redundant nested justify-between wrapper so the header actions sit directly under .page-header (space-between + bottom-align), matching the /admin/libraries header layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): hold poll marker when paths return but none resolve A freshly-enabled source whose path_rewrites aren't configured yet returns provider paths that resolve to zero library folders. PollOnce previously advanced the marker unconditionally on any successful poll, permanently skipping those imports. Now the marker advances only when there is nothing to do (zero paths) or at least one path resolved+enqueued; when paths come back but none resolve, the marker is held and an explaining error recorded so the operator can fix the rewrites and a later poll re-reads the same window. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): don't prune sources of disabled-but-installed plugins PluginScanSourceLister used the installation store's ListEnabled, so a temporarily-disabled plugin dropped out of the discovered set and PollOnce treated its sources as orphaned, skipping them with no last_error (silent vanish). Switch to List so only a fully-uninstalled plugin counts as orphaned; a disabled-but-installed plugin's sources are still attempted and surface a visible RecordError when the client fails to load. The Add-source picker shares the same all-installed set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): treat empty request_integration_id as no link ConnectionResolver.Resolve gated the linked-integration path on a non-nil RequestIntegrationID pointer, so a pointer-to-empty-string (from a both-NULL orphan or a stripped link) called requests.Get(""). Guard on a non-empty trimmed value so it falls back to the connection's own fields instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): align startup poll interval with reschedule computation Startup seeded the poll task by integer-dividing default_poll_interval_seconds by 60 (minutes), while HandleUpdateSettings reschedules with seconds*1000 ms; the two diverged for sub-minute and non-60-multiple intervals. NewAutoscanPollTask now takes the interval in milliseconds and main.go seeds it as seconds*1000, matching the reschedule path so both agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize stored rewrite From at poll time applyRewrites matched the stored From after only TrimSpace/TrimSuffix, while suggest.go coveredBy normalizes via normalizePath (backslash->slash, collapse '//'). A Windows-style or dup-slash stored rewrite was thus reported 'covered' at suggest time yet never matched at poll time. applyRewrites now normalizes From through normalizePath so poll-time and suggest-time agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): don't corrupt source poll interval on enable/connection change Add a `parseInterval` helper that maps empty input to null (use global default), valid positive integers to the integer, and any other mid-edit-invalid value to the source's currently-persisted `poll_interval_seconds` — so toggling the enable switch or changing the connection cannot silently overwrite the interval with 0 or NaN. Wire the helper through `fullBody()` (the single source of truth for PUT payloads) and remove the two inline duplications in `handleConnectionChange` and `handleRewriteSave` that both previously used raw `Number()`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): make the source connection optional (provider-agnostic) A host connection is the credential/endpoint for server-based providers (Sonarr/Radarr); other scan_source providers (e.g. a CephFS/filesystem watcher that reads ceph.dir.r* xattrs) need none. PollOnce now polls connection-less sources, passing an empty ResolvedConnection the plugin may ignore; a plugin that requires credentials surfaces the error at poll time. Drops the enable-requires-connection 400s. Provider-specific config lives in the plugin's own global_config_schema, not a host connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): provider-agnostic autoscan copy + optional source connection Replace arr-hardcoded framing in AdminAutoscan, SourcesPanel, and ConnectionsPanel with neutral scan-source language. Remove the connection-required gate on the source enable toggle so connectionless providers (e.g. filesystem watchers) can be enabled; soften the badge from "Needs connection" to "No connection". Sync-from-server button remains gated on a bound connection (it needs a server to query). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: repo-relative paths in autoscan plans Replace local absolute filesystem paths (/opt/silo, sibling checkouts, /tmp/go/bin) in docs/superpowers/plans with repository-relative wording per CLAUDE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): rune-safe last_error truncation Truncate RecordError messages on a UTF-8 rune boundary so a byte-bounded cut can't split a multi-byte rune and store invalid UTF-8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): advance marker when resolved-but-suppressed (not unresolved) resolveAndClaim now reports resolvedAny (whether any path mapped to a Silo library folder, independent of suppression). PollOnce gates the "none matched a Silo library folder" hold+RecordError on !resolvedAny instead of len(targets)==0, so a poll whose paths resolved but were all debounced/suppressed advances the marker instead of being treated as a misconfiguration. Adds a regression test for the suppressed case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): normalize request_integration_id Trim whitespace and collapse empty-after-trim request_integration_id to nil on connection create and update, so a pointer-to-"" or " " is never persisted as a bogus Requests link. Also corrects a stale migration-172 comment to 171 (the collapsed migration number). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(autoscan): provider-agnostic poll-task copy Rename the poll task to "Autoscan poll" with a provider-agnostic description and progress message; drop Sonarr/Radarr/arr wording. Key() (autoscan_poll) is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): fix typo in connectionless-source test name Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add scan source management * chore(deps): bump silo-plugin-sdk for structured scan source changes Pins silo-plugin-sdk to 0d78651, which adds source_config on PollChangesRequest plus the structured changes / ScanSourceChangeScope fields on PollChangesResponse that internal/autoscan/provider.go already consumes. Without this the branch fails to compile against the prior pin (807b07e). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): label scan sources by connection name in admin UI arr-plugin sources fan out one-per-connection under a single generic "arr" capability, so every row in the Sources and Activity panels rendered an identical "arr (plugin #N)" label. Lead with the bound connection name (Radarr/Sonarr/...) instead, demoting capability + plugin to a subtitle. Sources without a connection (e.g. cephfs) keep the capability fallback. Activity threads a source_id -> connection name lookup (built from the existing sources + connections queries) through the scan/poll tables the same way librariesByID is threaded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): spec for generic + operator-editable source labels Design for a shared label-resolution helper (operator label -> connection name -> manifest display_name -> capability_id) consumed by the Sources and Activity panels, plus an operator-editable per-source label backed by a new autoscan_sources.label column. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): implementation plan for source labels Task-by-task TDD plan: migration 174 (label column), Go domain/repo/handler wiring with server-side normalization, shared frontend label helper, and Sources/Activity panel integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): migration for source label column * feat(autoscan): source label domain field + normalizer * feat(autoscan): persist source label in repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): accept, normalize, and return source label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add label to source API types * feat(autoscan): shared source-label resolution helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): polish source-label helper per review Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): label sources via shared helper + operator label input Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): clarify source label naming per review * feat(autoscan): resolve activity source labels via shared helper Replace the sourceNames Map plumbing in ActivityPanel with SourceLabelLookups and delegate both name functions to resolveEventSourceName from @/lib/autoscanLabels, enabling the full label chain (operator label → connection name → manifest display_name → capability_id) for all Scan History and Poll log rows. * fix(autoscan): carry label on status source + guard poll label Final-review follow-ups: add the label field to the autoscanStatusSource response (and AutoscanStatusSource type) so the status view matches the source response per spec, and give pollSourceName a non-empty fallback for symmetry with scanSourceName. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): resolve source aria-labels through the label chain Replace the legacy capability-only sourceLabel() helper with resolveSourceName() (operator label -> connection -> display_name -> capability). Row controls now announce the row's resolvedLabel (reflecting in-progress edits) and the delete dialog announces the resolved name, so screen readers hear "4K Movies" instead of "arr (plugin #4)". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): paginate queue + history with a shared table pager Replace the card/table hybrid and 200-row "Load more" cap on the autoscan Activity panel with proper tables and real pagination. Backend: add offset + total-count to the scans/events list endpoints so history pages through the full set instead of a capped window. Extract shared event/scan WHERE-clause builders so list and count filter identically, and add CountEvents / CountAutoscanScans. Frontend: add a reusable TablePagination component (rows-per-page, "showing X-Y of Z", numbered window with ellipses, responsive) and reuse it for the server-paginated history (scans + polls) and the client-paginated live queue. Unify all three tables behind one DataTable shell so they read as one family. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> * fix(migrations): renumber PR 48 migrations * fix(migrations): tolerate stale device profile ids --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: fluxis <warmasterx555@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 22:43:20 -04:00
4 people	177cfdc485	feat(autoscan): pluggable scan-source autoscan category (Sonarr/Radarr) (#44) ... * docs: design spec for autoscan arr polling Periodic poller over autoscan-enabled Radarr/Sonarr instances (reusing request_integrations) that maps import paths to Silo media folders and enqueues targeted scans via the existing scantrigger + scanqueue. Lean single-service model: no cross-node fan-out guard or retry queue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan arr polling Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): settings and sources schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): core types Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): path rewrite helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): dedupe imported paths to parent folders Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): arr import-history client Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): settings + sources repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): redis scan-suppression seam Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): PollOnce poll cycle * feat(autoscan): poll task and wiring * feat(autoscan): admin API endpoints * feat(autoscan): admin API endpoints Adds ErrIntegrationNotFound sentinel (errors.Is) instead of string matching. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan types and hooks Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan admin tab * fix(autoscan): release suppression claim on enqueue failure; reconfigure trigger on interval change; skip source on key-resolution error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): update handler test for 3-arg NewAutoscanHandler * fix(autoscan): per-path suppression key, bounded poll window + overlap, boundary-safe rewrites, GREATEST cursor guard, async trigger, quiet unresolved-path skip, FK->404 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize Windows path separators, surface status errors, re-seed source editor on save Addresses minor code-review findings: Windows backslash paths now normalized before rewrite/dedupe; HandleStatus returns repository errors instead of 200; the per-source editor re-seeds from server data after a save. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan rewrite-sync from arr root folders Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan rewrite-sync Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): suffix-match rewrite suggester Add suggestRewrites / commonSuffixLen for Task 1 of the autoscan arr-polling feature. Pure function: matches arr root-folder paths to Silo media folder paths by longest common trailing segment count, adjusted for depth-delta so coincidental same-named segments at different structural levels don't inflate confidence. Categorises each arr root as Proposed, Ambiguous, Unmatched, or Covered by an existing PathRewrite rule. TDD: test file written first, verified failing, then implementation added. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): GetSource single-source lookup * feat(autoscan): arr root-folder client + Silo folder lister * feat(autoscan): Service.SuggestRewrites Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): rewrite-suggestions endpoint Add GET /autoscan/sources/{id}/rewrite-suggestions admin endpoint: extend the autoscanTriggerer interface with SuggestRewrites, wire SetRewriteResolvers in the router, and add handler + test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan rewrite-suggestions types and hook * feat(web): autoscan sync-rewrites preview * fix(autoscan): normalize covered-rule paths, dedup roots/folders, skip no-op suggestions Addresses final-review edge cases: coveredBy normalizes the existing rewrite's From (so a stored Windows/dup-slash rule still covers a root); duplicate arr roots and duplicate Silo folder paths are de-duplicated; an arr path that already equals its Silo path is not proposed as a no-op rewrite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): vitest 4 compatible fetch spy in recipes.test (unblocks build after vitest 4.1.0 bump) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): non-null suggestion slices + move Sync into rewrites card - suggestRewrites initializes Proposed/Unmatched/Ambiguous/Covered to empty slices so the JSON response is [] not null — fixes the 'Something went wrong' crash when every root is already covered (frontend mapped over null). - Move the sync button into the Path rewrites card beside 'Add rewrite' and rename it 'Sync rewrites'; guard the proposed map with ?? []. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): long root-folder timeout + sync spinner + collapse rewrites on load - Root-folder fetch for sync uses a 2-min timeout: Radarr/Sonarr compute unmappedFolders by scanning all roots, so a large library's /rootfolder takes 20-30s+ and tripped arrclient's 30s default (Sonarr 502'd at exactly 30s). - Spin the sync icon + show 'Syncing…' while the request is in flight. - Path rewrites card starts collapsed on page load. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): rescan on Sonarr/Radarr file renames History polling previously only tracked downloadFolderImported events. A rename in Sonarr/Radarr (episodeFileRenamed / movieFileRenamed) moves a file without an import event, leaving the library folder stale until the next full scan. Extend the history client to also surface renamed paths: both the new path and the old sourcePath, since a rename can move a file between folders and both parents may need rescanning. Delete events are still skipped — upgrade-deletes are covered by the paired import, and standalone deletes carry no file path in arr history. Renames the interface method ImportedPaths -> ChangedPaths to reflect the broader scope. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): synchronize trigger test with detached PollOnce goroutine HandleTrigger dispatches PollOnce on a detached goroutine and responds 202 immediately. The test read trig.called straight after the handler returned, racing the goroutine (usually 'PollOnce was not invoked') and reading the field without synchronization (a data race under -race). Signal completion through a channel the fake sends on when PollOnce runs; the test waits on it (bounded) before asserting. The channel send happens-before the receive, so the subsequent read of called is race-free. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan as a pluggable scan-source category Reframes autoscan from a Requests-coupled, arr-only feature into a standalone Autoscan category. Change-detection providers become out-of-process plugins via a new additive scan_source.v1 capability (client-pull, opaque marker); Sonarr/Radarr is the first provider. Host keeps a provider-agnostic resolve/suppress/enqueue engine; all arr-specific logic (and path rewrites) move into the plugin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for scan_source.v1 SDK capability First of the per-repo plans from the autoscan-plugin-architecture spec. Adds the additive scan_source.v1 capability to silo-plugin-sdk (proto + codegen + capability allowlist + runtime wiring), TDD per task, tagged as v0.5.0 so the host and arr-plugin plans can build against it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan host backend (part 1 of 2) Backend for the standalone Autoscan category: scan_source.v1 plugin plumbing (pluginhost client + plugins.Service resolver), generalized engine driven by a provider seam, autoscan_connections + autoscan_sources schema (decoupled from Requests), connection resolution (own or Requests-linked), admin API. Depends on silo-plugin-sdk v0.5.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plans for autoscan arr plugin + host UI arr plugin: new installable scan_source.v1 plugin (history imports+renames, rewrites, Silo-native paths), structured like silo-plugin-tmdb; ports the arr-specific logic from the closed PR #43. host UI (part 2 of 2): standalone Autoscan admin category (connections, sources, settings) extracted out of Requests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(autoscan): replace silo-plugin-sdk with local scan_source.v1 checkout Temporary dev replace so the host backend can build against the unreleased scan_source.v1 capability (silo-plugin-sdk PR #2). Finalize to v0.5.0 once the SDK is tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pluginhost): scan_source.v1 capability client wrapper Adds ScanSourceClient struct, the Client.ScanSource() accessor (mirrors ScheduledTask pattern), and a PollChanges method. Also introduces client_test.go with capability-gate tests for both scheduled_task.v1 and scan_source.v1 using a lazy gRPC ClientConn. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test+fix(pluginhost): cover capability-id gate, dedicated scan_source timeout Adds a "wrong id returns error" subtest to both capability-gate tests so the capability-ID component is exercised independently of the type. Introduces DefaultScanSourceTimeout (2m) for PollChanges, which polls an external arr API that can be slow, instead of the generic 10s control timeout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(plugins): expose scan_source.v1 client resolver Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(migrations): autoscan v2 schema (connections + sources, no requests FK) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): v2 types and repository Replace the request_integrations-coupled model with the decoupled v2 schema (autoscan_settings + autoscan_connections + autoscan_sources). Connection CRUD, source upsert/list/get, and AdvanceMarker/RecordError for opaque marker bookkeeping. ErrIntegrationNotFound becomes ErrNotFound. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): resolve connections (own credentials or Requests-linked) ConnectionResolver turns a stored Connection into concrete credentials, reading a soft-linked Requests integration's live base URL/key when RequestIntegrationID is set, then resolving the api-key ref to plaintext. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): scan-source provider seam over the plugin resolver ScanSourceProvider lets the engine poll changed paths without a live plugin; pluginProvider adapts plugins.Service.ScanSourceClient in production. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): generic engine drives sources via scan_source provider Rewrite PollOnce to iterate enabled sources, resolve each connection, poll the provider for changed paths, and run the salvaged resolve→suppress→enqueue loop (uniqueParentDirs, (folder,path) suppression key, RequestError quiet-skip, release-claims-on-enqueue-fail) verbatim. Store the opaque next marker via AdvanceMarker only after a successful enqueue; RecordError + keep marker on provider failure. Tests reworked onto a fakeProvider/fakeStore with an added opaque-marker-verbatim assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): drop conflicting connection CHECK, add connection resolver tests - migration 172: remove the autoscan_connections_source_present CHECK. It conflicted with request_integration_id ON DELETE SET NULL: deleting a Requests integration that a linked-only connection (base_url NULL) points at would null the FK and trip the CHECK, blocking the delete. The intended behavior is for the connection to survive as an orphaned 'needs attention' row. Creation-time validity is now enforced at the application layer. Verified on a throwaway DB: full chain applies and the delete-cascade leaves an orphaned (both-null) connection. - connection.go: TrimSpace the api key ref + resolved secret before the empty-string checks, matching requests.resolveAPIKey parity. - connection_test.go: fake-based tests for ConnectionResolver.Resolve (own creds, linked, linked-missing error, lookup error, trim/fallback). - repository.go: bound RecordError's stored last_error to 2048 chars. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): autoscan v2 admin endpoints Rewrite the autoscan admin HTTP handler against the v2 model: settings, connection CRUD, source update, manual trigger (detached PollOnce), and status. Connection/source responses omit api_key_ref and resolved keys (has_api_key flag only); unknown connection/source ids map to 404 via autoscan.ErrNotFound. Retire the host-side rewrite-suggestions endpoint (now lives in the arr plugin). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): wire v2 service, routes, retire rewrite-suggestions Export PollChangesClient/ScanSourceResolver from the autoscan provider so the api package can declare a structurally-conformant plugin adapter (Go has no return-type covariance, so the adapter must name the interface as its return type). Add api.BuildAutoscanService with the requests-integration lookup and plugin scan-source adapters, shared by the router (manual trigger) and the background poll task. Re-wire router routes to the v2 connections/sources/settings/trigger/status surface and drop the rewrite-suggestions route. Update cmd/silo to build the v2 poll task, seeding its interval from Settings.DefaultPollIntervalSeconds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): enforce connection requires own URL or a Requests link Migration 172 dropped the DB CHECK that required an autoscan connection to carry either its own base_url or a request_integration_id, delegating that invariant to the application layer — but the enforcement was never added, so HandleCreateConnection/HandleUpdateConnection accepted both-NULL orphans that ConnectionResolver.Resolve would hand a plugin as an empty base URL. Add a shared validateConnectionInput helper (whitespace-only request_integration_id counts as absent) and reject both-empty payloads with HTTP 400 on both the create and update paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): deliver resolved connection to plugin PollChanges now populates PollChangesRequest.Connection with the resolved {base_url, api_key} instead of dropping the conn param on the floor. Drops the stale doc comment claiming the connection was delivered out-of-band at upsert time -- that mechanism never existed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): auto-discover sources from installed scan_source plugins Auto-discovery seeds a disabled, connection-less source row per installed scan_source.v1 capability before an operator binds a connection, so connection_id is now nullable end to end: - migration 172: connection_id drops NOT NULL (still ON DELETE RESTRICT) - Source.ConnectionID becomes *string; repository scans/writes it as nullable and adds idempotent EnsureSource (INSERT ... ON CONFLICT DO NOTHING) - new ScanSourceLister seam + Service.DiscoverSources, called at the start of PollOnce (errors logged, non-fatal); production adapter enumerates ListEnabled -> ListCapabilities filtered to scan_source.v1 - PollOnce skips an enabled source with no connection bound, recording 'no connection bound' so the UI can surface it - HandleUpdateSource rejects enabling a source with no effective connection (400); source DTOs expose connection_id as nullable - BuildAutoscanService / NewService thread the installation store at both wiring sites (router + poll task) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): honor per-source poll interval PollOnce now skips an enabled source that ran too recently: the floor is source.PollIntervalSeconds when set, else settings.DefaultPollIntervalSeconds. The global poll task fires at the default cadence, so this makes the per-source interval a 'poll at most every N seconds' floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): reconcile spec + arr-plugin plan with credential-in-request + auto-discovery The credential-delivery mechanism changed during execution: the host now passes resolved {base_url, api_key} in PollChangesRequest.connection each poll (not plugin runtime config). Also records source auto-discovery, nullable connection_id, and the per-source interval floor decided at the final integration review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan v2 types and query hooks Replace v1 autoscan types and hooks with v2 DTOs matching the backend handler (autoscan.go): settings, connection (with has_api_key, no raw key), source (installation_id/capability_id/connection_id), status. Add connections CRUD hooks, useAutoscanStatus, update sources hook to v2 input shape. Retain deprecated shims for AutoscanPathRewrite, AutoscanRewriteSuggestions, and useAutoscanRewriteSuggestions so AdminRequests.tsx continues to compile until Task 6 removes that tab. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan connections panel (reuse or own) Card+Table listing connections with "Reused from Requests" / "Own" badges. Add/edit dialog with two modes: reuse a Sonarr/Radarr Requests integration or enter own name/URL/API-key credentials. Delete with alert-dialog confirm. Never renders key material — only has_api_key is sent by the backend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan sources panel Table of auto-discovered scan sources (one row per installed scan_source plugin capability). Operator can bind a connection via inline Select (auto-saved on change), set a per-source poll interval (saved on blur), and toggle enabled. Shows a "Needs connection" badge for unbound sources; attempting to enable without a connection lets the backend 400 surface via the existing toast in useUpdateAutoscanSource.onError. Status column shows last_run_at relative time or last_error with icon. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): standalone Autoscan admin page Tabs page (Sources | Connections | Settings) mirroring AdminRequests header/layout. Settings tab exposes global enable switch, default poll interval, and debounce — all auto-saved on blur or toggle. "Run now" button calls useTriggerAutoscan and toasts "Autoscan triggered" on 202. Route and sidebar nav are intentionally deferred to Task 5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): route and sidebar nav for Autoscan category Add /admin/autoscan route pointing to AdminAutoscan and a matching "Autoscan" item in the Content group of the admin sidebar (with RefreshCw icon), so the new standalone page is reachable from the nav. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(web): move Autoscan out of Requests into its own category Remove the Autoscan tab, AutoscanTab/AutoscanSourceEditor component definitions, and AutoscanSettingsFormState from AdminRequests.tsx. Delete the Task-1 compatibility stubs: AutoscanPathRewrite and AutoscanRewriteSuggestions types from api/types.ts, and the useAutoscanRewriteSuggestions no-op shim from useAutoscan.ts. The build confirms zero dangling references. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): allow unbinding a source connection (full-state source update) Change the source-update input struct's connection_id from string to *string so the UI can send null to unbind, a UUID to bind, or omit (null) to clear. Remove the fall-back-to-existing logic; the handler now sets the source's ConnectionID directly from the input. The enable-guard fires when the resulting connection is nil regardless of cause. Frontend sends the complete triple (connection_id, enabled, poll_interval_seconds) on every mutation site; selecting "— No connection —" sends null for a real unbind. Adds aria-label to connection Select and interval Input for accessibility. Backend tests cover bind, unbind, unbind while enabled → 400, and enable without connection → 400. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(migrations): backfill autoscan v1 settings+connections instead of dropping Migration 172 unconditionally DROPped the shipped v1 autoscan_settings/ autoscan_sources (migration 171), losing an upgraded operator's enable flag, poll cadence, debounce, and arr server list — autoscan came back OFF. Rewrite 172 up to be non-destructive of what can be carried: rename the v1 tables aside, create the v2 schema, backfill settings (poll minutes -> seconds) and seed a reusable LINKED connection per distinct v1 source integration, then drop the renamed v1 tables. v2 sources are keyed on a plugin (installation_id, capability_id) that did not exist in v1, so they are left to runtime discovery; path rewrites move to plugin config and are intentionally not carried. Verified against a throwaway DB: after 171 + v1 seed data, applying 172 yields enabled=true, default_poll_interval_seconds=300, debounce_seconds=30, and one autoscan_connections row linked to the v1 integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): preserve api key on metadata-only connection edit UpdateConnection unconditionally wrote api_key_ref = nullable(c.APIKeyRef), so a metadata-only edit (the UI omits the key when left blank — "leave blank to keep existing") NULLed the stored key and broke the next poll. Mirror requests' UpdateIntegration: api_key_ref = CASE WHEN $5 = '' THEN api_key_ref ELSE $5 END, passing the raw trimmed string so a blank incoming ref keeps the existing value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): skip orphaned sources + add source delete endpoint An enabled source whose scan_source plugin was uninstalled/disabled kept its autoscan_sources row, which errored every poll cycle, and there was no way to remove it. DiscoverSources now returns the set of currently-discovered (installation_id, capability_id) pairs; PollOnce skips any enabled source not in that set quietly (no RecordError), stopping the per-cycle error spam for orphans. A nil set (no lister / discovery failed) disables pruning so a transient discovery failure does not silence live sources. Adds DELETE /admin/autoscan/sources/{id} -> HandleDeleteSource -> repo.DeleteSource so an operator can clear orphans (unknown id -> 404). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reject reused connection when Requests integration is disabled RequestIntegrationLookup.Get returned a linked integration's base_url/api_key even when the integration was disabled or had a blank base_url (the v1 poll gate `WHERE ri.enabled = true` was dropped in v2). Now Get surfaces a disabled or unconfigured linked integration as an error, which the engine turns into a logged skip / RecordError instead of polling an unusable target. The gating is extracted into a pure checkRequestIntegrationUsable helper so it is unit-testable without a DB-backed repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reschedule poll task on settings change HandleUpdateSettings no longer rescheduled the poll task (the v1 triggerUpdater / UpdateTriggers wiring was dropped in v2), so a default_poll_interval_seconds change only applied after a restart. Re-add an optional triggerUpdater (taskmanager.UpdateTriggers) on AutoscanHandler, wired via SetTriggerUpdater from the router when a task manager is available. On a successful settings update the handler recomputes the interval trigger from default_poll_interval_seconds and calls UpdateTriggers("autoscan_poll", ...). The dependency is optional: a nil updater skips rescheduling so tests need no task manager, and a reschedule failure is non-fatal (the interval is persisted). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): disable enable toggle for unbound sources, add source delete + interval hint - Disable the Enable switch when a source has no effective bound connection (connection_id null and no pending edit selection), re-enabling once bound. - Add useDeleteAutoscanSource hook mirroring useDeleteAutoscanConnection pattern. - Add per-row delete button (Trash2 icon → AlertDialog confirm) to let operators remove orphaned/unwanted source rows. - Add interval floor helper text showing the global default poll interval so operators know values below it have no effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): consume source_paths from merged scan_source contract The merged plugin SDK renamed PollChangesResponse.changed_paths to source_paths and the plugin now returns RAW source-namespace paths. pluginProvider.PollChanges reads GetSourcePaths(); the host applies per-source path rewrites before resolving/enqueueing (separate commit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(migrations): add path_rewrites to autoscan_sources Add path_rewrites jsonb NOT NULL DEFAULT '[]' to the autoscan_sources CREATE in migration 172 (unreleased/branch-only, so amended in place). The host now owns per-source prefix rewrites. v1 path_rewrites cannot be backfilled (v2 sources key on a plugin installation/capability with no v1 mapping); documented that operators must re-enter rewrites post-upgrade. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-owned per-source path rewrites Rewrite ownership moved from the scan_source plugin to the host. The plugin returns raw source-namespace paths; the host now normalizes separators and applies the source's per-source prefix rewrites before dedupe/resolve/enqueue. - types: add PathRewrite{From,To} and Source.PathRewrites - rewrite: re-add applyRewrites/normalizeSeparators; apply the MOST-SPECIFIC (longest From) match, not first-match, so a broad rule can't shadow a nested one regardless of ordering - service.PollOnce: rewrite raw provider paths before resolveAndClaim - repository: marshal/unmarshal path_rewrites jsonb in UpsertSource and all source scans (EnsureSource discovery rows take the DB default []) - handlers: autoscanSourceInput/response + status DTO carry path_rewrites (full-state like connection_id); reject blank from/to with 400 - tests: rewrite unit tests, engine applies rewrites before enqueue, handler round-trips path_rewrites and 400s on a blank rewrite Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): discover installed scan_source plugins on sources-list view A scan_source plugin installed via the normal /admin/plugins flow must show up in the Autoscan component immediately, not only after a poll cycle (which runs only when autoscan is enabled). HandleListSources now runs discovery (seeding a disabled, connection-less source row per installed scan_source capability) before listing. Best-effort: discovery failure does not block listing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): per-source path rewrites editor + plugins-page install hint Add AutoscanPathRewrite type and path_rewrites fields to AutoscanSource/ AutoscanSourceInput. SourcesPanel gains an expandable rewrite editor per source row (from→to pairs, Add/Remove/Save) threaded into the full-state body so connection, interval, and rewrite changes always carry all fields. Adds a Plugins-page install hint in both the empty state and above the table for discoverability. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): host-owned path rewrites + install/discovery flow Reconcile the spec with the merged SDK decision (rewrites moved host-side; PollChangesResponse.source_paths carries raw provider paths). Document that scan-source plugins install via the normal /admin/plugins page and surface in Autoscan via discovery (run on poll cycles and on sources-list view). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: depend on merged silo-plugin-sdk via pseudo-version (drop local replace) PR #2 (scan_source.v1 + source_paths) is merged to silo-plugin-sdk main, so the host can resolve the canonical module at the merged commit (v0.4.1-0.20260603030807-807b07e785b2) instead of a local-path replace. The branch now builds off-machine (CI/Docker). Bump to a clean v0.5.0 once tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(migrations): single clean autoscan v2 migration (v1 never shipped) The v1 in-process autoscan (migration 171) was never released to origin/main, so no live system has v1 autoscan data to preserve. Collapse the v1-create + v2-rename/backfill/drop dance into one clean 171 that creates the v2 connections-based schema directly. Removes 172 entirely. The runner applies by version set-difference with no checksum validation, so the already-migrated test instance (171+172 recorded) skips both and is unaffected; fresh installs get the clean v2 schema in one step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): allow many sources per plugin + add-source enumeration Drop the one-source-per-(installation, capability) model. A single installed scan_source plugin capability can now back many sources, each bound to a different connection (e.g. one Sonarr plugin fronting four arr servers). - migration 171: remove the autoscan_sources UNIQUE(installation_id, capability_id) constraint; sources are operator-created, not auto-seeded. - repository: replace UpsertSource (relied on the unique conflict) with a plain CreateSource (fresh uuid) + a by-id UpdateSource; remove EnsureSource. - discovery: replace auto-seeding (DiscoverSources/RefreshDiscovered) with ListAvailableScanSources (the Add-source picker list, enriched with plugin id + display name) and an installedScanSources set used only for orphan-skip. - service: PollOnce stops seeding and instead fetches the installed-capability set for orphan detection; Store gains GetSource and drops EnsureSource. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): connection test endpoint (engine) Add Service.TestConnection / TestConnectionByID: resolve a connection (ad-hoc input or an existing stored connection) to concrete credentials and probe the arr GET /api/v3/system/status with a short timeout. A reachable/authorized target yields OK=true plus the reported version; an unreachable / 401 / non-200 target yields OK=false with a human-readable error (the probe failure is part of the result payload, never an error from the method itself). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-side rewrite suggester + admin API for new endpoints Port the path-rewrite suggester back host-side (it had moved into the plugin): suggestRewrites suffix-matches arr root folders against Silo media folders to propose path rewrites, reporting proposed / unmatched / ambiguous / covered. Service.SuggestRewrites resolves the source's bound connection, lists arr roots (GET /api/v3/rootfolder) and Silo folder paths, and runs the matcher; a source with no bound connection returns ErrNoConnection (400). Admin API (all admin-gated): - POST /admin/autoscan/sources create a source - GET /admin/autoscan/scan-source-plugins Add-source picker list - POST /admin/autoscan/connections/test probe a connection - GET /admin/autoscan/sources/{id}/rewrite-suggestions sync rewrites HandleListSources no longer auto-seeds; create validates the capability is currently installed and that enabling requires a connection. Wiring threads the arr root-folder/status client and the catalog folder lister through BuildAutoscanService; the lister now surfaces plugin id + display name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan hooks + types for sources, connection test, rewrites Add types and React Query hooks backing the autoscan admin UI batch: - AutoscanAvailableSource / useAvailableScanSources (scan-source plugins) - AutoscanSourceCreateInput / useCreateAutoscanSource (POST sources) - AutoscanConnectionTestResult / useTestAutoscanConnection (advisory test) - AutoscanRewriteSuggestions / useAutoscanRewriteSuggestions (on-demand) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): add-source dialog + sync-from-arr rewrites in SourcesPanel Add a "+ Add source" header action opening a dialog that creates a scan source from any installed scan-source plugin bound to an arr connection, so operators can add one source per connection (e.g. four arr instances). Empty state links to /admin/plugins when no plugins are installed. Add a "Sync from arr" button to each source's rewrite editor that fetches root-folder rewrite suggestions and renders a preview: checkbox-selectable Proposed rewrites plus collapsed Unmatched / Ambiguous / Already-mapped sections. "Apply selected" merges the checked rewrites (dedupe by `from`) and persists via the normal full-state source PUT. Sync is disabled until the source has a bound connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): test-connection button in autoscan ConnectionsPanel dialog Add an advisory "Test connection" button to the add/edit connection dialog. It probes the current dialog input — connection_id when editing, request_integration_id in reuse mode, or base_url/api_key_ref for own credentials — and renders the result inline: green "Connected (vX.Y)" on success, red error on failure. Never blocks save; stale results clear when credential fields change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan page polish + global enable toggle in header Surface a global Autoscan enable toggle and an enabled/disabled status badge next to the page title, alongside the existing "Run now" header action so primary controls are reachable without opening a tab. Remove the now-redundant enable switch from the Settings tab (it points at the header toggle instead). Tighten header layout for wrap on narrow widths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): right-align autoscan enable toggle + Run now in the page header Drop the redundant nested justify-between wrapper so the header actions sit directly under .page-header (space-between + bottom-align), matching the /admin/libraries header layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): hold poll marker when paths return but none resolve A freshly-enabled source whose path_rewrites aren't configured yet returns provider paths that resolve to zero library folders. PollOnce previously advanced the marker unconditionally on any successful poll, permanently skipping those imports. Now the marker advances only when there is nothing to do (zero paths) or at least one path resolved+enqueued; when paths come back but none resolve, the marker is held and an explaining error recorded so the operator can fix the rewrites and a later poll re-reads the same window. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): don't prune sources of disabled-but-installed plugins PluginScanSourceLister used the installation store's ListEnabled, so a temporarily-disabled plugin dropped out of the discovered set and PollOnce treated its sources as orphaned, skipping them with no last_error (silent vanish). Switch to List so only a fully-uninstalled plugin counts as orphaned; a disabled-but-installed plugin's sources are still attempted and surface a visible RecordError when the client fails to load. The Add-source picker shares the same all-installed set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): treat empty request_integration_id as no link ConnectionResolver.Resolve gated the linked-integration path on a non-nil RequestIntegrationID pointer, so a pointer-to-empty-string (from a both-NULL orphan or a stripped link) called requests.Get(""). Guard on a non-empty trimmed value so it falls back to the connection's own fields instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): align startup poll interval with reschedule computation Startup seeded the poll task by integer-dividing default_poll_interval_seconds by 60 (minutes), while HandleUpdateSettings reschedules with seconds*1000 ms; the two diverged for sub-minute and non-60-multiple intervals. NewAutoscanPollTask now takes the interval in milliseconds and main.go seeds it as seconds*1000, matching the reschedule path so both agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize stored rewrite From at poll time applyRewrites matched the stored From after only TrimSpace/TrimSuffix, while suggest.go coveredBy normalizes via normalizePath (backslash->slash, collapse '//'). A Windows-style or dup-slash stored rewrite was thus reported 'covered' at suggest time yet never matched at poll time. applyRewrites now normalizes From through normalizePath so poll-time and suggest-time agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): don't corrupt source poll interval on enable/connection change Add a `parseInterval` helper that maps empty input to null (use global default), valid positive integers to the integer, and any other mid-edit-invalid value to the source's currently-persisted `poll_interval_seconds` — so toggling the enable switch or changing the connection cannot silently overwrite the interval with 0 or NaN. Wire the helper through `fullBody()` (the single source of truth for PUT payloads) and remove the two inline duplications in `handleConnectionChange` and `handleRewriteSave` that both previously used raw `Number()`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): make the source connection optional (provider-agnostic) A host connection is the credential/endpoint for server-based providers (Sonarr/Radarr); other scan_source providers (e.g. a CephFS/filesystem watcher that reads ceph.dir.r* xattrs) need none. PollOnce now polls connection-less sources, passing an empty ResolvedConnection the plugin may ignore; a plugin that requires credentials surfaces the error at poll time. Drops the enable-requires-connection 400s. Provider-specific config lives in the plugin's own global_config_schema, not a host connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): provider-agnostic autoscan copy + optional source connection Replace arr-hardcoded framing in AdminAutoscan, SourcesPanel, and ConnectionsPanel with neutral scan-source language. Remove the connection-required gate on the source enable toggle so connectionless providers (e.g. filesystem watchers) can be enabled; soften the badge from "Needs connection" to "No connection". Sync-from-server button remains gated on a bound connection (it needs a server to query). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: repo-relative paths in autoscan plans Replace local absolute filesystem paths (/opt/silo, sibling checkouts, /tmp/go/bin) in docs/superpowers/plans with repository-relative wording per CLAUDE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): rune-safe last_error truncation Truncate RecordError messages on a UTF-8 rune boundary so a byte-bounded cut can't split a multi-byte rune and store invalid UTF-8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): advance marker when resolved-but-suppressed (not unresolved) resolveAndClaim now reports resolvedAny (whether any path mapped to a Silo library folder, independent of suppression). PollOnce gates the "none matched a Silo library folder" hold+RecordError on !resolvedAny instead of len(targets)==0, so a poll whose paths resolved but were all debounced/suppressed advances the marker instead of being treated as a misconfiguration. Adds a regression test for the suppressed case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): normalize request_integration_id Trim whitespace and collapse empty-after-trim request_integration_id to nil on connection create and update, so a pointer-to-"" or " " is never persisted as a bogus Requests link. Also corrects a stale migration-172 comment to 171 (the collapsed migration number). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(autoscan): provider-agnostic poll-task copy Rename the poll task to "Autoscan poll" with a provider-agnostic description and progress message; drop Sonarr/Radarr/arr wording. Key() (autoscan_poll) is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): fix typo in connectionless-source test name Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add scan source management * chore(deps): bump silo-plugin-sdk for structured scan source changes Pins silo-plugin-sdk to 0d78651, which adds source_config on PollChangesRequest plus the structured changes / ScanSourceChangeScope fields on PollChangesResponse that internal/autoscan/provider.go already consumes. Without this the branch fails to compile against the prior pin (807b07e). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): label scan sources by connection name in admin UI arr-plugin sources fan out one-per-connection under a single generic "arr" capability, so every row in the Sources and Activity panels rendered an identical "arr (plugin #N)" label. Lead with the bound connection name (Radarr/Sonarr/...) instead, demoting capability + plugin to a subtitle. Sources without a connection (e.g. cephfs) keep the capability fallback. Activity threads a source_id -> connection name lookup (built from the existing sources + connections queries) through the scan/poll tables the same way librariesByID is threaded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): spec for generic + operator-editable source labels Design for a shared label-resolution helper (operator label -> connection name -> manifest display_name -> capability_id) consumed by the Sources and Activity panels, plus an operator-editable per-source label backed by a new autoscan_sources.label column. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): implementation plan for source labels Task-by-task TDD plan: migration 174 (label column), Go domain/repo/handler wiring with server-side normalization, shared frontend label helper, and Sources/Activity panel integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): migration for source label column * feat(autoscan): source label domain field + normalizer * feat(autoscan): persist source label in repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): accept, normalize, and return source label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add label to source API types * feat(autoscan): shared source-label resolution helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): polish source-label helper per review Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): label sources via shared helper + operator label input Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): clarify source label naming per review * feat(autoscan): resolve activity source labels via shared helper Replace the sourceNames Map plumbing in ActivityPanel with SourceLabelLookups and delegate both name functions to resolveEventSourceName from @/lib/autoscanLabels, enabling the full label chain (operator label → connection name → manifest display_name → capability_id) for all Scan History and Poll log rows. * fix(autoscan): carry label on status source + guard poll label Final-review follow-ups: add the label field to the autoscanStatusSource response (and AutoscanStatusSource type) so the status view matches the source response per spec, and give pollSourceName a non-empty fallback for symmetry with scanSourceName. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): resolve source aria-labels through the label chain Replace the legacy capability-only sourceLabel() helper with resolveSourceName() (operator label -> connection -> display_name -> capability). Row controls now announce the row's resolvedLabel (reflecting in-progress edits) and the delete dialog announces the resolved name, so screen readers hear "4K Movies" instead of "arr (plugin #4)". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): paginate queue + history with a shared table pager Replace the card/table hybrid and 200-row "Load more" cap on the autoscan Activity panel with proper tables and real pagination. Backend: add offset + total-count to the scans/events list endpoints so history pages through the full set instead of a capped window. Extract shared event/scan WHERE-clause builders so list and count filter identically, and add CountEvents / CountAutoscanScans. Frontend: add a reusable TablePagination component (rows-per-page, "showing X-Y of Z", numbered window with ellipses, responsive) and reuse it for the server-paginated history (scans + polls) and the client-paginated live queue. Unify all three tables behind one DataTable shell so they read as one family. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>	2026-06-05 22:19:38 -04:00
QuickandClaude Opus 4.8	943b5dd9f0	fix(sections): harden trending refresher per PR review ... - Interleave Trakt movies/shows by rank so the mixed row shows both types instead of burying all series past the display limit. - Treat any Trakt sub-fetch failure as fatal (errors.Join) so a partial result never overwrites the last-good snapshot with a media type missing. - Skip non-title entries (TMDB trending/all returns media_type "person") in both ID batching and ordering so they can't match an unrelated library title. - Guard the refresh task against a nil refresher. - Tests: person skip, Trakt interleave, Trakt partial-failure preserves last-good, snapshot read error propagation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-05-29 11:28:59 -04:00
Quick	13081f7d07	feat(tasks): add refresh_trending_discover task	2026-05-29 10:44:15 -04:00
Quick	bba3177fc9	fix(metadata): break duplicate provider candidate ties ... - Score candidate metadata completeness and auto-match the richer duplicate when title/year/type tie - Enrich near-duplicate candidates via the provider chain before initial match selection - Seed both movie and series match queues for mixed-type libraries and wait for TV queue settle - Add taskmanager worker test coverage and a plan doc for the tie-breaker work	2026-05-26 20:07:43 -04:00
Silo Server Migration	246c9da6ab	feat(requests): add media request system with Radarr/Sonarr fulfillment ... - Add request domain, repository, service, and reconcile task - Add Radarr/Sonarr fulfillment adapters and TMDB discovery - Expose user and admin request APIs with quota and approval rules - Add web UI for browsing, requesting, and admin queue management - Migration 139 introduces media_requests and related tables	2026-05-24 13:58:12 -04:00
Silo Server Migration	c085b12fd1	Initial Silo migration	2026-05-22 23:26:56 -04:00