e18bc3c5680cf4ddeb91fb8657da16c0c7184bb5
78
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 rxwatcherandClaude Fable 5
|
e18bc3c568 |
docs(ebooks): add enrichment architecture plan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104andClaude Fable 5
|
1f9bd99990 |
fix(diagnostics): address round-4 review findings on PR #445
- schema: add crash/report.type conditionals (allOf if/then) so a crash/anr/native_crash/hang/abnormal_exit manifest requires `crash` and a `manual` manifest forbids it, matching ValidateManifest. - service: reject uploads where X-Profile-Id and manifest.report.profile_id are both present but differ (new ErrProfileMismatch, mapped to 400 profile_mismatch) instead of silently preferring the header; single-source and matching cases unchanged. Adds service tests for mismatch, match, and header-only attribution. - schema: require manifest.json as the first archive.entries element via prefixItems (contains retained for validators without prefixItems support). - schema: document that maxLength is a character-count bound while the server enforces UTF-8 byte length, via a top-level note and per-field notes on the free-text device_summary and crash fields. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh |
||
|
Quick104andClaude Fable 5
|
a6348b3dc5 |
fix(diagnostics): address PR #445 review findings
- bundle: reject tar entry names that differ from their trimmed form instead of normalizing padded names into the allowlist - repo: reserve expected bytes on receiving rows and count receiving+ready in the per-user byte quota so concurrent/multi-node uploads can't overshoot - contract: require the crash object for event report types and keep it absent for manual; add contract tests - settings/service: seed diagnostics.server_instance_id atomically via insert-if-absent and adopt the winning value across nodes - bundle/service: capture the embedded manifest.json during ValidateBundle and reject reports whose embedded manifest disagrees with the part-1 manifest (minus archive); add tests - admin: delete the DB row before the blob on DeleteReport; log bucket/key when the blob delete fails instead of leaving a visible report with a missing bundle - bundle: reject PAX/GNU tar formats and extension records that smuggle bytes past validation; add a PAX-archive rejection test - migration: add CHECK constraints for state, report_type, and platform - docs: add text/jsonc language identifiers to the two unfenced code blocks - cleanup: log-and-continue per report and aggregate errors so one poisoned report no longer blocks the whole run; update tests - tasks: give diagnostics its own cleanup interval key instead of reusing the opslog key, and bound the startup settings lookup Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh |
||
|
Quick104andClaude Fable 5
|
f965a489bd |
fix(diagnostics): align bundle contract with real tar writers
Two validator behaviors made the contract unimplementable for clients using standard tar libraries: - Any byte after the tar end-of-archive marker was rejected, but GNU tar, Python tarfile, and Apache Commons Compress all pad the archive with zero blocks to a record boundary. Accept up to 64 KiB of zero padding; any non-zero trailing data is still rejected. - uncompressed_bytes was computed as the sum of entry payloads, which no tar-producing client observes. Define it as the total decompressed tar stream (headers, end-of-archive marker, and padding included) — the byte count between a client's tar writer and gzip writer, and what gzip -l reports. Documented in the design doc and contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh |
||
|
Quick104andClaude Fable 5
|
4fa84a661a |
feat(diagnostics): client diagnostics server foundation
Implements slice 1 of docs/design/2026-07-19-client-diagnostics.md: the versioned contract (schemas, fixtures, Go validator), storage-validated diagnostics.uploads_enabled gate, account-scoped status endpoint, hardened streaming multipart ingest with quota reservation and a receiving/ready/ failed report state machine, S3 streaming puts, acting-admin report API (list/detail/download/delete with audit events), and the retention + orphan-reconciliation cleanup task. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XppCCycoaskCsW7ja1fZct |
||
|
Quick104andClaude Fable 5
|
7369d0afd8 |
docs(design): client diagnostics spec (crash reports + debug log upload)
Cross-repo spec and rollout plan for opt-in client crash reporting and debug-log upload to the user's own Silo server: silo-server ingest, storage, admin API, and retention; silo-android and silo-apple capture, consent, and upload. Disabled by default on both server and client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XppCCycoaskCsW7ja1fZct |
||
|
|
91e1164090 |
feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
854d07cf8f |
feat(playback): add protocol v3 planning and recovery (#398)
* docs(playback): plan protocol v3 server implementation * docs(playback): incorporate protocol v3 review * feat(playback): implement protocol v3 server * fix(playback): persist empty route diagnostics * feat(playback): harden protocol v3 HDR routing * feat(playback): complete protocol v3 client contract * fix(playback): harden protocol v3 recovery * fix(playback): restore dovi_rpu strip filter for DV remuxes The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg; the probe failed closed on every deployment, disabling the new validated DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1 remux path from main. Restore dovi_rpu across the probe, remux and HLS copy arguments, and the recipe-card constant. Also from review: validate the remux DV mode for every profile (garbage modes on non-P7 sources silently no-opped), reject preserve mode for P7 outright (a base-layer-only remux cannot preserve dual-layer DV), tag dvhe sample entries only for the explicit v3 preserve recipe so legacy web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the token-frozen DV mode in the proxy remux path instead of legacy-auto. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): correct v3 planner policy and contract validation Review fixes to the v3 planner and wire contracts: - Bar Profile 7 sources from the non-strip progressive remux route: a base-layer-only remux can never deliver native dual-layer DV, so the planner no longer emits plans claiming validated Dolby Vision while the executed remux drops the enhancement layer. - Accept the device-quirks feature flag from either capability location, matching every other dual-location feature check. - Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with a degradation warning instead of leaving them unplayable under v3. - Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and wire the previously dead Metered signal into conservative auto rungs. - Degrade to the validated source-quality route instead of a terminal when only an implicit quality reduction demanded an unsupported transcode; explicit user-selected rungs keep terminal behavior. - Bound inner capability lists and strings; compare attempt keys exactly instead of case-folded; make ParseTrackIDV3 strict about canonical numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising burn-in for unknown subtitle codecs; probe every h264 encoder rather than requiring libx264; normalize the file-level bitrate fallback. - Evaluate subtitle renderability against the engine each candidate route executes on, not always media3_direct. - Pin the with-quirks attempt-key preimage arity in the cross-language fixture so the Kotlin client stays in lockstep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 control-plane reliability Review fixes to the v3 session, store, and handler layer: - Bound concurrent replans with a slot semaphore: each replan pins a pooled connection for its advisory lock while issuing further store queries from the same pool, so an unbounded recovery storm could turn every connection into a lock holder and deadlock the server. - Make CompleteReplan a real compare-and-swap (base-revision predicate, ErrReplanSupersededV3) and map BeginReplan insert races to a replay instead of a raw unique violation. - Fingerprint start requests (request_digest column): an attempt ID reused with different input is now a 409-style conflict rather than a silent replay, and both replay paths check session liveness so dead sessions surface as retryable terminals. - Pre-delete expired attempt rows on SaveAttempt so a retry during the cleanup window cannot wedge on an unreachable conflict. - Align the in-memory store's semantics with Postgres and add DB-backed planstore tests (SILO_TEST_DATABASE_URL), including a regression test inserting every route-event name against the real CHECK constraint. - Session manager: v3 route-set updates own RemuxDVMode outright so a replan onto an SDR source clears a stale strip mode; replacement reservations survive unrelated legacy stream updates; replacement admission excludes the replaced session explicitly instead of decrementing totals it may no longer be part of; the admission CAS loop is bounded and decider errors are logged. - Map transient store failures to 500s instead of terminal 404/403s; authorize route events via identity-only projections after the rate limiter; keep sanitized diagnostics deterministic. - Merge the server-computed durable plan key into replan exclusions so unreproducible client history cannot re-select the failed route. - Remap tracks only when the effective edition changes (a same-file replan no longer switches audio to a lookalike track) and remap ID-only subtitle selections on edition fallback. - Cache the v3/shadow feature flags for five seconds instead of one settings SELECT per playback request; stop remote transports best-effort when the start call times out; carry dvm/tid claims and the transport-scoped job identity through the legacy audio-change re-mint; index playback_route_events(received_at) for the retention delete; run store maintenance for DB-less deployments too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): reap idle node jobs and gate WebVTT conversion - Add an idle reaper to the transcode node: a job untouched by manifest or segment requests for ten minutes is closed and unregistered. After a v3 replan retires a transport ID, a stale in-flight stream token could resurrect the old job via reconstruct and encode to end-of-file for nobody; jobs waiting on readiness count registration as access and are never reaped mid-wait, and reaping keeps the recipe so a still-valid token reconstructs on the next hit. - Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with 415 before headers are written instead of spawning an ffmpeg command that always fails mid-response, and make the extract-format override fall back to source-driven mapping for bitmap codecs. - Drain error bodies on non-202 node responses so the HTTP transport can reuse connections. - Pin the transcode-dir cleanup separator-boundary semantics with a regression test (a session ID sharing another's prefix must not retain foreign directories). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): close v3 planner policy gaps from review - Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has no rung below 480p/1500kbps, so lower caps were silently exceeded even though the cap is documented as a hard delivery ceiling. - Treat video-only media as audio-compatible instead of forcing an AAC conversion (or an audio_conversion_unsupported terminal) onto a file with no audio stream. Tracks whose codec failed to probe keep the gate. - Only promise a bitmap subtitle sidecar for embedded PGS with an engine that renders embedded bitmap: external/downloaded bitmap and embedded DVD/DVB published artifact URLs that always failed at fetch. They now fall through to burn-in or its terminal. - Accept client_video_transformations_v1 from either client_features or the nested context when validating client-executor transformations, matching the planner's dual-source reads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): probe and execute DV remuxes with one ffmpeg binary The v3 transformation registry probed the configured playback.ffmpeg_path while progressive remux execution resolved the process-global discovery path, so a deployment where only one binary carries dovi_rpu could plan a server_dv7_to_hdr10 route and then fail it at stream time. Resolution now goes through a shared ResolveFFmpegPath (configured path first, discovery fallback — the same rule the transcode pipeline already used), the dovi_rpu probe is cached per binary path, and the stream handler and proxy worker pass their configured path into ServeRemuxWithDVMode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 replan identity and control-plane limits - Seed failure-replan track selections from the durable current plan before overlaying the request: after an alternate-version fallback the normalized request still carries requested-edition track IDs, so a replan omitting unchanged tracks was rejected as a track/file mismatch. - Remap ID-only audio selections across edition changes (parse the ID to an index like the subtitle remap already does) instead of leaving a stale file-bound ID to fail validation. - Release the node planner reservation when a prepared remote transport rolls back after the node accepted the job; repeated failed starts could otherwise pin max-job/bandwidth budgets for the full reservation age. - Size the replan semaphore below the PostgreSQL pool via a store capacity advisor: with max_connections at or below the fixed bound, advisory-lock holders could starve the inner store queries they need to finish. - Contain shadow-planner panics with a recover boundary; it runs on a bare goroutine where an escaped panic kills the process for what is telemetry-only work. Document why the memory store's session lock is deliberately a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): serialize node job teardown against reconstructs - Look up and touch manifest/segment sessions in one critical section so the idle reaper cannot unregister a job between the lookup and its liveness refresh. - Re-validate each reap candidate under the per-session lifecycle lock before closing it: Close removes the output directory, and without the lock it could race a token reconstruct and wipe the segments the fresh ffmpeg is writing. - Take the lifecycle lock in handleStop so a stop racing a RequireReady start's readiness wait blocks until registration and tears the job down, instead of 404ing and orphaning the ffmpeg until the reaper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
b7292a9473 |
fix(streaming): stop killing healthy streams at the server WriteTimeout (#361)
The main API server's WriteTimeout (120s) is an absolute deadline from request start, so every streaming response still being written at T+120s was cut mid-body with a clean close. Clients saw multi-GB direct streams truncate every two minutes; the Apple client's cursor-resume reconnect absorbed most kills silently, but one landing during backpressure or a demuxer resync exhausted its retry budget and forced a full player teardown (visible stop + historical audio desync seeding). Fix: internal/httpstream.RollingDeadlineWriter pushes the connection's write deadline forward with progress via http.ResponseController — a response that keeps moving lives indefinitely, a stalled one is still reaped within the window (180s default, SILO_STREAM_WRITE_STALL_TIMEOUT to override). ReadFrom delegates in bounded slices so http.ServeContent keeps its sendfile fast path. Wired into direct play, remux, downloads, the transcode-node proxy, and ebook serving; the server-level 120s guard stays for every other route. The metrics and request-logger response writers now implement Unwrap — without it http.ResponseController cannot traverse to the connection and SetWriteDeadline fails, silently disabling the fix (exactly what the first dev deploy showed). A middleware-chain integration test locks the whole path down against future wrappers missing Unwrap. Validated on dev: 200s/512MB direct and 300s/768MB via CDN sustained range-GETs (previously dying at 120s), zero duration_ms=120000 stream entries since deploy. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandGitHub
|
10e15798e0 | feat(plugins): add approved community catalog hub (#355) | ||
|
|
d68e70bb47 |
feat(autoscan): Sonarr/Radarr webhook intake without arr API keys (#353)
* docs(autoscan): add arr webhook intake spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add webhook intake schema migration Adds delivery_mode to autoscan_sources, the autoscan_webhook_endpoints table, and delivery_mode/provider_event_type on autoscan_events. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add built-in arr-webhook source identity Host-discovered scan-source entry so webhook-mode sources need no plugin installation; composite lister appends it to plugin discovery. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): persist delivery mode, webhook endpoints, event metadata Sources carry delivery_mode; autoscan_webhook_endpoints CRUD with SHA-256 token lookup and AAD-bound encrypted redisplay; events record delivery_mode/provider_event_type; CreateEvent gains SkipRunningCheck so webhook deliveries are never dropped by the poll exclusion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): share the consume path and add webhook IngestChanges Extracts consumeSourceChanges from PollOnce (marker semantics preserved, existing poll tests unchanged); PollOnce skips webhook sources; IngestChanges feeds deliveries through the shared pipeline without markers and without the running-event exclusion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add Sonarr/Radarr webhook payload parser Host-side arrwebhook package: provider inference, import/rename/delete path extraction with vanished-path-friendly previous paths, subtree fallback, exact-path dedupe, and no-op unknown events. Fixture-backed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add public webhook delivery route and admin endpoint management Public POST /api/v1/autoscan/webhooks/{token} with per-IP rate limiting, 256KiB body cap, 202-for-noop semantics, and token/body kept out of logs; admin create/rotate/delete endpoint routes; source responses carry delivery mode + webhook status/URL; create/update validate delivery mode against source identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add webhook delivery mode to Autoscan admin UI Webhook sources get a generate/copy/rotate webhook URL section, provider selector, delivery status, and a connection-free Add-source flow; activity rows badge webhook deliveries with the arr event type. Path rewrites stay editable in both modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): redact secret path params from request and activity logs The request logger and activity-log middleware recorded raw URLs, so bearer credentials in secret path segments (autoscan webhook {token}, webhook-sync {secret}) were persisted to app logs and activity_log. Redact the secret segment via the chi route params in both sinks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(autoscan): make webhook delivery reliable --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
04c4344f52 |
feat(metadata): reconcile artwork cache after public S3 provider changes (#349)
* feat(metadata): reconcile artwork cache after public S3 provider changes Changing the public S3 provider previously broke every cached image permanently: the DB keeps bucket-relative keys, the image cache pipeline treats a cached path as its durable dedup marker and never re-enqueues, and clients eat the 404s straight from S3 so the server never notices. Add a storage identity fingerprint (s3.public_storage_identity, seeded via SetIfAbsent at boot) and a reconcile_artwork_cache task whose startup trigger only fires when the identity changed; manual runs always sweep, doubling as bucket-data-loss recovery. The task probes a random sample of cached objects, then either bulk-resets (near-total miss) or per-row verifies. Missing provider-sourced artwork is reset to its *_source_path so the existing enqueue loop re-caches it; surfaces without a re-downloadable source (chapter thumbnails, collection artwork, library posters, branding refs, embedded book covers) are cleared so their owning pipelines refill them. Small upload-holding tables are always per-row verified so bulk mode cannot blind-clear an upload that survived migration, and transport errors never reset rows. Users never see broken images during the transition: reset rows serve the provider's original URL via the existing absolute-URL pass-through and thumbhashes are preserved. The storage settings page now warns that uploads cannot be re-downloaded when the identity fields are edited. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): harden artwork reconcile per code review Address the confirmed findings from the PR review: - Fingerprint the key prefix case-sensitively and slash-trimmed exactly as s3client applies it (new exported NormalizeKeyPrefix): a case-only prefix edit is a real storage move and must reconcile; a slash-only edit is not and must not. - Certify the storage fingerprint immediately after the artwork sweep succeeds and make the 4-object branding check non-fatal (reported in the task message), so a transient branding error cannot discard a completed catalog sweep and force it to repeat every boot. - Fail closed on conditional-task preflight errors in the task manager (previously fail-open ran the task), and retry transient settings reads in ShouldRun since the startup trigger fires once per process. - Track probe HEAD errors against a separate baseline so a flaky probe cannot consume the sweep's error budget. - Probe before counting: bulk mode skips the per-surface count(*) full scans entirely, and probe sampling drops ORDER BY random() (plain LIMIT answers "is the cache in this bucket" just as well). - Verify chapter thumbnails across a whole 500-file batch in one HEAD fan-out instead of per file, keeping the worker pool saturated. - Replace the 10 inline non-provider-scheme ARRAY literals in the enqueue query with the shared nonProviderImageSchemesSQL constant. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): guard bulk reset against degraded probes, certify only clean sweeps Address bot review feedback on the reconcile hardening: - A probe where more than half the HEAD requests error aborts the run: errored requests are excluded from the sample, so a partial outage could otherwise present a handful of surviving 404s as a ~100% miss rate and bulk-reset the catalog. Bulk mode additionally requires a minimum number of successful samples; thinned probes and tiny catalogs take the safe per-row verify path. - Track sweep errors separately from probe/branding errors (stats.sweep_errors) and certify the storage fingerprint only when the sweep completed with zero of them — skipped rows were never verified, so the next startup retries. Applied resets stay durable. - Give each ObjectExists attempt its own timeout so a stalled HEAD fails that attempt instead of pinning the retry loop to the run context. - Report branding assets checked (not just cleared) in stats.Checked. - Drop the dead settingsRepo/brandingSvc nil guards in cmd/silo and sync spec numbers with the implementation constants. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
a2ef26bece |
perf: root-cause fixes for endpoints still slow after #292 (NextUp, series badges, resume tail, subtitle fonts) (#350)
* docs(plans): root-cause analysis for endpoints still slow after PR #292 Five endpoint groups stayed slow after the home/Continue Watching/Latest latency work shipped: Resume (110s p95), NextUp (17s p95), Latest (17s), /Items, and the home sections routes. The caps and caches from PR #292 are live in the deployed binary; they bounded how many rows the loops touch but not what each underlying query costs. Documents the four confirmed root causes (4.3M stale completed-with-position progress rows + missing resume index, unbounded next-up anchor scan, per-episode series rollup fanout, two index-starved history/scanner paths) with live EXPLAIN ANALYZE measurements and the fix plan implemented by the follow-up commits. AI-use disclosure: analysis and doc produced with AI (Claude) assistance. * perf(catalog): bound the global next-up anchor scan to recent completions The completed_episodes CTE in buildListNextUpQuery derived per-series anchors from the profile's ENTIRE completed history — DISTINCT ON over 233k rows joined to episodes for the worst bulk-import profile, then a per-series LATERAL that scans every episode of a fully-watched series before yielding nothing. 648 slow executions in a 19h window, 44.7s worst; this drove /Shows/NextUp (17.1s p95) and the next-up injection on the native home sections aggregate. Global queries now derive anchors from the profile's nextUpAnchorMaxRows (500) most recent completed rows — an ordered index walk on idx_uwp_profile_completed, with the hidden-items exclusion and date cutoff applied inside the bounded scan so hidden/old rows never consume the anchor budget. A next-up rail surfaces ~24 series; the 500 most recent completions cover every series that can realistically rank on it. Series-scoped calls (the show-detail tile) keep the unbounded shape: they must anchor on the series' last completed episode no matter how long ago it was watched, and are naturally bounded by one series. Measured on the live worst-case profile with the exact generated SQL: 44.7s worst / ~2.6s avg before; 10ms after (together with the one-time stale-resume-point data repair applied directly to the deployment DB — see docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md). AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat,userstore): aggregate series watch-state rollup in SQL The series Played/UnplayedItemCount badge on list rails (per-library Latest, library browse, search results) and series detail pages was computed by materializing EVERY episode of every series on the page (episodeRepo.ListBySeriesIDs) and then batching per-episode progress+history lookups in 500-id chunks. A 50-series page of an episode-heavy library (Sports) expanded to 32,467 episode rows and ~65 sequential queries — measured 17-18s per /Items/Latest request, and PR #292's cached Latest fast path pays it on every response for series libraries. The same fanout made /Items?searchTerm=... slow whenever the result set was mostly series (Meilisearch itself answers in milliseconds). New optional store capability userstore.SeriesEpisodeRollupStore, implemented by PostgresUserStore as one GROUP BY e.series_id aggregate with semantics identical to the chunked path (episode availability via episode_libraries, hidden-items visibility on progress rows, completed-history fold, in-progress = not watched with position > 0 — verified value-for-value against the old semantics on a real 1,586-episode series). enrichSeriesListUserData and enrichDetailUserData use it when present; SQLite-backed stores and rollup query failures keep the existing chunked path as fallback. catalog.SeasonUserDataFromCounts pins the counts-to-DTO mapping to EpisodeRollupUserData. Measured on the live worst-case profile against the real 50-series Sports Latest page: ~17s of chunked round-trips before, 119ms in one query after. Part of docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md. AI-use disclosure: implemented with AI (Claude) assistance. * perf(catalog): bound superseded-episode completed walk to recent history The Resume / Continue Watching superseded-episode filter loaded a profile's *entire* completed history into memory on every request that contained an in-progress episode: CompletedProgressSnapshots paged user_watch_progress WHERE completed=TRUE with no upper bound. The 2026-07-06 slow-query comparison showed this surviving as a 60-116s Resume tail even after the in-progress index landed live, because the 4.3M zeroed Plex-import rows are still completed=TRUE and were re-walked every load. A completed episode can only supersede an in-progress one it was finished more recently than (the query gates on done_progress.updated_at > ip_progress.updated_at), so only completed rows newer than the oldest in-progress entry can matter. Compute that cutoff in SupersededEpisodeProgressIDs and pass it to CompletedProgressSnapshots, which — since the completed listing is ordered updated_at DESC — stops paging as soon as it crosses the cutoff. Import-heavy profiles whose back-catalogue predates their current in-progress items now stop on the first page instead of paging hundreds of thousands of irrelevant rows. Correctness is unchanged: no relevant superseding row is excluded. * perf(catalog): hard-cap superseded-episode completed walk at 5 pages The updated_at cutoff added in the previous commit bounds the completed walk on the relevance axis, but a very old in-progress entry sitting behind a large volume of newer completions could still page deep. Add a 5-page (2,500-row) hard backstop on top of the cutoff: normal profiles still stop on page one via the cutoff, and only the adversarial tail hits the cap. When it engages the tail of the completed set goes unscanned, so a superseded episode could momentarily survive on Continue Watching — we log a warning when that happens (with profile_id + rows scanned) rather than mis-filter silently, and it self-corrects once the stale in-progress entry ages out of the scanned window. * perf(playback): extract subtitle fonts in a single ffmpeg pass Embedded ASS/SSA font extraction spawned one ffmpeg process per font attachment, each re-opening the (usually CephFS-backed) media file. Anime releases carry 15-47 fonts, so the per-spawn file-open cost dominated and pushed GET /api/v1/stream/{sid}/subtitles/{track}/fonts to a 17-60 s plateau (p95 ~33 s in the live logs). Collapse the N spawns into one ffmpeg invocation that dumps every attachment to a temp dir (-dump_attachment:idx path ... -i file -map 0:t? -c copy), then read the files back. The file is opened once instead of N times, taking p95 from ~30 s to ~1-2 s with no change to output. Safety is preserved. The 32-attachment / 32 MiB caps still apply: attachment size is stat'd before read so an over-limit font never enters memory, and a watchdog polls the dump dir and kills ffmpeg if its on-disk output crosses the cap -- restoring the hard bound the old pipe-per-attachment reader enforced by killing at maxBytes+1, so a container with oversized "font" attachments can't fill the disk. Part of the slow-endpoint follow-up; see slow-query-analysis/subtitle-fonts-extraction-findings.md. * fix(review): report enforced font-byte cap; correct doc subtitle scope Address PR #350 review: - dumpFontAttachments reported the maxSubtitleFontBytes package constant in both over-limit errors instead of the maxBytes argument the caller passed, so the message misstated the enforced bound whenever a different cap was in effect (as the tests use). Interpolate maxBytes in both messages. - The root-cause plan claimed subtitle extraction was 'out of scope' while the branch actually optimizes /subtitles/{track}/fonts. Scope the out-of-scope note to subtitle *track* conversion and record the fonts single-pass work as deliverable 5. |
||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
43d9056b01 |
fix(collections): repair broken builtin collection templates (#331)
* fix(collections): repair broken builtin collection templates A live audit of the builtin template catalog (all 40 MDBList URLs and all 10 TMDB franchise IDs fetched) found two dead sources, a silent bundle-apply collision, and several templates whose defaults contradict their descriptions: - Repoint mdblist_misc_a24 and mdblist_misc_criterion_collection to live lists; the original irvingbeano/shtluck lists were deleted on MDBList (404), so every sync of those collections failed. - Retitle mdblist_charts_popular_movies to "IMDb MovieMeter Top 100". It shared the "popular-movies" title slug with tmdb_popular_movies, and bundle apply dedupes by slug per library, so applying all_defaults silently skipped it. Poster regenerated from the raw plate with the new title; new handler test asserts builtin title slugs stay unique. - Raise the shared default limit 50 -> 100, give the IMDb Top 250 templates an explicit 250 (limit*4 fetch trim previously never scanned entries 201-250), and drop the limit on catalog lists (Criterion, A24) so they hold every owned title. - Correct IFC Films to MediaMovie (live list is 100% movies; as MediaMixed it was offered to TV libraries where it always synced empty) and fix the Trakt Popular descriptions (ratings-based, not "most-watched"). - Update stale limit docs in collection-templates.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(collections): raise import limit caps above IMDb Top 250 default The IMDb Top 250 templates now default to 250 items, but the template config forms rendered their Max Items input with max=200 and the user import API rejected limits above 200, so applying those templates from the direct galleries failed native validation or got a 400. Raise the cap to 500 on both sides, wired to shared constants: sync's fetch trim (collectionSourceFetchMax) never scans more than 500 source entries, so a larger explicit limit could never be satisfied anyway. collectionutil.MaxExplicitItemLimit backs validateOptionalLimit, and COLLECTION_MAX_ITEMS in lib/collectionTemplates backs all seven Max Items inputs (gallery forms + admin import/editor dialogs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e140bd9424 |
feat(metadata,scanner): trailers and extras for movies and series (#322)
* feat(metadata,scanner): trailers and extras for movies and series Remote provider videos (TMDB trailers/teasers/featurettes/...) are fetched through the unified match/refresh pipeline into the new item_videos table, filtered per-library via media_folders.trailer_kinds, merged across providers with site/provider dedup, and lockable via FieldVideos. The movie scanner stops discarding supplemental directories (Trailers/, Featurettes/, Behind The Scenes/, ...) and classifies them — plus Jellyfin-style filename suffixes (-trailer, -behindthescenes, ...) and series-root supplemental dirs — into the new media_extras entity backed by ordinary media_files rows (extra_id ownership, content_id/episode_id NULL so existing version/matching queries stay structurally blind to extras). Series Extras/SxxExx season-0 mapping is unchanged. Extras are playable watch targets via a GetWatchDetail fallback tier (episodes precedent), with contentid.ForLocal minting stable ids. API: ItemDetail gains additive videos/extras arrays (single + batch parity); library settings expose trailer_kinds. jellycompat now populates RemoteTrailers, LocalTrailerCount/SpecialFeatureCount, and serves real /LocalTrailers + /SpecialFeatures items playable through PlaybackInfo. Requires silo-plugin-sdk v0.9.0 (VideoRecord) before go.mod can bump; builds locally via go.work against the SDK feat/metadata-videos branch. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): trailers and extras sections, library trailer-kinds setting TrailersSection (YouTube thumbnails + youtube-nocookie modal) and ExtrasSection (plays extras through the standard watch controller) on movie and series detail pages; admin library form gains a trailer-kinds allow-list synced with the server default (all provider kinds), now also honored on library create. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): scan extra_id in scanMediaFiles; review cleanups scanMediaFiles (the plural row scanner behind GetByContentID/GetByFolder/ GetByExtraID and 20+ other queries) was missing the scan destination for the new extra_id column, which would have failed every media-file read at runtime with a column/destination count mismatch. Also: extend the batch equivalence test to seed item_videos/media_extras so the new videos/extras prefetch wiring is actually proven; drop the one-off pgxRows interface for the repo-wide pgx.Rows convention; reuse formatClock instead of a third duration formatter in ExtrasSection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): bump silo-plugin-sdk to v0.9.0 for VideoRecord Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(matching): exclude extras files from match queues and bulk content linking Dev verification caught extras media_files rows (content_id NULL by design) being swept into the movie/series match queues and the root-claim bulk relink: a '-featurette' suffix extra was matched onto its parent as a version, and a Trailers/ file minted a spurious local skeleton item that shadowed the extra's watch target. Add 'extra_id IS NULL' to the queue eligibility conditions, root/group claim relinks, observed-root content assignment, and the admin unmatched-files listing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): authorize local extras files through their parent item Dev verification: playback/start (and the shared MediaFileAuthorizer used by markers/subtitles/ebook reader) resolved file ownership only via episode_id/content_id, so extras files (extra_id only) 404ed. Add an ExtraLookup tier that resolves media_extras and gates on the parent item's access, mirroring the episode->series pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): resolve local extras through GetItemDetail for compat playback jellycompat PlaybackInfo (and any per-item consumer resolving arbitrary content ids) goes through GetItemDetail, which lacked the extras tier that GetWatchDetail has — so Jellyfin clients got zero MediaSources for extras. Add buildExtraItemDetail (minimal detail + ordinary playback surface, parent-gated access) as the fourth resolution tier, and map the extra type to Jellyfin's Video kind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): allow youtube-nocookie embeds in CSP; trailer modal a11y The frontend CSP's frame-src blocked the trailer modal's youtube-nocookie.com iframe (found on dev verification). Also add the missing sr-only DialogDescription and drop the redundant allowFullScreen attribute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address PR review findings for trailers/extras - Extras watch/item detail no longer stamp SeriesID/SeriesTitle for movie-owned extras (players key episodic post-roll flows off series_id); series-owned extras keep them (Codex). - processExtraFiles resolves the parent and upserts media_extras before the unchanged fast-path, and the fast-path now also compares mtime, so rematched parents / reclassified kinds / same-size replacements converge (Codex + CodeRabbit). - media_files upsert clears content/episode linkage atomically when extra_id is set (ownership mutual exclusion in one statement); the now-redundant MarkFileAsExtra helper is removed (CodeRabbit). - ScanFile's extras branch runs syncPresentLibraryState + reconcileLibraryMemberships so converting a primary file to an extra cleans stale library membership immediately (CodeRabbit). - media_extras migration adds the media_files FK as NOT VALID + VALIDATE to avoid a full-scan exclusive lock on large tables (CodeRabbit). - trailer_kinds input is trimmed/lowercased/deduped and unknown values are dropped instead of silently widening the allow-list to 'other' (CodeRabbit). - Extras authorization branches match the episode branch's posture: unconfigured lookup is a config error, nil extra is a 404 (CodeRabbit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
c29212b2c5 |
docs: handoff for account.capabilities_changed events
Design doc for a user-scoped capability-invalidation event on the existing events WebSocket, so clients refresh cached capability payloads (e.g. /downloads/capability) when admin permission or server settings change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0fb5afe479 |
feat(matching): split wrongly merged versions with watch-state reattribution; anchor group keys on provider tags (#319)
* feat(matching): split wrongly merged versions, reattribute watch state, anchor group keys on provider tags
Wrong merges (two titles normalizing to the same title+year key) stacked
different films as fake "versions" of one item with no in-app repair, and
explicit {tmdb-…}/[imdb-…] folder tags could not prevent it because the
content-group key ignored provider IDs entirely. Merges also silently
orphaned all per-user watch state.
- Anchor group keys on structured provider tags: same tag always groups,
different tags can never merge; untagged files keep title+year keys.
- media_identity_overrides: path-scoped (root/file) forced identities applied
during group inference, so admin splits survive rescans.
- internal/catalog/reattribute: shared user-state mover — exact moves for
file-linked rows, evidence-based user_watch_history classification via the
playback session log, newest-wins progress conflicts; wired into
rebindItemToExistingItem to stop merge orphaning (with S/E episode mapping).
- POST /admin/items/{id}/split (dry-run = full transaction + rollback, so
previews are exact), POST /admin/items/{id}/merge, GET /admin/items/{id}/files.
- Web admin: Split Versions dialog (files by folder → candidate search →
preview → split), Resolve link from ambiguous-roots diagnostics.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(reattribute): classify history before moving session log; cover managed downloads and series-scoped preferences
Review findings on #319, all reproduced against a migrated scratch database:
- moveFileSubset re-pointed playback_history_admin before the history
evidence query ran, erasing exactly the evidence proving a profile's plays
were all on moved files — their history stayed behind as ambiguous.
History classification now runs first; the pre-fix code demonstrably fails
TestRun_HistoryEvidenceClassification.
- Managed offline downloads (downloads.content_id/episode_id) were not
remapped on split or merge, stranding rows on the old id. Now moved per
file on splits and swept per id pair on merges/episode re-anchoring.
- Series merges left user_audio_preferences, user_subtitle_preferences,
user_series_playback_preferences (series_id-keyed) and the denormalized
user_home_item_dismissals.series_id behind. All four now move, mirroring
the provider-merge remap.
All five reattribute DB tests now verified green against PostgreSQL, with
new coverage for managed downloads, subtitle preferences, and dismissal
series ids.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
42602b7896 |
feat(policy): access groups + embedded OPA policy engine with decision audit log (#282)
* docs(policy): add OPA policy engine design spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(deps): add OPA v1.18.2 SDK for the policy engine Pulls github.com/open-policy-agent/opa v1.18.2 (policy engine core for the upcoming internal/policy subsystem) and the transitive upgrades go mod tidy applied (otel 1.44, grpc 1.81.1, prometheus/common 0.67.5). Full build verified. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add OPA engine core, vendor scope policy, and parity suite New internal/policy package (dead code — nothing wires into request paths yet): prepared-query Engine with 25ms eval timeout and fail-closed decode, typed PDP.ResolveViewerScope, go:embed vendor bundle, capabilities lockdown for future admin-authored Rego, and vendor scope.rego reproducing access.Resolver.Resolve (library intersection, disabled-library handling, quality/rating ceilings) with a narrowing-only silo_custom.scope.override extension hook. Parity proven by 1368 dual-execution subtests against the real access.Resolver, including the nil-vs-empty AllowedLibraryIDs battery and quality/rating variation; rank tables are test-pinned to internal/access. Rego unit tests run via opa/v1/tester inside go test. Bench: ~106µs/op per scope decision incl. input marshaling. Also restores the OPA requirement to go.mod (the earlier deps commit ran go mod tidy before any import existed, so tidy dropped it). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed, corrected (quality.allowed raw-file-rank divergence), and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy document store, foundation schema, and compile-check policy_foundation migration: policy_documents (one enabled doc per domain via partial unique index — two enabled docs would define override twice and conflict at eval), immutable policy_document_versions, single-row policy_generation counter, and the partitioned policy_decisions log table (daily range partitions, no FK, denial partial index). PolicyStore: transactional version numbering (FOR UPDATE), activation that verifies compiled_ok and bumps the generation in the same tx, enable/disable with typed ErrDomainAlreadyEnabled, and a delete guard for documents with an active version. CompileCheck sandboxes admin Rego: locked capabilities (no http.send/net.*/opa.runtime), enforced silo_custom.<domain> package path, vendor+stub layering, 2s budget, structured row/col errors. Engine gains NewEngineWithCustom / NewEngineFromStore with WARN-and-skip for invalid custom rows. DB-backed tests verified against a migrated Postgres (concurrent version numbering, atomic generation bumps, activation guards). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (domain constants extracted). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy System lifecycle with hot reload and cross-node invalidation policy.System owns one long-lived Engine and reloads it in place when policy documents change: EventPolicyChanged on the existing ChannelAdmin bus (new cache event constant) plus a 60s generation-poll fallback for Redis-less deployments, with a generation-consistent snapshot read. Vendor compile failure is startup-fatal; store/custom failures degrade to vendor-only and the poll loop heals them; runtime reload failures keep the last known-good engine. NotifyChanged gives the future admin handlers synchronous local reload + cross-node publish. Wiring: constructed in integrated/api modes only, PolicySystem field on api.Dependencies (unused by routes yet), policy.eval_timeout_ms setting (hot-reloaded via configWatcher.OnChange; default 25ms). Verified by a full server boot smoke and DB-backed convergence tests (event + poll paths, degraded boot, last-known-good). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add async decision logging with sampling, retention, and query repo DecisionLogger batch-inserts each node's policy decisions straight to the partitioned policy_decisions table via a non-blocking buffered channel (drop-and-count on overflow — logging never adds latency to or fails a decision). Scope decisions sample 1-in-N (default 50, setting policy.decision_log_scope_sample_rate); denials and eval errors always log; input/result JSON samples only at policy.decision_log_verbosity= verbose. Cursor-paginated DecisionRepository backs the upcoming admin log viewer. Retention via partman (daily partitions) and a PolicyDecisionLogCleanupTask honoring policy.decision_log_retention_days (default 14). PDP emits entries per evaluation; the System owns the logger lifecycle and settings hot-reload. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (removed an unused, unsynchronized PDP setter). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): add admin policy management API and capability endpoint /api/v1/policy/capability (authenticated feature detection) plus the acting-admin /api/v1/admin/policy surface: vendor Rego viewer, document CRUD with the one-enabled-per-domain conflict mapped to 409, immutable version creation (compile-checked; failed versions persist as audit history with structured row/col errors and can never activate), activate/rollback with synchronous reload + cross-node invalidation via System.NotifyChanged, stateless validate, throwaway-bundle simulate (never touches the live engine, never logs decisions), and cursor-paginated decision-log queries. Routes mount only when the policy system is wired, keeping proxy/transcode modes untouched. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (seeded the FK'd test user; replaced an unchecked fmt.Sscanf with strconv). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add /admin/policy workspace with Rego editor, simulate, and decision log New Policy admin page (System nav group): documents list with one-enabled-per-domain conflict handling, CodeMirror 6 Rego editor (hand-rolled StreamLanguage mode) with server compile issues rendered as inline lint diagnostics, explicit Save-version vs Activate flow with confirm, read-only vendor module viewer, simulate panel with seeded example inputs, version history with rollback, and a cursor-paginated decision-log browser. Capability-gated via /policy/capability. Adds the three decision-log settings to Log Retention. First code-editor dependency in web/ (@uiw/react-codemirror + @codemirror/*), decided in the design spec. Implementation drafted by Codex (GPT-5.5) via codex exec; verified here (lint, format:check, tsc --noEmit, vitest policy suites). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for viewer scope resolution policy.ViewerResolver implements the ViewerResolver interface backed by PDP.ResolveViewerScope and replaces access.Resolver at all five construction sites: router viewer middleware, notifications scopes, the reconciler, jellycompat's scope filter, and the ABS resolver (which now accepts a pre-built resolver, preserving its PIN-at-login semantics). PIN/profile-token verification and disabled-library loading are extracted into shared exported helpers used by both implementations, so the legacy resolver stays compiled as the parity reference with identical behavior. The adapter lives in internal/policy (which already depends on internal/access transitively) — direct typed PDP calls, no new import cycle. Sites without a policy system (proxy modes, bare test routers) keep the legacy resolver until the cleanup phase. Verified: full test suite green (jellycompat TestBeginWebOperation* and one playback GPU test are pre-existing failures, confirmed identical on main), 1368-case parity suite, dedicated ViewerResolver parity/PIN/ nil-vs-empty/fail-closed tests, and a full server boot smoke. Implementation drafted by Codex (GPT-5.5) via codex exec; a first-pass reflection-based adapter was rejected and reworked into the typed in-policy adapter; reviewed line-by-line and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for acting-admin and permission gates vendor/permission.rego reproduces the acting-admin rule (admin role + primary-profile-or-none), HasEffectivePermission semantics for marker_edit, and the metadata-curation rule including the subtle admin-past-refused-bypass case that requires the explicitly ASSIGNED permission. Policy-backed middleware in policy_gates.go keeps all Go-side lookups (declared-profile primary check, item->library resolution, the 404-on-unknown-item path) and preserves the legacy status/body taxonomy exactly — proven by dual-execution middleware tests that run every scenario through both implementations and assert byte-equal responses. Permission decisions always log (allowed flag populated); simulate and the capability endpoint gain the permission domain automatically via the domain registry. Router swaps behind single constructor choice points with the legacy gates retained for policy-less wiring. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for download and playback admission decisions vendor/action.rego decides download eligibility (downloads enabled + user allowed), download-transcode eligibility (transcode enabled + user allowed + artifacts available), and playback admission (stream/transcode counts vs limits, zero = unlimited), with a tightening-only silo_custom.action override that can also clamp a quality ceiling (never widen — merged via quality.min). Go keeps everything stateful: config loading, preset-ladder enumeration, and live session counting. Downloads consult an optional ActionDecider (nil = legacy logic) mapped back to the existing sentinel errors and capability response. Playback gains a minimal AdmissionDecider hook at the exact point of the legacy limit comparison: counts snapshot under the session mutex, PDP evaluated OUTSIDE the lock, then revalidated under lock before insert (retry on count drift) — no admission ever decided on stale counts and no eval under the mutex. Deny reasons map to the legacy ErrTooManyStreams / ErrTooManyTranscodes sentinels, pinned by tests. Parity: combination tables driven against the real PresetsFor / ensureTranscodeAllowed / SessionLimits math; full suite green (known pre-existing jellycompat flakes only). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (locking design verified line-by-line) and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): satisfy tsc -b strict return typing in the Rego stream tokenizer The production build (tsc -b) rejects assigning CodeMirror's string | void next() result to string | undefined; tsc --noEmit did not catch it. Restructured the string-literal loop. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): clearer error when a decision is undefined for partial input Vendor policies index required input fields directly, so a hand-written simulate payload missing fields yields an undefined decision. Surface that as 'decision X is undefined for this input (missing required input fields?)' instead of 'empty result' — found while exercising the simulate API against a live server. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(web): set changeOrigin automatically when the API proxy target is remote Remote dev backends sit behind vhost-routing proxies that reject a localhost Host header; local targets keep the existing pass-through behavior. Enables pointing the Vite dev server at a hosted backend via VITE_API_PROXY_TARGET in web/.env.local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): redesign the policy workspace around the decision pipeline The first-pass UI was structurally generic: a five-column document table squeezed beside the editor, three equal-weight action buttons with hidden preconditions, raw version IDs, and jargon copy — nothing taught the model. The page now teaches it: - A pipeline strip states the mental model up front: Silo decides the baseline -> your overrides narrow it -> every decision is logged. Tabs renamed to Overrides / Baseline / Decision Log (ids stay stable for bookmarked URLs). - The document table becomes one card per domain (Library visibility / Admin & permissions / Downloads & playback) with plain-language descriptions, example rules, status pills (Live vN / Draft / Disabled), inline creation, and the enable kill-switch in place. - Selecting an override drills into a full-width editor with a visible lifecycle rail (Draft -> Validated -> Saved -> Live) and one contextual primary action per step; the unedited live source shows no actions until edited. Version comments appear only at the save step. - Simulate is reframed as 'Test before going live' with a human verdict chip (Allowed / Denied — reason / ceiling summary) above the raw JSON; internal generation counters no longer surface. - History uses 'Make live' with plain go-live copy; authors read 'User N'; the baseline tab explains that upgrades never touch overrides. Hand-written redesign (no Codex); verified via vitest, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): present the policy baseline as readable rules, not raw Rego The Baseline tab dumped five Rego modules into read-only editors. It now leads with what the rules actually do: one card per domain with plain-language statements of the shipped behavior and a note on what an override may change, plus content-rating and playback-quality tier ladders parsed live from the lib module sources (so the tiers shown are the ones the server enforces, not a hardcoded copy). The Rego source stays one click away behind a per-module accordion and remains the stated source of truth; unrecognized modules fall back to source-only. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(policy): add access-groups design addendum Groups with permission toggles become the everyday admin surface; the Rego editor is demoted behind policy.editor_enabled (default off). Restriction-only composition: group grants are an upper bound, per-user settings tighten further — same rule as the existing account/profile merge, one layer up. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): add access groups — group defaults with restriction-only composition New access_groups table + users.access_group_id (one group per user, NULL = today's behavior). Group grants are an upper bound composed with the user's own settings by strictest-wins rules — library intersection, MinQuality, AND'd booleans, strictest positive stream/transcode limits, permission-mask intersection, and a requests toggle gating CreateRequest. The merge happens in Go (access.ApplyGroupPolicy / EffectivePolicyForUser) before policy inputs are built, so vendor Rego, the parity suites, and the decision log are untouched; every enforcement surface (viewer scope in both resolvers, permission gates, downloads, playback admission, requests) consumes the effective policy and fails closed on provider errors. Changing a group's quality ceiling bumps its members' access_policy_revision, mirroring the per-user rule. Additive admin API: /admin/access-groups CRUD with member counts; PUT /admin/users/{id} + user DTOs gain access_group_id. Also demotes the Rego editor: policy.editor_enabled (default off, hot-reloaded) drives the capability endpoint's editor_available and 403-gates editor endpoints while the engine and decision logging keep running. Design: docs/superpowers/specs/2026-07-02-access-groups-design.md. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (composition core + fail-closed call-site audit) and verified here. DB-backed group-store tests pending local Postgres recovery. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add Access Groups admin page and gate the policy editor New /admin/access-groups: a card grid summarizing each group (member count + key restrictions), drilling into an editor that reuses the same LibraryAccessSelector and quality presets as the user editor, with toggles for downloads/transcoded-downloads/requests, concurrent-stream and transcode limits, and a permissions mask (all-assignable by default, narrowable to specific permissions). Delete warns how many members fall back to the built-in defaults. Copy states the composition rule up front: a group grants the most a member can do; their own restrictions still apply on top. The user editor gains a Group picker and read-only row; the Policy nav entry is now hidden unless the capability reports the editor enabled. Plumbing (types, hooks, user-editor picker, nav gating) drafted by Codex (GPT-5.5); the Groups page hand-built. Verified: 25 tests across the touched suites, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): seed a Default Group and auto-assign newly created users Adds access_groups.is_default with a partial unique index (one default at most — the profiles is_primary pattern) and seeds a permissive 'Default Group' whose ceiling is a no-op, so assignment never changes anyone's effective access until an admin edits it. The seed is guarded against pre-existing defaults and name collisions; the Down migration only removes the row if it is still untouched. Assignment happens at the single INSERT INTO users choke point (UserRepository.Create): when no explicit group is given, access_group_id is filled by a scalar subquery on the default flag — NULL when no default exists. Every creation path (setup, signup, invites, OAuth, admin create) is covered by construction. Setting a new default via the API atomically clears the previous one in the same transaction. Deleting or unsetting the default is legal: new users then start with no group, which is pre-feature behavior. Implementation drafted by Codex (GPT-5.5); migration guards and the choke-point subquery reviewed line-by-line here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): surface the default access group Cards show a Default badge; the group editor gains a 'Default for new users' toggle (with copy noting existing users are never moved); the delete dialog warns when removing the default that new accounts will start with no group. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): ship the Default Group with house-rule ceilings Seed values per product decision: 5 concurrent streams, 5 transcodes, transcoded downloads off, and a permission mask of marker_edit only (metadata curation excluded). Plain downloads and requests stay on. The Down guard matches the new values so it still only removes an untouched seed row. Only newly created users are affected; existing users are never assigned. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): retire per-user defaults — the Default Group is the sole default policy Removes both legacy 'user defaults' mechanisms now that the seeded Default Group owns new-user policy: - users.max_streams / max_transcodes column defaults drop from 6/2 to 0 (= unrestricted at the user layer), so group ceilings apply to new signups/invites/OAuth users instead of fighting stale per-user numbers. Existing rows keep their stored values — nobody is silently uncapped on upgrade. - The dead defaults.max_playback_quality / defaults.max_profiles settings validation goes away with its only writer (the User Defaults dialog, removed on the web side). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): replace the User Defaults dialog with group-governed creation The Users page's 'User Defaults' dialog (defaults.* server settings) duplicated what access groups now do properly, and its values were only ever form prefill — no backend path applied them. The button now links to Access Groups, and the create-user form seeds unrestricted user-layer values (0 streams/transcodes, any quality, downloads allowed) so the member's group governs; per-user fields remain for tightening individual users. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): migrate existing non-admin users into the Default Group Existing users join the seeded Default Group on upgrade so one policy source governs the whole instance. Their per-user limits still holding the retired 6/2 column defaults are normalized to 0 in the same statement so the group's ceilings actually apply; deliberately customized values are preserved. Admin accounts stay ungrouped — scope/action decisions are role-blind, so grouping an admin would cap the server owner on upgrade. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): keep admins out of the Default Group and treat group moves as policy changes New-user creation now mirrors the migration's admin exclusion: the default access group is only auto-assigned to non-admin roles, so a fresh server owner no longer inherits the starter group's transcode denial and stream caps. Changing a user's access group now bumps access_policy_revision (the group carries permissions, quality, and limits, exactly like the per-user fields that already bump it) and triggers admin session revocation when the group actually changes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): enforce marker_edit through the PDP on marker write routes The Rego permission policy owned marker_edit but no Go caller ever consulted it: PUT/DELETE /markers went through a handler-local check that short-circuited admins and read only the user's own permissions, so group permission masks and custom policy overrides were ignored. Marker writes are now gated by router middleware like the other permission surfaces: a PDP-backed RequireMarkerEdit that evaluates the group-merged effective permissions (plus the legacy variant for proxy/test wiring without a policy system). The handler-local check and its user loader are gone. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert device/quality policy facts and honor the quality ceiling The download_transcode action check hard-coded an empty device ID and never asserted the requested quality, and no caller consumed ActionDecision.QualityCeiling — custom download policies keyed on those inputs were silently ineffective. Resolve now threads the request's device ID and requested quality into the action input, and a returned quality ceiling downscales the prepared transcode target (the ceiling applies to what is served, matching the serve-time rule in serveDownloadBytes). FileQuality and the content-rating pair stay intentionally empty for downloads — documented on downloadActionInput: those ceilings are enforced against the served artifact by the scope-derived access filter, and asserting the source's quality would wrongly deny capped transcodes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(access): align the default-group seed assertions with the migration The DB test still asserted the earlier no-op seed (transcode allowed, unlimited streams/transcodes, null permissions); the shipped migration seeds transcode denied, 5/5 limits, and marker_edit-only permissions, so the test failed on any database with the migration applied. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): lock the Rego sandbox by builtin purity and bound compile work Exclude every nondeterministic builtin from the admin sandbox instead of denylisting names, so OPA upgrades cannot silently expose impure builtins while pure helpers like net.cidr_contains stay usable. Apply the same capabilities to the runtime engine, cap concurrent compile checks, and reject oversized sources before they reach the uncancelable compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): require literal booleans in vendor override and input checks Bare object.get truthiness treated any non-false value as satisfied, so a malformed override 'allowed' value could fail to tighten a base grant and hand-crafted simulate input could flip flag predicates. Compare against literal true so anything else denies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): surface decision log cleanup failures to the task manager CleanupDecisionLogsOnce now returns the first error alongside the deleted count so a broken partition manager or DB outage marks the scheduled task failed instead of reporting 100% success while policy_decisions grows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): log admission decider errors before failing closed A policy-evaluation failure was silently mapped to the too-many-streams denial, making an engine outage indistinguishable from a real limit hit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): nil-guard the downloads user and restore the ABS legacy resolver effectiveDownloadUser dereferenced policy state before its nil-user check, and the ABS handler lost viewer-scoped filtering entirely when the policy system was unavailable because no legacy access.NewResolver fallback was wired like the other resolver paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): address admin policy review feedback - invalidate the version query by version_number, the key usePolicyVersion actually caches under - keep the goPrevious cursor-stack updater pure (Strict Mode double-invoke) - make version history rows keyboard-selectable like the document list - clamp download_transcode_allowed when downloads are disabled so groups cannot save a contradictory record Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): cap policy endpoint request bodies at 1 MiB The policy write endpoints (create document/version, set enabled, validate, simulate) decoded JSON bodies without a size limit, so an oversized payload buffered fully in memory before CompileCheck's 256 KiB source cap could reject it. Route all five through a shared decodePolicyRequest helper that wraps the body in http.MaxBytesReader and returns 413 with the repo's standard too_large error shape. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(access): forbid deleting or demoting the default access group Deleting the default group (or unsetting its is_default flag) left the server with no default: new non-admin users were then created ungrouped with max_streams/max_transcodes of 0 — unlimited — because the legacy per-user column defaults were retired in favor of the group's ceilings. The store now rejects both operations with ErrDefaultGroupRequired (mapped to 409); promoting another group remains the supported way to move the default, and atomically clears the previous one. The admin UI disables the delete button and the default toggle on the default group and explains the promote-another-group flow. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(web): keep unsaved policy drafts when a newer version activates elsewhere The editor state was keyed on the active version's id/sha, so a background refetch after another admin (or another tab) activated a version remounted the editor and silently discarded the dirty draft. PolicyEditorPanel now pins the seed it is editing against and only adopts an incoming seed when nothing can be lost: the editor is clean, the draft already equals the incoming source (the same-admin activate flow), or the selection moved to a different document. Otherwise the pinned editor stays mounted and an inline notice offers an explicit "Load live version" action. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(policy): fail reloads on invalid custom sources and surface degraded/apply state A stored custom source that stops compiling used to be silently skipped on reload: the bundle widened to vendor-only for that domain while the generation reported fully applied. Reload is now strict — a bad enabled source fails the reload and the last known-good engine keeps serving. Boot keeps its vendor fallback for availability, but skips are recorded on the engine and exposed (with store-outage reasons) through System.DegradedState and additive degraded fields on GET /policy/capability. Activate/SetEnabled re-run CompileCheck instead of trusting the stored compiled_ok flag. Mutation endpoints also no longer conflate persistence with live apply: activation/enable responses carry additive applied/failed_step/ loaded_generation fields and return 202 when the store change persisted but the local reload failed. Addresses review findings C1, C2, and the degraded-signal gap (6.1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): type deny reasons across the contract and enforce profile_verified Deny handling used to branch on exact free-text reason strings in three Go consumers, and playback reported ANY unrecognized reason — including custom override free text and engine failures — as a stream-limit error. Decisions now carry a stable reason_code (custom overrides always get custom_denial); downloads, the metadata-curation gate, and playback admission switch on codes, with a new ErrPlaybackNotAllowed -> 403 playback_not_allowed mapping for non-limit denials. Rego tests pin every vendor code. The scope contract's tighten-only profile_verified output was also emitted but never consumed; a policy revocation now surfaces as ErrProfileUnverified (403 profile_unverified) instead of silently proceeding. Addresses review findings 6.2 and C4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): close the dual-library disabled-scope bypass in direct item authorization EnsureAccessible, EnsureAccessibleIDs, and FilterAccessibleContentIDs gated library access with allow/deny predicates over a single joined media_item_libraries row, so an item linked to BOTH a passing library and a disabled one satisfied the disabled check via the passing row — a direct-ID bypass of disabled-library scope on the detail, media-file, playback, and download paths. All library access predicates now share one helper (libraryAccessConditions) emitting independent EXISTS / NOT EXISTS subqueries, the semantics GetByIDsWithAccess already used, including the orphan-item membership guard for disabled-only scopes. SQL-shape tests pin every builder and a DB-gated regression test covers the dual-library item end to end. Addresses review finding C3 (plus the same shape in buildFilterAccessibleContentIDsSQL, which the review did not flag). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): serialize quota check and row creation under a per-user advisory lock The concurrent-download quota was check-then-insert with nothing serializing the pair: parallel creates could all observe free quota before any row existed, bypassing the cap and stacking artifact encode jobs. All four check->insert spans (ephemeral original, artifact-backed, series batch, managed batch) now run inside Repository.WithUserQuotaLock — a pg_advisory_xact_lock keyed by user, so the serialization holds across nodes. The artifact path keeps the limiter-before-Ensure ordering (a rejected request must not leave an encode job behind) by holding the lock across Ensure. Managed-entry replacement stays quota-exempt and lock-free. A DB-gated barrier test races 8 creates against a cap of 1. Addresses review finding C5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert served quality at create time for original and remux downloads Direct-original and remux downloads serve the source resolution unchanged, but create-time policy checks left file_quality empty — an over-ceiling source registered a row serveDownloadBytes could never satisfy. Resolve now runs a final download action check with FileQuality populated on those two paths (capped transcodes keep the ceiling-on-artifact behavior), a custom override ceiling below the served resolution denies, and quality_ceiling_exceeded maps to ErrQualityUnavailable. The ActionInput contract now documents exactly when file_quality and the rating facts are supplied so custom policy authors are not misled. Addresses review finding C6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): guard activation against slow overrides and make eval timeouts observable A custom scope override that exceeds the 25ms eval budget compiled fine, activated fine, and then converted to 500s on every authenticated request — server-wide lockout authored in the admin editor. Activation and enable now run GuardEvalCost: the candidate source is evaluated on a throwaway engine against a canned representative input under the live budget, and a source that cannot complete is rejected 422 with ErrPolicySlowEval before it goes live. Runtime timeouts keep failing closed but now carry a distinct ErrPolicyEvalTimeout sentinel, an Error log, and a per-engine counter exposed as eval_timeouts on GET /policy/capability so intermittent near-budget policies are attributable. Addresses review finding C7. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt remediation files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
430224a1b9 |
perf: cut home-screen, Continue Watching, and Latest latency; cache shared home rails (#292)
* perf(jellycompat,sections): bound resume scan, batch leaf detail progress, widen section concurrency Three low-risk fixes from the section-fetch performance investigation (docs/superpowers/plans/2026-07-03-section-fetch-performance.md): - jellycompat: bound loadProgressPage at resumeScanMaxRows=300 so a single request never pages through more than that many in-progress rows. The cap is unconditional: it also covers the sparse-visible-set case (a heavy watcher whose recent rows are mostly dismissed/superseded, or a Series/Season-only request that matches no leaf in-progress row), where the page never fills and the loop would otherwise scan the entire history — previously an O(history) scan reaching tens of seconds. In the common case the loop exits far earlier, so the cap only bounds the pathological worst case; 300 leaves ample headroom to fill a ~20-item Continue Watching page. Beyond the cap the reported total is a clamped lower bound. Covered by TestLoadProgressPage_BoundsScanForSparseVisibleSet. - jellycompat: batch the leaf-item (movie/episode) progress lookup in GetItemDetailsByIDs via ListProgressWithCompletedHistory instead of a per-item GetProgressWithCompletedHistory (~100 sequential queries for a 50-item detail page). Series keep the per-item episode-rollup path (they own no progress row). Output is unchanged; a batch-lookup failure is now logged rather than silently dropping played state for the whole page. - sections: raise fetchAllMaxConcurrency 4 -> 6 to cut FetchAll wave count for large home layouts, staying within the default 20-conn pool. Part of the home/continue-watching latency work. AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat): keep Latest browse on the cross-library fast path under isPlayed /Items/Latest with isPlayed=false is the highest-frequency compat browse (~10.8k calls/day). The played overlay can't be pushed into SQL, so browse over-fetches and filters locally. The cross-library recently_added fast path (BrowseRecentlyAddedAcrossLibraries: one ~1ms index walk per library) was gated on Offset==0, so a heavy watcher who had already seen the newest items needed a 2nd chunk and fell through to BrowsePage — a whole-catalog MIN(first_seen_at) + GROUP BY HashAggregate over ~147k movies measured at ~755ms per call (0.8-1.6s observed end-to-end). Fetch the entire over-fetch budget (maxScannedRows) in a single merged fast-path walk instead of paging into BrowsePage, so the loop fills from one call. The clamp caveat (MaxLimit=1000 leaves a fall-through only for requestedLimit>200, off the Latest hot path) is documented inline. Part of the home/browse latency work. AI-use disclosure: implemented with AI (Claude) assistance. * fix(jellycompat): scope resume scan cap to resume path and bound the fast-path loop Addresses PR #292 review feedback: - Codex (P2): the resumeScanMaxRows cap was applied unconditionally in the general loop, which also paginates the completed (watched-items) list. Gate it on resumeFiltered so the completed path keeps exact TotalRecordCount and deep StartIndex pagination. Covered by TestLoadProgressPage_CompletedScanNotCapped. - CodeRabbit (Critical): the earlier raw-offset fast-path loop — the default Continue Watching shape and the sections-fallback route — had the same unbounded-scan bug and was not covered by the cap (the existing test forces EnableTotalRecordCount=true, routing around it). Bound it with the same resumeScanMaxRows guard. Covered by TestLoadProgressPage_BoundsFastPathScanForSparseVisibleSet. - CodeRabbit (Minor): tag the doc's fenced example blocks as text to satisfy markdownlint MD040. AI-use disclosure: implemented with AI (Claude) assistance. * perf(sections): cache shared user-agnostic home rails per access scope Home-screen rails that are identical for everyone who can see the same libraries (recently added, recently released, genre, trending on server, most watched, new to library, critically acclaimed, award winners, format showcase, seasonal, mood, trending discover, admin-curated lists, and library collections) were rebuilt from Postgres once per request, per user. Only the overlay on top of each row (watched flags, play position, presigned poster URLs) is actually per-user. Insert a process-global resolved-list cache at the FetchOne choke point in internal/sections. Each cacheable row is built once per access scope, held with a 15m TTL, and refreshed in the background 3m before expiry; singleflight collapses cold-miss stampedes into a single build. The per-user overlay still runs fresh in buildSectionsResponse, so no profile state is ever shared. Random and per-user rows (continue watching, next up, recommendations, hidden gems, forgotten favorites, activity feed, user collections) bypass the cache. The access-scope key captures every access boundary the fetch path enforces -- section identity (type + id + config hash) + item limit + accessible and disabled libraries + max content rating + excluded media types + name prefix + allowed-content-id allowlist -- and nothing per-user, so entries are safely shared. Empty membership is never cached (avoids freezing a transiently empty rail); background refreshes are bounded by a timeout. Scale (analytical, derived from the cache behavior -- not a measured latency): for the user-agnostic rows, Postgres section-query volume collapses from O(rows x concurrent requests) to O(rows x distinct access scopes) per 15m refresh window, because most users share a handful of access scopes. Illustrative -- 40 cacheable rows on a home screen, 1000 concurrent users falling into ~5 distinct access scopes: - before: ~40 x 1000 = ~40,000 section queries per wave of home loads - after: ~40 x 5 = ~200 builds per 15m window (plus one background refresh per row per scope), i.e. a warm home load runs zero section queries for these rows. That is a ~99% reduction in shared section-query load at that concurrency; the win grows with concurrency and shrinks as access-scope diversity rises. Design/plan doc added under docs/superpowers/plans/. * perf(jellycompat): serve per-library Latest via the cached recently-added section A jellyfin-compat per-library /Items/Latest rail is the same user-agnostic list as the native "recently added" library rail -- both order by mil.first_seen_at DESC. It was rebuilt on every request through directContentService.BrowseItems, missing the resolved-list cache entirely. Route per-library Latest for movies and series libraries through the native section fetch instead, so it reuses the shared cache. HandleLatest resolves the library's type once, and for a movies/series library builds a synthetic SectionRecentlyAdded with the same type + config + limit + access scope the native rail uses and calls FetchOne; the per-user overlay (favorites, progress, episode targets, presign) is extracted into buildLatestItemDTOs and shared by both the native and BrowseItems paths, so no overlay logic is duplicated. Cached *models.MediaItem values are read-only -- LocalizeItemModels deep-copies before any presign mutation. To let the two surfaces share one entry, resolvedListCacheKey no longer includes the arbitrary section ID: every cacheable section type derives its membership from type + config + limit + scope, never from its own ID (audited all 14 cacheable types plus the library-collection path; the sole s.ID read lives in the non-cacheable user-collection branch). A native recently-added rail and the compat Latest for the same library + scope now collapse to ONE cache entry, built once and reused. Access-scope isolation is unchanged -- the removed ID never carried access information, and every access boundary (libraries, rating cap, excluded types, content allow-list, name prefix) still keys the entry. Guardrails: the native path is restricted to movies and series libraries; every other library type (ebook, music, manga, mixed) is ignored and keeps its exact BrowseItems behavior -- important because an unfiltered recently-added fetch would otherwise surface non-video items to Jellyfin clients that only expect video. Deeper pages, played-filter and backdrop-required requests, a client asking for a type other than the library's own, and any FetchOne error also fall back to BrowseItems. Chosen over an alternative that gave the synthetic section a deterministic ID (which kept two separate cache entries): both returned identical data with similar complexity, so the shared-entry design won. * fix(sections,jellycompat): post-review fixes for the shared-list cache and Latest path Consolidates fixes from the branch's adversarial review and PR #292 review comments into one commit: - Latest fast path: fall back to BrowseItems when a request carries a genre, name-prefix, or person filter (the synthetic recently-added section cannot express these, so serving it unfiltered would return a wrong, broader set). Eligibility is decided by latestFastPathEligible and covered by a test. - Clamp the /Items/Latest page size to compatBrowseMaxLimit before building the section, matching the BrowseItems fallback, so a large client Limit can't drive an oversized recently-added fetch or explode the shared cache key with unbounded ItemLimit values. - Evict expired entries from the process-global resolvedListCache: resolvedListSet sweeps expired keys at most once per minute, bounding the map to scopes seen within one TTL window. Covered by TestResolvedListCacheEvictsExpiredEntries. - Log a short digest of the cache key (resolvedListLogKey) instead of the raw key in the background-refresh panic/error paths, since the key embeds user-controlled access-scope fields such as NamePrefix. Skipped review comments (verified already fixed or stale against current code): the resume fast-path scan bound and watched-items cap (04d2e795) and the docs fence-language tags (already addressed). Build, vet, and go test -race pass for internal/sections and internal/jellycompat. * perf(plugins): cache plugin installations in-memory, invalidated on lifecycle change ## Problem Every poster/image on a warm home rail re-read plugin_installations from Postgres to answer "is this plugin enabled?" and to acquire the plugin client (Source A: metadata chain buildProviders enabled-check; Source B: ensureClient -> loadInstallation). Plugin-resolved image URLs are never URL-cached, so the plugin source and the DB read behind it fired again on every identical warm request; 100% of images in the target library are plugin-backed. ## Solution - Guarded in-memory installation cache (map[int]*Installation + RWMutex) in plugins.Service. loadInstallation reads through it; the requireEnabled gate stays after the cache read so ErrInstallationDisabled semantics are unchanged. invalidateInstallationCache clears it and is self-registered as a lifecycle hook, so Service.OnLifecycleChange wipes it on install/enable/disable/update/ uninstall. - A generation counter closes an invalidate-vs-repopulate race: captured before installations.GetByID and re-checked under the write lock, so a row fetched before a lifecycle mutation is never written into a freshly cleared cache (would otherwise resurrect a just-disabled plugin). - Route the metadata chain enabled-check through the same cache via a structural InstallationEnabledChecker interface (nil-safe: falls back to the pool query when no checker is injected), wired in cmd/silo/main.go. ## Post-review fix (auto-update reliability blocker) AutoUpdateService mutated installations (new InstallPath, old dir deleted) on the default auto update policy without firing OnLifecycleChange, leaving the cache stale and breaking plugins with "stored plugin manifest mismatch" until restart. It now takes an onChange callback wired to Service.OnLifecycleChange and fires it once per Check run that mutated a row. ## Verification go build/vet, go test ./internal/plugins/... ./internal/metadata/... (-race). Tests: cache hit/invalidation, racing-invalidation guard, IsInstallationEnabled, auto-update fires onChange. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(jellycompat): batch per-item presign, and enrich series on the cached Latest path ## Problem List rails presigned each item's poster/backdrop/logo/still image individually (~160 singular resolver calls for a 40-item page where 4 batched calls suffice), and ItemsHandler carried a near-verbatim duplicate of the batch presigner. ## Solution (batching) Promote the batch presigner to a shared package-level presignCompatListItems (presign_list.go) with a generic collectImagePaths[T]; convert the per-item loops (cached home/Latest rail, favorites, batch loaders, userdata favorites) to one batched PresignImageURLsWithExpiry per image type per page; batch the season/episode collections; delete the three duplicate presign helpers. URL output is unchanged (verified byte-for-byte). ## Post-review fix (series Latest data-parity regression) The native cached Latest fast path built items via compatListItemsFromModels + buildLatestItemDTOs and never ran the series watch-state rollup, so a series library's Latest lost Played / UnplayedItemCount and page 1 disagreed with the BrowseItems fallback. enrichSeriesUserData is promoted to the ContentService interface and called on the native path (reused, not duplicated). ## Verification go build/vet, go test ./internal/jellycompat/... ./internal/catalog/... Tests: bounded presign invocation counts + per-item URL mapping; series rollup populated on the native Latest path. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(sections): gate personalized rails out of the shared cache; widen refresh lead ## Problem 1. The shared home-rail cache whitelisted custom_filter/genre sections by TYPE alone, but those route through fetchFiltered -> ParseQueryDefinition and can carry personalized (per-profile) rules/sorts (watched, favorited, in_watchlist, in_progress, last_watched; sorts progress/date_viewed/plays). Their membership is per-profile yet the cache key excludes userID/profileID, so a personalized rail built for one profile was served to others in the same access scope for up to 15m -- a cross-profile watchlist/watch-state leak. 2. The background-refresh lead was tuned so steady traffic is served a warm entry from a longer soft window. ## Solution - Add QueryDefinition.IsPersonalized() (reusing the existing QueryFieldRequiresProfile/QuerySortRequiresProfile helpers). isCacheableSectionType now parses the section QueryDefinition and refuses to cache custom_filter/genre when personalized; non-personalized definitions stay cacheable. Seasonal/mood/trending build their definitions server-side and stay unconditionally cacheable. - resolvedListRefreshLead 3m -> 10m (soft threshold builtAt+5min instead of builtAt+12min). ## Verification go build/vet, go test ./internal/sections/... ./internal/catalog/... (-race). Test: personalized custom_filter/genre not cacheable; non-personalized are. ## AI-use disclosure Implemented with AI assistance (Claude). * fix(sections,metadata): post-review fixes for shared cache and plugin chain staleness Addresses three review findings on PR #292: - sections: canonicalize section config JSON before hashing so configs differing only in whitespace/field order share a cache entry (native + jellycompat rail sharing). Added TestHashSectionConfigCanonicalizes. - metadata: invalidate the resolved-chain cache on plugin lifecycle changes; the installation-enabled check already reads the invalidated plugin cache, but resolveChainCached could serve a stale provider chain for up to chainCacheTTL after a provider's availability changed. - jellycompat: move ctx to the first parameter of presignCompatListItems for consistency with the other presign helpers. Skipped the episode-image presign batching nitpick: the resolver already dedupes+singleflights, so it is a Minor perf-only item not worth the two-pass refactor risk in this pass. * fix(sections,jellycompat): harden shared rail cache and Latest fast path per review Addresses the eight findings from the deep review of this PR: - Detach the blocking cold-miss rebuild from the singleflight leader's request context (context.WithoutCancel + the shared 30s build timeout) so one client disconnect no longer fails every collapsed waiter and leaves the entry uncached. - Stop client-controlled values minting unbounded cache entries: the compat Latest fast path now always fetches a fixed 100-row budget and slices to the requested limit (one entry per scope+library instead of one per Limit value), and an unrecognized MaxOfficialRating string disqualifies the fast path instead of entering the global cache key. - Add release_date to the sections item projection/scan so movies served via the Latest fast path keep PremiereDate (Jellyfin default-set field) in parity with the BrowseItems fallback. - Fall back to per-item progress lookups when the batched leaf progress query fails, restoring one-item-at-a-time degradation instead of blanking played state for the whole page. - Derive cache eligibility from a single source of truth: fetchSection and isCacheableSectionType now share the userAgnosticSectionFetcher table, whose no-userID/profileID signature makes a fetcher drop out of the cacheable set at compile time if it ever gains per-profile inputs. - Decide Latest fast-path eligibility off the actual browse params the fallback would receive, so any filter later added to buildBrowseParams automatically disqualifies the cached path; share one compatDefaultBrowseLimit constant between both paths. - Extract AccessFilter.WriteAccessScopeCacheKey as the shared, security- critical serializer for all access-scoped caches (resolved-list, editorial candidates, audiobook groups); the editorial key now captures ExcludedMediaTypes, which its loaders already applied in SQL. - Strip leaked agent-transcript markup from the section-fetch plan doc. go build ./..., go vet, gofmt clean; go test -race on internal/sections, internal/catalog, internal/jellycompat passes (TestBeginWebOperation* failures are the known pre-existing flakes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
604bbf1a0f |
feat(playback): unified restart-resilient playback (native + jellycompat) (#174)
* feat(playback): unified restart-resilient playback via shared TranscodeManager Make direct, remux, and native HLS transcode sessions survive a server restart through one shared flow instead of per-method paths. A missing in-memory session becomes a reconstruct trigger, not a 404: the server rebuilds the session from a tiny durable recipe card plus the position the client re-supplies on its next request. - internal/playback/transcode_manager.go: shared TranscodeManager owning the transcodes map, recipe-card lifecycle, reconstruct single-flight + concurrency cap, LoadOrReconstructSession front door, ReconstructSession / ReconstructTranscode, and orphan cleanup. ~90% is logic moved out of the native handler (no behavior change), not new surface. - internal/playback/recipecard.go + recipecard_postgres.go: RecipeCard with a PlayMethod discriminator (direct/remux/transcode; empty decodes as transcode for back-compat) behind a swappable, nil-safe RecipeStore interface backed by transcode_recipes. - internal/playback/session.go: RegisterReconstructed inserts a rebuilt Session under its existing id (no UUID mint, no limit double-count, race-yielding). - internal/playback/transcode.go: CloseProcess keeps the output dir so a reconstruct winner keeps serving; Close removes it. - internal/api/handlers: drain the transcode lifecycle into the manager; wire reconstruct into the stream/segment serve paths; re-bind ownership to the live caller (refuse userID==0/mismatch); card-aware orphan cleanup. - migrations: add transcode_recipes (expires_at TTL, filter-on-read, indexed). Ownership stays two-factor: an authenticated caller AND a session.UserID that matches; the card stores no secrets and identity is re-resolved per request. Tests: recipe-card round-trip/legacy-decode/disabled-noop, RegisterReconstructed insert/race/concurrency, close-vs-close-process dir semantics, the LoadOrReconstructSession status matrix, and the reconstruct concurrency cap. AI-use: implemented with AI assistance (design, implementation, adversarial review). * feat(jellycompat): reconstruct transcodes across restart via shared manager Bring Jellyfin (jellycompat) HLS playback onto the same restart-resilient flow as the native path. Previously jellycompat owned a separate PlaybackHandler with a private transcodes map and a duplicated transcode lifecycle that never grew the reconstruct half, so an in-flight Jellyfin transcode died on restart and the next segment request 404'd. - Embed the shared playback.TranscodeManager and delete the duplicate lifecycle, so jellycompat gets reconstruct, the concurrency cap, the node-affinity rule, and the card lifecycle for free. - internal/jellycompat/playback_sessions_postgres.go: DurableCompatPlaybackStore, a write-through cache over jellycompat_playback_sessions behind the new CompatPlaybackStore interface (nil pool degrades to cache-only). This persists the load-bearing PlaySessionId -> UpstreamSessionID mapping (plus media sources, route item id, seek) so it survives a restart instead of vanishing with the map. - Write a recipe card on compat transcode start keyed by the upstream session id, using the native StreamAppUserID so the ownership re-bind matches; reconstruct the upstream session and the transcode seeked to the requested seg_NNNNN. - migrations: add jellycompat_playback_sessions (expires_at TTL + compat_token index, full PlaybackSession in data JSONB). Auth is mapped to the native user id before reconstruct so the same two-factor ownership check and userID==0/mismatch refusal apply unchanged. Tests: DB-gated (SILO_TEST_DATABASE_URL) durable-store round-trip proving a session written by one instance reloads in a fresh one (the restart case), plus a nil-pool cache-only path; existing handler tests updated to the manager. AI-use: implemented with AI assistance (design, implementation, adversarial review). * docs(playback): consolidate unified playback reconstruction design Replace the three overlapping playback docs (the native Postgres restart-resilience spec, the jellycompat plan, and the unification spec) with a single self-contained design at docs/superpowers/specs/unified-playback-reconstruct.md. The doc leads with the unified design — the one-idea reconstruct model, a strong visual flow of a restart mid-playback, the shared TranscodeManager + recipe card, the two swappable durable stores, security, the concurrency cap and node-affinity constraint, preconditions, and verification. The design history and rationale (reconstruct-not-rehydrate, phased delivery, Redis-vs-Postgres, token-as- descriptor, failure analysis) move to an appendix. It references no other md file. AI-use: written with AI assistance. * fix(playback): address review on restart-resilient playback Four fixes from PR review of the unified reconstruction work: - Rewrite the recipe card on audio-track change. HandleChangeAudioTrack only updated the in-memory session/transcode, so after a restart reconstruct resumed with the stale AudioTrackIndex/TranscodeAudio (and stale play method) from the start-time card. Re-save the card (direct/remux/transcode) with the switched state, mirroring the start-card pattern. - Guard nil TranscodeManager in LoadOrReconstructSession and ReconstructSession. StreamHandler.TM is documented optional (tests/minimal setups); a missing session previously panicked in recipeEnabled instead of returning SessionMissing. ReconstructTranscode already guarded nil; make the two siblings consistent. - Reject direct/remux cards in doReconstructTranscode before spawning ffmpeg, so a non-transcode card id can never enter the HLS reconstruction path. - Log a non-success status from the remote transcode-node DELETE in CloseTranscodeSession; a 401/404/500 was previously silent. AI-use: implemented with AI assistance. * fix(playback): harden restart-resilient compat sessions * feat(playback): token-carried reconstruction across restarts Build on the shared TranscodeManager (introduced earlier in this branch) so a playback session survives an API-server or transcode-node restart without the client re-negotiating, and retire the Postgres transcode_recipes store in favor of a recipe carried inside the signed stream token. - RecipeCard encodes the byte-affecting encode parameters and rides inside the stream token; LoadOrReconstructSession rebuilds the in-memory Session (and, for integrated transcodes, the ffmpeg process) on a cold miss, single-flighted per session and paced by a spawn semaphore. Removes recipecard_postgres.go and the 20260617233705_add_transcode_recipes migration. - transcodenode reconstructs a lost ffmpeg node-side from the forwarded token. - TR-lease: proxy/streamauth enforce a revocation deny-marker on every served segment, with a 500ms Redis timeout, a bounded per-session "allowed" cache (3s TTL, expiry-first graceful eviction), and a degraded-fail-open counter. Review hardening folded in: - Manifest/segment handlers do the in-memory session lookup first and only verify the stream token on a reconstruct miss (token HMAC was per-segment). - Copy-mode reconstruct never applies the encoded-only seg*dur seek, at spawn time or via the recovery path: RestartSeekTarget reports "unresolved" for a copy session whose manifest cannot yet map the segment, so the client retries instead of seeking to a fabricated source time. - Crash teardown is a compare-and-delete (CloseTranscodeSessionIf returns whether it matched); the crash closure tears down the playback session only when it matched, so a session reconstructed under the same id is not killed. - Reconstruct enforces the same per-user stream/transcode caps as a fresh start (RegisterReconstructedWithLimits), closing a token-replay slot bypass. AI-use disclosure: implemented with AI assistance (Claude Code), including a two-round multi-agent adversarial review whose findings drove the hardening. * feat(jellycompat): node-side transcode reconstruct via shared recipe store Make Jellyfin-compat playback sessions survive a server or transcode-node restart by reusing the shared TranscodeManager reconstruct path and a durable recipe store, on top of the durable compat session store added earlier in this branch. - Node-side transcode reconstruct goes through the shared recipe store; the recipe is persisted to the control-plane store (Redis) when a dedicated transcode node is used so the node can rebuild ffmpeg after its own restart. - Adopt the shared manager's API (3-arg OnFFmpegCrash carrying the dead session, guarded CloseTranscodeSessionIf, RegisterReconstructedWithLimits). Review hardening folded in: - Recipe lifecycle: noderecipe.Store gains Delete, called on deliberate teardown (stop, method-switch discard, node stop/force-reload) so a stopped session cannot be resurrected by a buffered request after a node restart; crash paths intentionally keep the recipe so a resume can reconstruct. - Crash closure tears down the upstream session only when the guarded transcode close matched, so a reconstructed successor is never left orphaned. - Copy-mode segment recovery surfaces a retryable not-found instead of a wrong-position restart, matching the native and node paths. - Durable Update is now a SELECT ... FOR UPDATE transaction, removing the lost-update clobber that could silently drop a transcode recipe. - Empty-token route resolution no longer falls back to an unbounded full-table scan; DB expiry filters bind the injected clock; the redundant re-Get is gone. AI-use disclosure: implemented with AI assistance (Claude Code), including a two-round multi-agent adversarial review whose findings drove the hardening. * docs(playback): consolidate restart-resilient playback design Replace the superpowers spec with a single architecture record describing the token-carried recipe card, the shared TranscodeManager reconstruct path for direct/remux/transcode, the jellycompat durable session + node recipe store, and the revocation-lease model with its fail-open tradeoff. AI-use disclosure: written with AI assistance (Claude Code). * docs(playback): correct jellycompat node-recipe rationale in comments The noderecipe / transcode-node / jellycompat comments justified the Redis recipe store with "a Jellyfin client cannot round-trip a token". The real reason: the node-hop token is server-minted and could carry the recipe, but the recipe is mutated in place under a stable session id (a /Sessions/Playing/Progress audio switch restarts ffmpeg without re-minting the client's token) and a third-party Jellyfin client cannot be driven to refresh a stale token, so the node must reconstruct from a server-authoritative, node-reachable store. Aligns the comments with docs/architecture/restart-resilient-playback.md §10. Comment-only; no behavior change. * refactor(playback): remove deny-lease revocation, defer to future PR The deny-lease stream-revocation mechanism (the internal/streamauth package, its silo:streamauth:<sid> Redis markers, the proxy Allowed() enforcement, and the admin Stop/Terminate deny write) only ever enforced on the offload-proxy topology and was a silent no-op on the integrated single box and the dedicated transcode node. Rather than ship a partial revocation feature that looks complete but isn't, remove it wholesale and defer a uniform cross-topology revocation design to a dedicated follow-up. Removed: internal/streamauth (package + tests); the LeaseDenier field, StreamLeaseDenier interface, and denyStreamLease helper in playback.go; the admin deny write; the router/main wiring; and the proxy verifyToken Allowed() gate. The unified-reconstruct core (recipe-token, LoadOrReconstructSession) is orthogonal and untouched. Known limitation (now on every topology): admin Terminate and user Stop tear down the live in-memory session and ffmpeg producer, but a still-valid stream token can reconstruct the session until its 24h TTL expires. No node-side byte-withholding ships in this PR. docs/architecture/restart-resilient-playback.md is updated to mark the revocation/deny-lease sections as deferred and to drop the overstated "instant revocation on admin kill" claim. * fix(playback): allow zero-caller bearer on transcode reconstruct The authless HLS transcode delivery routes (master.m3u8 / segment) treat the session UUID as the bearer credential, so a real request carries requestUserID == 0. The live serve path already allows this, but ReconstructSession hard-rejected a zero caller, so a request that worked before a restart became SessionMissing -> 404 after the in-memory session was gone, breaking the restart resilience these routes advertise. Match the live-path contract in LoadOrReconstructSession: allow a zero caller (UUID-as-bearer) and refuse only a non-zero caller that mismatches the card owner. The reconstructed session is bound to card.UserID either way. Adds TestReconstructSession_Ownership covering both cases. * fix(jellycompat): re-persist recipe on local audio switch A Jellyfin client switching audio on an integrated/local compat transcode restarted live ffmpeg with the new track but did not re-persist PlaybackSession.Recipe. The remote branch already re-persists via startRemoteTranscode -> persistTranscodeRecipe. After a central restart, reconstruct rebuilt ffmpeg from the stale Recipe.AudioTrackIndex, so the integrated session resumed on the original audio track. Persist the updated recipe (best-effort) after a successful Restart in the local branch, mirroring the remote branch, so the durable Recipe.AudioTrackIndex tracks live ffmpeg. Adds a regression test. * fix(playback): strip stream token from proxied transcode-node URL proxyToTranscodeNode appended the client's raw query string to the internal transcode-node URL and logged that URL on transport failure. When a remote transcode runs without a separate proxy node, that query carries ?st=<signed JWT> — a 24h bearer reconstruction descriptor exposing the media path and recipe claims — placing the token into internal requests and error logs. Strip the "st" param before building targetURL, preserving any other query params. The token is neither forwarded to the node nor present in the logged URL. Header-forwarding of the token (so the node can reconstruct) is a separate follow-up (#6). * fix(playback): fail open on transient limit-provider error in reconstruct During the reconstruct wave right after a restart (Postgres under peak load), a transient limit-provider DB error was collapsed into a hard 404, permanently stopping playback for a user within their limits. limitsForUser wrapped any provider error, RegisterReconstructedWithLimits propagated it, and ReconstructSession mapped every error to SessionMissing -> 404 - indistinguishable from a genuine over-cap rejection. Distinguish the two: tag provider errors with a new ErrLimitProviderUnavailable sentinel and, during reconstruct, fail OPEN on a provider error (admit via RegisterReconstructed + log a degraded warning) rather than refuse - mirroring the reliability-first fail-open-on-dependency-error philosophy. A genuine ErrTooManyStreams / ErrTooManyTranscodes over-cap still refuses. Adds tests for both the fail-open and still-refused paths. * fix(playback): forward stream token to transcode node as header The dedicated transcode node's reconstruct path reads the stream token only from the X-Silo-Stream-Token header, but proxyToTranscodeNode forwarded only the node-API bearer token (and #5 now strips st from the URL). So when the central API proxied to the node and the node self-restarted, it could not reconstruct from the recipe-complete native token -> 404. Capture st before stripping it from the URL, verify it at the API boundary (streamtoken.Verify + SessionID match, mirroring the node's own check), and forward it as X-Silo-Stream-Token. Best-effort: a missing/invalid token never blocks the live proxy, and the token is still kept out of the forwarded URL and logs. * fix(playback): restart node ffmpeg on native remote audio switch A native audio-track switch on an offloaded/remote transcode was a no-op at the node yet returned 200 with a fresh URL: HandleChangeAudioTrack restarted ffmpeg only when the API owned a LOCAL TranscodeSession, so for an offloaded transcode the node kept serving the OLD audio (the node consults the token only on a session miss). The replacement URL was also minted from identity- only claims, so a later node restart 404'd. For the offloaded transcode case (detected via session.TranscodeNodeURL), POST a fresh /transcode/start to the node with the new AudioTrackIndex (handleStart tears down and restarts ffmpeg) and mint the replacement proxy URL from a full RecipeCard so reconstruct survives a node restart. The encode recipe is derived from the durable session target fields plus the file, mirroring HandleStartTranscode. A concrete SegmentDuration (playback.DefaultSegmentDuration) is embedded rather than 0: the node's token completeness gate treats SegmentDuration<=0 as incomplete and falls back to a recipe store the native path never populates, which would 404 on a node restart - the exact resilience this path provides. A failed node POST now surfaces 502 rather than a false 200. Remux and non-offloaded (local) transcode paths keep their prior identity-claim URLs unchanged. Known limitation: Session does not persist the original SegmentDuration or SubtitleTrackIndex/SubtitleBurnIn, so a remote audio switch resets subtitle selection to none and assumes the default segment length; a client that started with a non-default segment length will resegment on switch. Making that state durable on the session is a follow-up. * docs(playback): scrub stale deny-lease/revalidator comments The deny-lease revocation mechanism and its "central revalidator" were removed earlier in this branch, but four comments still described them as live (transcode_manager.go, noderecipe/store.go, streamtoken/token.go, proxy/server.go). Reword them to match the shipped behavior: ownership claims are re-resolved at reconstruct, the noderecipe store shares Redis only with the node-session tracker, and a sub-TTL hard cut depends on a node-side revocation mechanism that is deferred to a future PR. * fix(jellycompat): surface durable playback-session write failures DurableCompatPlaybackStore.Update applied the in-memory mutation and then swallowed every Postgres commit-failure path, returning nil. Callers that promise restart resilience (persistTranscodeRecipe's recipe write, the upstream-session binds in streams.go) were told the session was durably persisted when only the cache held it, so a transient DB hiccup could leave the next restart reloading a stale row (wrong audio track) or 404ing. updateDB now returns the genuine DB round-trip error (begin/query/unmarshal/ marshal/exec/commit); Update propagates it while still applying the in-memory mutation so live state stays correct. A nil pool and a genuinely absent/expired row remain best-effort (return nil) — only real infrastructure failures propagate, so existing rollback paths fire exactly when durability is lost. Part of #174 * fix(playback): re-inject stream token into proxied transcode manifests API-proxied remote transcode manifests dropped the reconstruct token from their segment URLs, so playback died after a node or API restart. When a remote transcode has no separate proxy node, the client loads its manifest via the API-local path; proxyToTranscodeNode strips the signed token ("st") from the forwarded URL (keeping it off node URLs and logs, forwarded only as the X-Silo-Stream-Token header), and the node builds relative segment URIs from that token-less query. The segment URLs the client received carried no token, and the proxy only re-attached the header when an incoming segment request already had "st" — which it never did — so a restart made those segments non-reconstructable and they 404'd. proxyToTranscodeNode now rewrites the manifest body at the boundary: every segment and #EXT-X-MAP init URI gets the client-facing, API-verified token re-appended (new playback.AppendManifestQueryParam helper), so the client's later segment fetches carry "st" again and reconstruct after a restart. The token still never reaches the node URL or its logs. Only 200 .m3u8 responses are rewritten (Content-Length corrected); segments stream through untouched. Part of #174 * fix(playback): preserve subtitle/cadence recipe across offloaded audio switch Switching audio on a remote (offloaded) transcode with burned-in subtitles silently dropped them, and reset a non-default segment cadence. The offloaded audio-switch restart rebuilt the node start request from Session state, but Session/SessionStreamState retained no subtitle or segment-duration state (only the live local ts.Opts() and the RecipeCard did), so the branch hard-coded SubtitleTrackIndex:-1, SubtitleBurnIn:false and SegmentDuration:Default — signing that altered recipe into the replacement stream token. An audio switch then changed bytes beyond audio selection, and any later reconstruct kept the wrong no-subtitle/wrong-cadence recipe. Persist the byte-affecting recipe on the session: SubtitleTrackIndex, SubtitleBurnIn and SegmentDuration are added to Session/SessionStreamState, populated at start (finalizeTranscodeStart) and on post-restart reconstruct (ReconstructSession from the card), carried forward on every audio-switch state update, and read back when rebuilding the offloaded node request and its recipe card. The restart now reproduces the exact live stream. Also resolves the M-4b non-default segment_duration reset. Part of #174 * fix(playback): serialize transcode spawn paths with a per-session lock Reconstruct was single-flighted only against other reconstructs, so a restart-driven segment reconstruct racing a quality/seek/audio fresh start could spawn two ffmpeg processes writing the same output directory at once — segment corruption, partial-write closes, orphaned processes, and skewed active-job accounting. The atomic register-after-spawn (GetOrRegister / the reconstruct compare-on-register) prevented a map leak but not the concurrent disk writers, because the losing path had already spawned. The dedicated transcode node had the same split between handleStart and spawnReconstruct. Add a refcounted per-session lifecycle lock to both TranscodeManager and the node Server, held across "check existing -> spawn -> register": - reconstruct (doReconstructTranscode / spawnReconstruct) re-checks under the lock and yields to any live session instead of spawning a duplicate; - the native and jellycompat fresh-start paths take the lock around their spawn+register (the native path also closes any session a reconstruct rebuilt in the meantime so its fresh ffmpeg is the sole writer); - the node handleStart holds it across teardown+spawn+register. The refcount drops the map entry once no path holds/waits, keeping it bounded. GetOrRegisterTranscodeSession is removed — the lock supersedes it and keeping a register-after-spawn primitive would invite reintroducing the race. Part of #174 * fix(playback): serialize restart re-spawn under the session lifecycle lock TranscodeSession.Restart() releases s.mu across cancel -> wait-for-done -> re-exec and spawns ffmpeg into opts.OutputDir without holding the per-session lifecycle lock. LockSessionLifecycle's contract (fresh start, restart, reconstruct) requires restart to hold it too, but all five callers invoked Restart unlocked: native audio-switch and segment-recovery, compat audio-switch and segment-recovery, and the transcode-node segment-recovery. A restart racing another restart (audio-switch vs segment-recovery) or a fresh-start/reconstruct could land two ffmpeg processes writing the same segment directory -- mixed timelines, init.mp4/segment mismatch, and an orphaned-but-still-writing ffmpeg -- the exact concurrent-writer corruption the lifecycle lock exists to prevent. Add RestartSessionLocked (TranscodeManager) and restartSessionLocked (node Server) that hold LockSessionLifecycle only across the cancel->respawn transition, re-check that the handle is still the live mapped session under the lock, and return ErrSessionSuperseded rather than re-spawning a stale handle. Route all five call sites through them. The lock is released before callers wait on segments so recovery latency is unchanged. Tests: gating (restart blocks until the lifecycle lock frees, then spawns), concurrent-restart serialization, and superseded re-check on both the manager (covers native + compat) and node lock owners. --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
9cae868a27 |
feat(downloads): offline sync for mobile — downloads v2 (#258)
* feat(downloads): offline sync for mobile (downloads v2) Replace internal/download with a unified internal/downloads package and add fully-offline download + watch-sync support for mobile clients, across five independently-shippable phases: - Phase 0: reshape the downloads table and the /downloads contract to be device- and format-aware; add GET /downloads/capability; extend DownloadConfig (default-off keys); update the web download hooks/components in lockstep. This is the one approved pre-lock exception to the additive-only /api/v1 rule (the web app is the only consumer and is updated together). - Phase 1: managed device-library entries (create/list/PATCH/delete/serve), keyed on the X-Silo-Device-Id header. - Phase 2: offline playback manifest plus artwork/subtitle proxy endpoints that strip every presigned URL (inline thumbhashes + authenticated proxies). - Phase 3: prepare-to-file (remux + transcode-to-single-file) as a durable, leased artifact queue with startup recovery, hosted on the task manager; playback.PrepareFile emits one +faststart MP4. Adds the admin transcode toggle and per-artifact LRU cleanup. - Phase 4: offline progress reconciliation -- a clamped event_at LWW key plus a server-assigned synced_seq cursor on watch_progress; an optional clamped updated_at on POST /sync/progress and an opaque ?since= cursor on GET /progress (additive; existing callers unaffected). Security & reliability invariants, each with an acceptance test: 1. Server-owned sync ordering: ?since= delta delivery is driven only by the server-assigned synced_seq; the client clock is bounded (event_at, clamped to now+skew) and used only for last-write-wins on the caller's own profile. 2. Full profile+device authorization on every managed endpoint, with a per-profile content/library access re-check before serving any bytes/assets. 3. Durable artifact recovery: a transactionally-claimed (FOR UPDATE SKIP LOCKED), lease-heartbeat, attempt-counted queue with a startup sweep, so no crash strands a download in preparing and concurrent workers never double-encode. Migrations are timestamped Goose files: reshape downloads (device/format); download_artifacts (durable queue); watch_progress event_at/synced_seq. DB-backed acceptance tests skip without SILO_TEST_DATABASE_URL and run in CI; the invariant-1 progress test also runs against the real SQLite backend locally. Client repos (silo-android, silo-apple) consume the reshaped /downloads/* contract and the updated_at/?since= progress fields and require coordinated follow-up. Implements the maintainer-approved v1 capability proposal for offline sync (downloads v2). AI-use disclosure: implemented by Claude (Claude Code) from the approved design doc under docs/superpowers/specs, with human review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(downloads): series & season downloads + client-pull monitoring Build season downloads and a "monitor a series" capability on top of the downloads v2 (offline sync for mobile) work. Season downloads: - POST /downloads accepts season_number (with series:true) to download one season. CreateSeries/CreateSeason share one body via a listEpisodes closure and register managed entries under a shared batch_id (original-only). Episode files are resolved in a single batched query. Series monitoring (auto-download), client-driven: - New device-scoped download_subscriptions table with a Sonarr-style mode (all | future | latest_season | specific_seasons), a client-enforced delete_watched flag, and a max_storage_bytes cap. The server never deletes on-device files; retention and the hard cap are the client's, the server only soft-gates registration. - The client calls POST /downloads/subscriptions/sync on open / background refresh; the server registers the in-scope, not-yet-downloaded episodes (idempotent via the managed-entry unique index) and the device pulls them on its own schedule. No background worker and no dependency on the notifications subsystem. latest_season follows new seasons (>= subscribe-time season); future excludes the back catalog via air date. - Subscription CRUD + sync are profile+device authorized (device id from the X-Silo-Device-Id header only) with a per-request content-access re-check. The capability endpoint advertises season_download / series_monitoring / monitoring_modes. Also lands the downloads-v2 work already present in the tree: durable artifact (remux/transcode) preparation and offline watch-progress reconciliation, plus the design-spec updates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WIP: epitaxy pre-switch from feat/downloads-v2-offline-sync * test(downloads): fix deterministic ID collision in reconcile test Artifact IDs are time-sortable, so two artifacts created in the same moment share their first 8 chars; combined with a captured timestamp the two preparing-download IDs collided on downloads_pkey. Use the full artifact ID, which is unique per row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): support sqlite userdb backend for managed downloads With the sqlite userdb backend, profiles live only in per-user SQLite stores and public.user_profiles stays empty, so user_devices' profile FK made every managed create/subscription/offline-sync request fail with an FK violation. Drop the FK (shared Postgres tables must not FK profile tables — same rule as notifications) and replace the lost cascade with an app-level purge on profile deletion, wired through ProfileHandler for both backends. DB-backed regression tests cover the no-Postgres-profile-row path and the purge cascade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): dispatch encode kick asynchronously triggerDrain invoked the kick inline, and the kick (taskmanager RunTask) executes the encode task on the caller's goroutine — so a POST /api/v1/downloads with a bitrate quality blocked the HTTP request on the entire queue drain, ffmpeg encodes included, delaying the 202 by minutes on an idle queue. Dispatch the kick on a goroutine; the task manager already serializes concurrent runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): enforce per-user quota on the encode pipeline Two gaps let a user bypass MaxConcurrentPerUser entirely for prepared downloads: artifact-backed rows are created in 'preparing' (never 'queued'/'downloading'), which CountActiveByUser didn't count, and createArtifactDownload enqueued the encode job before limiter.Check, so even a 429-rejected request left a job the worker would transcode. Count 'preparing' as active and check the limiter before Ensure; managed replacements stay quota-exempt since they don't add a row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): protect ephemeral artifact links from LRU eviction HasActiveLink only counted managed (device_id IS NOT NULL) rows, so under a byte budget Cleanup could delete an artifact still referenced by a ready-but-unfetched ephemeral web download — permanently 404ing a row the API kept listing as ready (the artifact row is gone, so recovery can't re-queue it). Any non-terminal link now protects the artifact; only artifacts whose links are all cancelled/failed/revoked are evictable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): batch manifests skip bad entries instead of failing whole batch One deleted or access-filtered episode made GET /downloads/batches/{id}/manifests 404 for the entire season, so a client could no longer fetch manifests for the still-valid entries. Report unbuildable entries in a skipped[] array (revoked | not_found | error) alongside the delivered manifests, mirroring the create path's skip idiom. Also cut the batch cost: the shared series detail is resolved once per batch instead of once per episode, and buildSubtitles reuses the already-loaded media file instead of re-querying it per manifest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(migrations): wrap DO block in StatementBegin/End markers Under NO TRANSACTION goose splits statements on semicolons, so the dollar-quoted DO block failed every fresh install with 'unterminated dollar-quoted string' (SQLSTATE 42601). Already-applied databases are unaffected. Same fix is being applied to main; identical content merges cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): allow season 0 (Specials) in season downloads season_number was a plain int dispatched with '> 0', so requesting the Specials season was indistinguishable from omitting the field and silently broadened to a full-series download. Dispatch on pointer presence, treat 0 as the Specials season, and reject negatives with 400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): capability quality_presets is never JSON null PresetsFor returned a nil slice when downloads are disabled or the user lacks the permission, and Capability's []string{} initialization was immediately overwritten by it — so GET /downloads/capability serialized "quality_presets": null where the contract documents an array. Normalize at the source so every caller inherits the guarantee. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): subscription sync correctness + batched registration Three subscription fixes: - A paused subscription no longer syncs: PATCHing scope (or pausing and changing scope in one request) registered episodes for a monitor the user had just stopped, inconsistently with SyncSubscriptions' guard. - SubModeFuture compares calendar days (UTC): air_date is date-only, so the strict instant comparison permanently excluded episodes airing the same day the user subscribed; episodes with no air date now fall back to their ingest time instead of never registering. - Registration is one batched fetch (GetManagedEntriesByKeys) plus one batched INSERT ... ON CONFLICT DO NOTHING RETURNING (CreateManagedEntriesBatch) instead of a SELECT+INSERT per episode — a 300-episode series cost ~600 sequential round trips per request and every no-op sync re-walked the full set. RETURNING yields exactly the new rows, so the sync response's 'registered' count now honestly reports 0 in the steady state instead of the full in-scope count on every app open. The now-unused InsertManagedEntryIfAbsent is removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(userstore): stamp triggers own the event_at LWW key MarkProgressBatch (jellycompat series mark-played) advanced updated_at but never event_at, and both stamp triggers only defaulted event_at when NULL — so a queued offline event with a client time between the row's old event_at and the mark could win SetProgressIfNewer and resurrect a stale resume position that then re-synced to every device. Make the triggers authoritative instead of adding a tenth hand-written SET clause: whenever an UPDATE changes updated_at without explicitly changing event_at, the trigger advances the LWW key; writes that do set event_at (offline sync's clamped client event time) keep their value. Postgres gets a CREATE OR REPLACE migration; SQLite gets a v12 userdb migration that drops and reinstalls the trigger bodies (CREATE TRIGGER IF NOT EXISTS never replaces). Conformance tests cover both batch paths, the preserved-client-time invariant, and the v11→v12 upgrade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): lifecycle hygiene — squash migrations, dead status, stale-row sweeps Migrations: fold the 20260621 corrective migration back into the base Downloads V2 migrations (its columns/constraints already exist there) and fix the reshape Down, which re-added the narrow status CHECK without collapsing managed-lifecycle rows first — rollback aborted on any DB with preparing/ready/revoked rows; validated against a live row. Branch databases that applied the corrective migration need its version row removed: DELETE FROM goose_db_version WHERE version_id = 20260621020459. Code: drop the dead 'registered' status (nothing ever wrote it; the lifecycle is preparing -> ready; 'revoked' stays reserved for the planned admin revoke flow) along with unused KindDirect and ErrInvalidFormat. Sweeps: Cleanup now runs an age-based hygiene pass independent of the byte budget — cold terminally-failed artifacts (with .part leftovers), orphaned ready artifacts no download row references, and ephemeral web rows older than their convenience-record lifetime (also unpinning their artifacts and bounding GET /downloads growth). The byte budget remains the disk quota per the limits & restrictions design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): sync API doc with v2 fixes; HEAD on file route; Android handoff Document the contract changes from the review fixes: batch-manifest skipped[] shape, honest subscription 'registered' semantics, season 0 = Specials, always-array quality_presets, bytes_sent actual behavior, ephemeral 7-day retention, header-pairing requirement, progress-delta deletion caveat, and the ready/failed push event schema (new §9.4). Add an Android client handoff section (§11) mirroring the Apple one, register HEAD on /downloads/{id}/file for download stacks that probe before ranged GETs, and add season_number to the web create-request type. Flag the /direct-download session-token-in-URL tradeoff; a short-lived download-scoped URL is a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: consolidate download/progress helpers, prune dead code, gate sweeps Behavior-preserving consolidation from the Downloads V2 review: - appendVideoFilterArgs: one home for the burn-in/hwaccel -vf selection, shared by the HLS builder and the single-file prepare builder (the drift pattern that already bit tone-mapping once). - userstore.ResolveProgressState: one home for the min-resume/watched threshold rule, replacing five identical copies across both store backends and the offline-sync ingest. - Download file selection ranks resolutions via access.CompareQuality (adds 4320p, agrees with playback) instead of a private switch. - writeSubtitle uses the shared subtitles.SubtitleContentType mapping. - config.DefaultTranscodeDir replaces three '/tmp/silo-transcode' literals. - Read-side quality/revision defaulting helpers removed: insertArgs plus the NOT NULL/CHECK schema already guarantee the invariant. - Dead code removed: Repository.ListByUser, SubscriptionRepository. ListActiveBySeries, and the stale auto-register-worker comments (the design is client-pull; no worker exists). - Redundant left-prefix indexes dropped from the base migrations (their unique indexes serve the same prefixes). - recover()'s disk-presence sweep and the stale-row hygiene sweep run on startup then hourly instead of every 30s tick (both are O(cache size)). - gofmt/prettier fixes for pre-existing drift in handlers/playback.go and pages/Profiles.tsx. Deferred (noted for follow-ups): quality-ladder preset table collides with the drafted download limits & restrictions design, which specifies its own ladder helper; Download-literal construction consolidation and the managed-identity value object remain open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): draft download limits & restrictions design Design input for the follow-up v1 capability proposal (quality ceiling, batch size cap, per-user quantity/bandwidth overrides). Committed with downloads v2 because the remediation work explicitly defers the quality ladder refactor and revocation wiring to this spec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(progress): reject malformed updated_at; clamp negative progress inputs Review findings on #258: - A malformed (non-RFC3339) updated_at in POST /sync/progress previously parsed to the zero time, which clampEventAt treated as "now" — letting a stale offline event win LWW as a fresh server-time write. The item is now rejected with a per-item error instead. - ResolveProgressState now clamps negative position/duration before classification so no backend can persist negative progress through UpdateProgress/SetProgress. - The online-write event_at invariant test is table-driven over both SetProgress and UpdateProgress, which share the same contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): close review gaps — permission gates, file-access recheck, artifact-true manifests Review findings on #258: - UpdateSubscription now applies the same feature/DownloadAllowed gate as CreateSubscription and SyncSubscriptions; a PATCH could previously re-activate or widen a monitor and register managed rows after an admin disabled downloads or revoked the user. - Serving download bytes (managed and ephemeral) and /direct-download now mirror playback's per-file authorization via catalog.FileAllowedByAccess: library scope and the profile's max playback quality are re-checked at serve time, with artifact-backed rows checked against the artifact's resolution (a 720p transcode of a 4K source stays servable under a 1080p ceiling). - Offline manifests for remux/transcode entries now describe the prepared artifact (container, codecs, resolution, single selected audio track) instead of the catalog source file the client never receives. - ArtifactRepository.Requeue reports ErrNotFound when the row was concurrently swept; ArtifactManager.Ensure recreates the job in that case instead of linking downloads to a dead artifact id. - "No downloadable episodes" is a sentinel (mapped to 404 no_downloadable_episodes) rather than a bare error that surfaced as 500. - Subscription season_numbers are bounds-checked (0–9999) before the int32 narrowing in the repo could silently wrap them. - HandlePatchDownload reuses requireManaged instead of hand-rolling the same managed-identity checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
QuickandClaude Opus 4.8
|
91b5105f3c |
docs(search): add hybrid semantic search hardening plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
87159b0a38 |
feat(collections): add profile-scoped display filters (#191)
* feat(collections): add profile-scoped display filters
* refactor(collections): dedup display-filter helpers per review
Address code-review feedback on the profile-scoped display filters
without changing behavior:
- Widen CompletedHistoryItemMap to accept ProgressCompletionStore and
drop the duplicate completedHistoryItemMapForProgress copy.
- Extract the duplicated MDBList candidate retry loop into a generic
collectionutil.FetchMDBListWithFallback helper, used by both the user
and library collection syncers, and cover it with unit tests.
- Reuse validateOptionalLibraryIDs in HandleUpdateCollection instead of
an inline positive-ID loop.
- Import the shared COLLECTION_{WATCH,MEDIA}_FILTER_OPTIONS in the
template config form rather than redefining them locally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(collections): sanitize query_definition library_ids fallback
readSourceConfigLibraryIDs validated source_config.library_ids (finite,
positive, truncated, deduplicated) but returned the query_definition
fallback raw, so legacy rows could surface zero/negative/duplicate IDs
that the backend now rejects on save. Extract a shared sanitizer and
apply it to both paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(docs): This makes the agents annoying to work with
* Improve playback session handling
* Support collection source order in catalog filters
* fix(collections): address display filter review feedback
* refactor(catalog): remove duplicate collection query params
* Hide episode media scope for collection overlays
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
QuickandGitHub
|
b3198276f7 |
[codex] fix(subtitles): stream live transcribe_translate cues (#177)
* fix(subtitles): stream live transcribe_translate cues * fix(subtitles): harden live AI transcription |
||
|
Quick
|
562ae635d7 | feat: improve audiobook groups and notification refresh | ||
|
|
e99079abf8 |
Server-side Kindle→EPUB conversion (mobi/azw/azw3) for in-app reading (#171)
* Kindle->EPUB conversion: design + proven wasm build pipeline
Server-side MOBI/AZW/AZW3 -> EPUB conversion so the Android in-app reader
can render Kindle-family ebooks. Conversion runs in-process via libmobi's
mobitool compiled to wasm32-wasi, executed by wazero (pure Go) -- no cgo,
no external binary, arch-independent, sandboxed untrusted input.
This commit lands the design + the validated build artifact (spike done):
- docs/.../2026-06-17-kindle-epub-conversion-design.md (Codex-reviewed;
9 review fixes folded in: failure contract, strong cache key + negative
cache, wazero command-module specifics, FS-sandbox tightening,
double-gated capability, serve headers, .wasm guardrails).
- tools/mobitool-wasm/{Dockerfile,README.md}: reproducible build of
mobitool.wasm (wasi-sdk 25, libmobi 9062742, zlib 1.3.1->wasm), with a
smoke-conversion gate. Build proven on native amd64.
- internal/ebookconvert/mobitool.wasm (+ .sha256): canonical artifact,
built on amd64. go:embed target for the converter package (next).
Spike proven on amd64: -e EPUB path works with --with-libxml2=no (internal
xmlwriter); converts MOBI6/KF8/HUFF-CDIC/unicode -> well-formed EPUB;
verified end-to-end under wazero (WASI preopen + argv + _start). Build
gotcha: link libmobi against real (wasm) zlib, not --with-zlib=no, to avoid
miniz duplicate-symbol clash with mobitool's zip miniz. DRM gotcha:
mobitool prints "Document is encrypted" to stdout but exits 0 -> detect via
stdout + output validation, not exit code.
Not yet implemented: internal/ebookconvert Go package (wazero harness +
cache + singleflight), read-handler wiring, admin flag, client capability.
v1-scope proposal required before PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ebookconvert: converter core + cache (Codex-reviewed)
internal/ebookconvert: in-process MOBI/AZW/AZW3 -> EPUB via the embedded
mobitool.wasm on wazero. Converter compiles the module once and instantiates
per conversion (isolated). Cache adds on-disk, singleflighted, size-bounded,
negative-cached conversion keyed by file identity + module fingerprint.
18 tests pass (DRM-free->valid EPUB, DRM->ErrDRMProtected + no output,
oversize/corrupt/missing/timeout/cancel/after-close, 6/8-way concurrent,
EPUB structural validation incl. stored-mimetype + container rootfile,
cache miss/hit/key-change/singleflight/eviction/negative-cache).
Codex review fixes folded in:
- timeout/cancel classified before generic nonzero exit (WithCloseOnContextDone
surfaces sys.ExitError special codes); no more bogus "exit <huge>".
- DRM detection scoped to known mobitool diagnostic LINES (Document is
encrypted / DRM key not found / Invalid DRM pid / DRM expired / DRM support
not included) -> no false-positive on book text; Print Replica -> clear fail.
- WithMemoryLimitPages cap; capped stdout/stderr writers; MaxOutputBytes.
- read-only fs.FS input mount + dedicated writable out dir; documented that
FS isolation ultimately relies on running as a non-root user (memory-safety
is the WASM boundary). validateEpub now requires STORED mimetype + verifies
the container.xml OPF rootfile exists. Atomic moveFile. Closed-guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ebookconvert: wire Kindle->EPUB into the read handler + capability endpoint
Server now transparently serves Kindle-family ebooks as EPUB when the admin
flag ebook.kindle_conversion_enabled is on and the WASM converter initialized.
- handlers.EbookConversion (converter + per-request flag predicate) on the read
handler; HandleReadFile -> h.serveEbook. Kindle + enabled -> cached EPUB with
X-Silo-Ebook-Conversion: converted, epub MIME, ETag = exact conversion cache
key, must-revalidate. Failure (DRM/corrupt/oversize/unservable) -> raw
original + X-Silo-Ebook-Conversion: failed + no-store, so the client opens
externally. Context cancel propagates (not a conversion verdict).
- GET /api/v1/ebooks/capability advertises {enabled, source_formats,
served_format, header contract}; enabled only when flag on AND converter
wired (double gate) so the Android client can decide whether to flip
mobi/azw/azw3 to in-app.
- router: buildEbookConversion compiles the module once at startup (feature off
if it fails), cache dir is a sibling of TranscodeDir, flag read per request.
Codex review fixes folded in: ETag derived from the exact SourceKey cache key
(id+size+mtime+oshash+module version), not a weaker hash; no-store on the raw
fallback; open/stat failure of a produced EPUB falls back to raw per the
contract instead of 500. 10 handler tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ebookconvert): harden conversion cache, HEAD path, and artifact verification
Addresses adversarial review + CodeRabbit findings on the Kindle->EPUB feature.
Correctness:
- Stop poisoning the negative cache on transient timeouts. Introduce
ErrConversionTimedOut (distinct, non-wrapping ErrConversionFailed); classify
the per-call timeout as transient and propagate a caller's cancel/deadline
verbatim instead of reclassifying it as a conversion failure. remember() now
only caches deterministic verdicts (DRM / failed), so a one-off timeout under
load no longer wedges a convertible book onto raw-fallback for 6h.
- Detach the singleflight conversion from any single caller's context (DoChan +
context.WithoutCancel), so one caller cancelling no longer aborts the shared
work for the others; the cache is still populated for the next reader.
- enforceBudget never evicts the entry it is about to return, and skips other
conversions' in-flight "converting-*" temp files.
- Cache hits refresh mtime so the mtime-ordered budget eviction is a real LRU,
not FIFO.
Read path:
- HEAD is now cache-only via Cache.Lookup: a hit serves real converted headers,
a negatively-cached source serves the failed contract, a miss advertises the
converted representation cheaply without triggering a (minute-long, ~1 GiB)
conversion. The GET still delivers the body + authoritative verdict.
- The admin flag is read through a short-TTL predicate so the read path and the
capability endpoint no longer hit the DB per request.
Artifact / build:
- Add an in-code provenance test (embedded mobitool.wasm matches its recorded
sha256) and a self-hosted CI job that runs the ebookconvert smoke conversions
+ provenance check, so the committed wasm can't silently rot.
- Pin + checksum-verify wasmtime in the build Dockerfile (drop curl|bash).
Docs: correct the design doc cache-key + setting-name descriptions, document the
HEAD/timeout/LRU semantics and resource limits, note DRM-marker brittleness, and
fix the README markdown table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci: remove ebookconvert workflow
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
|
||
|
|
e084cdd1d6 |
Add unified literary works for ebooks and audiobooks (#107)
* docs: add literary works design and plan * feat(literary): add work link schema * feat(literary): add work domain primitives * feat(literary): persist work links * feat(literary): score work matches * feat(catalog): include literary work summary on item detail * feat(literary): expose work detail API * feat(literary): assemble work detail * feat(literary): add admin work linking primitives * feat(catalog): group literary items by work * feat(literary): auto-link works during book scans * fix(literary): narrow work match candidates * fix(literary): address work merge blockers --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
13c5e0ba2f |
feat(catalog): deterministic cross-server content_id (#155)
* feat(catalog): deterministic cross-server content_id Replace per-server Sonyflake content_id with a structured natural key derived from provider IDs (movie:tmdb:…, series:tvdb:…, episode:…, local:… fallback), so two servers holding the same title share one anchor for artwork, watch history, progress, favorites and ratings. - internal/contentid: derivation core, SeriesIDFromContentID transform, frozen precedence, SchemeVersion=1, embedded-series-anchor invariant. - internal/metadata/service.go: deterministic id at every mint site. - internal/catalog/history_source.go: resolve show via string transform for anchored episode ids; skip the episodes_pkey probe. - migrations/sql/20260612130000: collision-safe value remap across the 65-column reference graph + COLLATE "C", FK/trigger handling, audit map, working down. Benchmarked against an exact-cardinality copy of cprod-postgres (1.93M episodes, 775k history rows): 2.57x faster history page, 1.7x throughput at 100 concurrent users, 2.7x cheaper per content_id probe. * feat(catalog): re-ID untagged items to deterministic content_id at first match Untagged libraries get a path-derived local: content_id at scan time and only learn their provider IDs later, when the match worker confirms a result. Previously that id was never folded back in, so untagged-then-matched items kept a per-server local: placeholder forever and never converged across servers (re-ID was deferred to a migration rerun). mergeAndPersist now promotes a local: skeleton to its deterministic provider-anchored id at the moment of first confirmed match, via a single new gate (canonicalizeLocalContentID): - target id already taken -> merge onto it (existing rebind machinery) - target id free -> rename in place The rename is a single SQL function (silo_rename_content_id); FK children follow via ON UPDATE CASCADE added to the content_id family, so a fresh skeleton moves a handful of rows rather than the full-table remap the bulk migration does. The guard is one IsLocal prefix check, so tagged content and all refreshes pay nothing, and the move is self-healing under retry. Verified: gofmt/vet/build clean; migrate-validate passes; migration applies on the real schema (up/down/up), FKs gain ON UPDATE CASCADE while keeping ON DELETE; functional test confirms series PK move + series_id cascade + provider-id sweep, and movie rename. Follow-ups (noted in docs): recomposeSeriesChildIDs for a series that accumulated episodes before matching; a lockstep test for the soft-ref list. * fix(catalog): harden content_id parsing and merge per review Address review feedback on the deterministic content_id work: - history_source.go: gate the anchored-episode display-id transform on the full five-part episode shape (split_part parts 2-5 non-empty), not just the 'episode:' prefix, so a malformed id can't transform to 'series:broken:' and vanish at the media_items join. Shared anchoredEpisodePredicate drives both the null-poisoned join key and the series-recovery expression. - contentid.go: unexport the provider-precedence slices so no package can mutate the frozen SchemeVersion ordering at runtime. - contentid.go: add parseAnchored to validate the exact per-kind arity and numeric season/episode suffixes; SeriesIDFromContentID and IsProviderAnchored now fail closed on truncated/malformed ids (e.g. "episode:tvdb:296762"). - canonicalize.go: distinguish catalog.ErrItemNotFound from transient lookup errors (a real error no longer masquerades as "target free"), and allow a matched local source to be consolidated onto the canonical row instead of orphaning a duplicate. * refactor(contentid): URL-safe "-" separator in content_id Use "-" instead of ":" to join content_id components (movie-tmdb-228064, episode-tvdb-296762-1-5, local-<hex>). "-" is an RFC 3986 unreserved character, so a content_id is URL-safe verbatim: encodeURIComponent is a no-op and the id is its own tidy path segment (/item/series-tvdb-296762) with no %3A escaping. The stored value equals the URL value, so there is no encode/decode boundary and an operator can grep the id straight out of a URL or log. Every component is [a-z0-9]+ (or "tt"+digits), so "-" is unambiguous. Pre-release format finalization: this branch is unmerged, so no deployed data carries ":" ids — the migration mints the "-" form fresh and no re-migration is needed. Still SchemeVersion 1. - contentid.go: single `sep` constant drives construction and parsing so the two can never drift; all constructors/parsers and doc examples updated. - history_source.go: split_part transform and the anchored-episode predicate use '-'; kept in lockstep with the package via a code comment. - 20260612130000_deterministic_content_id.sql: derivation and season/episode composition emit '-'; LIKE filters match 'series-%'. - docs/architecture/deterministic-content-id.md: format spec + rationale for the separator choice; this is the design doc the change is derived from. Client-side: the web frontend treats content_id as an opaque string (no splitting/regex), so no client changes are required; existing encodeURIComponent call sites simply stop emitting %3A. * docs(contentid): show why hash/bigint rejected in probe-cost table Add Cross-server deterministic / Zero-join show transform / Human-readable columns to the index-probe-cost comparison so the trade-off is legible at a glance: the 128-bit hash and bigint surrogate are faster but each give up a load-bearing property, and the structured key is the only all-checkmark row. * docs(contentid): order probe-cost table to end on the structured key * docs(contentid): label fenced blocks and drop stray EOF tags Per CodeRabbit review: add 'text' language to three fenced code blocks (MD040) and remove accidental </content></invoke> artifacts at EOF. * fix(catalog): remap array-valued content_id soft references in deterministic id migration The value-remap migration (20260612130000) enumerates the reference graph by FK plus a scalar name+type sweep (text/varchar/bpchar). That misses trending_discover_snapshots.content_ids: it is text[] (excluded by the type filter), named content_ids not content_id (excluded by the name list), and cannot carry an FK — so the bulk remap left those arrays holding stale Sonyflake ids that resolve to nothing until the snapshot regenerates. A counterexample to the migration's "self-protecting, cannot orphan" invariant. Remap the array element-wise in both directions (Up old->new, Down new->old), preserving order and leaving collision/unmatched elements untouched; a WHERE EXISTS guard skips empty/unaffected arrays so array_agg never collapses the NOT NULL column to NULL. Mirror the gap in silo_rename_content_id (20260614120000) with array_replace for the single-value runtime rename so the two stay in lockstep. Verified on PG18: mixed/collision/empty arrays remap correctly and round-trip clean; runtime array_replace preserves order. Surfaced reviewing #155. The jellycompat restart-decode regression and the atomicity-wording nit are posted as review comments, not addressed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(jellycompat): pack content_id into compat UUID reversibly so item ids survive restarts Addresses the restart-decode regression raised in review of #155. With content_id now a structured string instead of a numeric Sonyflake, EncodeStringID sent every item/season id down the one-way SHA1 path, making decode depend on an in-memory reverse map. That map is cold after a process restart (the codec is a process-lifetime singleton), so a client presenting a previously-issued item UUID — resume-from-home, deep link, detail page, image, userdata — got "unknown compat id" until the item was re-listed. Make the encoding reversible instead of stateful: - internal/contentid: add Pack/Unpack, a bit-packed, fixed-budget (<=15 byte) binary form of a structured or local content_id. digitCount preserves provider-id leading zeros (e.g. imdb tt0944947); structured forms are self-delimiting; the local form fills the budget exactly. Provider ids that overflow uint64 return ok=false. - Shrink ForLocal to a 112-bit (sha256(path)[:14]) hash so a local id packs losslessly into the 15-byte UUID payload. 112 bits is far beyond any single server's local-item count. No other code assumed the old width. - internal/jellycompat: EncodeStringID packs item/season content_ids into the UUID (byte 0 = kind, bytes 1..15 = packed, non-zero tag distinguishes it from the numeric encoding); DecodeStringID unpacks first and re-packs to confirm, so an opaque id whose bytes merely parse is rejected and falls through to the map. Numeric ids and arbitrary names (genres, studios) are unchanged. Net: item/season ids decode by pure computation — stable across restarts and across instances — with no lookup table. Only the rare unpackable content_id and non-content names still use the in-memory map. TDD: round-trip property tests in contentid (all kinds, leading zeros, reject cases) and a cross-instance decode test in jellycompat that fails on the old hash+map path. Full contentid + jellycompat suites green; production code golangci-clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(migrate): make the migration run timeout configurable (SILO_MIGRATE_TIMEOUT) The boot-path migration runner hardcoded a 5-minute context timeout. The deterministic-content-id value-remap (20260612130000) does a full-table COLLATE rewrite + 65-column remap that needs ~20 min on a real dataset (615k items / 2M episodes), so it was cancelled at 5 min. Worse, Postgres keeps the orphaned backend running (holding AccessExclusive locks) until it notices the dead client at a statement boundary, while the goose session advisory lock releases on disconnect — so each 5-min boot retry piled a new attempt behind the previous one's locks. The migration never applied; the server boot-looped. Make the timeout configurable via SILO_MIGRATE_TIMEOUT (a Go duration like "60m"); 0 or negative disables the deadline for a one-off heavy migration. Default stays 5m. All three entry points (migrate-status, --migrate-only, boot) honor it. Required for the deterministic-content-id migration to apply on any real-sized database, not just dev — the 5m cap made the PR undeployable at scale. Follow-up (not here): on cancellation the runner should actively terminate its backend so a future timeout cannot orphan a lock-holding statement. TDD: MigrationTimeout parsing (default/override/zero/invalid) + MigrationContext deadline behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(contentid): require exact length for local ids in Unpack Tighten the tagLocal branch of Unpack from `len(body) < localHashLen` to an equality check. The local form fills the compat-UUID payload exactly (no padding), so a body of any other length is non-canonical; matching it exactly keeps Unpack a strict fail-closed inverse of Pack for the fixed-length branch, which decodes client-supplied UUIDs. Not applied to the structured branch (a review suggestion proposed the same change there): structured ids are self-delimiting and the compat layer pads them with trailing zeros to fill the 15-byte UUID payload, so ignoring trailing bytes is intentional and documented. Rejecting them would make every structured id fail to decode — the jellycompat cross-instance test guards against that. Not a live bug today (the only caller passes u[1:] from a 16-byte UUID, so body is always exactly localHashLen, and idcodec re-packs to verify), but it is the correct contract and zero-risk. Adds a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
88aa769fe6 |
feat(collections): surface server collections on the user Collections tab (#156)
* feat(collections): surface server collections on the user Collections tab The user-facing Collections tab only showed personal collections, which are usually empty — leaving most users with a confusingly blank page. Server (admin-curated) collections were reachable only inside each individual library's tab. Add a new GET /collections/server endpoint that aggregates visible library collections across every accessible library (honoring access scope, capped per library with a total_count for a See all link), and restructure Collections.tsx into two titled sections: Your collections (personal) and Server collections (horizontal teaser rows per library, linking into each library's Collections tab). Extract the shared CollectionPosterCard so the per-library grid and the new rows share one implementation. * fix(collections): match server-collections loading skeleton to row layout The Server collections section renders as one horizontal teaser row per library, but the loading skeleton showed a poster grid — so data arriving visibly reflowed the page from a grid into rows. Mirror the final layout (section header + per-library rows of poster cards) in the skeleton, and drop the now-unused COLLECTION_POSTER_GRID_CLASSES import. Addresses CodeRabbit review comment on PR #156. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Align server collections with shared carousel behavior - Add opt-out edge padding to reusable media carousels - Render server collection rows with shared carousel controls and spacing --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
5afe56cfc0 |
feat(jellycompat): add runtime-managed Jellyfin Web compatibility (#77)
* feat(jellycompat): install web assets at runtime * fix(jellycompat): recover stale web operation locks * fix(jellycompat): harden web component management * feat(admin): refine compat settings and restart status * chore(dev): add hot-reload docker compose stack * fix(dev): include npm in hot-reload backend * feat(admin): refine Jellyfin compatibility settings * feat(settings): improve jellyfin proxy summary * feat(settings): improve jellyfin web controls * fix(settings): update jellyfin web removal status * fix(settings): enable jellyfin web after install * feat(jellycompat): auto-select web ui version * test(api): update rate limit handler setup * feat(jellycompat): refine web ui install onboarding * fix(jellycompat): address web ui install review issues * fix(onboarding): mirror jellyfin api runtime status * fix(admin): remove global restart banner * fix(settings): gate restart required tracking * fix(jellyfin): ignore live settings for restart status * fix(jellyfin): avoid restart for live compat settings * fix(subtitles): normalize AI language codes * fix(catalog): support partial title search tokens * feat(branding): add white-label customization * Add push relay engineering plan - Document relay API contracts, APNs/FCM behavior, auth, storage, and ops - Capture implementation plan, provider references, decisions, and README --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
6f9427d613 |
docs(process): v1 scope-lock process — proposal template, CODEOWNERS gate, agent instructions (#145)
Implements the process layer of the v1 feature-lock planner: capability proposals arrive uniform via issue form; the lock artifact (docs/architecture/v1-scope.md) is CODEOWNERS-gated; the shared CLAUDE.md/AGENTS.md guidelines gain the scope gate, additive-only API rules, and pre-push checklist for agent-driven contributions. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
df95e3cb95 |
feat(notifications): email notification channel
Adds email as a notification channel built on the shared SMTP core (mail.Sender). Email mode is a per-account preference (off, daily digest, or per-episode) stored in notification_email_prefs; delivery is an account-watermark sweep over notification_deliveries that dedupes cross-profile duplicates, advancing the watermark only after a successful send. Admin controls cover the channel kill switch, the per-episode allowance (off coerces those accounts to the digest), digest hour, and an external URL for deep links inside emails. Availability is advertised through /notifications/capability and the user settings page gains an Email section for opt-in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
a24279e3e3 |
docs(notifications): import notification design docs, add web push + v1.5 specs
Imports the notification system design folder (architecture overview, release-events/inbox foundation, APNs/FCM relay specs, outbound webhooks) and adds the Web Push spec (05, implemented in this branch), the shared outbound-email architecture note, and the v1.5 roadmap (06) covering the remaining work after APNs/FCM were deferred to v2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
adeda3c87c |
docs(notifications): add notification system design plans
Imported the Continuum-era notification specs (durable inbox + websocket foundation, APNs relay, FCM relay, outbound webhooks) and amended them for Silo. Amendments from the 2026-06-11 review: wire contracts normalized to Silo naming, back-catalog seeding and per-series burst suppression, durable dispatch outbox, cross-library episode dedupe, forward-sync wake API, ticket-based websocket handshake, relay threat-model additions (egress IP, keyed collapse IDs), and webhook 4xx/SSRF hardening. Includes a self-contained design-decisions.html visual overview. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
39ba284c9d |
feat(ai): shared AI core — metadata translation, Whisper ASR, per-profile language, on-view translation (#127)
* docs: design + plan for shared AI core, metadata translation, Whisper ASR Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai): shared LLM client, segment translator, and job runner packages internal/ai/llm: OpenAI-compatible chat client moved out of subtitles/ai, plus /v1/audio/transcriptions (verbose_json) for the ASR work; one shared retry/backoff loop for both. internal/ai/translate: the batched indexed-JSON translation protocol generalized to text segments. internal/ai/jobrunner: dispatch/heartbeat/reaper/cancel lifecycle extracted behind a minimal store interface, with a semaphore shareable across job services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(subtitles): consume shared AI core LLMTranslator becomes a thin cue<->segment adapter over aitranslate; the service delegates dispatch/heartbeat/reaper/cancel to jobrunner; the local OpenAI client is gone in favor of internal/ai/llm. Behavior (prompts, wire protocol, job rows, recovery semantics) is unchanged. NewService now takes the dispatch semaphore so all AI job services can share one bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(config): shared ai.* settings, metadata translation job table, localization provenance columns ai.* connection keys (chat + optional separate ASR endpoint) load with a fallback to the legacy subtitle_ai.* rows — those are never renamed in SQL because encrypted values are GCM-bound to their setting key. New toggles: subtitle_ai.transcribe_enabled, metadata_ai.enabled. Migration adds metadata_translation_jobs, per-field provenance (provider|ai|manual) on the localization tables, and media_folders.auto_translate_metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): localization field provenance with provider/ai/manual precedence Provider upserts keep manual values and never blank a field with an empty incoming value; new UpsertAITranslation/UpsertAIOverview methods write AI fields only over empty or ai-sourced values (force adds provider, never manual) — all enforced in single-statement SQL. Serving now merges only non-empty localized fields onto the base item, since localization rows are legitimately partial (AI rows carry no titles/artwork). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(metadata): AI translation service, refresh auto-fallback, and admin API internal/metadata/translation: job service over the shared AI core that expands an item to its season/episode overviews, skips already-localized fields (zero model calls on repeat runs), batches paragraphs through the generic translator, and persists per batch with provenance-aware upserts. MetadataService gains an AutoTranslator seam invoked after each refresh for libraries with auto_translate_metadata. Admin endpoints under the metadata curation guard: enqueue, list (poll), cancel; plus a status probe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(subtitles): Whisper ASR transcribe and transcribe_translate jobs New WhisperTranscriber: one ffmpeg pass extracts the audio track to 10-min 16kHz mono WAV chunks (temp dir cleaned on every exit path), each chunk goes to the OpenAI-compatible /v1/audio/transcriptions endpoint (verbose_json, per-request timeout sized to 3x chunk duration), segment timestamps are offset and built into wrapped cues. Chunks process playhead-first and stream live to the requesting session. The transcript is stored as an ordinary downloaded subtitle (provider 'transcribed'); transcribe_translate chains the existing translator and stores the translated track as the job result. Enqueue accepts an optional kind; status reports transcribe_enabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): AI services settings, metadata translate action, library auto-translate, generate-from-audio New AI Services admin page hosts the shared endpoint config (reads fall back to legacy subtitle_ai.* values, writes target ai.*) and the three feature toggles; the AI card moves out of Subtitles settings. The metadata editor gains a Translate-with-AI panel with job polling and force/re-translate. The library form gains the auto-translate toggle (threaded through the libraries API). The player translate modal gains a From-audio mode that lists audio tracks and submits transcribe / transcribe_translate jobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt import grouping in router and translation tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): per-profile metadata language and viewer-triggered description translation user_profiles.preferred_metadata_language threads through the access scope into catalog serving: presentation language now resolves explicit param -> profile preference -> library metadata language (native API and jellycompat). ItemDetail gains pending_translation_language when the viewer's language is missing a localized overview. New metadata_ai.on_view setting (off|button| auto) gates POST /items/{id}/translate-description: any profile with item access may request its language, with in-flight dedup and a 15-minute failure cooldown so page views never hammer a broken endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): on-view description translation with per-profile metadata language Profile playback settings gain a Metadata language picker (library default inherit). Detail pages: when the server reports pending_translation_language and metadata_ai.on_view is 'auto', the description translates on view with a pulse animation until the refetched detail comes back localized (45s timeout); in 'button' mode a small Translate chip triggers the same flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): expose metadata_ai.on_view in AI Services settings The on-view translation mode had no UI control, so it could only ever be 'off' — viewers got neither the auto translation nor the fallback button. Adds the off/button/auto selector to the Features card, and the config loader now warns and falls back to 'off' on a bad row instead of refusing to start. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): clear configuration hint when the transcription endpoint is chat-only A blank Transcription base URL falls back to the chat endpoint; chat-only gateways reject the multipart upload with an opaque 400 that reads like a pipeline bug. 400/404/405 transcription failures now carry a hint to set a Whisper-compatible endpoint in AI Services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): wrap ASR cue text by rune count, not bytes Arabic/Cyrillic/Greek text is 2+ bytes per character in UTF-8, so byte-based wrapping broke lines at roughly half the intended visual width. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(web): steer transcription base URL hint away from chat-only gateways Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): block chat-only gateways for transcription, add endpoint presets llm.IsChatOnlyGateway (OpenRouter et al — no timestamped transcription API) is enforced in three layers: the settings API rejects ai.asr_base_url values pointing at one, the router disables ASR with a warning when the blank-URL fallback would land on one, and llm.Transcribe refuses outright. The AI Services page gains one-click transcription presets (Groq turbo/accurate, OpenAI, self-hosted speaches) plus the mirrored client-side check, and the settings API now also validates metadata_ai.on_view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): tighten ASR subtitle sync Three systematic timing-error sources addressed: cue offsets now use the segment muxer's exact per-chunk start times (segment_list CSV) instead of assuming index*chunk_seconds; the audio stream's start delay relative to the container timeline (common in TS remuxes) is probed via ffprobe and added to every cue; and the chunk length is now operator-tunable via subtitle_ai.asr_chunk_seconds (60-600s, default 600) since shorter chunks bound Whisper's within-chunk timestamp drift. Playhead-first ordering now pivots on real chunk starts, and a beyond-end playhead starts at the final chunk instead of restarting from zero. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): tolerate base URLs that already include the /v1 segment Providers like DeepInfra expose their OpenAI-compatible API under a base that contains the version segment (api.deepinfra.com/v1/openai); always appending /v1/... mangled those. endpointURL now appends bare paths when the base already carries /v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): prefer self-hosted transcription in presets and hints Preset order becomes self-hosted (recommended) -> Groq turbo -> Groq large-v3 -> OpenAI, and the settings hint plus the job-error hint lead with the self-hosted option. The self-hosted preset now fills the turbo CT2 model to match the recommended speaches setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): request VAD and word timestamps for ASR cue accuracy Without vad_filter, faster-whisper servers report wall-to-wall segment times: cues linger on screen through silence (verified up to 91s) and paragraph-length segments become single 400+ char cues. Request vad_filter=true (skipped for hosted providers that reject non-OpenAI fields and run VAD server-side) plus timestamp_granularities word+segment, and rebuild cues from word timings: split at speech pauses, sentence ends, text capacity, and a 7s max duration; cap word-less segments instead of trusting their reported end; stretch sub-second cues to a readable minimum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
54b20f57d7 |
feat(audiobooks): player QoL — shortcuts, smart rewind, per-book speed, volume, chapter nav (#125)
Brings the web audiobook player to parity with best-in-class audiobook apps (Audiobookshelf, Audible, BookPlayer) on the quality-of-life axis, per docs/superpowers/plans/audiobook-player-qol.md: - Keyboard shortcuts active on every route while the player is mounted (space/K, arrow skips, volume, M, N/P chapters, shift+./, speed, E expand, Esc collapse), inert while a video session exists. - Configurable skip intervals with asymmetric defaults (back 10s, forward 30s) via a new player settings popover. - Speed control expanded to 0.5x-3x with a 0.05 stepper, presets, and per-book memory (LRU-capped, device-local). - Chapter prev/next transport buttons (prev restarts the chapter when more than 3s in, the universal player convention). - Volume/mute via the shared VolumeControl with a new theme-token "surface" tone; persistence shared with the video player. - Smart rewind: backs up 3-30s on resume scaled by pause length, plus a flat 10s on cold resume from the detail page; explicit seeks are never second-guessed. Toggleable in player settings. - Now Listening time label cycles total / remaining / remaining at current speed. Chapter flattening is deduplicated into lib/audiobooks/chapters.ts, the SkipIcon is shared between both player views, and new pure modules (smartRewind, chapter nav, prefs) come with unit tests. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick
|
ef78a1c5da | Merge branch 'main' of https://github.com/Silo-Server/silo-server | ||
|
|
8b70357703 |
feat(ebooks): first-class ebook libraries, scanner, and reader (#124)
* docs: define ebook architecture matching audiobooks * docs: plan ebook audiobook-parity implementation * feat: add ebook scanner parser foundation * fix: harden ebook scanner foundation * fix: handle ebook isbn labels * fix: guard ebook subtree scans * feat: scan ebook libraries in core * fix: preserve ebook scan people credits * fix: refresh ebook scan metadata safely * feat: persist ebook series membership * test: cover ebook series persistence decisions * fix: address ebook scanner PR review * docs: clarify ebook foundation PR scope * feat: add ebook metadata enricher * fix: harden ebook poster cache * feat: wire ebook metadata sync task * feat: expose ebook library metadata setup * feat: add ebook catalog scope support * feat: add ebook detail view * feat: label ebook file versions by format * feat: use file-size copy for downloads * feat: use file language in download dialog * test: cover ebook detail authors and downloads * fix: drop narrator credits from ebook scanner merges * fix: align ebook collection filters with book media * fix: drop asin provider ids from ebook enrichment * fix: force ebook people refresh for stale narrators * chore: omit ebook planning docs from branch * feat: add ebook detail related content * feat: add ebook reader file entrypoint * feat: render ebooks with foliate reader * feat: persist ebook reader progress * feat: add ebook reader controls * feat: extract ebook pdf metadata * feat: favor scanner isbn during ebook enrichment * feat: extract fbz ebook metadata * feat: count cbz ebook pages * feat: show ebook file page counts * feat: show ebook download summaries * feat: switch ebook reader files * feat: prefer epub for ebook read action * feat: surface ebook reader progress * feat: sync ebook reader progress cache * feat: hide ebook read action for unsupported files * feat: filter ebook reader file selector * fix: serve fbz ebook archives with reader mime type * fix: detect fbz ebooks from compound filename * fix: authorize fbz ebooks from compound filename * fix: scope ebook catalog facets * fix: reject narrator queries for ebooks * fix: build ebook recommendation text from authors * fix: include ebooks in embedding eligibility * fix: include ebooks in recommendation media mix * fix: include ebooks in recently added recommendations * feat: include ebook progress in recommendation signals * feat: include ebooks in continue watching sections * feat: include ebooks in catalog progress metrics * fix: read ebook isbn from epub metadata * fix: filter ebook asin provider aliases * fix: fall back from unsupported ebook reader files * fix: sort ebook catalogs by reader progress * fix: filter ebook catalogs by reader progress * fix: include ebooks in last watched catalog filters * feat: reflect ebook reader progress in item user state * feat: share ebook progress state across item surfaces * feat: report ebook scan progress * fix: include ebook activity in recommendations * fix: expose ebook reader progress on item detail * fix: support ebook subtree scans * fix: honor profile header for ebook item progress * fix: add ebook library default sections * fix: route ebook continue cards to reader * fix: hide watched toggle for ebooks * fix: route ebook watch tonight cards to reader * fix: route ebook hero actions to reader * fix: detect archive ebook reader formats by filename * feat: cache embedded ebook covers during scan * fix: encode ebook hero reader links * fix: persist non-epub ebook reader progress * fix: scope narrator catalog badges to audiobooks * fix: merge ebook reader progress during item repair * fix: label ebook progress filters as read * fix: show ebook related rails as book covers * fix: remove txt ebook reader support * fix: reject txt ebook reader files * fix: label ebook advanced filters as read * fix: label ebook personalized sorts as read * fix: remove plain text reader loader path * test: cover ebook unread catalog rules * fix: preserve ebook reader library context * fix: link ebook genres with library scope * fix: encode related rail item links * fix: encode catalog card item links * fix: encode hero and continue item links * fix: encode watch tonight item links * fix: encode recommendation and search item links * test: cover ebook scan format set * fix: label ebook search results clearly * fix: make global search prompt media neutral * fix: encode catalog read API ids * fix: encode item API ids * fix: include ebook reader vendor in docker build * fix: make ebook reader build clean * fix: clean ebook embedded descriptions * docs: plan ebook reader shell parity * feat: add ebook reader shell controls * fix: widen ebook scrolled reader flow * fix: remove scrolled reader content width cap * docs: plan ebook reader full parity * feat: persist ebook reader config * feat: add ebook annotations and bookmarks * feat: add ebook reader tools and aids * feat: add ebook advanced reader settings * fix: keep ebook reader panel in viewport * fix: use foliate sizing units for ebook scroll flow * fix: keep ebook settings controls readable * fix: simplify ebook reader settings controls * feat(ebooks): extract local covers during scan (#98) * feat(ebooks): extract local covers during scan * fix(ebooks): read nullable poster paths during cover scan * fix(catalog): coalesce nullable media artwork fields * fix(ebooks): group sibling formats by book identity * fix(ebooks): tolerate legacy ebook metadata encodings * fix(ebooks): decode PDF hex metadata strings * fix(ebooks): harden local cover extraction and format grouping Address review findings on the local cover scan: - Restrict generic sidecar covers (cover.jpg, folder.png, ...) to single-book directories, always accept images named after the book file, and apply exactly one cover per reconcile with sidecar taking precedence over the embedded cover. - Replace the read-then-write poster update with an atomic conditional UPDATE (ItemRepository.SetLocalPoster) so provider/admin artwork is never clobbered by concurrent writers, and refresh locally owned posters when the extracted cover bytes change (thumbhash compare). - Preserve UTF-8 PDF Info strings (including a UTF-8 BOM) instead of forcing everything through Windows-1252; the cp1252 fallback now only applies to non-UTF-8 bytes. - Select EPUB covers by manifest media-type with properties="cover-image" outranking the EPUB2 meta name="cover" id, so XHTML cover pages no longer shadow the real image. - Order CBZ pages naturally (2.jpg before 10.jpg, ch2/ before ch10/) when picking the cover page, via a single O(n) min-scan. - Bump the ebook content group key scheme to version 2 and reprocess rows written under older versions so pre-existing libraries gain sibling-format grouping instead of accumulating duplicates. - Group different formats only (a same-format sibling with colliding sparse metadata stays a separate item) and stop a joining sibling's embedded metadata from overwriting a provider-matched item. - Decode any IANA-labelled OPF/FB2 XML charset (windows-1251, koi8-r, shift_jis, ...) via x/net/html/charset, and wire the charset reader into FB2 parsing which previously had none. - Strip the full .fb2.zip double extension from filename-derived titles and group keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * feat(ebooks): add reader profiles and ruler (#99) * feat(ebooks): extract local covers during scan * fix(ebooks): read nullable poster paths during cover scan * fix(catalog): coalesce nullable media artwork fields * fix(ebooks): group sibling formats by book identity * fix(ebooks): tolerate legacy ebook metadata encodings * fix(ebooks): decode PDF hex metadata strings * feat(ebooks): add reader profiles and ruler * fix(ebooks): address reader ruler and profile review findings - skip renderer setStyles/render when computed styles and attributes are unchanged, so ruler position updates no longer re-style the book view - drag the ruler via a local draft that commits on release, with the surface rect cached at pointer-down - migrate font values persisted before the generic stacks (Inter, Georgia, Merriweather, legacy serif) so the font select never renders blank, with a Custom fallback option for unknown values - make the ruler band click-through and move dragging to a dedicated keyboard-accessible slider handle so links and text selection keep working under the band - share font stacks between options and profiles via READER_FONT_STACKS - surface the active reading profile, move presets to the top of the settings panel, and drop the redundant profile button aria-labels Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ebooks): resolve prefer-const lint error in readest document lib `pnpm run lint` failed on the branch because `direction` is never reassigned in getDirection; split the destructure so only the reassigned `writingMode` stays mutable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * Merge branch 'main' into work/ebooks-reader-base Brings the ebook integration branch up to date with main (audiobook library redesign, continue-watching rework and card affordances, quic-go bump, jellycompat fixes). Conflict resolutions favor main's generalized mechanisms and register ebooks with them: - media scope validation goes through IsValidMediaScope (now including "ebook" alongside main's "video" group scope), in Go and in the web filter/search types - continue-watching uses main's typed rails; reading-type sections pull resume points from ebook_reader_progress and the ebook library default section is wired to ContinueTypeConfig(ContinueTypeReading) - item_repo keeps main's derived select-list machinery (itemColumnExpr) and both poster accessors (GetPoster/SetLocalPoster for ebook covers, GetPosterPath for audiobook covers) - web cards/hero/watch-tonight adopt main's buildMediaPlayHref helpers, which now route ebooks to /reader/ebook and encode content ids; ebook affordances (BookOpen icon, Read verb, percent-read subtitle) carry over onto main's reworked components - LibraryForm ebook support ported into main's refactored useLibraryForm/libraryTypes modules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docker): copy foliate-js vendor into Dockerfile.dev frontend stage foliate-js is a file:vendor/foliate-js dependency, so pnpm install needs the vendor directory before the lockfile install layer. The production Dockerfile already copies it; the dev image was missed, breaking make dev-deploy with ENOENT on /app/web/vendor/foliate-js. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ebooks): render Continue Reading sections as upright poster cards All-ebook continue sections previously fell through to the horizontal 16:9 wide card; include ebooks in the poster-variant check so book covers render in their natural 2:3 framing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): stop related-rail highlight ring clipping on detail pages Move the current-item ring onto the cover artwork with a themed ring-offset color (matching the sidebar profile highlight) and give the scroll container top headroom so the ring is not cut off by overflow-x-auto. Applies to both ebook and audiobook detail rails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): harden ebook scanning against data loss and bad metadata - Reconcile missing ebook files like video/audio, with real per-root walk failure tracking (failed/unmounted roots are excluded from deletion), symlinked-root support via the shared logical walker, and the empty-root cleanup allowance before any destructive reconciliation. - Create ebook items as 'pending' so enrichment can promote them to 'matched' (backfill migration included), and protect matched items from re-scan clobbering: title/year skipped, people/series fill-empty only. - PDF metadata: scan head + tail windows (non-linearized PDFs keep the Info dict at the end), require proper key delimiters, head values win. - Cap plain .fb2 reads like .fbz entries; drop .md as an ebook format. - gofmt internal/scanner/audiobook.go (pre-existing drift). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ebooks): make enrichment failures non-terminal with dedicated backoff state - Provider errors now record a failure (capped retries) instead of stamping last_refreshed, which permanently excluded items after transient outages. - Unconfigured metadata chains and the scan-window membership race skip the item without stamping or burning a retry. - Failure tracking moves to a new ebook_enrichment_state table, decoupling it from media_items.refresh_failures (shared with metadata refresh debt). - Preserve non-author people credits when persisting enrichment results. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): gate ebook progress on hidden history and centralize threshold - Apply user_history_hidden_items gating (video semantics) to the ebook watched/in-progress filters, progress sort plan, and Continue Reading. - Continue Reading pages past dismissed items via the shared collector and dedupes items across pages (also fixes the video path's latent exposure). - Centralize the 0.9 finished threshold as models.EbookFinishedProgressThreshold with a single SQL-interpolated mirror in catalog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(recommendations): correct watcher counting and wire ebook taste signals - itemWatchersQuery dedupes to distinct (watcher, item) rows so one binge-watcher can no longer satisfy minWatchers; the eligibility floor now counts distinct accounts rather than profiles. - Hidden-history gating on GetEbookReaderProgressForUser (signal reader). - Ebook reading produces canonical implicit taste signals (weighted like the equivalent movie progress ratio); ebooks join taste-seed candidates. - Stale GetRecentlyAddedItems doc comment corrected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): harden ebook reader endpoints and serve a Content-Security-Policy - Serve a CSP on all SPA HTML responses: blob/srcdoc book iframes inherit it, so script-src 'self' 'wasm-unsafe-eval' blocks script execution from malicious book content (sandbox alone is defeated by the WebKit allow-scripts requirement). Threat model documented on the constant. - X-Content-Type-Options: nosniff on frontend, jellycompat, and ebook file responses; MIME resolution can no longer fall through to octet-stream for an admitted ebook file. - Annotation PATCH: presence-aware field semantics (absent keeps, present sets/clears), invariant re-validation on the merged row, and an atomic SELECT ... FOR UPDATE read-merge-write. - Request size caps (413) on progress/config/annotation writes; Content-Disposition via mime.FormatMediaType; hidden-history gating in the shared ebook progress lister; FK-cascade indexes for reader tables. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): native read-state endpoints for ebooks - POST/DELETE /watched/{id} accepts ebook content IDs: mark read upserts progress 1.0 preserving the reader's file/location (or picks the preferred reader file for never-opened books); mark unread mirrors video unwatch semantics and deletes the progress row. - /history/remove accepts ebooks: hides via user_history_hidden_items without touching the reading position (hidden != unread; next reading activity resurfaces the book, mirroring video re-watch). - Access-filter checks match the video branch; shared logic lives in ebook_read_state.go. Sort metrics/user-state thresholds use the shared constant; profile-header fallback deduplicated. Clients: response is {type: "ebook", affected_count: 1, played: bool}; the existing watched SSE event fires. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): harden the ebook reader UI - Open-flow race: cancellation checked after every await with full stale-run teardown (no wrong-file progress saves, no leaked views/blob URLs); book.destroy() on cleanup. - Progress: monotonic stale-response guard; visibilitychange flush uses the refresh-capable client, pagehide uses keepalive; per-book cross-format progress documented as deliberate. - Settings: side effects out of the setState updater; local edits no longer clobbered by late server config; pending saves flushed on unmount/pagehide. - TTS: generation token so Stop actually stops (Chromium/Firefox synthetic events); Media Session uninstalled on unmount. - External book links: http(s) only, opened with noopener,noreferrer. - apiBlob 512 MiB guard with a user-facing error; fraction bookmarks navigable; search-result key collisions fixed; dead e-ink code removed; getLibrarySortRelevanceScope deduplicated; md format dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): mark read/unread affordances for ebooks - Item detail gets a Mark Read/Unread button; card menus drop the ebook gate and share type-aware labels/toasts (also dedupes audiobook wording). - Watched-state invalidation includes the reader progress query key so the Continue button and percent refresh after toggling. - Continue Reading dismiss copy for ebooks; dismissal path now URL-encodes item IDs (ebook content IDs can contain reserved characters). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: record the PR #124 review and hardening pass Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0bd4f8cb3b |
fix(jellycompat): restore CanDownload with a real Download route for Infuse (#123)
dd81a7ef set CanDownload=false to stop Wholphin's screensaver from
404ing on the nonexistent /Items/{id}/Download route — but the flag is
load-bearing for Infuse, which refuses Direct Play (Static=true
streaming) of items it believes it cannot download. With omitempty the
field vanished from the JSON entirely and Infuse playback broke, while
PlaybackInfo-negotiating clients were unaffected.
Resolve the underlying inconsistency instead of trading one client for
the other: implement GET/HEAD /Items/{id}/Download serving the original
file (range support, Content-Disposition, optional mediaSourceId for
multi-version items) under stream-group auth, and restore
CanDownload=true now that the route exists. Fixes Infuse playback and
keeps Wholphin's download callers working.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
c5f21cb10d |
fix(jellycompat): parse repeated Fields query params (#110)
* fix(jellycompat): parse repeated Fields query params
parseItemsQuery read the Fields parameter via q.Get("Fields"), which
returns only the first value when a client sends Fields as repeated
query params (Fields=A&Fields=B&...) instead of comma-separated in a
single param (Fields=A,B,C).
The jellyfin-sdk-kotlin (used by Wholphin) sends repeated params. When
such a request listed a detail-only field like MediaSources after other
fields — e.g. the episode-playlist request
/Shows/{id}/Episodes?Fields=PrimaryImageAspectRatio&...&Fields=MediaSources&...
silo saw only the first value (PrimaryImageAspectRatio), so
needsDetailFields stayed false, the request took the list path, and the
response came back without MediaSources. Clients then could not start
playback of the returned episodes ("no media sources").
Join all repeated Fields values before splitting on commas so field
order and delimiter style no longer matter. Comma-separated single-param
clients (e.g. VidHub) are unaffected.
* fix(jellycompat): stop advertising CanDownload and stub ThemeSongs
Wholphin (jellyfin-sdk-kotlin) audit surfaced two reachable gaps:
- mapping.go set CanDownload=true on every playable item while no
/Items/{id}/Download route exists, sending clients that honor the flag
(e.g. Wholphin's screensaver/slideshow) into 404s. Advertise false until
a download route exists.
- GET /Items/{id}/ThemeSongs 404'd, so enabling theme songs in Wholphin
silently failed on every detail page. Stub it with an empty
ThemeMediaResult. This cannot reuse the generic item stub: the SDK
models OwnerId as non-nullable, so the response must include it even
when empty.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(architecture): add Wholphin endpoint coverage audit
Cross-references every Jellyfin endpoint the Wholphin client can call
against the routes jellycompat serves, with gating evidence for each
missing-but-unreachable endpoint and prioritized recommendations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Quick
|
9a240edb17 | Merge branch 'main' of https://github.com/Silo-Server/silo-server | ||
|
|
9e29e7b330 |
feat(security): encrypt server-owned credentials at rest (#45) (#95)
* feat(security): encrypt server-owned credentials at rest Introduce AES-256-GCM at-rest encryption (HKDF-derived from a required SECRET_KEY) for server-owned credentials, with row-bound AAD, a versioned enc:v1: envelope, and an idempotent startup backfill. - internal/secret: cipher + RowAAD/SettingsAAD + the startup backfill engine. - SECRET_KEY required at bootstrap; cipher threaded as an explicit dependency. - server_settings: EncryptedSettingsRepo decorator over the audited SensitiveSettingKeys (also drives admin redaction); the config watcher and watch-sync settings reads decrypt too. - Arr keys inline-encrypted; the ambiguous SecretResolver indirection removed from requests/autoscan. - Per-table columns encrypted: subtitles, watch-sync, webhook-sync (not webhook_secret), history-import, and the jellycompat session's bridged Silo access/refresh tokens. - Startup backfill (resolve-then-encrypt for arr refs) is best-effort and primary-node gated. Equality-looked-up secrets and plugin_runtime_configs.config_value are out of scope (need hashing / cross-repo design) — see docs/architecture/secret-encryption.md. Refs #45 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(compose): require SECRET_KEY in docker-compose The server now fatals without SECRET_KEY, so the integrated service (and the commented distributed proxy/transcode examples) pass it through with a fail-fast guard matching the existing MEDIA_ROOT pattern. Distributed worker nodes must use the SAME key as the primary to decrypt shared data. Generate with: openssl rand -base64 48. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): encrypt history import session credentials --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
QuickandClaude Opus 4.8
|
ed8f63c886 |
docs(plans): add credential encryption at-rest plan (#45)
Plan for issue #45: a reusable internal/secret AES-256-GCM package keyed by a required SECRET_KEY (HKDF), inline-encrypted api_key_ref columns that replace the SecretResolver indirection, an EncryptedSettings decorator for sensitive server_settings, and an idempotent startup backfill with resolve-then-encrypt for legacy arr references. Plan doc only; no code changes. Refs #45 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
eb6024573e |
feat(audiobooks): make audiobook libraries first-class catalog items (#73)
* docs(audiobooks): design spec for plugin absorption Plan to absorb silo-plugin-audiobooks into silo-server as a first-party feature. Audiobooks land in silo's existing SPA; ABS clients connect directly. Hard constraints: reuse existing tables (media_items, media_files, user_watch_progress, user_playback_sessions, people, item_people, library_collections); only two new tables (abs_sessions, podcast_feeds) and at most one column add (media_libraries.kind); silo's main :8080 listener handles ABS Socket.io natively. Out of scope: audiobook requests flow, smart collections, share links, external recommender, custom metadata providers, separate audiobook SPA. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 1 (discovery + schema) First of six sub-plans for the absorption. Six tasks: a discovery audit that resolves the spec's Risk questions, four idempotent SQL migrations (abs_sessions, podcast_feeds, media_libraries.kind, audiobooks.enabled feature flag), and an empty-but-compiling internal/audiobooks package scaffolded into cmd/silo. Lands as a strict no-op for users (feature flag defaults to false). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): discovery findings for absorption sub-plan 1 Locks schema/code decisions for migrations 139-142 and downstream sub-plans. Resolves open Risk questions from the absorption design spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 139 add abs_sessions table Parallel of jellycompat_sessions for Audiobookshelf-compatible clients. Lets ABS mobile/desktop apps maintain a device-bound session that silo's audiobooks/abs handlers will validate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): match codebase conventions in migration 139 Lowercases type keywords in the abs_sessions CREATE TABLE body to match neighboring migrations, fixes the client_version column alignment, and replaces the misleading "parallel to jellycompat_sessions" header comment with a more accurate description of the table's role. Cosmetic only — the running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 140 add podcast_feeds table Side table on media_items for RSS-subscribed podcasts. Holds feed URL, ETag/Last-Modified for conditional fetches, last-refresh timestamp, and the per-feed refresh interval consumed by the upcoming podcastfeed.Refresher scheduled task. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): uppercase PRIMARY KEY in migration 140 Aligns with the codebase convention (type keywords lowercase, constraint keywords uppercase) established in migration 139's post-style-fix form. Cosmetic only — running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(audiobooks): migration 141 no-op for media_folders.type Sub-plan 1 originally reserved migration 141 to add a 'kind' column to media_libraries discriminating audiobook/podcast libraries. Discovery audit (sub-plan 1 Task 1) found that the actual table is media_folders and it already has a type text NOT NULL column with no CHECK constraint or enum, so 'audiobooks' and 'podcasts' can be added as future values without DDL. Landing this migration as a documented no-op preserves the version numbering audit trail and pins the decision in git history. The matching down migration is also a no-op. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 142 add audiobooks.enabled flag Server-settings row that gates the absorbed audiobooks feature. Defaults to 'false' so sub-plan 1 lands as a strict no-op; subsequent sub-plans branch on this flag and operators flip it to 'true' at cutover. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scaffold internal/audiobooks package Empty-but-compiling Service that reads the audiobooks.enabled feature flag from server_settings. Wired into cmd/silo so the package is referenced from the binary; no routes mounted, no scheduled tasks registered, no DB writes. Subsequent sub-plans hang scanner branches, ABS handlers, Socket.io, podcast refresher, and SPA pages off this Service. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): cosmetic cleanups in scaffolded package Two pre-emptive cleanups flagged by code review before sub-plan 2 copies the patterns: 1. Sort the internal/audiobooks import after internal/adminjob in cmd/silo/main.go (alphabetical). 2. Drop the redundant "audiobooks: " prefix from the Enabled() error wrap; matches how every other top-level service package (watchstate, scanqueue, metadata, etc.) formats errors. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 2 (scanner) Second of six sub-plans. 10 tasks: PersonKind constants for Author and Narrator, audio-extension recognizer, library-type helpers, a walkLogicalTree refactor (movieLibrary bool -> typed walkMode), chapter extraction via ffprobe, single-file and multi-file audiobook parsers, scanner write path producing media_items.type='audiobook', and a filesystem podcast parser (RSS deferred to sub-plan 5). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add Author and Narrator PersonKind constants Discovery audit confirmed item_people.kind is unconstrained smallint with values 1-6 in use. Reserve 7 = Author, 8 = Narrator for audiobook people-links written by the upcoming scanner branches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add audio-extension recognizer for scanner Mirrors the existing videoExtensions/SupportsVideoFile pair. Used by upcoming audiobook and podcast scanner branches to filter directory walks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): library-type recognizers for scanner dispatch isAudiobookLibraryType and isPodcastLibraryType match singular and plural forms case-insensitively, mirroring isMovieLibraryType. Used by upcoming scanner walk branches (Task 4) that filter audio files into audiobook and podcast libraries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(scanner): replace movieLibrary bool with typed walkMode Lets walkLogicalTree dispatch on multiple library shapes (video, movie, audiobook, podcast) without proliferating boolean flags. Behavior for existing video and movie libraries is unchanged; audiobook and podcast modes will be consumed by the upcoming audiobook.go and podcast.go parsers in later tasks of this sub-plan. walkModeFor() derives the mode from a media_folders.type string; unknown types default to walkModeVideo to preserve prior behavior for any caller still passing a raw type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose ffprobe format tags on ProbeData The audiobook scanner needs format-level tags (title, artist, album, date) for media_items metadata; ffprobe already parses them in ffprobeFormat.Tags but ProbeData previously discarded them. Add FormatTags map[string]string to ProbeData, populate it in convertProbeData via a new normalizeFormatTags helper that lowercases keys and trims values. Adds a fixture audiobook .m4b with embedded chapters (Intro/Outro) and format tags, and a test that verifies ProbeFile() returns both correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): parser for single-file audiobook folders parseAudiobookFolder reads tags + chapters via the existing ProbeFile (now that Task 5 exposes FormatTags on ProbeData) and produces a parsedAudiobook struct. Title falls back from "title" tag to "album"; author from "artist" -> "album_artist" -> "composer"; series from "album" -> "series" -> "mvnm" (Movement Name, used by some MP4 tools). Year parsed from "date" or "year" tags, tolerating ISO dates and parenthesized forms. Single-file case only; multi-file folders (one audio file per chapter) return a placeholder error and arrive in Task 7. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): multi-file audiobook folder support Folders containing N audio files (one per chapter/part) get one parsedAudiobookFile per file; each file's chapter list is synthesized as a single chapter with title = filename stem. Title/author/series/ year come from the first file's tags. Also drops the duplicate pickFirstNonEmpty helper added in Task 6 in favor of the existing firstNonEmpty already in probe.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scanner write path produces audiobook media_items ScanAudiobookFolder walks an audiobooks-typed media folder and treats each immediate subdirectory as one audiobook. For each parsed audiobook it upserts: - one media_items row with type='audiobook' - one media_files row per audio file (with chapters JSONB) - author/narrator links in item_people (kind=7, kind=8) Adds itemRepo and personRepo to the Scanner struct, wired from fileRepo.Pool() in NewScanner — no constructor signature change needed. ScanFolder dispatches to this path when folder.Type='audiobooks', bypassing the per-file movie/TV pipeline because audiobooks are folder-scoped entities. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): filesystem podcast scanner ScanPodcastFolder walks a podcasts-typed media folder, treating each subdirectory as a podcast show and each audio file inside as an episode. Writes media_items.type='podcast' + episodes rows + media_files rows. RSS-subscribed feeds (podcast_feeds table) arrive in sub-plan 5; this task covers filesystem-only ingestion. ScanFolder dispatches to this path when folder.Type='podcasts'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 5 (podcasts) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose audiobooks/podcasts library types in admin UI Adds 'Audiobooks' and 'Podcasts' options to the library-type dropdown in the admin libraries page so operators can flag a folder as an audiobook or podcast library. Extends contentLevelsForType() so the admin UI's downstream filtering treats those types correctly (audiobook -> ['audiobook'], podcasts -> ['podcast', 'podcast_episode']). Backend scanner branches for these types were already wired in sub-plan 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 139_abs_sessions to 147 for origin/main merge origin/main adds 139_media_requests at the same number our local audiobook branch had used for abs_sessions. Renumber ours to 147 to free up 139 for the upstream migration. The schema_versions row is updated in lockstep on the running database so the migrator sees the abs_sessions migration as already applied at its new version. Migrations 140-146 (podcast feeds, media_folders kind noop, audiobook feature flag, abs playback sessions, podcast episode guid, audiobook series, audiobook title cleanup) stay where they are — they don't collide with anything on origin/main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 140_podcast_feeds to 157 for origin/main merge origin/main added 140_user_permissions at the same version this branch had used for podcast_feeds. Renumber ours to 157 (next free above the collections-unify migration at 156) so 140 is free for the upstream migration. schema_versions on the running database is updated in lockstep so the migrator sees podcast_feeds as already applied at its new version. Same pattern as |
||
|
QuickandGitHub
|
31b2716089 | feat(database): adopt goose migrations (#62) | ||
|
|
0163df3683 |
[codex] Add IntroDB marker integration and dialogue-aware Chromaprint refinement (#57)
* docs(markers): design + implementation plans for multi-source markers & TheIntroDB contribution * fix(markers): TheIntroDB read-path correctness (TVDB, real confidence, best candidate) Honor TVDB ids in /media lookups (previously dropped — anime/TheTVDB-first libraries got no markers), decode and use the real per-segment confidence and submission_count instead of a hardcoded 0.9, and pick the most-submitted / highest-confidence candidate when several are returned. Adds httptest coverage for the introdb client and provider. Phase 1 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): multi-source dispatch, per-provider config, per-segment provenance Add marker_provider_config (per-provider fetch enable/priority + contribute gates, contribution off by default) and a cached ProviderConfigStore. Add Registry.FetchMerged: query all fetch-enabled providers concurrently and keep the best candidate per segment (submission_count, then confidence, then fetch priority), stamping each winning marker with its provider/algorithm. Thread per-segment provenance through MarkerUpdatePayload and scanner.MarkerUpdate (additive SegmentProvenance overrides) so a merged result writes correct per-segment provider/confidence/algorithm; the legacy shared columns keep a summary. The lazy-playback path now uses FetchMerged. With only TheIntroDB enabled, behavior is unchanged. Phase 2 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): TheIntroDB submission client, contribution audit, service engine Add a markers.Submitter capability and implement it on the introdb provider (POST /v3/submit, GET /v3/user/stats; key required, usage-limit aware, applies the null start/end conventions). Add the marker_contributions audit table and a value-hash-keyed ContributionStore for idempotency. Add ContributionService: resolves enabled submitter providers, gates eligibility (never re-submit online-sourced markers; auto runs require contribute_auto_local + scanner-intro above the per-provider confidence threshold), checks idempotency, submits, and records. Wired in main.go; no trigger yet (admin API and task follow). Phase 3 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): admin marker editing, contribution, and provider config endpoints Add the RequireAdmin marker API: GET/PUT /admin/files/{id}/markers (read with provenance; manual upsert where a segment object sets and null clears), DELETE .../markers/{segment}, POST .../contribute and GET .../contributions, plus GET/PUT /admin/markers/providers[/{provider}] and a .../validate key-check returning user stats. Manual writes go through the priority-gated UpsertMarkers (source=manual) and notify live sessions; a new FileRepository.ClearMarkers nulls a segment's columns. Validation mirrors the contribution rules. Phase 4 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): daily auto-contribution task for local intro markers Add ContributeMarkersTask (daily 04:00, after local detection): when a provider has contribute_enabled + contribute_auto_local, page through episode files with a scanner intro marker at/above the provider's confidence threshold (new ContributionStore.CandidateLocalIntroFiles keyset query) and run them through ContributionService with Auto=true. No-op when no provider opts in; idempotent and resumable across runs. Phase 5 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(intromarkers): refine chromaprint starts with dialogue cues * feat(markers): finish marker management backend * feat(web): add marker editing UI * feat(markers): use plugin marker providers * fix(markers): address PR review feedback * feat(player): show marker labels on seek hover * fix(markers): type nullable marker mutation params * feat(markers): audit marker edits and add permission --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
177cfdc485 |
feat(autoscan): pluggable scan-source autoscan category (Sonarr/Radarr) (#44)
* docs: design spec for autoscan arr polling Periodic poller over autoscan-enabled Radarr/Sonarr instances (reusing request_integrations) that maps import paths to Silo media folders and enqueues targeted scans via the existing scantrigger + scanqueue. Lean single-service model: no cross-node fan-out guard or retry queue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan arr polling Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): settings and sources schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): core types Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): path rewrite helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): dedupe imported paths to parent folders Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): arr import-history client Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): settings + sources repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): redis scan-suppression seam Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): PollOnce poll cycle * feat(autoscan): poll task and wiring * feat(autoscan): admin API endpoints * feat(autoscan): admin API endpoints Adds ErrIntegrationNotFound sentinel (errors.Is) instead of string matching. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan types and hooks Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan admin tab * fix(autoscan): release suppression claim on enqueue failure; reconfigure trigger on interval change; skip source on key-resolution error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): update handler test for 3-arg NewAutoscanHandler * fix(autoscan): per-path suppression key, bounded poll window + overlap, boundary-safe rewrites, GREATEST cursor guard, async trigger, quiet unresolved-path skip, FK->404 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize Windows path separators, surface status errors, re-seed source editor on save Addresses minor code-review findings: Windows backslash paths now normalized before rewrite/dedupe; HandleStatus returns repository errors instead of 200; the per-source editor re-seeds from server data after a save. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan rewrite-sync from arr root folders Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan rewrite-sync Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): suffix-match rewrite suggester Add suggestRewrites / commonSuffixLen for Task 1 of the autoscan arr-polling feature. Pure function: matches arr root-folder paths to Silo media folder paths by longest common trailing segment count, adjusted for depth-delta so coincidental same-named segments at different structural levels don't inflate confidence. Categorises each arr root as Proposed, Ambiguous, Unmatched, or Covered by an existing PathRewrite rule. TDD: test file written first, verified failing, then implementation added. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): GetSource single-source lookup * feat(autoscan): arr root-folder client + Silo folder lister * feat(autoscan): Service.SuggestRewrites Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): rewrite-suggestions endpoint Add GET /autoscan/sources/{id}/rewrite-suggestions admin endpoint: extend the autoscanTriggerer interface with SuggestRewrites, wire SetRewriteResolvers in the router, and add handler + test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan rewrite-suggestions types and hook * feat(web): autoscan sync-rewrites preview * fix(autoscan): normalize covered-rule paths, dedup roots/folders, skip no-op suggestions Addresses final-review edge cases: coveredBy normalizes the existing rewrite's From (so a stored Windows/dup-slash rule still covers a root); duplicate arr roots and duplicate Silo folder paths are de-duplicated; an arr path that already equals its Silo path is not proposed as a no-op rewrite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): vitest 4 compatible fetch spy in recipes.test (unblocks build after vitest 4.1.0 bump) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): non-null suggestion slices + move Sync into rewrites card - suggestRewrites initializes Proposed/Unmatched/Ambiguous/Covered to empty slices so the JSON response is [] not null — fixes the 'Something went wrong' crash when every root is already covered (frontend mapped over null). - Move the sync button into the Path rewrites card beside 'Add rewrite' and rename it 'Sync rewrites'; guard the proposed map with ?? []. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): long root-folder timeout + sync spinner + collapse rewrites on load - Root-folder fetch for sync uses a 2-min timeout: Radarr/Sonarr compute unmappedFolders by scanning all roots, so a large library's /rootfolder takes 20-30s+ and tripped arrclient's 30s default (Sonarr 502'd at exactly 30s). - Spin the sync icon + show 'Syncing…' while the request is in flight. - Path rewrites card starts collapsed on page load. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): rescan on Sonarr/Radarr file renames History polling previously only tracked downloadFolderImported events. A rename in Sonarr/Radarr (episodeFileRenamed / movieFileRenamed) moves a file without an import event, leaving the library folder stale until the next full scan. Extend the history client to also surface renamed paths: both the new path and the old sourcePath, since a rename can move a file between folders and both parents may need rescanning. Delete events are still skipped — upgrade-deletes are covered by the paired import, and standalone deletes carry no file path in arr history. Renames the interface method ImportedPaths -> ChangedPaths to reflect the broader scope. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): synchronize trigger test with detached PollOnce goroutine HandleTrigger dispatches PollOnce on a detached goroutine and responds 202 immediately. The test read trig.called straight after the handler returned, racing the goroutine (usually 'PollOnce was not invoked') and reading the field without synchronization (a data race under -race). Signal completion through a channel the fake sends on when PollOnce runs; the test waits on it (bounded) before asserting. The channel send happens-before the receive, so the subsequent read of called is race-free. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan as a pluggable scan-source category Reframes autoscan from a Requests-coupled, arr-only feature into a standalone Autoscan category. Change-detection providers become out-of-process plugins via a new additive scan_source.v1 capability (client-pull, opaque marker); Sonarr/Radarr is the first provider. Host keeps a provider-agnostic resolve/suppress/enqueue engine; all arr-specific logic (and path rewrites) move into the plugin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for scan_source.v1 SDK capability First of the per-repo plans from the autoscan-plugin-architecture spec. Adds the additive scan_source.v1 capability to silo-plugin-sdk (proto + codegen + capability allowlist + runtime wiring), TDD per task, tagged as v0.5.0 so the host and arr-plugin plans can build against it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan host backend (part 1 of 2) Backend for the standalone Autoscan category: scan_source.v1 plugin plumbing (pluginhost client + plugins.Service resolver), generalized engine driven by a provider seam, autoscan_connections + autoscan_sources schema (decoupled from Requests), connection resolution (own or Requests-linked), admin API. Depends on silo-plugin-sdk v0.5.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plans for autoscan arr plugin + host UI arr plugin: new installable scan_source.v1 plugin (history imports+renames, rewrites, Silo-native paths), structured like silo-plugin-tmdb; ports the arr-specific logic from the closed PR #43. host UI (part 2 of 2): standalone Autoscan admin category (connections, sources, settings) extracted out of Requests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(autoscan): replace silo-plugin-sdk with local scan_source.v1 checkout Temporary dev replace so the host backend can build against the unreleased scan_source.v1 capability (silo-plugin-sdk PR #2). Finalize to v0.5.0 once the SDK is tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pluginhost): scan_source.v1 capability client wrapper Adds ScanSourceClient struct, the Client.ScanSource() accessor (mirrors ScheduledTask pattern), and a PollChanges method. Also introduces client_test.go with capability-gate tests for both scheduled_task.v1 and scan_source.v1 using a lazy gRPC ClientConn. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test+fix(pluginhost): cover capability-id gate, dedicated scan_source timeout Adds a "wrong id returns error" subtest to both capability-gate tests so the capability-ID component is exercised independently of the type. Introduces DefaultScanSourceTimeout (2m) for PollChanges, which polls an external arr API that can be slow, instead of the generic 10s control timeout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(plugins): expose scan_source.v1 client resolver Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(migrations): autoscan v2 schema (connections + sources, no requests FK) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): v2 types and repository Replace the request_integrations-coupled model with the decoupled v2 schema (autoscan_settings + autoscan_connections + autoscan_sources). Connection CRUD, source upsert/list/get, and AdvanceMarker/RecordError for opaque marker bookkeeping. ErrIntegrationNotFound becomes ErrNotFound. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): resolve connections (own credentials or Requests-linked) ConnectionResolver turns a stored Connection into concrete credentials, reading a soft-linked Requests integration's live base URL/key when RequestIntegrationID is set, then resolving the api-key ref to plaintext. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): scan-source provider seam over the plugin resolver ScanSourceProvider lets the engine poll changed paths without a live plugin; pluginProvider adapts plugins.Service.ScanSourceClient in production. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): generic engine drives sources via scan_source provider Rewrite PollOnce to iterate enabled sources, resolve each connection, poll the provider for changed paths, and run the salvaged resolve→suppress→enqueue loop (uniqueParentDirs, (folder,path) suppression key, RequestError quiet-skip, release-claims-on-enqueue-fail) verbatim. Store the opaque next marker via AdvanceMarker only after a successful enqueue; RecordError + keep marker on provider failure. Tests reworked onto a fakeProvider/fakeStore with an added opaque-marker-verbatim assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): drop conflicting connection CHECK, add connection resolver tests - migration 172: remove the autoscan_connections_source_present CHECK. It conflicted with request_integration_id ON DELETE SET NULL: deleting a Requests integration that a linked-only connection (base_url NULL) points at would null the FK and trip the CHECK, blocking the delete. The intended behavior is for the connection to survive as an orphaned 'needs attention' row. Creation-time validity is now enforced at the application layer. Verified on a throwaway DB: full chain applies and the delete-cascade leaves an orphaned (both-null) connection. - connection.go: TrimSpace the api key ref + resolved secret before the empty-string checks, matching requests.resolveAPIKey parity. - connection_test.go: fake-based tests for ConnectionResolver.Resolve (own creds, linked, linked-missing error, lookup error, trim/fallback). - repository.go: bound RecordError's stored last_error to 2048 chars. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): autoscan v2 admin endpoints Rewrite the autoscan admin HTTP handler against the v2 model: settings, connection CRUD, source update, manual trigger (detached PollOnce), and status. Connection/source responses omit api_key_ref and resolved keys (has_api_key flag only); unknown connection/source ids map to 404 via autoscan.ErrNotFound. Retire the host-side rewrite-suggestions endpoint (now lives in the arr plugin). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): wire v2 service, routes, retire rewrite-suggestions Export PollChangesClient/ScanSourceResolver from the autoscan provider so the api package can declare a structurally-conformant plugin adapter (Go has no return-type covariance, so the adapter must name the interface as its return type). Add api.BuildAutoscanService with the requests-integration lookup and plugin scan-source adapters, shared by the router (manual trigger) and the background poll task. Re-wire router routes to the v2 connections/sources/settings/trigger/status surface and drop the rewrite-suggestions route. Update cmd/silo to build the v2 poll task, seeding its interval from Settings.DefaultPollIntervalSeconds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): enforce connection requires own URL or a Requests link Migration 172 dropped the DB CHECK that required an autoscan connection to carry either its own base_url or a request_integration_id, delegating that invariant to the application layer — but the enforcement was never added, so HandleCreateConnection/HandleUpdateConnection accepted both-NULL orphans that ConnectionResolver.Resolve would hand a plugin as an empty base URL. Add a shared validateConnectionInput helper (whitespace-only request_integration_id counts as absent) and reject both-empty payloads with HTTP 400 on both the create and update paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): deliver resolved connection to plugin PollChanges now populates PollChangesRequest.Connection with the resolved {base_url, api_key} instead of dropping the conn param on the floor. Drops the stale doc comment claiming the connection was delivered out-of-band at upsert time -- that mechanism never existed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): auto-discover sources from installed scan_source plugins Auto-discovery seeds a disabled, connection-less source row per installed scan_source.v1 capability before an operator binds a connection, so connection_id is now nullable end to end: - migration 172: connection_id drops NOT NULL (still ON DELETE RESTRICT) - Source.ConnectionID becomes *string; repository scans/writes it as nullable and adds idempotent EnsureSource (INSERT ... ON CONFLICT DO NOTHING) - new ScanSourceLister seam + Service.DiscoverSources, called at the start of PollOnce (errors logged, non-fatal); production adapter enumerates ListEnabled -> ListCapabilities filtered to scan_source.v1 - PollOnce skips an enabled source with no connection bound, recording 'no connection bound' so the UI can surface it - HandleUpdateSource rejects enabling a source with no effective connection (400); source DTOs expose connection_id as nullable - BuildAutoscanService / NewService thread the installation store at both wiring sites (router + poll task) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): honor per-source poll interval PollOnce now skips an enabled source that ran too recently: the floor is source.PollIntervalSeconds when set, else settings.DefaultPollIntervalSeconds. The global poll task fires at the default cadence, so this makes the per-source interval a 'poll at most every N seconds' floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): reconcile spec + arr-plugin plan with credential-in-request + auto-discovery The credential-delivery mechanism changed during execution: the host now passes resolved {base_url, api_key} in PollChangesRequest.connection each poll (not plugin runtime config). Also records source auto-discovery, nullable connection_id, and the per-source interval floor decided at the final integration review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan v2 types and query hooks Replace v1 autoscan types and hooks with v2 DTOs matching the backend handler (autoscan.go): settings, connection (with has_api_key, no raw key), source (installation_id/capability_id/connection_id), status. Add connections CRUD hooks, useAutoscanStatus, update sources hook to v2 input shape. Retain deprecated shims for AutoscanPathRewrite, AutoscanRewriteSuggestions, and useAutoscanRewriteSuggestions so AdminRequests.tsx continues to compile until Task 6 removes that tab. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan connections panel (reuse or own) Card+Table listing connections with "Reused from Requests" / "Own" badges. Add/edit dialog with two modes: reuse a Sonarr/Radarr Requests integration or enter own name/URL/API-key credentials. Delete with alert-dialog confirm. Never renders key material — only has_api_key is sent by the backend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan sources panel Table of auto-discovered scan sources (one row per installed scan_source plugin capability). Operator can bind a connection via inline Select (auto-saved on change), set a per-source poll interval (saved on blur), and toggle enabled. Shows a "Needs connection" badge for unbound sources; attempting to enable without a connection lets the backend 400 surface via the existing toast in useUpdateAutoscanSource.onError. Status column shows last_run_at relative time or last_error with icon. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): standalone Autoscan admin page Tabs page (Sources | Connections | Settings) mirroring AdminRequests header/layout. Settings tab exposes global enable switch, default poll interval, and debounce — all auto-saved on blur or toggle. "Run now" button calls useTriggerAutoscan and toasts "Autoscan triggered" on 202. Route and sidebar nav are intentionally deferred to Task 5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): route and sidebar nav for Autoscan category Add /admin/autoscan route pointing to AdminAutoscan and a matching "Autoscan" item in the Content group of the admin sidebar (with RefreshCw icon), so the new standalone page is reachable from the nav. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(web): move Autoscan out of Requests into its own category Remove the Autoscan tab, AutoscanTab/AutoscanSourceEditor component definitions, and AutoscanSettingsFormState from AdminRequests.tsx. Delete the Task-1 compatibility stubs: AutoscanPathRewrite and AutoscanRewriteSuggestions types from api/types.ts, and the useAutoscanRewriteSuggestions no-op shim from useAutoscan.ts. The build confirms zero dangling references. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): allow unbinding a source connection (full-state source update) Change the source-update input struct's connection_id from string to *string so the UI can send null to unbind, a UUID to bind, or omit (null) to clear. Remove the fall-back-to-existing logic; the handler now sets the source's ConnectionID directly from the input. The enable-guard fires when the resulting connection is nil regardless of cause. Frontend sends the complete triple (connection_id, enabled, poll_interval_seconds) on every mutation site; selecting "— No connection —" sends null for a real unbind. Adds aria-label to connection Select and interval Input for accessibility. Backend tests cover bind, unbind, unbind while enabled → 400, and enable without connection → 400. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(migrations): backfill autoscan v1 settings+connections instead of dropping Migration 172 unconditionally DROPped the shipped v1 autoscan_settings/ autoscan_sources (migration 171), losing an upgraded operator's enable flag, poll cadence, debounce, and arr server list — autoscan came back OFF. Rewrite 172 up to be non-destructive of what can be carried: rename the v1 tables aside, create the v2 schema, backfill settings (poll minutes -> seconds) and seed a reusable LINKED connection per distinct v1 source integration, then drop the renamed v1 tables. v2 sources are keyed on a plugin (installation_id, capability_id) that did not exist in v1, so they are left to runtime discovery; path rewrites move to plugin config and are intentionally not carried. Verified against a throwaway DB: after 171 + v1 seed data, applying 172 yields enabled=true, default_poll_interval_seconds=300, debounce_seconds=30, and one autoscan_connections row linked to the v1 integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): preserve api key on metadata-only connection edit UpdateConnection unconditionally wrote api_key_ref = nullable(c.APIKeyRef), so a metadata-only edit (the UI omits the key when left blank — "leave blank to keep existing") NULLed the stored key and broke the next poll. Mirror requests' UpdateIntegration: api_key_ref = CASE WHEN $5 = '' THEN api_key_ref ELSE $5 END, passing the raw trimmed string so a blank incoming ref keeps the existing value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): skip orphaned sources + add source delete endpoint An enabled source whose scan_source plugin was uninstalled/disabled kept its autoscan_sources row, which errored every poll cycle, and there was no way to remove it. DiscoverSources now returns the set of currently-discovered (installation_id, capability_id) pairs; PollOnce skips any enabled source not in that set quietly (no RecordError), stopping the per-cycle error spam for orphans. A nil set (no lister / discovery failed) disables pruning so a transient discovery failure does not silence live sources. Adds DELETE /admin/autoscan/sources/{id} -> HandleDeleteSource -> repo.DeleteSource so an operator can clear orphans (unknown id -> 404). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reject reused connection when Requests integration is disabled RequestIntegrationLookup.Get returned a linked integration's base_url/api_key even when the integration was disabled or had a blank base_url (the v1 poll gate `WHERE ri.enabled = true` was dropped in v2). Now Get surfaces a disabled or unconfigured linked integration as an error, which the engine turns into a logged skip / RecordError instead of polling an unusable target. The gating is extracted into a pure checkRequestIntegrationUsable helper so it is unit-testable without a DB-backed repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reschedule poll task on settings change HandleUpdateSettings no longer rescheduled the poll task (the v1 triggerUpdater / UpdateTriggers wiring was dropped in v2), so a default_poll_interval_seconds change only applied after a restart. Re-add an optional triggerUpdater (taskmanager.UpdateTriggers) on AutoscanHandler, wired via SetTriggerUpdater from the router when a task manager is available. On a successful settings update the handler recomputes the interval trigger from default_poll_interval_seconds and calls UpdateTriggers("autoscan_poll", ...). The dependency is optional: a nil updater skips rescheduling so tests need no task manager, and a reschedule failure is non-fatal (the interval is persisted). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): disable enable toggle for unbound sources, add source delete + interval hint - Disable the Enable switch when a source has no effective bound connection (connection_id null and no pending edit selection), re-enabling once bound. - Add useDeleteAutoscanSource hook mirroring useDeleteAutoscanConnection pattern. - Add per-row delete button (Trash2 icon → AlertDialog confirm) to let operators remove orphaned/unwanted source rows. - Add interval floor helper text showing the global default poll interval so operators know values below it have no effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): consume source_paths from merged scan_source contract The merged plugin SDK renamed PollChangesResponse.changed_paths to source_paths and the plugin now returns RAW source-namespace paths. pluginProvider.PollChanges reads GetSourcePaths(); the host applies per-source path rewrites before resolving/enqueueing (separate commit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(migrations): add path_rewrites to autoscan_sources Add path_rewrites jsonb NOT NULL DEFAULT '[]' to the autoscan_sources CREATE in migration 172 (unreleased/branch-only, so amended in place). The host now owns per-source prefix rewrites. v1 path_rewrites cannot be backfilled (v2 sources key on a plugin installation/capability with no v1 mapping); documented that operators must re-enter rewrites post-upgrade. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-owned per-source path rewrites Rewrite ownership moved from the scan_source plugin to the host. The plugin returns raw source-namespace paths; the host now normalizes separators and applies the source's per-source prefix rewrites before dedupe/resolve/enqueue. - types: add PathRewrite{From,To} and Source.PathRewrites - rewrite: re-add applyRewrites/normalizeSeparators; apply the MOST-SPECIFIC (longest From) match, not first-match, so a broad rule can't shadow a nested one regardless of ordering - service.PollOnce: rewrite raw provider paths before resolveAndClaim - repository: marshal/unmarshal path_rewrites jsonb in UpsertSource and all source scans (EnsureSource discovery rows take the DB default []) - handlers: autoscanSourceInput/response + status DTO carry path_rewrites (full-state like connection_id); reject blank from/to with 400 - tests: rewrite unit tests, engine applies rewrites before enqueue, handler round-trips path_rewrites and 400s on a blank rewrite Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): discover installed scan_source plugins on sources-list view A scan_source plugin installed via the normal /admin/plugins flow must show up in the Autoscan component immediately, not only after a poll cycle (which runs only when autoscan is enabled). HandleListSources now runs discovery (seeding a disabled, connection-less source row per installed scan_source capability) before listing. Best-effort: discovery failure does not block listing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): per-source path rewrites editor + plugins-page install hint Add AutoscanPathRewrite type and path_rewrites fields to AutoscanSource/ AutoscanSourceInput. SourcesPanel gains an expandable rewrite editor per source row (from→to pairs, Add/Remove/Save) threaded into the full-state body so connection, interval, and rewrite changes always carry all fields. Adds a Plugins-page install hint in both the empty state and above the table for discoverability. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): host-owned path rewrites + install/discovery flow Reconcile the spec with the merged SDK decision (rewrites moved host-side; PollChangesResponse.source_paths carries raw provider paths). Document that scan-source plugins install via the normal /admin/plugins page and surface in Autoscan via discovery (run on poll cycles and on sources-list view). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: depend on merged silo-plugin-sdk via pseudo-version (drop local replace) PR #2 (scan_source.v1 + source_paths) is merged to silo-plugin-sdk main, so the host can resolve the canonical module at the merged commit (v0.4.1-0.20260603030807-807b07e785b2) instead of a local-path replace. The branch now builds off-machine (CI/Docker). Bump to a clean v0.5.0 once tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(migrations): single clean autoscan v2 migration (v1 never shipped) The v1 in-process autoscan (migration 171) was never released to origin/main, so no live system has v1 autoscan data to preserve. Collapse the v1-create + v2-rename/backfill/drop dance into one clean 171 that creates the v2 connections-based schema directly. Removes 172 entirely. The runner applies by version set-difference with no checksum validation, so the already-migrated test instance (171+172 recorded) skips both and is unaffected; fresh installs get the clean v2 schema in one step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): allow many sources per plugin + add-source enumeration Drop the one-source-per-(installation, capability) model. A single installed scan_source plugin capability can now back many sources, each bound to a different connection (e.g. one Sonarr plugin fronting four arr servers). - migration 171: remove the autoscan_sources UNIQUE(installation_id, capability_id) constraint; sources are operator-created, not auto-seeded. - repository: replace UpsertSource (relied on the unique conflict) with a plain CreateSource (fresh uuid) + a by-id UpdateSource; remove EnsureSource. - discovery: replace auto-seeding (DiscoverSources/RefreshDiscovered) with ListAvailableScanSources (the Add-source picker list, enriched with plugin id + display name) and an installedScanSources set used only for orphan-skip. - service: PollOnce stops seeding and instead fetches the installed-capability set for orphan detection; Store gains GetSource and drops EnsureSource. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): connection test endpoint (engine) Add Service.TestConnection / TestConnectionByID: resolve a connection (ad-hoc input or an existing stored connection) to concrete credentials and probe the arr GET /api/v3/system/status with a short timeout. A reachable/authorized target yields OK=true plus the reported version; an unreachable / 401 / non-200 target yields OK=false with a human-readable error (the probe failure is part of the result payload, never an error from the method itself). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-side rewrite suggester + admin API for new endpoints Port the path-rewrite suggester back host-side (it had moved into the plugin): suggestRewrites suffix-matches arr root folders against Silo media folders to propose path rewrites, reporting proposed / unmatched / ambiguous / covered. Service.SuggestRewrites resolves the source's bound connection, lists arr roots (GET /api/v3/rootfolder) and Silo folder paths, and runs the matcher; a source with no bound connection returns ErrNoConnection (400). Admin API (all admin-gated): - POST /admin/autoscan/sources create a source - GET /admin/autoscan/scan-source-plugins Add-source picker list - POST /admin/autoscan/connections/test probe a connection - GET /admin/autoscan/sources/{id}/rewrite-suggestions sync rewrites HandleListSources no longer auto-seeds; create validates the capability is currently installed and that enabling requires a connection. Wiring threads the arr root-folder/status client and the catalog folder lister through BuildAutoscanService; the lister now surfaces plugin id + display name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan hooks + types for sources, connection test, rewrites Add types and React Query hooks backing the autoscan admin UI batch: - AutoscanAvailableSource / useAvailableScanSources (scan-source plugins) - AutoscanSourceCreateInput / useCreateAutoscanSource (POST sources) - AutoscanConnectionTestResult / useTestAutoscanConnection (advisory test) - AutoscanRewriteSuggestions / useAutoscanRewriteSuggestions (on-demand) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): add-source dialog + sync-from-arr rewrites in SourcesPanel Add a "+ Add source" header action opening a dialog that creates a scan source from any installed scan-source plugin bound to an arr connection, so operators can add one source per connection (e.g. four arr instances). Empty state links to /admin/plugins when no plugins are installed. Add a "Sync from arr" button to each source's rewrite editor that fetches root-folder rewrite suggestions and renders a preview: checkbox-selectable Proposed rewrites plus collapsed Unmatched / Ambiguous / Already-mapped sections. "Apply selected" merges the checked rewrites (dedupe by `from`) and persists via the normal full-state source PUT. Sync is disabled until the source has a bound connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): test-connection button in autoscan ConnectionsPanel dialog Add an advisory "Test connection" button to the add/edit connection dialog. It probes the current dialog input — connection_id when editing, request_integration_id in reuse mode, or base_url/api_key_ref for own credentials — and renders the result inline: green "Connected (vX.Y)" on success, red error on failure. Never blocks save; stale results clear when credential fields change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan page polish + global enable toggle in header Surface a global Autoscan enable toggle and an enabled/disabled status badge next to the page title, alongside the existing "Run now" header action so primary controls are reachable without opening a tab. Remove the now-redundant enable switch from the Settings tab (it points at the header toggle instead). Tighten header layout for wrap on narrow widths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): right-align autoscan enable toggle + Run now in the page header Drop the redundant nested justify-between wrapper so the header actions sit directly under .page-header (space-between + bottom-align), matching the /admin/libraries header layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): hold poll marker when paths return but none resolve A freshly-enabled source whose path_rewrites aren't configured yet returns provider paths that resolve to zero library folders. PollOnce previously advanced the marker unconditionally on any successful poll, permanently skipping those imports. Now the marker advances only when there is nothing to do (zero paths) or at least one path resolved+enqueued; when paths come back but none resolve, the marker is held and an explaining error recorded so the operator can fix the rewrites and a later poll re-reads the same window. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): don't prune sources of disabled-but-installed plugins PluginScanSourceLister used the installation store's ListEnabled, so a temporarily-disabled plugin dropped out of the discovered set and PollOnce treated its sources as orphaned, skipping them with no last_error (silent vanish). Switch to List so only a fully-uninstalled plugin counts as orphaned; a disabled-but-installed plugin's sources are still attempted and surface a visible RecordError when the client fails to load. The Add-source picker shares the same all-installed set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): treat empty request_integration_id as no link ConnectionResolver.Resolve gated the linked-integration path on a non-nil RequestIntegrationID pointer, so a pointer-to-empty-string (from a both-NULL orphan or a stripped link) called requests.Get(""). Guard on a non-empty trimmed value so it falls back to the connection's own fields instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): align startup poll interval with reschedule computation Startup seeded the poll task by integer-dividing default_poll_interval_seconds by 60 (minutes), while HandleUpdateSettings reschedules with seconds*1000 ms; the two diverged for sub-minute and non-60-multiple intervals. NewAutoscanPollTask now takes the interval in milliseconds and main.go seeds it as seconds*1000, matching the reschedule path so both agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize stored rewrite From at poll time applyRewrites matched the stored From after only TrimSpace/TrimSuffix, while suggest.go coveredBy normalizes via normalizePath (backslash->slash, collapse '//'). A Windows-style or dup-slash stored rewrite was thus reported 'covered' at suggest time yet never matched at poll time. applyRewrites now normalizes From through normalizePath so poll-time and suggest-time agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): don't corrupt source poll interval on enable/connection change Add a `parseInterval` helper that maps empty input to null (use global default), valid positive integers to the integer, and any other mid-edit-invalid value to the source's currently-persisted `poll_interval_seconds` — so toggling the enable switch or changing the connection cannot silently overwrite the interval with 0 or NaN. Wire the helper through `fullBody()` (the single source of truth for PUT payloads) and remove the two inline duplications in `handleConnectionChange` and `handleRewriteSave` that both previously used raw `Number()`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): make the source connection optional (provider-agnostic) A host connection is the credential/endpoint for server-based providers (Sonarr/Radarr); other scan_source providers (e.g. a CephFS/filesystem watcher that reads ceph.dir.r* xattrs) need none. PollOnce now polls connection-less sources, passing an empty ResolvedConnection the plugin may ignore; a plugin that requires credentials surfaces the error at poll time. Drops the enable-requires-connection 400s. Provider-specific config lives in the plugin's own global_config_schema, not a host connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): provider-agnostic autoscan copy + optional source connection Replace arr-hardcoded framing in AdminAutoscan, SourcesPanel, and ConnectionsPanel with neutral scan-source language. Remove the connection-required gate on the source enable toggle so connectionless providers (e.g. filesystem watchers) can be enabled; soften the badge from "Needs connection" to "No connection". Sync-from-server button remains gated on a bound connection (it needs a server to query). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: repo-relative paths in autoscan plans Replace local absolute filesystem paths (/opt/silo, sibling checkouts, /tmp/go/bin) in docs/superpowers/plans with repository-relative wording per CLAUDE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): rune-safe last_error truncation Truncate RecordError messages on a UTF-8 rune boundary so a byte-bounded cut can't split a multi-byte rune and store invalid UTF-8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): advance marker when resolved-but-suppressed (not unresolved) resolveAndClaim now reports resolvedAny (whether any path mapped to a Silo library folder, independent of suppression). PollOnce gates the "none matched a Silo library folder" hold+RecordError on !resolvedAny instead of len(targets)==0, so a poll whose paths resolved but were all debounced/suppressed advances the marker instead of being treated as a misconfiguration. Adds a regression test for the suppressed case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): normalize request_integration_id Trim whitespace and collapse empty-after-trim request_integration_id to nil on connection create and update, so a pointer-to-"" or " " is never persisted as a bogus Requests link. Also corrects a stale migration-172 comment to 171 (the collapsed migration number). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(autoscan): provider-agnostic poll-task copy Rename the poll task to "Autoscan poll" with a provider-agnostic description and progress message; drop Sonarr/Radarr/arr wording. Key() (autoscan_poll) is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): fix typo in connectionless-source test name Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add scan source management * chore(deps): bump silo-plugin-sdk for structured scan source changes Pins silo-plugin-sdk to 0d78651, which adds source_config on PollChangesRequest plus the structured changes / ScanSourceChangeScope fields on PollChangesResponse that internal/autoscan/provider.go already consumes. Without this the branch fails to compile against the prior pin (807b07e). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): label scan sources by connection name in admin UI arr-plugin sources fan out one-per-connection under a single generic "arr" capability, so every row in the Sources and Activity panels rendered an identical "arr (plugin #N)" label. Lead with the bound connection name (Radarr/Sonarr/...) instead, demoting capability + plugin to a subtitle. Sources without a connection (e.g. cephfs) keep the capability fallback. Activity threads a source_id -> connection name lookup (built from the existing sources + connections queries) through the scan/poll tables the same way librariesByID is threaded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): spec for generic + operator-editable source labels Design for a shared label-resolution helper (operator label -> connection name -> manifest display_name -> capability_id) consumed by the Sources and Activity panels, plus an operator-editable per-source label backed by a new autoscan_sources.label column. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): implementation plan for source labels Task-by-task TDD plan: migration 174 (label column), Go domain/repo/handler wiring with server-side normalization, shared frontend label helper, and Sources/Activity panel integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): migration for source label column * feat(autoscan): source label domain field + normalizer * feat(autoscan): persist source label in repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): accept, normalize, and return source label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add label to source API types * feat(autoscan): shared source-label resolution helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): polish source-label helper per review Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): label sources via shared helper + operator label input Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): clarify source label naming per review * feat(autoscan): resolve activity source labels via shared helper Replace the sourceNames Map plumbing in ActivityPanel with SourceLabelLookups and delegate both name functions to resolveEventSourceName from @/lib/autoscanLabels, enabling the full label chain (operator label → connection name → manifest display_name → capability_id) for all Scan History and Poll log rows. * fix(autoscan): carry label on status source + guard poll label Final-review follow-ups: add the label field to the autoscanStatusSource response (and AutoscanStatusSource type) so the status view matches the source response per spec, and give pollSourceName a non-empty fallback for symmetry with scanSourceName. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): resolve source aria-labels through the label chain Replace the legacy capability-only sourceLabel() helper with resolveSourceName() (operator label -> connection -> display_name -> capability). Row controls now announce the row's resolvedLabel (reflecting in-progress edits) and the delete dialog announces the resolved name, so screen readers hear "4K Movies" instead of "arr (plugin #4)". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): paginate queue + history with a shared table pager Replace the card/table hybrid and 200-row "Load more" cap on the autoscan Activity panel with proper tables and real pagination. Backend: add offset + total-count to the scans/events list endpoints so history pages through the full set instead of a capped window. Extract shared event/scan WHERE-clause builders so list and count filter identically, and add CountEvents / CountAutoscanScans. Frontend: add a reusable TablePagination component (rows-per-page, "showing X-Y of Z", numbered window with ellipses, responsive) and reuse it for the server-paginated history (scans + polls) and the client-paginated live queue. Unify all three tables behind one DataTable shell so they read as one family. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
ea3b5a2e29 |
feat(requests): multi-instance Sonarr/Radarr routing with HD/4K defaults and anime overrides (#39)
* docs: design spec for multi-instance Sonarr/Radarr request routing Seerr-style multi-instance arr management inside Silo's request system: many instances per kind, HD/4K default routing, entitlement-driven dual-quality fan-out, per-instance anime overrides (keyword 210024), and a one-to-many media_request_targets model. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for multi-instance arr request routing Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): migration for multi-instance arr routing Adds migration 169 to convert request_integrations from a one-row-per-kind table keyed on `kind` to a multi-instance table keyed on `id`, with HD/4K defaults, anime overrides, and a new one-to-many media_request_targets table for per-quality fulfillment tracking. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(requests): instance, target, and dual-quality types Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(requests): id-based integration CRUD Replace upsert-by-kind (UpsertIntegration/UpsertIntegrations) with GetIntegration, CreateIntegration, UpdateIntegration, DeleteIntegration, and ClearDefault. Rewrites scanIntegration and integrationColumns to cover all new multi-instance columns (id, name, is_4k, is_default, is_default_4k, anime_* fields). Updates the Store interface accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(requests): target persistence and aggregate status * feat(tmdb): expose keyword ids and original language on detail * feat(requests): Seerr-exact anime detection (keyword 210024) * feat(requests): quality/anime routing engine Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(requests): force_dual_quality setting Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(requests): multi-target fulfillment, reconcile, retry, and instance CRUD Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): request integration CRUD endpoints, targets in responses, entitlement wiring Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): multi-instance request integration types and CRUD hooks Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): multi-instance arr manager, dual-quality toggle, per-target queue Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): UX review fixes for arr manager (delete confirm, switch hints, test feedback, dirty + target status) * fix(requests): address code-review findings (test-connection by id, HD-only default ceiling, retryable partial failure, idempotent submit, transactional defaults, presence/target reconcile, auto-approve gate) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review (anime override fallback, non-null slices, save gate, a11y, DeleteTarget not-found) - routing: anime fields only override standard root/profile/tags when set, so enabling anime with blank fields reuses standard values instead of clearing them into an invalid submission - api: normalize nil Tags/AnimeTags to [] so they serialize as arrays not null - web: require an API key before saving a NEW instance; add aria-expanded/ aria-controls to the anime-overrides disclosure toggle - repo: DeleteTarget returns ErrNotFound when no row was deleted Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(requests): address PR review findings --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
QuickandClaude Opus 4.8
|
f113b32de0 |
docs(calendar): add presets design spec and implementation plan
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |