1d4f5dbce2e928ceebe071e5c8a60c2cb350c15c
743
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Quick104andClaude Fable 5
|
1d4f5dbce2 |
fix(diagnostics): bind empty playback_session_ids and log ingest failures
pgx binds a nil Go slice as SQL NULL, which bypasses the column's '{}'
default and violates its NOT NULL constraint, so reports without
playback session ids failed to insert. Bind an empty slice instead.
The ingest path also swallowed the underlying insert and bundle
validation errors, logging only a generic rejection reason; both sites
now log the real error so failures are diagnosable from server logs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh
|
||
|
Quick104andClaude Fable 5
|
21ee543012 |
fix(s3client): upload unsized streams via multipart manager for R2 compatibility
PutObjectStream omitted Content-Length because client-reported sizes are untrusted, but Cloudflare R2 rejects unsized PutObject bodies with 411 MissingContentLength. Route streaming uploads through the SDK's multipart manager, which buffers fixed-size parts (8 MiB, sequential) and sends each with a known length, preserving bounded memory for untrusted stream sizes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh |
||
|
Quick104andClaude Fable 5
|
c5a5b02221 |
feat(web): admin client-diagnostics viewer
List/detail/download/delete for uploaded diagnostic reports at /admin/diagnostics, beside the Logs viewer: server-filter parity with the admin API, cursor pagination, manifest + device summary detail sheet, playback-session links into the filtered logs view, presign/stream download handling, and feature-state banner. Extracts a shared authenticated apiResponse() helper used by api() and apiDownload(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XppCCycoaskCsW7ja1fZct |
||
|
Quick104andClaude Fable 5
|
4fa84a661a |
feat(diagnostics): client diagnostics server foundation
Implements slice 1 of docs/design/2026-07-19-client-diagnostics.md: the versioned contract (schemas, fixtures, Go validator), storage-validated diagnostics.uploads_enabled gate, account-scoped status endpoint, hardened streaming multipart ingest with quota reservation and a receiving/ready/ failed report state machine, S3 streaming puts, acting-admin report API (list/detail/download/delete with audit events), and the retention + orphan-reconciliation cleanup task. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XppCCycoaskCsW7ja1fZct |
||
|
Quick104andClaude Fable 5
|
7369d0afd8 |
docs(design): client diagnostics spec (crash reports + debug log upload)
Cross-repo spec and rollout plan for opt-in client crash reporting and debug-log upload to the user's own Silo server: silo-server ingest, storage, admin API, and retention; silo-android and silo-apple capture, consent, and upload. Disabled by default on both server and client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XppCCycoaskCsW7ja1fZct |
||
|
|
8044eb84dd |
feat(activity): refine play-method tags and add a Jellyfin-client pill (#387)
* feat(activity): refine play-method tags and add a Jellyfin-client pill
Two related tagging improvements to the admin activity views, squashed:
Split audio transcodes into their own tag. The Play Method summary and
Server Activity popover bucketed every session by its raw play_method,
lumping real video transcodes together with video-copy HLS repackages and
having no separate tag for audio-only transcodes. Classify each session by
the per-stream decisions the backend already reports:
- video re-encoded -> "transcode" (yellow)
- only audio re-encoded -> "audio" (red)
- streams only repackaged -> "remux" (blue, incl. video-copy HLS)
- nothing touched -> "direct" (green)
ordered direct -> remux -> transcode -> audio across the distribution bar,
legend, method filter/sort, the per-row badge, and the Server Activity
stream counts.
Add a Jellyfin-client "JF" pill. Sessions from a Jellyfin-ecosystem client
(Jellyfin Web, Findroid, Swiftfin, Infuse, etc.) get a purple "JF" pill
next to the play-method tag. Detection is UI-only: isJellyfinSession()
positively matches client_name (set from the Jellyfin MediaBrowser auth
header) and then the raw user agent against the known Jellyfin client
tokens, mirroring the server's client-labeling list. The pill is orthogonal
to the method classification — a session can be both "transcode" and JF.
Pure UI/presentation change; no backend behavior changes.
AI-use disclosure: implemented with AI assistance (Claude Code).
* fix(web): cache-control on SPA shell so deploys bust stale UI
The frontend handler served index.html with no cache directives, leaving
freshness to browser/CDN heuristics. A stale index.html at a CDN edge kept
serving old content-hashed bundles, so a client-side hard refresh couldn't
recover — one browser would show the new UI while another showed the old.
Apply the standard SPA cache policy:
- index.html (and SPA-route fallbacks): no-cache + a truncated-SHA-256
ETag, so the shell is cached but revalidated on every load and answers
an unchanged request with a cheap 304.
- /assets/* (Vite content-hashed bundles): public, max-age=31536000,
immutable — cached indefinitely; a new build changes the filename hash,
which busts them automatically.
- other stable-named bundled files (sw.js, icons, fonts): no-cache, so a
changed service worker or icon can't stay stuck in a cache.
Caching is preserved (no no-store anywhere); only the tiny HTML shell is
revalidated, which is what busts a stale UI on deploy.
* fix(activity): compute the method bucket server-side and unify every session surface
Review follow-ups for the play-method tags (PR #387):
- The server now emits effective_play_method (additive field) from the same
per-stream decisions that drive the badges, so all consumers — web, realtime
popover, and the Android/Apple admin views later — agree on the bucket
instead of each client re-reducing raw play_method. Rows with an unknown
play_method (stale rows from older nodes) stay unbucketed rather than being
misreported as audio transcodes off the bare transcode_audio flag; the web
fallback classifier mirrors that and reports "unknown".
- Jellyfin-ecosystem detection moved server-side as is_jellyfin_client, owned
next to the client-labeling rules so the two lists cannot drift; the web
token list is gone. Adds kodi/mpv/delfin/finamp, which reach Silo only
through the Jellyfin compat surface.
- The dashboard stream cards, stats session table, and household streams panel
now use the same classification as the activity page and popover — they
previously showed contradictory tags for the same live session.
- One shared method->label/color table in adminActivityPresentation.ts
replaces the four independent copies (METHOD_META + three switches); the
method column sort now uses the shared cost-order comparator instead of
alphabetical; dead "copy"/"hls" order entries removed and the reachable
"unknown" bucket is styled.
* fix(server): make SPA revalidation RFC-compliant and stop rebuilding the shell per request
Review follow-ups for the SPA cache policy (PR #387):
- Stable-URL bundled files (sw.js, icons, vendor bundles) now carry a content
ETag. The embedded FS has no modtimes, so http.FileServer emits no validator
of its own — no-cache alone forced a full re-download of multi-megabyte
vendor trees on every use because there was nothing to revalidate against.
- Shell and favicon conditional requests go through http.ServeContent, which
implements RFC 9110 If-None-Match semantics (weak comparison, ETag lists).
The previous exact string compare never matched once a fronting proxy
compressed the response and weakened the ETag to W/"...", silently killing
the 304 path in the most common deployment topology.
- The rendered shell (index read + branding render + SHA-256) is cached per
branding snapshot via the new Snapshot.RenderKey instead of being rebuilt on
every request — the 304 revalidation that no-cache makes the common case now
costs two header writes. The misnamed weakContentETag (it emits a strong
validator) is renamed contentETag.
* fix(activity): show the JF pill on every session surface, not just the mobile row
Review comments on PR #387: the JF pill only rendered inside Admin
Activity's sm:hidden mobile row, so the desktop table — and the other
session surfaces that now share the method classification — never
identified Jellyfin-compat sessions.
Extract the pill into a shared JellyfinSessionPill component (renders
nothing for native sessions) and drop it into the Admin Activity desktop
client line, the dashboard stream cards, the household streams panel,
and the stats active-session table.
* fix(playback): sync real encode decisions and client identity for compat transcodes
Review comments on PR #387:
- Jellyfin HLS sessions that copy video and re-encode only audio synced as
full video transcodes: ensureUpstreamPlayback resets transcodeAudio for the
transcode transport method, and the TargetCodecVideo "copy" decision lived
only in TranscodeOpts. A new SessionManager.SetTranscodeStreamDetails
mirrors the actual decisions onto the upstream session when the transcode
starts (local and remote-node paths, via an optional interface so test
fakes are unaffected), so these sessions now bucket as "audio"/"remux".
- Transcode recipe cards now record TranscodeAudio derived from the opts
(only an explicit "copy" leaves audio untouched — empty runs ffmpeg's aac
default), so a session rebuilt after a restart keeps the same bucket.
- Recipe cards carry client name/version/user-agent, and reconstruction
restores them, so the admin client label and the JF pill survive server
restarts; the compat fallback card populates them from the live
MediaBrowser request. Deliberately not projected into stream-token claims,
where a user agent would bloat every stream URL.
* feat(api): capability endpoint for the live-session activity fields
Review comment on PR #387: effective_play_method and is_jellyfin_client are
omitempty, so an independently deployed client cannot distinguish an older
server from a supported one reporting an unknown method or a non-Jellyfin
session. GET /admin/sessions/capabilities advertises both fields plus the
closed bucket vocabulary, following the additive capability-endpoint rule
(same pattern as /collections/capabilities).
* fix(playback): treat empty target audio codec as an AAC re-encode in live state
ffmpeg defaults an empty target audio codec to AAC (appendAudioArgs), and the
new recipe logic already records that as an audio transcode — but the live
native path computed transcodeAudio=false for an empty codec, so the running
stream reported remux until a restart flipped it to audio. Extract the
predicate into playback.TranscodesAudio, share it across the live path, the
recipe card, and the compat mirror, and make appendAudioArgs case-insensitive
so the ffmpeg switch agrees with the predicate for any spelling.
Part of #387 review follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(jellycompat): re-sync sessions after recording compat encode decisions
ensureUpstreamPlayback flushes the session (compat_start) before
ensureTranscodeSession / startRemoteTranscode record the actual codec
decisions, and that later mutation triggered no sync — so the admin view
showed a video-copy stream as a full video transcode until the periodic
reconciler ran. Trigger syncSessionsNow after the details are recorded
successfully; the helper is shared, so both the local and remote-node
paths are covered.
Part of #387 review follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
b96e359b4e |
feat(catalog): typo-tolerant Postgres search fallback (did-you-mean) (#386)
* feat(catalog): typo-tolerant search for the Postgres (non-Meilisearch) path
## What this does (plain language)
When someone searches the library and misspells a title — "intersteller",
"godfathr", "jurasic" — the Postgres-backed search used to return nothing,
because it only did exact full-text matching. This adds a "did you mean"
fallback: when the normal search finds little or nothing, we run a second,
typo-tolerant lookup and surface the closest titles.
This only affects deployments that search via Postgres (the fallback path).
Meilisearch already does its own typo tolerance and is left untouched.
## Why not just make the main query fuzzy
The obvious approach — OR a trigram similarity match into the main search — is a
performance trap. The trigram operator is "lossy", so Postgres re-checks every
near-miss candidate by rebuilding three title search-vectors per row. On a real
library that turned routine searches into multi-second queries.
Measured on a 175k-title dev database:
- exact full-text only: ~60 ms
- fuzzy OR'd into the main query: ~217 ms (and far worse on prod-sized data)
## How it works
The fuzzy arm is a completely separate query (buildFuzzySearchSQL). It matches
only on the trigram-indexed title_normalized column and ranks only by
similarity() on that same column — it never touches the title search-vectors, so
it pays no per-row rebuild. It runs only when the exact search is "sparse" (fewer
than 5 hits) and the query is long enough for the trigram index to help (>= 4
characters), so the common case stays on the fast exact path. It is wired into
SearchPage (not just the thin Search wrapper) so the catalog search provider
benefits too.
## Measured on the live 183k-title catalog (read-only EXPLAIN ANALYZE)
- exact query for a typo: ~0.8 ms (0 hits -> triggers the fallback)
- fuzzy fallback query: ~5-27 ms, always via the trigram index, with no
search-vector rebuild
- "intersteller" -> Interstellar (similarity 0.63)
- "godfathr" -> GodFather (0.58), The Godfather
- "breakin" (134 exact hits) -> fuzzy correctly does NOT fire
Shared scope predicates (type / library / access / manga-exclusion) are extracted
into appendSearchScopeFilters so the exact and fuzzy queries filter identically.
Adapted from the earlier feat/search-fuzzy-fallback prototype onto main's current
SearchPage / includeTotal architecture.
* refactor(catalog): correct fuzzy-search pagination and parse the query once
Follow-up to the fuzzy fallback, from an adversarial code review. Two things: a
pagination correctness fix and a small performance/readability cleanup. Both were
validated against the live 183k-title catalog.
## The pagination bug (plain language)
Fuzzy results are shown after the exact results, as one combined list. The first
version stitched that list together with page-offset math, and got the math wrong
past the first page:
- the reported result count grew as you paged (page 1 said "31 results",
page 2 said "33");
- titles shown on page 1 could reappear on page 2;
- paging far past the end still ran the (pointless) fuzzy query every time;
- a tiny page size (e.g. an autocomplete asking for 3) could hide the fuzzy
results behind a page the client was told did not exist.
## The fix
Because the fuzzy fallback only runs when exact results are sparse (< 5) and the
fuzzy part is capped at 50, the whole combined list is tiny. So instead of
fragile per-page offset math, we now fetch that small combined list once and take
the requested slice in memory. Every page is then correct by construction: stable
total, no repeats, no wasted work past the end.
Before -> after, typo search "intersteller" (21 results, page size 5):
- total reported on page 2: 31 then 33 (drifting) -> 21 (stable)
- repeated titles across pages: yes -> none
- request past the end (offset 500): 2-3 DB queries -> 0 extra queries
- autocomplete (page size 1): fuzzy hidden -> paginates correctly
Cursor-style callers (that don't ask for a total) can't locate the boundary
between the two blocks on a later page, so they now get the fuzzy results as a
single terminal first page — no misleading "more results" flag.
## The cleanup
The raw query string was being parsed three times per search (once for the
eligibility check, once in each SQL builder). It is now parsed once in SearchPage
and passed down; the shared search-text derivation is extracted so the two
builders can't disagree; and the normalized form the eligibility gate needs is
precomputed at parse time. ("Performance first", per the repo guidelines.)
Also considered and rejected: excluding exact hits from the fuzzy query with a
NOT(full-text) clause instead of by id. It reintroduced the search-vector rebuild
the whole design avoids — measured ~51 ms vs ~20 ms on the worst case — so
id-based exclusion stayed.
Known limitation: the fuzzy path re-reads the small exact block in a second
query, so a title written in the sub-millisecond gap between the two reads could
be missed until the next search. Harmless and inherent to a multi-query design.
* fix(catalog): close fuzzy-search library-scope leak and restore small-limit cursor recall
Addresses two findings from the PR #386 review bots.
## Library-scope leak (Codex P1)
The search scope helper shared by the FTS query and the trigram fuzzy fallback
filtered libraries with `JOIN media_item_libraries mil` +
`NOT (mil.media_folder_id = ANY($disabled))`. An item linked to BOTH a disabled
and a non-disabled library fans out to two joined rows; the non-disabled row
satisfies the deny check, GROUP BY collapses the item back, and it surfaces in
search results despite the disabled library. Because the new fuzzy fallback
reuses this helper, typo searches could leak disabled-library items too.
appendSearchScopeFilters now delegates to the leak-safe
appendLibraryAccessConditions (access_filter.go), which emits item-scoped
EXISTS/NOT EXISTS subqueries — the same form GetByIDs/EnsureAccessible already
use — and needs no membership JOIN. The disabled-only path keeps its
argument-free positive-membership EXISTS so orphan items don't slip through a
vacuous NOT EXISTS. The scored CTEs keep GROUP BY (now required only for the
MAX() ranking aggregates). New regression test pins the EXISTS/NOT EXISTS shape
and the absence of a JOIN for both the FTS and fuzzy builders.
## Small-limit cursor recall (Codex P2)
In cursor mode (include_total=false) the FTS probe fetched only limit+1 rows.
For a tiny caller limit (e.g. an autocomplete asking for 2) with a few incidental
exact hits, that made ftsHasMore true, so the block never looked "sparse" and the
typo fallback never fired — and subsequent offsets are barred from triggering it,
so the fuzzy results were unreachable entirely.
SearchPage now floors the cursor-mode probe at fuzzyFallbackThreshold rows, and
execSearchBlock returns the pre-trim row count so sparsity is judged as
`fetched < threshold` independent of the caller's page size. The returned page is
still trimmed to limit with correct hasMore. Exact mode is unchanged (it judges
sparsity by the page-independent window count).
* fix(catalog): harden fuzzy-search fallback per adversarial review
Addresses the confirmed findings from a deep review of the fuzzy-search
fallback:
- Cursor mode now enters the fallback only when the whole sparse FTS
block fits the caller's page, so the terminal fuzzy page can never
hide exact matches the plain hasMore path would have surfaced
(jellycompat clients with EnableTotalRecordCount=false lost matches).
- execSearchBlock takes a querier and returns its untrimmed rows;
SearchPage hands the already-fetched block to the fallback instead of
re-running an identical FTS query on every sparse search.
- Fuzzy truncation is detected with LIMIT cap+1 instead of a
COUNT(*) OVER () window count that only fed a debug log; truncated
exact-mode responses now report total_exact=false rather than
presenting the cap as an exact count.
- The fuzzy query runs in a transaction pinning
pg_trgm.similarity_threshold via SET LOCAL, so match quality cannot
drift with cluster configuration.
- When the FTS block has real hits, fuzzy augmentation demands
similarity >= 0.45 so correctly-spelled sparse queries only gain
near-identical titles instead of base-threshold trigram noise.
- filterCatalogSearchItems no longer erases fuzzy matches on the
filtered/sorted/prefix resolver path: a typo token is never a
substring of the titles it matched, which left typo search returning
zero results there while the plain search box showed matches.
- The cursor probe floor applies only when the fallback can fire;
cursor fuzzy fetches no more rows than the terminal page can serve.
- slog.Debug -> slog.DebugContext (sloglint); reuse
contentIDsFromMediaItems instead of a duplicate helper; document the
title-only fuzzy scope.
Verified against the dev deployment: stable totals across pages, no
duplicates, small-limit cursor recall restored, filtered-path typo
search working, ~160ms typo-path latency.
Part of PR #386.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(catalog): reach long titles via strict word similarity in fuzzy search
Full-string trigram similarity is diluted by every extra trigram a long
title contributes, so a typo of one word could never reach titles like
"Avengers: Endgame" ("avegners" scores ~0.38 against "avengers" but far
below threshold against the full title). Swap the fuzzy predicate from %
to <<% (strict_word_similarity), which scores the query against the best
word-boundary extent of the title. At equal thresholds <<% is a strict
superset of %, and the existing gin_trgm_ops index serves both — no
migration needed.
The SET LOCAL pin moves to pg_trgm.strict_word_similarity_threshold and
is load-bearing: the 0.6 server default would reject ordinary one-edit
typos outright.
Ranking is strict word similarity first with whole-title similarity()
as tie-break, so near-identical short titles ("The Avengers") sort above
long titles that merely contain the matched word.
The 0.45 augmentation floor deliberately stays on whole-title
similarity(): word similarity rates embedded prefix words far too high
("coral" scores 0.5 against "coraline"), which dev testing showed would
flood a correctly-spelled sparse query with 27 noise rows. Zero-hit
(true typo) queries skip the floor, so the new long-title recall applies
where it matters.
Dev-verified: "avegners" now returns The Avengers first, then Avengers
Grimm / Avengers: Endgame; "coraline" still returns exactly its 4 real
titles; cursor small-limit recall, filtered-path typo search, pagination
stability, and ~160ms typo-path latency all unchanged.
Part of PR #386.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
2fb5e10de4 |
fix(scanner): recover malformed video durations (#416)
* fix(scanner): recover malformed video durations * fix(scanner): preserve probe failure semantics * fix(scanner): harden malformed-duration recovery after review Remediates the deep-review findings on the duration recovery machinery: - Ignore attached_pic cover-art streams when deciding whether a file is video: audiobooks/music with embedded artwork no longer fail import or persist 1-second durations, and never trigger the packet scan. - Cap the audio-only duration path (raised ceiling, not removed) so malformed audio containers cannot persist multi-year durations. - Keep the parsed probe when the packet fallback fails instead of discarding codecs/tracks/resolution with a hard ProbeFile error. - Apply the implausibly-short guard to the end-minus-start fallbacks so collapsed absolute-timestamp spans reach the packet scan too. - Make the legacy-duration repair one-shot: rows reprobed by the fixed parser are authoritative, ending the infinite reprobe loop for genuinely short large clips. - Teach the library-scan repair predicate (needsCriticalProbeRepairScanState) the same legacy signature so scans repair collapsed durations instead of deferring to request-time repair. - Grant the packet-scan timeout to any reprobe likely to hit the fallback (Duration<=0 video files), not just the legacy shape. - Share the implausibly-short thresholds between probe and repair layers (closes the 100-500MiB repair gap) and deduplicate frame-rate parsing within the scanner package. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
91e1164090 |
feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
1664c60425 |
fix(metadata): publish artwork revisions atomically (#399)
* fix(metadata): publish artwork revisions atomically * fix(metadata): harden artwork revision cleanup * fix(metadata): address artwork revision review findings - restore image applies for all media_items types and reject unsupported target/image combinations with 400 before uploading; episodes coerce to stills and the web dialog no longer offers image tabs episodes can't use - add WHEN clauses to displacement triggers and hoist to_jsonb so bulk catalog upserts that assign unchanged artwork columns skip the trigger - make artworkkey the single variant-ladder owner: imagecache derives its widths from it and triggers store image_type instead of hardcoded variant arrays, expanded by the collector at deletion time - sweep dormant registry rows periodically so references lost through untriggered surfaces degrade to slow cleanup instead of leaking - park just-published revisions dormant, keep dormant rows dormant on re-cache, and batch the GC reference pre-check per run - heal rows re-referencing a just-deleted revision via reconciler-style resets after the deletion commits - share a per-URL image-loaded hook across DetailHero, ItemCard, SectionItemCard, GlobalSearch, and CollectionPosterCard - deduplicate Cache/CacheBytes finalization and drop unused VariantPaths plumbing Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): cast reused timestamp parameter in revision upsert Postgres cannot deduce one type for $3 used both as a plain value and inside a CASE arm; the dev deploy surfaced it as SQLSTATE 42P08 on every publication. Cast both uses and cover the arm/park/track upserts with database-backed tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): address artwork revision review comments - keep a durable heal path: deletion marks deleted_at instead of removing the registry row, so a failed post-delete heal retries with backoff and broken references never park; trackers clear the marker on re-upload - never treat bare existence as an immutable-content match; backends without content verification rewrite the object - exercise revisioned cover keys in scanner/enrichment fakes, compare the tracked manifest exactly, and honor cancellation in the blocking test deleter Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
95779c47a9 |
fix(web): update metadata for selected version (#414)
* fix(web): update metadata for selected version * refactor(web): simplify selected duration lookup * refactor(web): share the hero runtime formatter via mediaFormat MovieContent and EpisodeContent carried identical local formatDuration helpers; move the minutes-based badge formatter into lib/mediaFormat as formatRuntimeMinutes alongside the other canonical media-spec helpers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
b3722dac58 |
fix(playback): open the listener before sweeping stale transcode dirs (#413)
* fix(playback): run orphaned-transcode cleanup in the background at startup The native and Jellyfin-compat routers swept stale per-session transcode dirs synchronously during NewRouter, before the listener bound. On a slow network filesystem this blocked startup for 80+s (64 leftover dirs on the last deploy), so restart-reconnect clients were turned away and the health check reported the server unhealthy the whole time. Move both sweeps into a background goroutine (StartBackgroundOrphanCleanup) so the listener comes up immediately and the cleanup runs concurrently. The delete logic is unchanged: same active-session snapshot and MaxTokenTTL age-sparing, only later. A package-level mutex serializes concurrent sweeps of the shared transcode root so the two background sweeps can't race on os.RemoveAll. Part of #412 * fix(transcode): background the node boot-time transcode-dir sweep A dedicated transcode node swept leftover transcode dirs synchronously in NewServer, before startStandaloneServer bound its listener. On a slow network filesystem that delete blocked the node from coming online at boot, the same startup-stall class as the main server. Move the sweep into the shared StartBackgroundOrphanCleanup goroutine so the node's listener binds immediately. Backgrounding required an age guard: the sweep previously ran as a full wipe (minAge=0) with an empty active-set, which was only safe because it completed before any request could arrive. Run concurrently that would race a token-carried reconstruct writing into TranscodeDir/<sessionID>, deleting segments a fresh ffmpeg is producing. Passing MaxTokenTTL spares any dir younger than the max token lifetime — exactly the ones a still-valid reconnect could reconstruct — while dirs older than any surviving token (never reconstructable) are still reclaimed. Part of #412 * feat(playback): reclaim orphaned transcode dirs periodically, not just at boot The orphaned-transcode sweep only ran at startup on both the central server and transcode nodes, so it only ever reclaimed dirs left by an ungraceful prior shutdown. During a long uptime the in-memory session reapers delete the dirs of sessions they still track, but a dir whose owning session was dropped without its RemoveAll succeeding becomes an "untracked orphan" with no runtime GC — on a box that runs for weeks these accumulate until the next restart. Add StartPeriodicOrphanCleanup: an immediate background sweep followed by an hourly re-run bound to a lifecycle context. Wire it on all three surfaces — native API and Jellyfin-compat (via deps.AppContext) and the transcode node (via a new Server.StartOrphanSweeper(appCtx), replacing its boot-only sweep). When no context is supplied (tests) it degrades to a single boot-time sweep so no ticker goroutine outlives the caller. The sweep stays age-guarded at MaxTokenTTL, so nothing reconstructable is ever reaped. Because the node sweep now runs during live traffic, it snapshots the live job set (Server.activeSessionIDs) and spares those dirs by id rather than by age alone — a long-lived session that only re-serves already-written segments stops advancing its dir mtime, which age could otherwise misclassify. In integrated mode the native and compat sweeps share one TranscodeDir but each snapshots only its own manager's live set; the resulting cross-manager reap of a >24h idle dir is bounded (rebuilds from token/recipe) and documented at both call sites. Part of #412 |
||
|
Jeremy SmithandGitHub
|
6ece120f77 |
fix(playback): stop orphan cleanup deleting the subtitle cache (#379)
The orphaned session sweep in CleanupOrphanedTranscodeDirs treats every subdirectory of the transcode dir as a dead session, but the subtitle cache lives in there too. Cache hits only bump the entry file mtimes, never the directory's, so an active cache that had not seen a new extraction in 24 hours looked stale and got wiped in one go. That forces a full ffmpeg demux on the next PGS subtitle selection, which is the exact cost the cache exists to avoid, and the transcode node's boot wipe removed it unconditionally. Skip the cache directory in the sweep. It validates entries on lookup and runs its own LRU eviction, so leaving it alone is safe. |
||
|
|
255b1be89c |
fix(history-import): import Emby favorites (#378)
* fix(history-import): import Emby favorites * fix(history-import): tolerate Emby favorite errors * fix(history-import): count atomic favorite inserts --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
8fc054c15d |
fix(scanner): never purge files under unreachable library roots (#372)
* fix(scanner): never purge files under unreachable library roots An unreachable root is not a removed root. When one root of a multi-root library dies (unmounted share, dead drive) while another root still has files, the whole-library empty-root guard does not fire — the surviving root produced files — so the scan marks everything under the dead root missing_since (desired: hides it from browse/playback) and then, with the default scanner.empty_trash_after_scan=true + 24h file_removal_grace, the next scan after the grace hard-deletes every row under the dead root. A week-long drive outage silently destroys the root's entire catalog state: probe data, intro/credits markers, file hashes. Worse, membership reconciliation immediately purges media_items whose only files lived on the dead root, cascading user collections (library_collection_items has ON DELETE CASCADE) and deleting cached artwork. This change makes "temporarily offline" survivable: - Probe each configured root at scan start (os.Stat + IsDir + ReadDir, factored into the new internal/rootcheck package and shared with the admin mount-check endpoint). Unreachable roots are skipped by the walk but their scopes still reconcile, so files are still marked missing. - The trash sweep (DeleteMissingByFolder) now excludes rows whose path sits under an unreachable root, using the same exact-path + escaped prefix-LIKE matching as ListIDsOutsideRoots (a sibling root that merely shares a string prefix is never protected). With all roots reachable the emitted SQL is unchanged. - Membership removal still happens — browse/home hide items via media_item_libraries, so removal is what keeps a dead-root-only title out of the catalog — but the orphan media_items purge exempts items whose files sit under an unreachable root. Their metadata, artwork, and collection links survive; when the root returns, the upsert clears missing_since and syncPresentLibraryState re-inserts the membership, restoring the item with zero re-probing or re-matching. - The folder surfaces scan_warning_code='dead_root' with a message naming the unreachable roots; a fully healthy scan or a successful mount check clears it, mirroring empty_root. The admin UI shows a badge and banner. - Deliberate deletion is untouched: removing a path from the library config still purges via ListIDsOutsideRoots, files under reachable roots keep the exact 24h-grace purge, the empty-root guard and the autoscan dead-mount guard are unchanged. The audiobook/podcast/ebook reconcile paths share the same folder-wide sweep and orphan purge, so they get the same guard. Covered by tests: an end-to-end two-root scan (root dies -> rows survive a zero-grace sweep and warning is set; root returns -> rows resurrect with their original ids and the warning clears; deleting a file under a reachable root still purges), repo-level sweep-protection and sibling-prefix tests, orphan-purge exemption, and rootcheck unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scanner): probe uncompacted roots and take dead-root path on full outage Review follow-ups: (1) probe every configured path instead of the compacted traversal roots, so a nested child mount that dies under a reachable parent is still protected from the sweep; (2) when every configured root is unreachable, bypass the empty-root confirm flow (without consuming the one-time cleanup allowance), mark files missing, and raise dead_root instead of empty_root; (3) dead_root warning banner no longer shows empty-root confirm-deletion guidance as its fallback hint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(scanner): simplify dead-root protection plumbing - extract pathscope.CoverageClauses as the single builder for the exact-path + escaped prefix-LIKE root predicate; scanner's rootCoverageClauses delegates to it and catalog's excludeOrphansUnderProtectedPrefixes reuses it instead of hand-rolling the same clause loop - extract Scanner.sweepMissingAndReconcile to replace the identical trash-sweep + membership-reconcile + S3-image-cleanup block that was triplicated across the audiobook, ebook, and podcast scans (callers keep their flavor-specific log lines so messages stay constant) - add unreachableConfiguredRoots helper for the repeated probeUnreachableRoots(ctx, folder.ID, cleanScanRoots(folder.Paths)) expression in scanPaths and ScanFile - drop the unread Path field from rootcheck.Result - move the dead/empty-root warning text constants in AdminLibraries.tsx out of the middle of the import block Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): close dead-root protection gaps found in review Remediates the confirmed findings from the deep review of this PR: - Scoped audiobook scans (autoscan file events, subtree scans) ran the folder-wide sweep while probing only the scoped clone's Paths, so a healthy-subtree event could hard-delete a dead sibling root's rows. sweepMissingAndReconcile now reloads the folder's configured roots from the DB and probes them uncompacted, which also protects nested child mounts in the audiobook/ebook/podcast reconcilers. - A lost mount that leaves an empty, stat-able mountpoint probed as reachable and kept the historical purge timeline. A reachable root that is a literally empty directory while cataloged rows remain under it is now treated as suspect: rows are only marked missing, the sweep and orphan purge exempt it, dead_root is raised, and the mount-check endpoint reports it (additive suspect_empty field) instead of clearing the warning. Arming the one-time empty-cleanup allowance completes the deletion, including in the mixed case where other roots are healthy. Roots that still have directory entries keep the historical grace-then-purge path. - Confirmed empty cleanup (allow_empty_cleanup_once) no longer force-deletes rows under probe-dead roots: an outage is not a confirmation, so a dead sibling root's catalog survives a confirmed cleanout of a reachable empty root. - Root probes are now bounded (rootcheck.ProbeWithTimeout, 5s): a hung network mount degrades into the protected unreachable path with a probe_timeout error code instead of stalling every scan of the folder indefinitely. - Documented the cross-library limitation of the orphan-purge exemption next to the query it applies to. All behavior is pinned by new DB-backed tests (suspect-empty protection + confirmed completion, confirmed-cleanup dead-root survival, scoped/nested-root sweep protection, suspect-root query, probe timeout). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): address dead-root review findings --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
1f2125b920 |
feat(plugins): group Apps sidebar by plugin manifest category (#366)
* feat(plugins): group Apps sidebar by plugin manifest category Implements the plugin SDK's documented PluginManifest.category semantics (silo-plugin-sdk proto/silo/plugin/v1/common.proto): a slash-delimited path that groups plugins in the user-facing Apps section, e.g. "Books/Audiobooks" lands under Apps -> Books. The field existed in the manifest proto but silo-server never surfaced it. Server: the user plugin-settings list/detail responses now include an additive-only `category,omitempty` string sourced from the already-loaded manifest via GetCategory(); no new parsing paths. Web: PluginSettingsSummary gains `category?: string`, and AppSidebar groups Apps entries by the FIRST segment of the category path (one level of grouping for now; deeper segments intentionally ignored, documented against the SDK contract). When fewer than 2 distinct categories exist among the visible app links, today's flat list under the single "Apps" header is kept; with 2+ categories, per-category sub-headers render via the existing SidebarSectionHeader (labels hide in the collapsed sidebar the same way other section headers do). Uncategorized plugins fall under "Other", which always sorts last. Tests: Go unit tests for the summary converter (category passthrough and JSON omission when empty) and vitest coverage for the pure groupAppNavLinks helper plus grouped/flat/collapsed sidebar rendering. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(plugins): use generic category examples in comments and tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(web): simplify Apps sidebar link list rendering - fold the duplicated <ul> list markup in the grouped and flat Apps branches into a single renderAppNavList helper so the list styling cannot drift between the two render paths Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
0f9584ef42 |
fix(overlays): derive dynamic-range badges from Dolby Vision metadata (#365)
* fix(overlays): derive dynamic-range badges from Dolby Vision metadata
Dolby Vision files were labeled with a generic "HDR" badge everywhere
outside the Media Info dialog, because badge sites only consulted the
bare FileVersion.hdr boolean even though the payload carries
video_tracks[].dolby_vision / dv_profile / video_range_type /
color_transfer.
Web: add a shared helper (web/src/lib/videoRange.ts) that derives a
display label ("DV", "DV HDR10", "DV HLG", "HDR10+", "HDR10", "HLG")
from the probed video tracks, mirroring the server-side normalizeHDR
vocabulary, with the hdr boolean kept as the last-resort fallback for
stale pre-DV probe rows. Use it in QualityBadges, VersionDropdown,
VersionFlyout, and the player HUD (playback-info) in place of the
hardcoded "HDR" literal. Badge styling is unchanged.
Server: in internal/overlays/summary.go, break bestFile resolution
ties by richness of dynamic-range metadata (DV > explicit HDR10/HLG
via color_transfer > bare hdr boolean > SDR) so a first-scanned
generic-HDR file no longer masks a Dolby Vision sibling on card
overlays. Full ties still keep the earliest file, so the selection
stays deterministic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(overlays): detect Dolby Vision via dv_profile and DOVI range type
Probed rows set DolbyVision and DVProfile together, but catalog-seeded
or imported tracks can carry only dv_profile or a DOVI* video_range_type
with an empty dolby_vision string. Share one hasDolbyVision predicate
between rangeRank and normalizeHDR so those rows rank and label as DV,
matching the web videoRange helper.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(overlays): read video_range_type and hdr10_plus in hdrTypeFromTracks
Review follow-up: hdrTypeFromTracks only inspected color_transfer, so
server card overlays could never label HDR10+ and a track whose only
signal is video_range_type (e.g. catalog-seeded rows without probed
color metadata) fell through to the bare-boolean tier of rangeRank —
inconsistent with hasDolbyVision, which already reads the range type.
The server now shares the web helper's detection order exactly, making
the web/server vocabulary mirror claim true.
Also drops the unused video_range field from the web VideoRangeTrack
interface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
|
||
|
|
f377be9d6e |
feat(catalog): include per-episode overlay summaries on season episode listings (#364)
* feat(catalog): include per-episode overlay summaries on season episode listings Episode cards already render user-configurable overlay badges everywhere except the series/season detail pages: the season-episodes endpoint never included overlay_summary, so SeasonEpisodeGrid and EpisodeRow had nothing to render. Server: add overlay_summary (omitempty) to episodeResponse and populate it in buildEpisodeResponses via overlays.BuildSummary over the episode's already-loaded media files, access-filtered with FilterMediaFilesByAccess to match the browse/sections paths. No extra queries; the files were already batch-fetched for the files[] payload. Additive-only API change. Web: extend EpisodeListItem with overlay_summary, add overlayDataFromEpisodeListItem (shared extract helper), and render CardOverlays (variant="wide") on the episode stills in SeasonEpisodeGrid and EpisodeRow using useOverlayPrefs, matching the ContinueWatchingCard pattern. The hook already honors the admin kill switch and per-user prefs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(catalog): apply library restrictions in FilterMediaFilesByAccess without a quality ceiling FilterMediaFilesByAccess short-circuited whenever MaxPlaybackQuality was empty, skipping the AllowedLibraryIDs/DisabledLibraryIDs checks that FileAllowedByAccess enforces. For viewers with library restrictions but no quality ceiling, callers (episode listings, browse/section overlay summaries, item detail versions) received files from restricted libraries. Short-circuit only when no access criteria are set at all. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
035ef457a4 |
fix(web): derive settings language options from canonical language list (#363)
The profile and device settings language dropdowns (preferred subtitle language, spoken language, metadata language, device audio language, and per-library audio/subtitle overrides) were built from two hardcoded 12-language lists, so Dutch and 24 other languages supported by the canonical list in languageNames.ts could not be selected and stored codes like "nl" rendered as raw codes instead of names. Derive both settings option lists from the shared LANGUAGES array and route getLanguageLabel through getLanguageName, so every supported language is selectable everywhere and stored values always render a proper display name (e.g. "nl" -> "Dutch"). Options are now alphabetized by label, matching the other language dropdowns that already use LANGUAGES; the "No preference" sentinel stays first. Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Quick104andClaude Fable 5
|
c36d70dcba |
feat(playback): enable protocol v3 by default
The playback.protocol_v3_enabled flag shipped seeded to 'false' as a rollout safety valve and was never exposed in any settings UI. Current Android clients are v3-only for video, so against a server with the flag off they refuse playback with a misleading "update your server" error even on the newest image. Flip the flag to 'true' everywhere. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104andClaude Fable 5
|
be3bfafeaa |
fix(api): log events websocket upgrade failures with handshake shape
The events handler silently swallowed gorilla upgrade errors, which hid a client bug that produced 19k+ failed upgrades in a week (the Android client's auth plugin was demoting wss to https, arriving here as a plain GET). Log the error plus the upgrade-relevant request headers so a failing client is diagnosable from the server alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104andClaude Fable 5
|
18283c2c9b |
fix(playback): HLS-safe audio policy and surround-preserving transcodes
Two audio fixes on the V3 planner and transcode pipeline: - Copied DTS in an HLS route drags Media3's audio clock (device stall corrections, ~0.3x pacing, frozen position reports). DTS/TrueHD/PCM are not HLS-native codecs regardless of the client's progressive decode claims, so HLS remux routes now convert them to AAC. Validated on the Shield: the copy-remux fallback went from ~0.3x pacing to exactly real time. - Transcodes no longer hard-downmix to stereo: multichannel sources keep 5.1 through the AAC re-encode (384k, -ac 6), plumbed through TranscodeOpts, the planner result, and the transcode-node protocol (new optional target_audio_channels field, ignored by older nodes). Also logs one "playback plan decided" line per V3 start (decision reason, delivery, play method, DV profile, quality inputs) so route selection is reconstructible from server logs alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104andClaude Fable 5
|
3ddc74f784 |
fix(playback): apply DV7 RPU strip on client-driven copy restarts
HandleStartTranscode built its ffmpeg recipe purely from the client request, so a Dolby Vision Profile 7 source restarted with copy video (the V3 recovery fallback after a progressive failure) shipped raw BL+EL+RPU NALs labeled as plain HEVC. The V3 start path derives the strip from the plan and the audio-switch restart derives it from the durable session route; this endpoint now derives it the same way (session RemuxDVMode or source DV profile 7 + copy video) for both the local transport and the pooled-node dispatch, including the node's reconstruct recipe card. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a196b0844e |
feat(notifications): add Android FCM push delivery via the relay (#409)
Extend the push pipeline to Android devices through the Silo push relay's /v1/fcm/send endpoint. push_devices gains platform-conditional FCM token columns (encrypted at rest with row AAD, hashed like APNs tokens), the generic POST /notifications/push/devices endpoint the Android client already calls registers FCM tokens, and fanout, operational dispatch, retries, and terminal UNREGISTERED device disabling all reuse the existing Apple machinery. Delivery is gated by a new notifications.android_push_delivery_enabled setting, advertised through the capability endpoint's android_push block, and testable via POST /admin/notifications/push/fcm/test. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
075e217477 |
feat(playback): plan v3 routes from pooled node capabilities (#408)
* feat(playback): plan v3 routes from pooled node capabilities Protocol v3 planning previously gated every server transformation on the API host's local ffmpeg probe, so deployments whose toolchain lives on transcode nodes (libx264/aac/dovi_rpu on nodes, minimal binary locally) received conversion terminals before transport preparation ever consulted the selected node's capabilities. Planning now draws on two registries split by executor pool: - Registry stays the local probe and keeps gating progressive remux routes, which execute in this process and can never offload. - HLSRegistry widens availability for HLS deliveries with the pooled transcode nodes' advertised transformations (name and recipe version pinned to the local specs), fetched concurrently under a short planning deadline through the existing TTL cache. Failures are now negatively cached so an unreachable node costs one timeout per window rather than one per start. The remux family picks the executor per branch: a recipe needing transformations only nodes carry skips the progressive remux and ships the same recipe on the HLS remux delivery instead. The local-fallback path in prepareTransportV3 now validates plans against the local registry's advertised set — mirroring the per-node validation — and returns the existing retryable transcode_node_capability_unavailable terminal when no executor can run the recipe, instead of spawning an ffmpeg that would fail at runtime. Deferred from PR #398 review (comment 3579105380). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden union capability planning from review Addresses all four review findings on the capability-union feature: - Select capability-matching nodes: plans carrying server transformations now restrict node selection to nodes whose advertised capabilities validate against the plan (nodepool.PlanSessionWith with a set-lookup predicate), so heterogeneous pools cannot load-balance a recipe onto a node that would reject it while a capable sibling exists. Transformation-free plans keep pure load-based selection. - Split the capability cache by consumer: planning honors negatively cached fetch failures (one timeout per window), while the transport path fetches through them — a memoized 3s planning deadline must not reject an already-selected node that the 10s transport budget could still validate. - Gate node-widened availability on the HLS engine: a progressive-only client that needs audio conversion keeps its specific retryable audio_conversion_unsupported terminal instead of falling through to a non-retryable adaptation_unavailable for routes it can never run; the DV strip union flag is gated identically. - Make HLSRegistry a lazy, memoized producer: the planner only builds the widened registry when a route decision depends on node capabilities, so direct-play and other source-preserving starts never wait on node capability fetches (or their dead-node deadlines). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
854d07cf8f |
feat(playback): add protocol v3 planning and recovery (#398)
* docs(playback): plan protocol v3 server implementation * docs(playback): incorporate protocol v3 review * feat(playback): implement protocol v3 server * fix(playback): persist empty route diagnostics * feat(playback): harden protocol v3 HDR routing * feat(playback): complete protocol v3 client contract * fix(playback): harden protocol v3 recovery * fix(playback): restore dovi_rpu strip filter for DV remuxes The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg; the probe failed closed on every deployment, disabling the new validated DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1 remux path from main. Restore dovi_rpu across the probe, remux and HLS copy arguments, and the recipe-card constant. Also from review: validate the remux DV mode for every profile (garbage modes on non-P7 sources silently no-opped), reject preserve mode for P7 outright (a base-layer-only remux cannot preserve dual-layer DV), tag dvhe sample entries only for the explicit v3 preserve recipe so legacy web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the token-frozen DV mode in the proxy remux path instead of legacy-auto. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): correct v3 planner policy and contract validation Review fixes to the v3 planner and wire contracts: - Bar Profile 7 sources from the non-strip progressive remux route: a base-layer-only remux can never deliver native dual-layer DV, so the planner no longer emits plans claiming validated Dolby Vision while the executed remux drops the enhancement layer. - Accept the device-quirks feature flag from either capability location, matching every other dual-location feature check. - Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with a degradation warning instead of leaving them unplayable under v3. - Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and wire the previously dead Metered signal into conservative auto rungs. - Degrade to the validated source-quality route instead of a terminal when only an implicit quality reduction demanded an unsupported transcode; explicit user-selected rungs keep terminal behavior. - Bound inner capability lists and strings; compare attempt keys exactly instead of case-folded; make ParseTrackIDV3 strict about canonical numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising burn-in for unknown subtitle codecs; probe every h264 encoder rather than requiring libx264; normalize the file-level bitrate fallback. - Evaluate subtitle renderability against the engine each candidate route executes on, not always media3_direct. - Pin the with-quirks attempt-key preimage arity in the cross-language fixture so the Kotlin client stays in lockstep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 control-plane reliability Review fixes to the v3 session, store, and handler layer: - Bound concurrent replans with a slot semaphore: each replan pins a pooled connection for its advisory lock while issuing further store queries from the same pool, so an unbounded recovery storm could turn every connection into a lock holder and deadlock the server. - Make CompleteReplan a real compare-and-swap (base-revision predicate, ErrReplanSupersededV3) and map BeginReplan insert races to a replay instead of a raw unique violation. - Fingerprint start requests (request_digest column): an attempt ID reused with different input is now a 409-style conflict rather than a silent replay, and both replay paths check session liveness so dead sessions surface as retryable terminals. - Pre-delete expired attempt rows on SaveAttempt so a retry during the cleanup window cannot wedge on an unreachable conflict. - Align the in-memory store's semantics with Postgres and add DB-backed planstore tests (SILO_TEST_DATABASE_URL), including a regression test inserting every route-event name against the real CHECK constraint. - Session manager: v3 route-set updates own RemuxDVMode outright so a replan onto an SDR source clears a stale strip mode; replacement reservations survive unrelated legacy stream updates; replacement admission excludes the replaced session explicitly instead of decrementing totals it may no longer be part of; the admission CAS loop is bounded and decider errors are logged. - Map transient store failures to 500s instead of terminal 404/403s; authorize route events via identity-only projections after the rate limiter; keep sanitized diagnostics deterministic. - Merge the server-computed durable plan key into replan exclusions so unreproducible client history cannot re-select the failed route. - Remap tracks only when the effective edition changes (a same-file replan no longer switches audio to a lookalike track) and remap ID-only subtitle selections on edition fallback. - Cache the v3/shadow feature flags for five seconds instead of one settings SELECT per playback request; stop remote transports best-effort when the start call times out; carry dvm/tid claims and the transport-scoped job identity through the legacy audio-change re-mint; index playback_route_events(received_at) for the retention delete; run store maintenance for DB-less deployments too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): reap idle node jobs and gate WebVTT conversion - Add an idle reaper to the transcode node: a job untouched by manifest or segment requests for ten minutes is closed and unregistered. After a v3 replan retires a transport ID, a stale in-flight stream token could resurrect the old job via reconstruct and encode to end-of-file for nobody; jobs waiting on readiness count registration as access and are never reaped mid-wait, and reaping keeps the recipe so a still-valid token reconstructs on the next hit. - Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with 415 before headers are written instead of spawning an ffmpeg command that always fails mid-response, and make the extract-format override fall back to source-driven mapping for bitmap codecs. - Drain error bodies on non-202 node responses so the HTTP transport can reuse connections. - Pin the transcode-dir cleanup separator-boundary semantics with a regression test (a session ID sharing another's prefix must not retain foreign directories). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): close v3 planner policy gaps from review - Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has no rung below 480p/1500kbps, so lower caps were silently exceeded even though the cap is documented as a hard delivery ceiling. - Treat video-only media as audio-compatible instead of forcing an AAC conversion (or an audio_conversion_unsupported terminal) onto a file with no audio stream. Tracks whose codec failed to probe keep the gate. - Only promise a bitmap subtitle sidecar for embedded PGS with an engine that renders embedded bitmap: external/downloaded bitmap and embedded DVD/DVB published artifact URLs that always failed at fetch. They now fall through to burn-in or its terminal. - Accept client_video_transformations_v1 from either client_features or the nested context when validating client-executor transformations, matching the planner's dual-source reads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): probe and execute DV remuxes with one ffmpeg binary The v3 transformation registry probed the configured playback.ffmpeg_path while progressive remux execution resolved the process-global discovery path, so a deployment where only one binary carries dovi_rpu could plan a server_dv7_to_hdr10 route and then fail it at stream time. Resolution now goes through a shared ResolveFFmpegPath (configured path first, discovery fallback — the same rule the transcode pipeline already used), the dovi_rpu probe is cached per binary path, and the stream handler and proxy worker pass their configured path into ServeRemuxWithDVMode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 replan identity and control-plane limits - Seed failure-replan track selections from the durable current plan before overlaying the request: after an alternate-version fallback the normalized request still carries requested-edition track IDs, so a replan omitting unchanged tracks was rejected as a track/file mismatch. - Remap ID-only audio selections across edition changes (parse the ID to an index like the subtitle remap already does) instead of leaving a stale file-bound ID to fail validation. - Release the node planner reservation when a prepared remote transport rolls back after the node accepted the job; repeated failed starts could otherwise pin max-job/bandwidth budgets for the full reservation age. - Size the replan semaphore below the PostgreSQL pool via a store capacity advisor: with max_connections at or below the fixed bound, advisory-lock holders could starve the inner store queries they need to finish. - Contain shadow-planner panics with a recover boundary; it runs on a bare goroutine where an escaped panic kills the process for what is telemetry-only work. Document why the memory store's session lock is deliberately a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): serialize node job teardown against reconstructs - Look up and touch manifest/segment sessions in one critical section so the idle reaper cannot unregister a job between the lookup and its liveness refresh. - Re-validate each reap candidate under the per-session lifecycle lock before closing it: Close removes the output directory, and without the lock it could race a token reconstruct and wipe the segments the fresh ffmpeg is writing. - Take the lifecycle lock in handleStop so a stop racing a RequireReady start's readiness wait blocks until registration and tears the job down, instead of 404ing and orphaning the ffmpeg until the reaper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
28c6ddc237 |
feat(playback): add per-user transcoding controls (#375)
* feat(playback): add per-user transcoding controls * fix(playback): enforce forced video transcode permission * chore: address transcode control review feedback * fix(playback): recheck transcode permission on audio switch --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
QuickandGitHub
|
3fd0912cb3 |
fix(notifications): align with Worker push relay (#374)
* fix(notifications): align with Worker push relay * fix(notifications): disable invalid APNs tokens |
||
|
Quick
|
df9cacc8da | fix(web): keep playback running during next up | ||
|
QuickandGitHub
|
02b6648995 | feat(notifications): harden push relay credentials (#373) | ||
|
|
c1c110e3d2 |
feat(playback): improve web subtitles and track selection (#362)
* fix(player): keep text subtitles in sync across copy-mode restarts and sparse cue windows - Rebase already-loaded cues in place when streamOriginSeconds changes (copy-mode session restart) instead of leaving them offset by the delta. - Stop inferring end-of-input from where a window's cues stop; only the known media duration marks EOF, so a dialogue gap no longer silently ends prefetching for the rest of playback. - Anchor the first window fetch to the intended start position (resume target or pending seek) while the element still reports currentTime=0, and reset coverage on forward seeks past the fetched window. * fix(playback): align encoded transcode start with the declared segment boundary A mid-segment start (resume, seek restart, audio switch) spawned ffmpeg at the raw seek position while labeling its first segment with the grid number, whose synthetic-manifest start is up to one segment earlier. hls.js aligns the first fragment's content to that declared position, shifting the whole session's timeline late by seek mod segment_duration (0-2s): subtitles trail dialogue by a constant per-session offset and progress/resume positions drift by the same amount. Snap the ffmpeg start position down to the segment boundary for encoded sessions so declared and produced timelines match exactly; the player still seeks to the precise requested position. Copy-mode sessions serve ffmpeg's real manifest and keep the raw seek. * fix(api): forward http.Flusher through response-writer middleware wrappers Streamed subtitle extracts (and any progressive response) flush per chunk via an http.Flusher assertion, but none of the status-capturing middleware wrappers implemented Flush, so the assertion failed and cues sat in Go's response buffer until ffmpeg finished. On large remuxes where a 600s window takes 20s+ to demux, captions appeared only when the whole window completed instead of within the first seconds. Give every wrapper a Flush() (satisfies plain assertions, including chi's Compress) and Unwrap() (satisfies http.ResponseController). The jellycompat image-proxy tag rewriter flushes only in passthrough mode since it buffers JSON bodies for rewriting. Regression test asserts the API chain forwards Flush end to end. * feat(player): PGS subtitles honor size, position, and background settings Port the tvOS/iOS bitmap-cue styling to the web player. libpgs now decodes in worker mode but draws to a hidden source canvas on the main thread; a compositor detects cue regions from the frame's alpha channel and re-places them on a visible overlay canvas per the shared subtitle appearance settings: size scale (with the 0.85 authored-size compensation), vertical position preset (dialogue-band cues only — floating signs keep authored placement, matching the Apple implementation), and the background box. Font family, text color, and outline are baked into the source pixels and remain inapplicable. * fix(player): anchor PGS position presets to the text overlay's reference frame The initial port used silo-apple's 30/1080 bottom margin and video-relative lower-third/top anchors; the web text overlay anchors to a 16:9 reference frame with 7%/18% offsets that extends into the letterbox for wide content. Use the same anchors so PGS dialogue lands exactly where SRT text does. * feat(player): size PGS cues to match the text subtitle line height Replace the authored-size ladder (0.85 × font-size ratio) with per-cue text-line matching: the region detector reports the tallest text line inside each cue, and the compositor scales the cue so one line of bitmap text renders at the same pixel height as the SRT overlay's font at the current preset. Authored size differences between discs no longer leak through; upscaling is capped at 2.5× to keep small bitmaps from going blurry. * feat(subtitles): opt-in windowed PGS extraction to cut mid-file load latency PGS extracts always demuxed the source from byte 0, so starting a large remux mid-file meant minutes before the first bitmap cue. The web player now opts in to windowed extraction (?windowed=1&position=&duration=) and re-points libpgs at a fresh window on seeks and near coverage end; ffmpeg input-side -ss with -copyts keeps absolute source timestamps. Without the explicit opt-in the endpoint behaves byte-identically, so Apple/Android and other single-fetch consumers are unaffected. ASS remains unconditionally non-windowed (its header only exists at offset 0). * feat(playback): cache extracted PGS subtitle tracks Every selection of an embedded PGS track re-ran a full ffmpeg extract that demuxes the entire source file from byte 0 — minutes for a large remux — and responses were Cache-Control: no-store, so repeat selections, re-watches, and multiple viewers all paid full price. Add a disk cache for full-track .sup extracts under <transcode_dir>/subtitle-cache, created lazily: - Keyed by source path hash + subtitle stream ordinal + source mtime+size (encoded in the filename), so a replaced source file implicitly invalidates its entries; the source is stat'ed on every lookup. - Cache miss: ffmpeg stdout is teed to the response (first viewer still streams progressively, first-byte latency unchanged) and into a temp file that is fsynced and atomically renamed into the cache on clean ffmpeg exit. Any error — ffmpeg failure, client disconnect, tee write failure, or the source changing mid-extract — discards the temp file, so a partial entry is never served. - Cache hit: served via http.ServeContent (Range support, Content-Length, Last-Modified from the source mtime) with a revalidatable Cache-Control instead of no-store. - Concurrent requests for the same in-flight track run their own uncached extract (mutex + in-flight key set) rather than blocking on another client's connection. - Scan-on-commit LRU eviction under a 2 GiB cap (recency tracked by bumping entry mtime on hit; atime is unreliable under relatime), plus sweep of crash-orphaned .part temp files. - Windowed PGS requests (?windowed=) bypass the cache in both directions: their output covers only a slice of the track. Both the integrated API handler and the standalone proxy subtitle path share the same playback.SubtitleCache.ServeSUPExtract helper. VTT (already windowed and fast) and ASS (small) stay uncached. No API surface change. AI-use disclosure: implemented with Claude Code. * fix(playback): check Close error returns in subtitle cache paths Silence errcheck on the cache-hit defer and the test's simulated disk-full Close. AI-use disclosure: implemented with Claude Code. * feat(playback): warm PGS cache in background and window from cached track Windowed PGS requests bypassed the cache entirely, so every window fetch re-demuxed the multi-GB original file. Now a windowed miss kicks off a detached background warm (full-track extract into the cache, at most 2 concurrent server-wide, coalesced with client-driven fills), and once the entry exists windowed extracts read the 15-80MB cached .sup instead — seeks and re-enables become near-instant after the first load. Verified empirically that ffmpeg preserves absolute PTS when windowing a sup input. * feat(player): hold playback while PGS subtitle cues load When a PGS track is enabled (or a seek lands outside the fetched window), extraction takes seconds and dialogue could play unsubtitled. The player now pauses until the renderer's parsed data covers the playhead — tracked via libpgs' parsed-timestamp watermark, the exact predicate it renders by — showing a "Loading subtitles…" indicator after 500ms. User play/pause always wins over the hold, a 20s safety timeout prevents stranding playback, and background prefetch never pauses. If future libpgs versions reshape the observed internals the hook degrades to the old play-through behavior. * perf(player): shrink uncached PGS window to 600s Draining a windowed extract from the source reads the full interleaved container across the window (~1GB per 100s of remux on measured hardware); a 3600s window cost ~12GB of reads per fetch while cold. Once the server cache is warm a window costs milliseconds regardless of size, so smaller windows only add trivially cheap re-fetches. * feat(subtitles): burn in PGS/bitmap subtitles for the web player The web player rendered PGS client-side via libpgs, which required extracting the .sup track — a cold ffmpeg demux that took seconds even windowed, since c:s copy still reads the whole interleaved container up to the playhead. Every other server (Plex, Jellyfin default, Emby) burns image subtitles into the video instead, and that is the only path with no per-seek extraction cost. Selecting a bitmap subtitle (PGS/DVD/DVB) now restarts the transcode with subtitle_burn_in at the current aligned position, reusing the same restart machinery as an audio/quality switch so the segment-boundary timeline alignment holds. The server composites the decoded subtitle onto the video with an overlay filter_complex graph (libass's subtitles= filter is text-only); overlay runs at native resolution before any target scaling, and hardware pipelines round-trip through CPU like the text path. Burn-in forces a video encode, so copy-video recipes are upgraded to h264 both client- and server-side. Text subtitles keep the instant, styled, client-side path. The .sup streaming endpoints, cache, and windowing are retained for the Apple client, which renders PGS natively. The now-dead web PGS stack (usePGSSubtitles, pgsPlacement, libpgs dep) is removed. Tradeoff: bitmap subtitles no longer honor web appearance settings (baked into the video) and toggling one restarts the transcode (~1-2s buffering), matching Plex behavior. * fix(player): rebuild text subtitle track when turning off PGS burn-in Selecting an SRT track that turned off bitmap burn-in restarted the transcode, and the client TextTrack built in that same moment was orphaned when the <video> element reloaded, so the subtitles never rendered (and a seek could not recover the dead track). Rebuild the text track once the new stream settles, gated on the burn-in-off transition so quality/audio switches and copy-mode seek restarts keep their subtitles without a needless re-extract. * fix(player): render web subtitles behind the control HUD The text subtitle overlay sat at z-20, above the controls layer (z-10), so cues painted over the bottom HUD and cluttered the control bar. Drop it to z-[5] — above the video, below the controls — so the HUD paints over the cues while it is visible. When controls are hidden the whole controls layer is opacity-0, so cues remain fully visible. * feat(player): lift web subtitles above the control bar while it's visible Rather than hiding bottom-anchored cues behind the HUD, raise them just above the control bar (measured height + a small gap) whenever the bar is visible in the foreground player, then settle them back when it hides. The bar is a roughly fixed pixel height while the cue offset scales with the player, so the bar is measured via ResizeObserver rather than hardcoded. Top-anchored cues never collide with the bottom HUD, so they stay put. z-[5] is retained as a safety so any residual overlap tucks behind the bar. * fix(player): coalesce same-tick transcode restarts into one dispatch Starting playback with a persisted bitmap subtitle fired transcode/start twice within milliseconds: the auto-start effect dispatched before subtitle auto-selection restored the burn-in, whose effect then forced a second start. The first request was already on the wire (no abort signal was passed to fetch), so the server spawned an ffmpeg only to kill it for the second start — visible in production as an ffmpeg exit error ~1ms after every such session start, and slowing time to first frame. Defer the network dispatch by one macrotask so back-to-back restart calls in a tick collapse into a single request carrying the final parameters; state updates stay synchronous. Pass the abort signal into playerFetch so a superseded in-flight request is actually cancelled, and drop any deferred dispatch on unmount so a stray transcode/start cannot land after the session's exit DELETE. * fix(catalog): resolve effective subtitle defaults for movie item details Movie pre-play subtitle selectors were missing the effective defaults (including per-item overrides saved from a previous play) that episodes and watch payloads already resolve. Extract applyToItemDetail/ applyToWatchDetail helpers and apply defaults for movies in buildMediaItemDetail. The SubtitlesPopover now also eagerly loads downloaded subtitles when the saved preference points at one so the closed trigger's Auto summary reflects the override. * feat(player): scale subtitle font size with the rendered video Replace fixed rem font sizes with px values defined at a 720px 16:9 reference height, scaled proportionally with the actually-rendered video (object-fit: contain) so subtitles keep the same relative size as the window grows or shrinks, with a 12px legibility floor. Rename useSubtitlePositionStyle to useSubtitleLayout, returning both the position style and the font scale, and add unit tests for the appearance helpers. * fix(player): satisfy strict index checks in transcode quality test * feat(player): let the pre-play Auto option clear the saved subtitle override A manual in-player subtitle selection persists as an 'always' override for that movie/series, but nothing in the UI could undo it — auto selection stayed pinned to the chosen track forever. Choosing 'Auto' in the pre-play subtitles popover now also deletes the stored preference (movie content ID / episode series ID) and invalidates item details so profile-level auto selection applies again. * feat(player): persist pre-play subtitle selections as the item override Choosing a track (or Off) in the pre-play subtitles popover only lived in component state: it applied to that playback session but vanished on returning to the detail page. Persist it through PUT /subtitle-prefs — the same 'always'/'off' override a manual in-player selection saves — keyed by movie content ID or episode series ID, and invalidate item details so the effective defaults reflect it immediately. * feat(ui): show the saved subtitle override and richer pre-play pill summaries A stored per-item override displayed as 'Auto: <language>', hiding both that an override exists and which track it is. The pre-play subtitle pill now shows the resolved track directly (name with (SDH)/(Forced) markers plus format, skipping markers the name already carries), the matching list row gets the checkmark instead of Auto, and the Auto row reads 'Reset to profile defaults'. Subtitle, audio, edition, and version pill summaries also truncate much later (max-w-44/sm:max-w-64). * fix(player): recover text subtitles from stream reloads and failed window fetches Three failure modes could silently freeze or stop web text subtitles: - A stream restart (seek-triggered transcode restart, quality/audio switch) reloads the <video> element and can orphan the programmatic TextTrack — cuechange stops firing and the last cue freezes on screen. Only the PGS-burn-in-off transition rebuilt the track. Now every settled stream URL change bumps the generation, and the rebuild carries loaded cues (converted back to source time) and window coverage over so it costs no refetch. - The sliding-window fetcher committed windowEnd before the fetch ran, so a failed or hung window counted as covered and was never retried — subtitles silently stopped for up to 10 minutes. Coverage now commits only after the window streams in fully; failures leave the range uncovered and retry after a 5s backoff. - A hung extraction (one fetch in flight at a time, no deadline) blocked every future window for the session. Reads now arm a 30s stall timer that aborts a response which stops delivering chunks; slow-but- progressing streams keep resetting it. Diagnosed from a session where ffmpeg took 69s to stream one subtitle window and a transcode restart landed mid-fetch, freezing the active cue. * fix(playback): keep subtitle selections stable across file changes * fix(web): clarify subtitle labels and positioning * fix(player): hide HUD when pointer leaves * fix(web): tidy subtitle track badges * fix(playback): remap audio tracks across file versions * fix(web): tidy audio track labels * docs(playback): clarify bitmap subtitle appearance * fix(playback): preserve selection state on restart * fix(http): preserve response state across flushes * fix(web): preserve pending quality for burn-in * fix(playback): preserve subtitle inventory identity --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
QuickandGitHub
|
9cfd01e9c2 | Add remote playback identity handoff (#360) | ||
|
|
b7292a9473 |
fix(streaming): stop killing healthy streams at the server WriteTimeout (#361)
The main API server's WriteTimeout (120s) is an absolute deadline from request start, so every streaming response still being written at T+120s was cut mid-body with a clean close. Clients saw multi-GB direct streams truncate every two minutes; the Apple client's cursor-resume reconnect absorbed most kills silently, but one landing during backpressure or a demuxer resync exhausted its retry budget and forced a full player teardown (visible stop + historical audio desync seeding). Fix: internal/httpstream.RollingDeadlineWriter pushes the connection's write deadline forward with progress via http.ResponseController — a response that keeps moving lives indefinitely, a stalled one is still reaped within the window (180s default, SILO_STREAM_WRITE_STALL_TIMEOUT to override). ReadFrom delegates in bounded slices so http.ServeContent keeps its sendfile fast path. Wired into direct play, remux, downloads, the transcode-node proxy, and ebook serving; the server-level 120s guard stays for every other route. The metrics and request-logger response writers now implement Unwrap — without it http.ResponseController cannot traverse to the connection and SetWriteDeadline fails, silently disabling the fix (exactly what the first dev deploy showed). A middleware-chain integration test locks the whole path down against future wrappers missing Unwrap. Validated on dev: 200s/512MB direct and 300s/768MB via CDN sustained range-GETs (previously dying at 120s), zero duration_ms=120000 stream entries since deploy. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandGitHub
|
98a22dafd2 | docs: use canonical plugin repository names (#359) | ||
|
QuickandGitHub
|
648e5c76ef | test(plugins): use canonical metadata repository name (#358) | ||
|
QuickandGitHub
|
58eed826a4 | test(plugins): cover repository rename compatibility (#357) | ||
|
QuickandGitHub
|
10e15798e0 | feat(plugins): add approved community catalog hub (#355) | ||
|
|
d68e70bb47 |
feat(autoscan): Sonarr/Radarr webhook intake without arr API keys (#353)
* docs(autoscan): add arr webhook intake spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add webhook intake schema migration Adds delivery_mode to autoscan_sources, the autoscan_webhook_endpoints table, and delivery_mode/provider_event_type on autoscan_events. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add built-in arr-webhook source identity Host-discovered scan-source entry so webhook-mode sources need no plugin installation; composite lister appends it to plugin discovery. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): persist delivery mode, webhook endpoints, event metadata Sources carry delivery_mode; autoscan_webhook_endpoints CRUD with SHA-256 token lookup and AAD-bound encrypted redisplay; events record delivery_mode/provider_event_type; CreateEvent gains SkipRunningCheck so webhook deliveries are never dropped by the poll exclusion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): share the consume path and add webhook IngestChanges Extracts consumeSourceChanges from PollOnce (marker semantics preserved, existing poll tests unchanged); PollOnce skips webhook sources; IngestChanges feeds deliveries through the shared pipeline without markers and without the running-event exclusion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add Sonarr/Radarr webhook payload parser Host-side arrwebhook package: provider inference, import/rename/delete path extraction with vanished-path-friendly previous paths, subtree fallback, exact-path dedupe, and no-op unknown events. Fixture-backed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add public webhook delivery route and admin endpoint management Public POST /api/v1/autoscan/webhooks/{token} with per-IP rate limiting, 256KiB body cap, 202-for-noop semantics, and token/body kept out of logs; admin create/rotate/delete endpoint routes; source responses carry delivery mode + webhook status/URL; create/update validate delivery mode against source identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add webhook delivery mode to Autoscan admin UI Webhook sources get a generate/copy/rotate webhook URL section, provider selector, delivery status, and a connection-free Add-source flow; activity rows badge webhook deliveries with the arr event type. Path rewrites stay editable in both modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): redact secret path params from request and activity logs The request logger and activity-log middleware recorded raw URLs, so bearer credentials in secret path segments (autoscan webhook {token}, webhook-sync {secret}) were persisted to app logs and activity_log. Redact the secret segment via the chi route params in both sinks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(autoscan): make webhook delivery reliable --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
8b9149a789 |
feat(catalog): remove smart collection item limit cap (#340)
* feat(catalog): remove smart collection item limit cap MaxSmartCollectionItemLimit silently clamped every smart collection to 500 items, and an unset limit defaulted to 100. On servers with large libraries this truncates collections with no indication to the admin — a per-language smart collection (original_language is fr) has 10,625 real matches and showed under 1% of them. Remove the cap and make an unset limit resolve the full result set: the default becomes a large sentinel so every consumer's int plumbing (SQL LIMIT) keeps working. An explicit positive limit in the query definition is still honored as-is — membership size is an admin decision, and every read path that serves collection items either paginates in SQL or scales with the actual collection size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(jellycompat): page smart collection browse in SQL The Jellyfin BoxSet children path resolved a smart (live-query) collection's entire membership and sliced one page locally. With the item cap removed (smart collection membership is now uncapped, an admin decision), a large collection browsed from a Jellyfin client materialized tens of thousands of rows per request — latency and memory scaling with collection size. For the default browse (no explicit SortBy), page the smart query directly in SQL via QueryExecutor.PreviewPage(offset, limit), hydrating only the requested page. Per-request work is now proportional to page size, not collection size. TotalRecordCount still reports true membership (the count query caps at the query's item limit), and past-the-end pages preserve the real total. Shared query prep is extracted into prepareSmartCollectionQuery, used by both the new paged path and the existing full-resolution path (still used by the explicit-sort allowlist branch, whose output is already SQL-paginated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> |
||
|
|
04c4344f52 |
feat(metadata): reconcile artwork cache after public S3 provider changes (#349)
* feat(metadata): reconcile artwork cache after public S3 provider changes Changing the public S3 provider previously broke every cached image permanently: the DB keeps bucket-relative keys, the image cache pipeline treats a cached path as its durable dedup marker and never re-enqueues, and clients eat the 404s straight from S3 so the server never notices. Add a storage identity fingerprint (s3.public_storage_identity, seeded via SetIfAbsent at boot) and a reconcile_artwork_cache task whose startup trigger only fires when the identity changed; manual runs always sweep, doubling as bucket-data-loss recovery. The task probes a random sample of cached objects, then either bulk-resets (near-total miss) or per-row verifies. Missing provider-sourced artwork is reset to its *_source_path so the existing enqueue loop re-caches it; surfaces without a re-downloadable source (chapter thumbnails, collection artwork, library posters, branding refs, embedded book covers) are cleared so their owning pipelines refill them. Small upload-holding tables are always per-row verified so bulk mode cannot blind-clear an upload that survived migration, and transport errors never reset rows. Users never see broken images during the transition: reset rows serve the provider's original URL via the existing absolute-URL pass-through and thumbhashes are preserved. The storage settings page now warns that uploads cannot be re-downloaded when the identity fields are edited. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): harden artwork reconcile per code review Address the confirmed findings from the PR review: - Fingerprint the key prefix case-sensitively and slash-trimmed exactly as s3client applies it (new exported NormalizeKeyPrefix): a case-only prefix edit is a real storage move and must reconcile; a slash-only edit is not and must not. - Certify the storage fingerprint immediately after the artwork sweep succeeds and make the 4-object branding check non-fatal (reported in the task message), so a transient branding error cannot discard a completed catalog sweep and force it to repeat every boot. - Fail closed on conditional-task preflight errors in the task manager (previously fail-open ran the task), and retry transient settings reads in ShouldRun since the startup trigger fires once per process. - Track probe HEAD errors against a separate baseline so a flaky probe cannot consume the sweep's error budget. - Probe before counting: bulk mode skips the per-surface count(*) full scans entirely, and probe sampling drops ORDER BY random() (plain LIMIT answers "is the cache in this bucket" just as well). - Verify chapter thumbnails across a whole 500-file batch in one HEAD fan-out instead of per file, keeping the worker pool saturated. - Replace the 10 inline non-provider-scheme ARRAY literals in the enqueue query with the shared nonProviderImageSchemesSQL constant. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): guard bulk reset against degraded probes, certify only clean sweeps Address bot review feedback on the reconcile hardening: - A probe where more than half the HEAD requests error aborts the run: errored requests are excluded from the sample, so a partial outage could otherwise present a handful of surviving 404s as a ~100% miss rate and bulk-reset the catalog. Bulk mode additionally requires a minimum number of successful samples; thinned probes and tiny catalogs take the safe per-row verify path. - Track sweep errors separately from probe/branding errors (stats.sweep_errors) and certify the storage fingerprint only when the sweep completed with zero of them — skipped rows were never verified, so the next startup retries. Applied resets stay durable. - Give each ObjectExists attempt its own timeout so a stalled HEAD fails that attempt instead of pinning the retry loop to the run context. - Report branding assets checked (not just cleared) in stats.Checked. - Drop the dead settingsRepo/brandingSvc nil guards in cmd/silo and sync spec numbers with the implementation constants. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
4d99597966 |
feat(watchlist): honor provider list order by default (#352)
The watchlist catalog page silently returned the stored list order while its sort dropdown claimed "Date Added", and mirroring a provider's list order (e.g. MDBList) was off by default, so synced watchlists appeared in first-sync-time order with no way to tell what was happening. - Web: the watchlist source now uses the same source-order sentinel as collections — the dropdown shows "List Order" as the default, and an explicit "Date Added" pick sends sort=added_at instead of being stripped (previously indistinguishable from the default). - Server: on personal lists (watchlist/favorites) an explicit added_at sort now takes the source-order path, where added_at means "date added to the list"; the query executor path sorted by the library's created_at instead. History keeps the executor path since its ID loading ignores the sort. - watchsync: new connections default sync_watchlist_order_enabled to true, with a migration flipping existing rows to match. Providers without the provides_watchlist_order capability ignore the flag at sync time. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e96a8a0cf8 |
feat(search): binary-quantized embedder vectors — optional, default-on for fresh installs (#351)
* feat(search): binary quantization setting for the Meilisearch embedder
New server setting catalog.search.meilisearch.binary_quantized
(default false) threads into the embedder index settings
("binaryQuantized": true) and into the schema-version hash, so flipping
it closes the sync gate and mandates a rebuild in both directions —
Meilisearch cannot de/re-quantize an index in place.
With 3072-dimensional embeddings this cuts vector storage ~32x
(≈12KB → 384B per document), keeping the whole vector store in page
cache: rebuilds and hybrid queries get sharply cheaper. Hybrid search
(keyword + semantic) cushions the small relevance cost of sign-only
vectors.
The hash token is appended only when the flag is set, so indexes built
before this change keep their schema version while it stays off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(search): binary quantization default-on for fresh installs + admin toggle
- Migration seeds catalog.search.meilisearch.binary_quantized=true only
when no active catalog search index exists. Existing deployments stay
unset (= off): flipping quantization changes the index schema-version
identity, which closes the incremental-sync gate until a full rebuild
runs — that must never happen implicitly on upgrade. Fresh installs
have no index yet, so their first rebuild simply starts quantized.
- Search settings page gains the toggle with an explicit
"requires a full index rebuild" warning, a status row, and settings-
search keywords.
Prod benchmark (607.9k docs, 3072-dim vectors, N=10 medians, replicated):
hybrid 0.5 unchanged (7.5ms float vs 8.0ms quantized, within ±2ms
keyword-control jitter); pure semantic 9ms → 4ms; on-disk index 18G →
8.3G; rebuild duration unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(search): address review feedback on binary-quantized embedder
- Add catalog.search.meilisearch.binary_quantized to the restart-required
registry. The provider freezes BinaryQuantized into MeilisearchProviderConfig
at construction, so without this a toggle-then-rebuild in the same process
builds a quantized schema while the live provider still compares against the
old value and falls back until restart (Codex P2).
- Validate binary_quantized in HandleUpdateSetting, mirroring semantic_enabled.
A raw API write of a non-bool previously persisted unnormalized, then failed
CatalogSearchSettingsFromMap on load and silently reverted the entire search
config to Postgres defaults.
- Gate the binary_quantized token in the schema-version identity on
semanticEnabled: with semantic off the index has no embedders, so the flag
has no on-index effect and must not force a pointless rebuild. Stays
byte-identical to a pre-flag index. Covered by a new test.
- Clarify the seed migration comment (guard is "no active index", which also
covers Meilisearch-configured-but-never-indexed deployments) and the UI hint
(~30x smaller raw vectors, index roughly halves; only applies with semantic).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
3e340552a8 |
fix(search): raise Meilisearch task-wait timeout from 5m to 2h (#347)
Semantic (vector-embedded) index batches can take Meilisearch many minutes on modest hardware, and Meilisearch auto-batches consecutive queued document tasks — with rebuild_task_queue_depth tasks in flight the oldest task's completion covers the whole fused unit. WaitTask's 5-minute default deadline killed four prod rebuilds mid-batch (2026-06-29 x2, 2026-07-08 x2) with "context deadline exceeded", each time leaving incremental sync blocked on a stale schema version and an unbounded event backlog (1.6M+ events). The wait still exits immediately when the task reaches a terminal state (succeeded/failed/canceled); the deadline only bounds tasks genuinely still processing, where giving up guarantees rebuild failure and gains nothing. Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
a2ef26bece |
perf: root-cause fixes for endpoints still slow after #292 (NextUp, series badges, resume tail, subtitle fonts) (#350)
* docs(plans): root-cause analysis for endpoints still slow after PR #292 Five endpoint groups stayed slow after the home/Continue Watching/Latest latency work shipped: Resume (110s p95), NextUp (17s p95), Latest (17s), /Items, and the home sections routes. The caps and caches from PR #292 are live in the deployed binary; they bounded how many rows the loops touch but not what each underlying query costs. Documents the four confirmed root causes (4.3M stale completed-with-position progress rows + missing resume index, unbounded next-up anchor scan, per-episode series rollup fanout, two index-starved history/scanner paths) with live EXPLAIN ANALYZE measurements and the fix plan implemented by the follow-up commits. AI-use disclosure: analysis and doc produced with AI (Claude) assistance. * perf(catalog): bound the global next-up anchor scan to recent completions The completed_episodes CTE in buildListNextUpQuery derived per-series anchors from the profile's ENTIRE completed history — DISTINCT ON over 233k rows joined to episodes for the worst bulk-import profile, then a per-series LATERAL that scans every episode of a fully-watched series before yielding nothing. 648 slow executions in a 19h window, 44.7s worst; this drove /Shows/NextUp (17.1s p95) and the next-up injection on the native home sections aggregate. Global queries now derive anchors from the profile's nextUpAnchorMaxRows (500) most recent completed rows — an ordered index walk on idx_uwp_profile_completed, with the hidden-items exclusion and date cutoff applied inside the bounded scan so hidden/old rows never consume the anchor budget. A next-up rail surfaces ~24 series; the 500 most recent completions cover every series that can realistically rank on it. Series-scoped calls (the show-detail tile) keep the unbounded shape: they must anchor on the series' last completed episode no matter how long ago it was watched, and are naturally bounded by one series. Measured on the live worst-case profile with the exact generated SQL: 44.7s worst / ~2.6s avg before; 10ms after (together with the one-time stale-resume-point data repair applied directly to the deployment DB — see docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md). AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat,userstore): aggregate series watch-state rollup in SQL The series Played/UnplayedItemCount badge on list rails (per-library Latest, library browse, search results) and series detail pages was computed by materializing EVERY episode of every series on the page (episodeRepo.ListBySeriesIDs) and then batching per-episode progress+history lookups in 500-id chunks. A 50-series page of an episode-heavy library (Sports) expanded to 32,467 episode rows and ~65 sequential queries — measured 17-18s per /Items/Latest request, and PR #292's cached Latest fast path pays it on every response for series libraries. The same fanout made /Items?searchTerm=... slow whenever the result set was mostly series (Meilisearch itself answers in milliseconds). New optional store capability userstore.SeriesEpisodeRollupStore, implemented by PostgresUserStore as one GROUP BY e.series_id aggregate with semantics identical to the chunked path (episode availability via episode_libraries, hidden-items visibility on progress rows, completed-history fold, in-progress = not watched with position > 0 — verified value-for-value against the old semantics on a real 1,586-episode series). enrichSeriesListUserData and enrichDetailUserData use it when present; SQLite-backed stores and rollup query failures keep the existing chunked path as fallback. catalog.SeasonUserDataFromCounts pins the counts-to-DTO mapping to EpisodeRollupUserData. Measured on the live worst-case profile against the real 50-series Sports Latest page: ~17s of chunked round-trips before, 119ms in one query after. Part of docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md. AI-use disclosure: implemented with AI (Claude) assistance. * perf(catalog): bound superseded-episode completed walk to recent history The Resume / Continue Watching superseded-episode filter loaded a profile's *entire* completed history into memory on every request that contained an in-progress episode: CompletedProgressSnapshots paged user_watch_progress WHERE completed=TRUE with no upper bound. The 2026-07-06 slow-query comparison showed this surviving as a 60-116s Resume tail even after the in-progress index landed live, because the 4.3M zeroed Plex-import rows are still completed=TRUE and were re-walked every load. A completed episode can only supersede an in-progress one it was finished more recently than (the query gates on done_progress.updated_at > ip_progress.updated_at), so only completed rows newer than the oldest in-progress entry can matter. Compute that cutoff in SupersededEpisodeProgressIDs and pass it to CompletedProgressSnapshots, which — since the completed listing is ordered updated_at DESC — stops paging as soon as it crosses the cutoff. Import-heavy profiles whose back-catalogue predates their current in-progress items now stop on the first page instead of paging hundreds of thousands of irrelevant rows. Correctness is unchanged: no relevant superseding row is excluded. * perf(catalog): hard-cap superseded-episode completed walk at 5 pages The updated_at cutoff added in the previous commit bounds the completed walk on the relevance axis, but a very old in-progress entry sitting behind a large volume of newer completions could still page deep. Add a 5-page (2,500-row) hard backstop on top of the cutoff: normal profiles still stop on page one via the cutoff, and only the adversarial tail hits the cap. When it engages the tail of the completed set goes unscanned, so a superseded episode could momentarily survive on Continue Watching — we log a warning when that happens (with profile_id + rows scanned) rather than mis-filter silently, and it self-corrects once the stale in-progress entry ages out of the scanned window. * perf(playback): extract subtitle fonts in a single ffmpeg pass Embedded ASS/SSA font extraction spawned one ffmpeg process per font attachment, each re-opening the (usually CephFS-backed) media file. Anime releases carry 15-47 fonts, so the per-spawn file-open cost dominated and pushed GET /api/v1/stream/{sid}/subtitles/{track}/fonts to a 17-60 s plateau (p95 ~33 s in the live logs). Collapse the N spawns into one ffmpeg invocation that dumps every attachment to a temp dir (-dump_attachment:idx path ... -i file -map 0:t? -c copy), then read the files back. The file is opened once instead of N times, taking p95 from ~30 s to ~1-2 s with no change to output. Safety is preserved. The 32-attachment / 32 MiB caps still apply: attachment size is stat'd before read so an over-limit font never enters memory, and a watchdog polls the dump dir and kills ffmpeg if its on-disk output crosses the cap -- restoring the hard bound the old pipe-per-attachment reader enforced by killing at maxBytes+1, so a container with oversized "font" attachments can't fill the disk. Part of the slow-endpoint follow-up; see slow-query-analysis/subtitle-fonts-extraction-findings.md. * fix(review): report enforced font-byte cap; correct doc subtitle scope Address PR #350 review: - dumpFontAttachments reported the maxSubtitleFontBytes package constant in both over-limit errors instead of the maxBytes argument the caller passed, so the message misstated the enforced bound whenever a different cap was in effect (as the tests use). Interpolate maxBytes in both messages. - The root-cause plan claimed subtitle extraction was 'out of scope' while the branch actually optimizes /subtitles/{track}/fonts. Scope the out-of-scope note to subtitle *track* conversion and record the fonts single-pass work as deliverable 5. |
||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
76b10b822e |
feat(jellycompat): surface audio profile and spatial format on media streams (#344)
* feat(scanner): capture audio stream profile metadata Store ffprobe audio stream profiles in the existing audio track metadata JSON. This mirrors the video track profile field and is additive for clients that want to display or inspect codec profile details. * feat(catalogseed): carry audio profile in seed export records Keep AudioTrackRecord in parity with VideoTrackRecord, which already mirrors the video profile field, so seed exports do not drop the new audio profile metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(jellycompat): surface audio profile and spatial format on media streams Set MediaStream.Profile from the captured ffprobe audio profile and derive AudioSpatialFormat (DolbyAtmos / DTSX) from it, matching Jellyfin's case-insensitive substring semantics. Audio display titles now prefer the profile over the codec name (except the AAC "LC" profile), matching Jellyfin, so DTS-HD MA and Atmos tracks read correctly in clients like Infuse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Silo Contributor <silo-contributor@example.invalid> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
7f4f609948 |
feat(scanner): capture audio stream profile metadata (#337)
* feat(scanner): capture audio stream profile metadata Store ffprobe audio stream profiles in the existing audio track metadata JSON. This mirrors the video track profile field and is additive for clients that want to display or inspect codec profile details. * feat(catalogseed): carry audio profile in seed export records Keep AudioTrackRecord in parity with VideoTrackRecord, which already mirrors the video profile field, so seed exports do not drop the new audio profile metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Silo Contributor <silo-contributor@example.invalid> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
3df5ed4e09 |
feat(web): show audio stream profile in Media Info (#339)
Expose the optional audio stream profile field in the web API type and include it in the Media Info audio section when present. This mirrors the existing video profile row and is additive for servers that do not emit audio profiles yet. Co-authored-by: Silo Contributor <silo-contributor@example.invalid> |
||
|
|
0deff23985 |
fix(profiles): enforce per-account profile name uniqueness (#342)
* fix(profiles): enforce per-account profile name uniqueness Profile create and rename accepted any name, so one account could hold unlimited profiles all called "Laura" (every client allowed it too). Reject a create or rename whose trimmed, case-insensitive name matches another profile on the same account with 409 name_conflict. Scoping is per account by construction — the check runs against a single user's profile store, so different accounts can still each have a "Laura". Renames exclude the profile being updated, so re-saving a profile under its own name (e.g. avatar-only edits that resubmit the name) still works. Also reject whitespace-only names on create and rename; a name of " " previously passed the blank check. Additive-only per the v1 API rules: new 409 error code on existing endpoints, following the profile_limit_reached pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(profiles): store the trimmed profile name The conflict check compared trimmed names but create/rename persisted the raw input, so " Laura " could land with stray whitespace and render inconsistently. Normalize to the trimmed form before storage on both paths. Addresses the CodeRabbit review finding on PR #342. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(profiles): cover whitespace-only rejection and rename trimming Also document the check-then-write race in profileNameConflicts: the userstore backends carry no unique index on name, so concurrent creates can still race past the guard, same as profile_limit_reached. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
CoffeeKnyteandGitHub
|
021e54a03c |
fix(scanner): fix slow-scan regressions from #319 and #322 (#341)
* fix(scanner): stop classifying "other" content folders as extras Regression from #322 (trailers and extras for movies and series), which introduced the extrasDirKinds map. The extras directory classifier mapped the generic labels "other" and "others" to ExtraKindOther. These are not part of the Jellyfin/Plex extras folder convention the map claims to mirror, and they collide with real content-scope folder names. A library organized as "movies/other/<Title (year) {ids}>/<file>" tripped the depth-2 ancestor lookup in classifyExtraPath: every title two levels under the scope folder "other" was classified as an "other"-kind extra. Such files are partitioned out of primary root/group inference and matching, then deferred in processExtraFiles because their parent cannot resolve (they are the primary titles, not children of one). The result on one deployment was ~10k movies under a folder named "other" funneled through the slow extras path every scan (parent-unresolved deferrals at ~9.5/s), stalling the scan and freezing that scope for new/changed primary content. Remove "other"/"others" from extrasDirKinds. The ExtraKindOther kind stays reachable through genuine convention labels (extra/extras/interviews/ scenes/shorts). Add regression coverage asserting titles under a scope folder named other/others stay primary. * perf(scanner): rewrite identity-only changes without re-probing A pure identity/grouping change on an already-probed file — a root_assignment_changed or group_assignment_changed reason with nothing else — used to fall into the full update branch, which unconditionally ran ffprobe (probeFile) and then upserted every column, including probe columns, from the freshly built row. When a group-key or root scheme changes across the library (see #319), this reprobed nearly every file on the next scan: an incremental scan that normally takes ~1h ran 7h+ as a full-library ffprobe storm, even though the media bytes were untouched. Add a metadata-only update path in processFile: when identityOnlyUpdateReasons reports every reason is a root/group reassignment, rewrite just the derived identity columns via the new FileRepository.UpdateIdentity and skip ffprobe, OSHash, and marker fetch entirely. UpdateIdentity issues a targeted UPDATE of the root/group/identity and edition/presentation columns only, mirroring Upsert's column handling, and leaves probe data, file bytes/mtime/hash, subtitles, chapters, markers, and content/episode/extra linkage intact. The stored group key converges to the recomputed value on the next scan, so the file takes the unchanged fast-path thereafter — without a probe storm. The shared identity-column population is extracted into populateScanIdentity so the full path and the metadata-only path stay in lockstep. Verification: unit test for the identityOnlyUpdateReasons classifier; a DB-backed test (skipped without SILO_TEST_DATABASE_URL) asserting UpdateIdentity rewrites grouping while preserving probe/linkage columns; the UPDATE statement was also exercised against the live schema inside a rolled-back transaction. * fix(scanner): harden identity fast path and extras scope classification Review follow-ups for the two scan-regression fixes on this branch, addressing both Codex review comments on PR #341 plus adversarial-review findings. Identity fast path (processFile/UpdateIdentity): - Gate the metadata-only path on existing.ExtraID == "": a row still linked as an extra reaching processFile is being reclassified as primary, and only the full upsert clears extra linkage; UpdateIdentity would have frozen it out of matching forever (match backlog filters extra_id IS NULL). - Gate on existing.FileHash != "": the full path backfills the OSHash and fetches hash-keyed S3 intro/credits markers, which no later scan reason would repair; hash-less legacy rows now take the full path once instead of silently losing that repair channel. file_hash is added to the scan-state row shape to support the gate. - Clear match_suppressed_at like every other scan write, so files with fresh identity re-enter the match backlog (suppression is documented as lasting "until retried or seen by a new scan"). - Write media_folder_id, mirroring Upsert's ON CONFLICT reassignment. - Return ErrFileNotFound when the row vanished mid-scan (concurrent delete) and fall through to the full upsert path instead of surfacing a per-file scan error. - Return only the row id instead of RETURNING all ~75 columns: the fast path fires once per file during library-wide grouping migrations, and dragging the track/chapter JSONB payloads along for a million rows dominated the cost of the path built to be cheap. - Extract identityColumnDefaults shared by Upsert and UpdateIdentity so the defaulting rules cannot drift, and drop the no-op editionConfidence indirection copied between them. - Use populateScanIdentity in the new-file insert path too; it still carried a verbatim copy of the extracted block (with a provably dead existingByPath lookup). Extras classification: - Restore "other" to extrasDirKinds: it is part of both the documented Jellyfin and Plex extras-folder conventions (the removed-label fix overshot and broke "movies/<Title>/Other/<file>" libraries, ingesting their extras as bogus primary titles). "others" stays removed - it is in neither convention. - Replace label removal with the structural guard the PR had deferred: classifyExtraPath now rejects a supplemental-named directory sitting at library-scope depth (the dir, any supplemental ancestor, or the first non-supplemental ancestor is a configured library root). This fixes the original "movies/other/<Title>" defer-storm generically, covering every convention label (shorts, scenes, extras, ...) used as a content-scope folder. - Scope extras parent binding by folder.Paths instead of the walk roots, so a subtree scan targeting a single movie folder still binds that movie's own extras instead of deferring them. Tests: eligibility-gate unit tests, scope-guard classifier cases (convention Other/ inside a title binds; scope-level other/shorts stay primary), and the DB-backed UpdateIdentity test now also covers folder moves, suppression clearing, and ErrFileNotFound. Full scanner suite ran green against a migrated scratch PostgreSQL 17 container. * refactor(scanner): simplify extras scope guard to title-folder rule Replace the ancestor-walking supplementalDirAtScopeDepth loop with the plain rule it was approximating: a convention-named directory counts as an extras dir only when it sits inside a title folder — it must not be a configured library root or directly under one. Same outcome for the layouts that matter (movies/other/<Title> stays primary, <Title>/Other classifies), less machinery. * test(scanner): assert all rewritten identity columns in UpdateIdentity test * fix(scanner): make extras scope classification structure-aware The title-folder rule from 53632022 anchored on library roots, so it missed both directions: chained convention dirs at the root ("movies/extras/behind the scenes/clip.mkv") classified as extras with an unresolvable parent (deferred forever), and category folders nested below the root ("movies/4K/other/<Title>/") still misclassified their titles. Replace the root-distance heuristic with the structural property that actually distinguishes the two cases: a convention-named directory only counts as an extras dir when its owner (first non-supplemental ancestor) is a title folder — a directory that holds media of its own. The new extrasClassifier derives that from the scan's walked path list (no extra I/O): movie folders must hold a file directly beside the extras dir; series folders may hold episodes one level down in season folders (media hiding inside a folder's own extras dirs doesn't count). Library roots never qualify. Watch-event scans, which have no walked list, probe ownership with bounded os.ReadDir instead. This handles title folders at any depth below the root and keeps scope/category folders primary at any depth, with two known edges: a title folder holding only extras (its media file missing) stays primary until the file appears, and a mixed dir holding both loose media and a category folder degrades to deferral, never wrong linkage. resolveExtraParent's inline supplemental-chain walk is extracted into the shared firstNonSupplementalAncestor. |