codex/bound-transcode-segments
139
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 SuspenseandGitHub
|
6e105af256 |
fix(catalog): preserve access filters for match-any queries (#563)
* fix(catalog): preserve access filters for match-any queries * test(sections): report seasonal fixture cleanup failures |
||
|
|
e77d9e933c |
fix(catalog): bound Next Up anchors by distinct series, not rows (#593)
* fix(catalog): bound Next Up anchors by distinct series, not rows The global Next Up query capped its anchor scan at the 500 most recently completed rows (nextUpAnchorMaxRows, #350). Bulk mark-watched writes hundreds of completed rows with the newest timestamps, so one series could flood the whole window and evict every other series from the rail - observed in production wiping a user's Next Up row entirely. Replace the row-capped CTE with a recursive skip-scan over idx_uwp_profile_completed that emits the newest completed row of each not-yet-seen series and stops after nextUpAnchorMaxSeries (96) distinct series. A compound (updated_at, media_item_id) cursor keeps the walk total when bulk writes share one timestamp across series. The series-scoped branch (show-detail tile) keeps its unbounded shape. DB-backed regression tests cover the flood shape (600 same-timestamp rows of one series must not evict others) and same-timestamp anchors across series; the flood test fails against the previous query. Fixes #592 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): continue Next Up anchor walk until enough eligible series found The 96-series walk budget counted visited series, but eligibility (next episode exists with a present file, no newer partial progress) is only decided downstream, so 96 consecutive ineligible anchors - all caught up, unavailable, or blocked - still emptied the rail and silently capped /Shows/NextUp pagination at whatever survived one walk. Run the walk in batches: each batch keeps the 96-series budget, reports its frontier (compound cursor position, seen-series array, rows walked), and ListNextUp resumes the next batch below that frontier until the requested limit is filled, history is exhausted (frontier short of the budget), or a 10-batch runaway guard trips (logged, never silent). Batches emit anchors in strictly descending (updated_at, media_item_id) order, so appending preserves rail order. Series-scoped queries keep their single-shot shape. DB-backed regressions: 100 fully-watched series must not hide older series with eligible next episodes (fails on the single-batch code), and an all-caught-up profile returns empty without spinning to the batch cap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): anchor Next Up on the highest episode, not the highest ID Review follow-up to the anchor walk. Two issues remained after the batching fix. The walk ordered anchors by (updated_at, media_item_id) and then by (season, episode). media_item_id is unique, so no two rows ever tie on that pair and the season/episode clause was unreachable. For a bulk mark-watched series — every row written with one timestamp — the anchor was therefore whichever content_id sorted highest. With production-shape 18-digit IDs where season 2 was scanned before a season-1 backfill, season 1 sorts higher, so the rail surfaced s01e04: an episode following one the user had already watched. Choosing the series and choosing its anchor episode are now separate steps. The walk still advances on the compound cursor, which is the total order it needs; a lateral then picks the highest (season, episode) among that series' rows sharing pick.updated_at, matching what the series-scoped branch already did. Pinning updated_at instead of re-sorting keeps it an index probe, and both rows carry the same updated_at, so cursor order and the reported CompletedAt are unchanged. Also adds the index the compound cursor needs. idx_uwp_profile_completed stops at updated_at, so the media_item_id half of the seek was a filter and every step re-read the rows tied on one timestamp — the exact shape a bulk mark-watched profile has. Measured by dropping and recreating the index around the shipped query on a synthetic 40k-row profile: 341ms -> 224ms, same plan shape otherwise. Verified against Postgres 18.3. The new DB regression seeds the scanned-out-of-order ID shape and fails on the previous ordering (returns s01e04, wants s02e04); the SQL-shape test pins that media_item_id can no longer gate the episode choice. Part of #592 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2dedf3ff26 |
feat(metadata): user-triggered trailer refresh with weekly per-item cooldown (#531)
* feat(metadata): user-triggered trailer refresh with weekly per-item cooldown
Adds POST /api/v1/items/{id}/trailers/refresh so any viewer with access to a
movie or series can ask the server to fetch its remote trailers, bounded by a
one-week per-item cooldown enforced server-side.
The cooldown lives in a new nullable media_items.trailers_refresh_requested_at
column rather than the refresh debt queue, whose last_attempt_at evaporates on
success (MarkTargetSuccess deletes the row when the reason mask clears). The
gate is a single UPDATE that writes NOW() only when the stored timestamp is
NULL or older than the window, so concurrent viewers cannot both win it; a
losing caller reads the stored timestamp back to compute next_allowed_at.
MetadataService.RequestTrailersRefresh resolves the per-library trailer_kinds
allow-list first: a non-nil empty map means every containing library disabled
remote videos, which answers "disabled" without consuming the cooldown slot
(a nil map is allow-all and must not short-circuit). On winning the gate it
reuses startOnDemandMetadataRefresh, whose scheduled mode merges fill-empty,
so this non-admin trigger cannot clobber unlocked admin edits while found
videos still persist.
The handler checks item access before calling the service, so an unauthorized
caller can never burn an item's slot, and rejects non movie/series types since
those detail responses never carry videos. cooldown and disabled are expected
client-rendered states and answer 200; 429 is reserved for the per-user
in-memory limiter.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): release trailer-refresh slot on failed refresh; resolve episode ids to 400
Three review findings on the viewer-facing trailer fetch.
The weekly per-item slot was consumed unconditionally on winning the gate,
but the refresh it started ran detached and only logged on failure — nothing
ever put the slot back. A brief TMDb outage therefore answered 202 queued,
failed 30s later, and then answered cooldown for seven days over work that
never happened. The repository gains an equality-guarded release
(trailers_refresh_requested_at = NULL only while it still equals the
timestamp this request wrote, so a later claim is never clobbered), and
TryClaimTrailersRefresh now RETURNINGs the timestamp it stored so a winner
holds the key to its own slot. startOnDemandMetadataRefresh splits into a
claim step and runOnDemandMetadataRefresh, which takes an optional failure
hook; only the trailer path passes one, so the existing callers are
unchanged. A timeout counts as failure. A refresh that succeeds but finds
nothing still keeps the slot — that semantics was chosen deliberately.
The in-process dedup claim (shared with the item-detail view's stale nudge)
silently dropped the start while the slot had already been consumed, so the
caller was told queued for a refresh that never began. It is now taken
before the durable slot: a request landing while an equivalent refresh is
already in flight reports queued without consuming the slot, which is both
honest and retryable if that refresh fails.
Real episode and season content IDs answered 404 rather than the contracted
400, because neither is a media_items row and GetByID queries media_items
alone. The handler now falls through to the same season/episode lookups
HandleTranslateOnView uses, authorizing through the parent series, so a
genuine episode ID reports unsupported-type and only unknown content 404s.
The type-check test no longer fabricates a MediaItem{Type: "episode"} row
that production never writes; it covers the types that do exist as
media_items rows, with the episode and season paths tested through the
lookups.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): address PR review on the trailer refresh action
Six review findings on the viewer-facing trailer fetch, all verified against
the current code before changing anything.
Durable claim no longer rides the request context. A cancellation landing
after Postgres commits the gate UPDATE but before pgx returns would consume
the item's weekly slot with no refresh started and nothing holding the
timestamp needed to release it. The claim now runs on
context.WithoutCancel with its own deadline, mirroring the release.
The cooldown gate retries once when the follow-up read finds the slot free.
Classification spans two statements, so a concurrent failure-release can
land between them; the old code reported that as a cooldown with no
next_allowed_at while the slot was in fact free. A NULL read now retries the
claim, and the doubly-lost case answers "queued" (an equivalent refresh is
running) rather than an undateable cooldown.
A failed item_videos write now releases the slot. mergeAndPersist logs and
continues when the write fails, so the refresh reported success and the
viewer was locked out for a week having stored nothing. A context-scoped
observer, installed only by this action, surfaces that failure to the
existing release hook.
Winning the gate also records durable refresh debt, so a restart that kills
the detached goroutine leaves work the refresh worker picks up instead of a
consumed slot and no fetch. Uses a new reason bit rather than the generic
failure reason: nothing is wrong with the item, so it must not sit in the
failure band ahead of real debt or count as a failure in operator metrics.
Any library lookup failure now degrades the video-kind scope to unknown. An
item in two libraries where one resolved with trailers off and the other
could not be read reported "disabled" — a guess made on behalf of a library
that might be the one enabling trailers. A library that is genuinely gone is
still skipped.
Adds GET /api/v1/items/trailers/capability, following the existing
per-subsystem probe convention. The action route is registered conditionally,
so "this build has the feature" is not the same question as "this deployment
serves it", and a 404 on the POST is indistinguishable from a missing item.
The probe is registered unconditionally and answers refresh:false when
unwired.
Not changed: content-ID canonicalization mid-refresh stranding the cooldown
on the old row. The re-anchor path is manual-refresh only and this action
runs in scheduled mode, so only local-skeleton promotion can fire, and the
rename carries the timestamp and the debt row to the new id along with
everything else.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style(web): format overlays schema after merging main
The line came in over-length from main's card_overlays merge and the Web
CI format check runs prettier across all of src, not just changed files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(metadata): address second review round — recovery-debt lease, locked-videos preflight, shared limiter
The restart mitigation added in the first round reintroduced two of the
problems it was closing, and the reviewer was right to push again.
Lease the recovery debt behind the fast path. The row was enqueued due
now, so refresh_metadata could claim it while the detached goroutine was
still running the same refresh — RefreshScheduledTarget does not consult
the in-process claim, so both would fetch the item at once. It is now due
5 minutes out, comfortably past the 2-minute on-demand timeout, and the
goroutine settles the row on success so it fires only when the fast path
really did not finish. Settling clears just the trailers-requested bit,
keeping any real debt the item still carries.
Release the cooldown after a failed recovery. A recovery runs in a worker
that never saw the claim, so a failure left the viewer blocked for the
week having stored nothing. RefreshScheduledTarget now adopts the claim
when the debt row carries the trailers-requested reason, reading the
stored timestamp so the release stays equality-guarded, and hands the
slot back on the same failures the fast path's hook covers — including a
videos write that failed and was only logged.
Preflight the videos lock. locked_fields containing FieldVideos makes
mergeAndPersist skip the item_videos write, so the refresh "succeeded"
and kept the cooldown while never being able to save trailers. It now
answers disabled before consuming the claim; reusing that status rather
than adding one is deliberate, since clients treat an unknown status as a
dead end and "trailers cannot be fetched for this item" is what disabled
already means to a viewer.
Use the shared limiter. A private MemoryLimiter gave every instance an
independent per-user allowance on Redis deployments, and the per-item
cooldown cannot compensate — it bounds one item, while this budget bounds
how many distinct items a user can start refreshes for. The action now
takes the middleware's configured limiter, with namespaced keys, and
falls back to a private one only when rate limiting is off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
9aab2ead57 |
feat(settings): add user-facing device settings (#527)
* feat(settings): let users manage device settings across their devices
Adds the server half of the user-facing device settings screen: a viewer can
see the devices they watch on and change settings for any of them from
whichever device they are holding, and the household parent can do the same for
everyone on the account.
No schema change. user_devices and user_setting_values are already keyed
(user_id, profile_id, device_id), and both list queries are already
account-wide, so this is authorization plus routes.
Two identity widenings on the canonical settings API, each behind a guard:
- A caller may name a device_id other than the request's own. Authorized
against user_devices for that profile, which is why DeviceExists lands
first: completeIdentity validated an identity's shape but never that the
device belonged to the caller, safe only while the id came from the header.
- A household parent may name a profile_id other than their own. Guarded by
canManageHousehold, extracted from ProfileHandler so profile management and
settings management cannot drift apart. Existence resolves through the
caller's own store, so a foreign profile is 404 and the cross-account
boundary holds for free.
Both default to today's behavior when the parameter is absent, so existing
clients need no change.
New self-service routes: GET /devices, DELETE /devices/{id}, and
DELETE /devices/{id}/settings. The list filters to the calling profile in the
handler — ListDevices is account-wide by construction in both backends (no
WHERE at all in the per-user SQLite), so a passthrough would have shown every
household member's devices to everyone. ?scope=household is opt-in and guarded.
Also fixes a bug the widening exposed: registerWritingDevice fired on every
device write, so writing to another device — or on another profile's behalf —
would have registered the actor's browser under the target, inventing a device
nobody holds.
Cross-profile and admin mutations are audited. The record carries identity
only, never the value, for the same reason user_settings.changed does: admins
receive other accounts' events. Ordinary self-service writes are not audited —
a trail that records everything answers nothing.
Part of #215
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(settings): add the user-facing device settings screen
Adds "Your devices" under Settings: a searchable device list and an editable
detail pane, so someone can fix how Silo behaves on any device they watch on
without borrowing that device. The household parent gets an "Everyone" switch
covering every profile on the account.
Layout is master-detail, and holds at eleven devices: fixed-height rows
carrying a name, when it was last used, and the one number that matters — how
many settings differ there. A device with nothing changed shows a dash rather
than a zero, so "which one did I change?" is answerable by scanning. Rows group
by recency, or by person in the household view.
Settings are grouped by what they affect — Picture, Sound, Subtitles,
Episodes — rather than in manifest order, and no raw key is ever shown: labels,
descriptions, controls, bounds and options all come from the contract. A test
asserts every device-scoped key lands in exactly one group or is deliberately
hidden, so a key added to the manifest cannot silently vanish from the screen.
Values round-trip as typed JSON rather than through strings, unlike the admin
console: a slider re-parsed from text is a hazard on a screen a viewer drives.
Policy caps are explained rather than hidden. A capped setting renders only the
permitted options and says which value the household limit displaced; a locked
one says so instead of presenting a disabled control with no reason.
Acting for someone else is stated, never implied — a persistent banner, and
reset actions that name the person ("Use Robin's setting"). The household view
also states what it does not show: this is how Silo is set up per device, not
what anyone watched.
Two fixes the work turned up:
- effectiveSettingsQueryKey was namespaced by active profile only, so reading
another device's values would have collided with the current device's cache
entry and served one device's settings as another's.
- The settings shell caps content at max-w-3xl, which is right for a single
column of rows and squeezes a two-pane page. Pages that manage their own
layout now opt out.
Part of #215
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(settings): filter the household device list by profile
Adds profile chips above the device list in the household view: Everyone, then
one per person, each with a device count. Picking a person narrows the list to
their devices; picking the active chip again clears it.
The chips only appear in the household view, where more than one profile is on
screen. A viewer looking at their own devices has exactly one profile, so a
filter with a single option would be chrome that explains nothing.
Three details the interaction needs to be honest:
- Counts come from the unfiltered list, so a chip keeps saying how many devices
it would reveal instead of collapsing to zero once another chip is active.
- Grouping falls back to recency once a person is chosen, because a person
heading would only repeat the chip above it.
- The detail pane follows the filter. Leaving someone else's device open while
the list shows another person would make the list and the pane disagree about
whose settings are being edited — the one thing this screen cannot be vague
about. Leaving the household view clears the filter for the same reason.
The chip's count sits in its own element, so its accessible name is set
explicitly: without it a screen reader announces "Everyone3".
Part of #215
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(settings): lead the profile filter with the viewer's own chip
Seeding a realistic eight-profile household made the ordering problem obvious:
chips were in device-arrival order, which put the person actually using the
screen last. Their own profile now comes first and the rest sort by name, so a
chip stays where it was last seen rather than moving as devices are used.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(settings): make the device screen work on a phone
The screen was built two-pane and stacked those panes vertically on narrow
viewports, which is the wrong shape for a phone: the device list ran past a
thousand pixels before the first setting, so reaching "turn HDR off" meant
scrolling through every other device to get there. The whole page measured 4726
CSS pixels — 5.6 screens — for four devices.
Below xl the list and the settings are now two screens rather than two panes.
Picking a device swaps to it and a back control returns; the page header and
scope switch belong to the list screen, and the detail screen's own header says
which device it is about. The list page is 1050px, and scroll position resets on
each swap so a tap does not land mid-settings.
Touch targets were 32-36px throughout. Rows, chips, the scope switch, the reset
link and the header actions now clear 44px on a phone and keep their compact
desktop sizing from xl. Device rows carry a chevron below xl, because there they
navigate rather than select in place. The device search input goes to 16px on
mobile — iOS Safari zooms the viewport for anything smaller and does not zoom
back out.
Profile chips wrapped to three lines at eight profiles and pushed the list off
screen; they scroll horizontally on one line instead, the same trade the
settings shell's own mobile tab bar makes. Switches now sit beside their labels
rather than below, saving a row on each of ~18 toggles, while selects and
sliders still take the full width they need.
Two fixes the pass turned up, neither mobile-specific:
- playback.max_bitrate_kbps is declared as an integer range with a select
control and no members, so it rendered as a dropdown with one blank entry —
unusable, and silent about the value it was storing. It now offers real
bandwidth choices bounded by the definition's own range, and keeps a
non-preset stored value selectable.
- Select triggers had no accessible name, announcing as bare comboboxes.
Forget is destructive and rare, so it no longer sits as a full-width sibling of
the common action.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(settings): keep the device list usable at hundreds of devices
A real account carries 260 devices; the test fixtures had 14. Every browser
profile, private window, reinstall and test build registers a device identity
and nothing prunes them, so the list grew without limit — 13,681 CSS pixels,
sixteen screens, and the settings themselves never came into view. That is the
same failure the mobile pass just fixed, at a scale the fixtures never showed.
Three changes:
- The list is a bounded scroll area rather than an unbounded column. The page
is now ~1,300px whatever the device count, and section headings stick while
scrolling so the recency or person grouping stays legible.
- Devices nobody has used for 90 days that carry no settings of their own
collapse behind "Show N unused devices". Over half of the real fleet is that:
one-off sessions that never changed anything. The current device and anything
with settings always stay visible, however old.
- Search spans everything including the hidden tail, because searching means
looking for something specific and hiding a device from its own name would
read as the device having vanished.
The 90-day threshold matches the settings contract's own rule for removing
empty device records, which is specified but not yet implemented server-side —
until it is, this keeps the screen usable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(settings): offer the whole bandwidth range the contract allows
The bandwidth cap stopped at 40 Mbps because the ladder behind it was a
hardcoded list I wrote, not anything the contract said. The definition allows
up to 200,000 kbps, and remuxed 4K HDR or an untouched Blu-ray rip needs well
past 40 — so the picker was silently capping people below what their own server
could already send them.
The ladder now runs to the definition's own ceiling. Its low end mirrors the
in-player quality switcher, so a cap chosen here lines up with what the player
offers mid-playback, and entries outside a definition's declared range are
filtered out as before.
Also stops duplicating the label format a third time: the player's
formatQualityBitrate is now exported and reused, since both surfaces pick from
the same ladder and should not disagree about how to spell a number.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* style(settings): use the app's overlay scrollbar in the device list
The bounded device list kept the browser's default scrollbar, which reads as a
heavy grey slab against a rounded dark panel. The app already has
.overlay-scroll for exactly this — a thin, low-contrast thumb over a
transparent track — and it now sits in the panel's gutter rather than flush
against the rounded edge.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* perf(settings): stop effective-values from stalling on live catalog language scans
GET /settings/values/effective computed suggested language values with
three sequential full-catalog scans on every request. On a 439k-item
deployment that took ~25s, dominated by the subtitle listing whose UNION
deduplicated ~5M unnested track rows.
- Deduplicate each subtitle arm before merging (UNION ALL of two
DISTINCT arms instead of UNION across all rows) and bound the result
with the facet LIMIT: 22.8s -> ~8.5s of per-arm work on that catalog.
- Cache the observed lists per (list kind, access scope) for 15 minutes
and collapse concurrent misses with singleflight.
- Run the three lookups concurrently, and cap a cold-cache wait at 2s:
the response ships with the contract floor while the detached scan
finishes and fills the cache for the next request.
Part of the device settings screen work; the regression itself shipped
in #526.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(settings): replace track-language catalog scans with picker free entry
Deployment-observed suggestions now decorate catalog.metadata_language
only — original_language is one indexed DISTINCT scan (~0.2s on a 439k
item catalog) and feeds the metadata-exceptions panel, where observed
data is load-bearing. The audio and subtitle track listings (up to tens
of seconds of media-file walking) are no longer queried for settings at
all, which also removes the TTL cache and singleflight added to manage
them.
Those pickers keep the contract's authored floor and gain an explicit
escape hatch instead: a shared LanguageSelect with an "Other…" entry
that accepts a BCP 47 tag, previews the resolved language name, and
refuses invalid tags. The settings are open language_tag values, so a
typed tag needs no server change, and a stored off-floor value already
renders through the current-value merge. The device screen hides the
free entry when policy pins permitted_values.
The browse facet queries keep the subtitle UNION ALL rewrite from the
previous commit; catalog browse still lists observed track languages.
suggested_values on playback.audio_language and
playback.subtitle_language now returns only the contract floor plus the
stored value — Android and Apple pickers need the same free-entry
affordance as follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
QuickandGitHub
|
73488d1bfa |
feat(metadata): add original-language preferences (#526)
* feat(metadata): add original-language preferences * fix(settings): show metadata exceptions immediately * docs(settings): add metadata language screenshot * fix(settings): make language exceptions responsive * fix(settings): standardize language display names |
||
|
|
dc4b9a0909 |
feat(settings): add the cross-platform settings contract and its manifest (#479)
* docs(settings): define the cross-platform settings contract Turns the audit in #376 into a decision-complete design for how user settings work across the server, bundled web client, Apple clients, and Android clients. Today there are three partial contracts - the server registry, the web client's own manifest, and independently owned key constants in each native client - and they have measurably drifted. The root enabler is that keyUsesUserScope returns true for any unregistered key, so a client can invent a production setting unilaterally and the server stores it as an unvalidated string. The design decides: Ownership. Every production user-facing setting needs a server-owned manifest entry, even when the value is stored only on one client. The single exception is private local.<client>.* diagnostics, bounded by five conditions. Types and scopes. Native JSON values instead of strings. Five remote scopes plus client_local, and each definition declares its own resolution order rather than inheriting a global precedence. Preferences versus restrictions. internal/policy already resolves max_playback_quality and metadata-language limits over the same controls this contract resolves preferences for. Definitions declare constrained_by, the effective response reports the permitted value alongside the user's stored one, and a mutation exceeding a restriction is stored rather than rejected - a capped 4K preference should take effect the day the cap lifts, not be destroyed by it. Compatibility. Widening a scope, adding an enum member, or widening a range is additive and revision-tagged; narrowing anything needs a new key. introduced_in is a manifest revision attached to individual enum members and scopes, not just whole definitions, so a newer client never offers a choice an older server will reject. Rollout. One coordinated breaking release, with no compatibility shim, projection, or client fallback. After the cutover no future setting requires coordination. No settings version check goes in the authenticated middleware and nothing returns 426: deleting the old routes already produces the break, and a gate would be more code in four repos for the same outcome while permanently coupling every endpoint to one subsystem's versioning. Scope placement. Appearance and date/time move from account to profile scope. Account scope was an artifact of pre-profile storage; leaving it there means a household shares one theme and text size, and any non-child profile can restyle everyone else. Read path. Batched context resolution, index requirements, a session-snapshot rule, and a no-regression benchmark gating storage consolidation - profile_series resolution is per-item, so a season view would otherwise issue one request per episode. Verified against the current server, Apple, and Android implementations. Two findings shape it: the unknown-key extension bag is real, and v1 scope reads NOT LOCKED, so removing the legacy surface needs no amendment if it lands before lock. Related to #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical settings contract manifest First implementation step for the cross-platform settings contract (#376). Adds the artifact everything else depends on: the manifest, its JSON Schema, the object value schemas, and a Go loader that validates the whole thing at load time. No routes, no storage, no behavior change — nothing reads this yet. contracts/settings/v1/ holds the artifact at a stable path because clients vendor it and generate bindings from it. The embed directive has to sit beside it (go:embed cannot reach outside its own directory), so that directory is a tiny Go package containing nothing else; loading and validation live in internal/settingscontract. 38 definitions: 35 remote, 3 contract-known client_local. That covers every key the legacy registry accepts, every unregistered key the extension bag was silently accepting from the web client, every unregistered device key Android writes, and the profile preference columns that become settings. Registering the previously-unregistered keys is where the drift shows up, and the manifest records each case in a notes field: - ui_theme, ui_text_scale, ui_text_weight, ui_high_contrast, ui_custom_theme_vars, and ui_custom_css reached the server only because keyUsesUserScope returns true for any unregistered key. They are now typed, renamed to the dotted convention every other key uses, and moved to profile scope per the design. - player.match_frame_rate and player.sleep_timer_default_minutes are written by Android against a server that does not register them, so every write and reset is currently rejected. Registered. - player.next_up_prompt_seconds is Android's alias for playback.next_up_prompt_seconds and does not become a definition; the test matrix pins it as a migration alias. - player.playback_speed is capped at 3.0, matching the server rather than Android's 4.0. - subtitle_appearance becomes playback.subtitle_appearance. Every other canonical key carries a domain prefix, and preserving accidental key names is an explicit non-goal of the design. Validation is deliberately stricter than the schema can express. Beyond shape, it enforces that a resolution order ends in "default", that it only resolves scopes the definition allows, and — the one most likely to bite — that every writable scope is actually read, so a setting cannot accept writes at a scope it will never honor. Defaults are validated against their own value schema, so a default that violates its own range or enum fails at load. Revision tags are checked to never run ahead of the manifest revision, which is what makes revision-aware client filtering trustworthy. Ceiling and floor policy constraints are rejected on unordered types, where capping would silently do nothing; playback.preferred_quality's enum is therefore ordered ascending. ValidateValue is the single validation path, so the mutation endpoint, the migration, and the manifest's own default checks cannot diverge later. Numbers decode through json.Number so an integer setting rejects 30.5 rather than truncating, and object values validate against their referenced JSON Schema instead of accepting arbitrary JSON the way validateJSONSetting does today. Canonicalization implements RFC 8785 over the value domain the contract uses: sorted keys, no insignificant whitespace, ECMAScript number formatting. The digest is the ETag, and PublicBytes strips maintainer notes so the served manifest never carries internal commentary. Promotes santhosh-tekuri/jsonschema/v6 from indirect to direct. Verification: 124 tests pass across 16 cases; golangci-lint clean; make verify-local-paths passes. Two failures in internal/api/handlers (TestRemoveJellyfinCompatWebDisablesWebSetting, the playback v3 seek recovery test) reproduce unchanged on main and are unrelated. Part of #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): give ui.theme a device override Theme joins text scale, text weight, and high contrast as a profile default with an optional per-device override, resolving profile_device -> profile -> default. The right theme is partly a function of the screen and the room — a light theme on a phone in daylight, a dark one on a TV at night — which is the same reasoning the other three appearance keys already used. All four appearance settings now cascade consistently, which also means one rule to explain in the UI rather than "these three follow the device, that one does not". ui.custom_theme_vars and ui.custom_css stay profile-wide. They are authored styling rather than a contextual preference, so a profile's custom tokens still apply on top of whichever theme a device resolves to. Recorded in the definition notes because it is a visible consequence: vars tuned against a dark theme will sit on top of a light one if a device overrides the theme. Widening those to profile_device later is an additive revision bump if it turns out to matter. Part of #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(web): tag local appearance caches with their owning account The theme, text scale, text weight, high contrast, custom theme variable and custom CSS caches in localStorage were untagged, so on a shared browser a second account inherited the first account's appearance: with no server value of its own, every fallback resolved to whatever the previous account had stored, and the leftover `silo-theme` key also suppressed the admin-configured default theme for the new account. DateTimeFormatProvider already solved this by stamping its cache with the authenticated user id and refusing another account's values. Extract that mechanism into `createOwnedCache` in utils/storage.ts (where key namespacing lives) and put all three groups behind it, so appearance and custom theme get the same protection instead of a third copy of the rule. - Each group carries its own owner stamp. A shared stamp would be unsafe: the groups are written by hooks nested inside each other, and effects run inner-first, so whichever hook stamped first would vouch for the other's still-stale values. - A null owner (auth bootstrapping, or signed out) still trusts the cache, which keeps the warm start and the login screen's last look. - An unstamped cache is not trusted once an account is known, so existing users take a one-time appearance reset on first load rather than a chance of seeing someone else's settings. - When a foreign cache is detected the values are dropped and the empty cache is handed to the new account, so a later single save cannot re-trust the rest of the previous account's state. Owner is the user id because /settings is user-scoped server side; it lives in one helper (`appearanceCacheOwner`) so it can be widened if appearance moves to profile scope. `shouldLoadApiTheme` is gone: it had become a synonym for `appearanceCacheOwner(...) !== null` with no callers left. Part of #376 AI-use disclosure: implemented with Claude Code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): make the settings contract enforceable and fix the appearance cache The contract manifest landed as a document nothing checked. This makes it a mechanism, and fixes the one defect in the change set that hurt users on merge rather than at cutover. Web appearance cache. useTheme cleared the cache for any account whose stamp did not match and never repopulated it — the only writers were the four user-action setters — so every upgrading user lost their warm start on every load, not once, and x-large-text and high-contrast users lost theirs too. The owner-stamp protocol is replaced with per-account key namespacing (`silo-theme:7`): a foreign value is absent rather than present-and-distrusted, so nothing has to be deleted, the first account keeps its warm start, and there is no shared stamp for a second tab, a stale debounce timer, or an out-of-order effect to race on. Widening ownership to profile scope, which this manifest requires, is now a change to appearanceCacheOwner alone. Adds the API-to-cache mirror useTheme was missing, cancels pending debounced writes across an account change, and re-seeds provider state during render so no frame paints the previous account's look. Canonicalization. writeCanonical used json.Marshal, which HTML-escapes < > and &, and canonicalNumber used Go's 'g' format — both diverge from RFC 8785, so the first label containing an ampersand or bound below 1e-4 would have forked the server's ETag from every conforming client. Output is now byte-identical to ECMAScript String() across the edge cases, verified against node. The ETag also covers the value schemas, which decide what the server accepts and previously could change while the tag stood still. All four derived representations are memoized; a conditional GET no longer costs a full parse and re-serialize. Validation. strictUnmarshal's decoder.More() answered false for a stray ] or }, so `true]` validated as a boolean. Enum matching compared fmt.Sprintf tokens, so the string "3" satisfied an integer member. Declared steps were never enforced. The language pattern rejected tags both mobile platforms emit unprompted (en_US, ca-ES-valencia, ar-EG-u-nu-latn) and never normalized case, so en-US and en-us were two rows for one preference; NormalizeValue now canonicalizes on the shared path. Manifest. show_forced_subtitles defaulted false where the server column is NOT NULL DEFAULT true, which would have turned forced subtitles off for every profile that never touched it. preferred_quality declared 13 members where the planner speaks 6 and collapses the rest to auto. metadata_language's allowlist was bound to the very column it migrates from. subtitle-appearance pinned fontFamily to three families while Apple stores any installed system font. Registers five user-facing settings the clients already ship, and corrects three notes that described Android behaviour that was not true. Enforcement. The package had no non-test callers, so MustLoad never ran; it now loads and logs at startup. The inventory test compared the manifest against a hand-copied map and could not see the drift it named; it now iterates settingsRegistry and checks defaults too — both verified to fail on injected drift. Adds .github/workflows/ci.yml, the repo's first CI that runs go test, go vet, gofmt, and the frontend suite. Known pre-existing failures are named individually in the Makefile so everything else stays gated and the list can only shrink. Part of #135 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): align the sleep timer default and range with the shipped client Android is the only client that implements this setting. It clamps to 0..240 and defaults to 30. The manifest said 0..480 with a default of 0, so a manifest-driven UI would have offered durations no client can store, and every user who never opened the picker would have had the preset silently turned off at cutover. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: give the new workflow the deps it actually needs The first run exposed two gaps in the workflow itself. go build ./... fails without libvips headers, because h2non/bimg binds libvips through cgo and pkg-config; the Dockerfile installs the same package. And pnpm/action-setup resolves its version from package.json, but there is no package.json at the repo root — the packageManager field lives in web/package.json, and a job's defaults.run.working-directory does not apply to an action's inputs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(web): stop the diagnostics download test depending on the Node version new Response(blob) reads the body through blob.stream(), which jsdom's Blob does not implement on Node 22 — the version the Dockerfile builds with. The test passed locally on Node 24 and threw "object.stream is not a function" in CI. Nothing in it asserts on the body, only that the object URL and filename reach the anchor, so a string body is equivalent and works on both. Surfaced by the CI workflow added in this branch, which is the first thing in this repo to run the frontend suite anywhere but a developer's machine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(build): copy the settings contract into the container build context Both Dockerfiles copy cmd/, internal/, migrations/ and web/embed.go, but the manifest lives in contracts/settings/v1 — an embedded Go package that sits outside internal/ because clients vendor those files. The image build therefore fails with "no required module provides package .../contracts/settings/v1". Caught deploying to the dev box. Nothing had built an image since the manifest landed: the Docker workflow only runs on pushes to main and workflow_dispatch, and CI's go build runs against a full checkout, so neither gate covers the container context. This would have broken the published image on merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): enforce the language-tag and step constraints the manifest declares A sweep of all 43 manifest definitions against the running server (160 checks: declared default, both boundaries, and deliberate violations for each remote key) found two places where the live registry accepts what the contract forbids. Both are fixed by calling the contract's own validators rather than adding a second implementation. playback.audio_language was checked as "32 characters or fewer", so the server stored "!!!" for a field the manifest declares as language_tag — a value track matching would then silently never match. It now requires a well-formed tag via settingscontract.NormalizeLanguageTag. The empty string is still accepted: the string-only endpoint has no way to send null, and both Android and web send "" to clear the choice, so rejecting it would break clearing the preference. player.playback_speed declared step 0.05 and nothing enforced it, so 0.26 was stored — a value no client's stepper can represent and that every client would silently snap on the next write. settingscontract.StepAligned is now exported and used by both the contract validator and the registry, so there is one definition of "on step" rather than two that can drift. This gives the contract its first production consumer beyond the startup load, which is the direction Phase 2 continues in. Also fixes a genuinely flaky test that the new CI gate would have hit intermittently: TestRemoveJellyfinCompatWebDisablesWebSetting used t.TempDir as the install root, but the endpoint returns 202 and its goroutine keeps writing there after the test body returns, so cleanup tripped "directory not empty" roughly one run in four. Confirmed pre-existing and unrelated to settings; the suite now passes six consecutive full-package runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): keep widened numeric bounds resolvable at older revisions A bound was one scalar plus the revision that introduced it, which discards the value it replaced. Widening a maximum from 240 to 480 at revision 3 left a revision-3 client with no correct answer against a revision-1 server: honoring 480 offers values that server rejects, and filtering the tagged bound out leaves the setting unbounded. Since clients are specified to filter their pinned contract against the server's advertised revision, the bound has to carry what it used to be. Bounds now hold their full history, oldest first, and AtRevision hands back the limit a given peer actually enforces. A bound nobody has widened still serializes as a bare number, so the manifest reads the same and untouched entries do not churn the ETag. Validation gains the rules the representation makes checkable: a maximum may only grow and a minimum may only shrink, history is strictly ordered, later entries must say when they arrived, and the first entry cannot predate the definition. That last rule is the lower bound allowed_scopes already enforced; the same gap is closed for enum members, which could previously claim to predate the definition containing them. Reported by Codex review on #479. * fix(settings): accept the partial subtitle appearance objects already stored The schema required all nine properties, but the current API accepts and round-trips sparse objects — settings_device_test.go stores {"fontSize":"xxlarge"} and reads it back — and the web client has always merged whatever it gets over DEFAULT_SUBTITLE_APPEARANCE. Requiring the full object would have made the cutover migration quarantine preferences users really set, or block on them. Every property is now optional and a stored value is documented as a sparse override merged over the definition's complete default. An empty object is still rejected: an override that overrides nothing is the same state as no override, which the contract represents as unset. Cross-scope resolution is deliberately unchanged. A device override still replaces the profile's object rather than merging into it, because a device override means "draw subtitles this way on this screen", not "amend the profile" — and that is what the server does today. Reported by Codex review on #479. * fix(jellycompat): scan the parent directory when a sidecar changes Autoscan matched scantrigger rejections by comparing RequestError.Message against literal strings. One of those messages became "Unsupported media file extension for library type" and the copy in handlers_autoscan.go did not, so the comparison silently stopped matching. The effect is user-visible: a Jellyfin client posting a change for Movie.nfo or poster.jpg gets a 400 and the batch is abandoned, when the sidecar should have resolved to a scan of the directory containing it. Three tests covered exactly this and had been excluded rather than read. RequestError now carries a Reason the caller can switch on. Message stays prose for the client reading the response — it is meant to be reworded, and nothing should break when it is. Also makes two tests honest about asynchronous work. The Jellyfin Web teardown deleted its install root while the operation goroutine was still writing to it, where a late write recreates a path RemoveAll already walked past; it now waits for the operation's terminal state, which required exporting CurrentWebOperation. And the direct-play If-Range test pinned size and mtime so ctime was the only remaining validator, then read it back inside a single coarse-clock tick — it failed about 85% of the time on main for a reason unrelated to what it tests, and now rewrites until the stamp moves. With those fixed, GOTEST_KNOWN_FAILURES is empty and gone: make test-go runs the whole Go suite. The one test that cannot pass yet — TestHandleReplanPlaybackV3SeekFailureRecoveryNeverChangesMediaVersion, which has failed since the commit that introduced it and describes unimplemented v3 planner behavior — carries a t.Skip explaining that where the test is, rather than a regex in the Makefile. Reported by CodeRabbit review on #479. * fix(settings): reject JSON the decoder would otherwise rewrite Two cases where encoding/json accepts input by quietly changing it, which is the one thing a contract promising byte-identical agreement between peers cannot tolerate. Duplicate object properties. jsonschema.UnmarshalJSON keeps the last occurrence, so {"fontSize":"small","fontSize":"large"} validated and stored "large". Which one wins is a property of the parser, not of the contract: a client generated against a different JSON library can disagree about what it just sent, and the canonical form cannot represent the duplicate at all. Lone surrogates. An unpaired \ud800 became U+FFFD and canonicalization reported success, so the server would issue canonical bytes and an ETag for an artifact a conforming implementation must refuse — RFC 8785 requires terminating here. Substitution also means the value read back is not the value written. Both checks run before the decode that would hide them, on the shared decodeJSON path that the manifest, its public projection and every value schema go through, and again on the object branch of ValidateValue, which uses a different decoder. Reported by Codex review on #479. * ci: gate Go lint on the lines a branch changes AGENTS.md told contributors CI ran the same checks as `make lint`, and the Go job ran only gofmt and vet. A change failing the documented Go lint gate passed all three jobs. Running the linter as-is is not an option: the tree has ~296 findings today, which is why this half of `make lint` was never enforced. Blocking every PR on a cleanup nobody has scheduled gets the gate deleted again, so CI runs with --new-from-merge-base and only the lines a branch touches have to be clean. The count can then only fall. golangci-lint is built from source at a pinned version rather than downloaded. A released binary refuses to run against a Go newer than the one it was built with, and go.mod here tracks Go closely enough that the current release already fails that way on 1.26.4. .golangci.yml declared version 2 while still using v1's issues.exclude-rules key. Current golangci-lint ignores it, so the "allow repeated strings and unchecked cleanup errors in tests" exclusions silently did not apply — 16 findings in test files that the config says to skip. Moved to linters.exclusions, which `golangci-lint config verify` accepts. The four lines this surfaced in scantrigger are fixed rather than excluded: its repeated status codes and messages are now named constants, so one condition cannot end up worded two ways. Also drops the workflow token to contents:read and stops persisting credentials in the three checkouts, neither of which any job needs. Reported by CodeRabbit and Codex review on #479. * docs(v1): record the settings removal as a pre-lock exception The design removes the legacy /api/v1/settings routes and the profile DTO preference fields, while AGENTS.md states /api/v1 is additive-only and removals go through Deprecation/Sunset. Read together those contradict. They do not actually conflict: v1-scope.md scopes the additive-only rule to "when the scope locks", and the scope is still open, so a removal taken now is in scope and there is no amendment process to invoke yet. But that reasoning lived only in the settings design, where nobody checking the API policy would find it. v1-scope.md now carries a pre-lock removals table naming what goes and why waiting is worse, and states the deadline the argument depends on: a removal listed there must ship before lock or fall back to Deprecation/Sunset. AGENTS.md points at the table and says to treat an unlisted removal as a mistake. Reported by CodeRabbit review on #479. * fix(settings): clear the remaining review findings Small, unrelated except that each was raised on #479. compileObjectSchemas parsed every non-directory file under schemas/ as a JSON Schema, so a stray editor backup or .DS_Store would panic the server at startup through MustLoad. schema_ref can only name a .json file; anything else is skipped. cmd/silo used MustLoad while the ETag check beside it and every other startup failure use log.Fatalf. It now fails the same way, so a bad contract prints an error instead of a stack trace. TestRegistryDefaultsMatchTheContract called scalarDefault before handling null, and scalarDefault rejects null as non-scalar — so the subtest skipped and the comparison after it was unreachable. A nullable contract default could disagree with a non-empty registry default and nothing failed. Confirmed by injecting that drift, which now reports it. The three appearance providers each adapted the auth context to AppearanceAuth with identical code, putting the shape of auth back in three places that widening cache ownership would have to find. useAppearanceCacheOwner now does it once. useTheme.test.ts cleared storage.KEYS between cases, but appearanceCache writes namespaced keys and an owner pointer that are not in that list, so both survived and the suite was order-dependent. It clears the store, as storage.test.ts already did. The abs_smart_collection_store comment is reworded rather than given back its SQL quotes: gofmt folds a pair of apostrophes in a doc comment into a typographic quote, which is how it became one in the first place. Reported by CodeRabbit review on #479. * feat(settings): add canonical typed storage for the settings contract The cross-platform settings contract needs one typed store behind it before a resolver, routes or a migration can exist. This adds that storage to both user-store backends and holds them to identical behavior. PostgreSQL gets user_setting_values with the scope CHECK constraints, the five partial unique indexes that enforce one explicit value per identity, and the covering indexes the one-query read path needs, plus user_setting_mutations for mutation_id idempotency and the inert user_setting_migration_rejects audit table. The per-user SQLite store gets the same shape minus user_id, since that database is already user-scoped. The UserStore interface grows the typed operations: read one explicit value at one scope, collect every candidate row for a resolution request in a single query, upsert with a revision increment, unset, and the idempotency receipt operations. The resolution read deliberately returns unranked candidates so the resolver can rank in Go — one query per request, never one per scope, which the pgx query-count test pins. Delete behavior is application-enforced. Neither backend can inherit it from constraints: the SQLite store declares no foreign keys, and library, series and device columns are not FK targets in Postgres either. Profile deletion cascades to profile-anchored values while account scope survives, forgetting a device clears its profile_device values alongside the legacy overrides, and the library/series purges remove only what is scoped to that entity. The shared conformance suite covers all of it, including the set-versus-unset distinction for false, 0, "" and null, so a divergence between the two backends fails a test rather than reaching a client. Part of #376 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(settings): pin the settings-value schema constraints in both backends Completes the storage track. The conformance suite exercises the store API, which validates identities in Go before any SQL runs — so nothing noticed whether the CHECK constraints and partial unique indexes actually existed. The one-time migration writes these rows in bulk without going through the per-request path, so the schema is the only thing guarding it. Adds constraint tests to both backends covering every scope's column requirements, rejection of an unknown scope, a profile that does not exist, non-JSON values, and each of the five partial unique indexes. Also clears the lint the storage commit did not get to: sql.ErrNoRows and pgx.ErrNoRows compared with == rather than errors.Is (which fails on a wrapped error), an unchecked rows.Close, and repeated fixture literals in the shared suite now named so a backend that confuses two scope columns fails on the assertion rather than on a typo. * fix(settings): close the review findings in the validator and the theme cache Four defects the existing tests did not reach. The web theme resolver compared the server's value against the appearance cache and fell back when they agreed, but the mirroring effect writes the server's value into that same cache — so the comparison held on the first render and stopped holding on the second, reverting an explicitly chosen theme to the default. The server's value is this account's own stored choice, so it now simply wins. The regression test re-renders rather than asserting on the first paint, which is why the original one passed. golangci-lint's exclusions.paths is a path regex, not a directory list, so a bare `web` also excluded internal/jellycompat/web_component.go, internal/webhooksync/, internal/notifications/webhook*.go and eleven other non-test files that were being linted before. Anchored. json.Number is a string kind, so `"1.5"` unmarshalled into it happily and Float64 parsed the quoted digits: a numeric setting validated as a JSON string and NormalizeValue stored the quoted form into jsonb. Rejected. The lone-surrogate check ran only on the object branch, so a lone surrogate in ui.custom_css decoded to U+FFFD on SQLite and was refused outright by Postgres jsonb — the two backends disagreeing about whether the same value could be stored. Hoisted to cover every type. The strict language-tag validation this branch added is correct, but it rejects what the shipped Android client sends; the companion fix is silo-android 4aeb78b4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(auth): stop TestJWT_TamperedToken passing a valid signature The test overwrote the last character of the signature with "X". An HMAC-SHA256 signature is 32 bytes, so its base64url encoding is 43 characters and the final one carries only four significant bits — U, V, W and X all decode to the same trailing byte. Roughly one token in sixteen was therefore left byte-identical and validly signed, and the test failed because ValidateToken correctly accepted it. Measured at 3098/50000 (6.2%) over distinct signatures; it just failed the Go job on this branch for reasons unrelated to the branch. Flipping a character in the middle of the signature is 0/50000. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): reject raw invalid UTF-8, not just escaped surrogates The previous commit hoisted the lone-surrogate check to cover every value type, but that only closes the escaped path. A raw 0xff byte inside a quoted string — what an HTTP body carries when a client encodes text in the wrong charset — is not an escape, so the surrogate scan never sees it, while encoding/json still substitutes U+FFFD and reports success. NormalizeValue then stores the original bytes, which SQLite's json_valid accepts and Postgres jsonb refuses: the same backend divergence, reached the other way. Found by the Codex review bot on the previous commit's own diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): size the library page state bound to what the web client writes ui.library_page_state's `search` was bounded at 256 characters. The web client serializes an advanced library view as URLSearchParams, encoding each filter rule as three groups[i][rules][j][field|op|value] keys — measured at 216 characters for one rule, 518 for three, 820 for five. The current endpoint validates this key by checking only that it parses, so those oversized values are already stored in production. Typing them at the declared bound would have failed the migration for anyone who had saved a view with more than one filter rule, and rejected the equivalent write afterwards. Raised to 4096, which clears ten rules with room to spare while staying a real bound. The test pins it against the key shapes libraryPageSearchParams.ts actually emits rather than a round number. Reported by the Codex review bot; the lengths above were measured by calling serializeLibraryPageSearchParams, not estimated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): split quality into two axes and register the orphan keys Two manifest changes the cutover needs. **Quality becomes resolution + bitrate.** The legacy ladder values (1080p-high, 720p-medium, 1080p-8, 420p, 328p) were never a third dimension — they are a bitrate spelled into the resolution string. The web player already decomposes them: useTranscodeQuality.ts defines 1080p-high as {resolution: 1080p, bitrate: 10000} and sends the two separately, so the compound form never reached the wire. Downloads went further and kept only a bitrate ladder. So playback.preferred_quality keeps the six clean resolutions and playback.max_bitrate_kbps becomes the second axis, nullable because "uncapped" is a real answer and a numeric sentinel would need widening every time hardware improves. Clients compose their own presets from the pair, which means retuning what "High" means is a client release rather than a contract break. Migration decomposes each legacy value losslessly, so none of them lands in the rejects table. **The five extension-bag keys are now definitions.** card_overlays, next_up_mode, sidebar_pins, disabled_library_ids and library_order reached the server only through the unknown-key path, stored as unvalidated strings. Two of them the server reads back — next_up_mode decides home section assembly and card_overlays falls back to an admin default — so they cannot be demoted to client-local. Registering them is what lets the extension bag close. Adds three schemas for their shapes and a test that exercises every schema_ref against a real value: each of these is nullable with a null default, so the existing default-validation test returns at the null branch without ever compiling the reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical resolution engine One answer to "what is this setting, for this profile, on this device, for this content". Before this, each caller carried its own ladder: catalog/detail.go resolved subtitles across four levels by hand and audio across three, handlers/settings.go had a two-level device/user resolution with a lazy write-back inside a GET, and jellycompat read profile columns directly. Those disagreed about precedence, which is the drift the contract exists to remove. Resolution is one batched read regardless of how many keys, libraries, or series are in play — ranking happens in Go against each definition's declared resolution_order. Five sequential index lookups per key per item is the implementation the design rejects, and a season view is exactly where it would have shown up. An absent identity drops its scope rather than erroring, so one code path serves an identified client, an anonymous jellycompat seed, and a batch spanning many series. Rows for a foreign profile, device, library or series are ignored even though the batched read returns them. Constraints narrow without destroying: a capped 4K preference resolves to the cap, reports itself constrained, and keeps the authored value so it takes effect the day the cap lifts. Two cases needed care — null on a nullable numeric means unbounded, so a ceiling must cap it rather than rank it equal and let the value that most needs capping slip past; and an allowlist falls back to a permitted member rather than the definition's default, which may itself be outside the list. Adds ValueSchema.CompareValues to the contract package, since ordering values is what makes a ceiling or floor mean anything and value semantics belong with the schema that declares them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the one-time migration planner The conversion rules from legacy settings storage to canonical values, as ordinary Go rather than twice in two SQL dialects. Both backends read their own rows, hand them to Plan, and write what comes back — so the decisions are testable without a database and SQLite and Postgres cannot drift apart in what they decide. The rules that needed care, each pinned by a test: Column defaults are not choices. quality_preference is NOT NULL DEFAULT '1080p' while the contract defaults to auto, so migrating the column unconditionally would pin every profile in the install to 1080p having never chosen it — and that stored value would then outrank the contract default forever. Same for language 'en', subtitle_mode 'auto', and show_forced_subtitles true. The empty string is unset, not a value. The legacy string API had no way to send null, so both Android and web spell "clear my choice" as "". Storing that would make a cleared setting outrank the default. Legacy quality decomposes rather than rejects. Every compound value maps to a resolution and a bitrate from the ladder in useTranscodeQuality.ts, so nothing lands in the rejects table. Account rows fan out to every profile, which is the account-to-profile move the contract makes for appearance and search scope: a household that shared one theme each end up owning theirs. Legacy strings become typed JSON — "true" to true, "30" to 30 — or every generated binding would fail to decode what the migration wrote. Nullability differs per backend, so profile columns arrive as pointers and the caller resolves "chose the default" versus "never written" when it reads. jellycompat's DisplayPreferences blobs ride the same table under synthetic keys and are left alone; they are that subsystem's storage. Everything that cannot convert is recorded with a reason rather than dropped, and a final test asserts every planned row would be accepted by the mutation endpoint's own validation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): run the one-time migration on the SQLite backend Wires the planner to real storage as userdb migration V15. V14 created the tables; this fills them. It runs inside runMigrations' existing transaction, so a database either comes out fully migrated or untouched — a partial migration is the one state neither the operator's backup nor a rollback covers. Pinned by a test that rolls back and asserts nothing was left behind. Two things the wiring had to get right that the planner could not see: Reject identities are JSON. Postgres declares that column jsonb NOT NULL and SQLite guards it with a json_valid CHECK, so the free-form "profile=p1 device=d1" the planner emitted would have failed to insert — on exactly the rows the table exists to record. They are structured documents now, which is also queryable. Subtitle and audio preferences are two tables keyed the same way, so they merge into one per-series record before planning. Converting them independently would have produced two rows racing for the same identity. Every legacy read tolerates a missing table, since this runs against databases created at any schema version, and preferred_metadata_language is deliberately absent: that column exists only in the Postgres schema. Tested end to end against a real database rather than only through the planner — the rows land, satisfy the scope CHECK and the partial unique indexes, and hold valid JSON. Also covers the empty-install case and asserts a second run fails rather than silently doubling every value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): run the one-time migration on the Postgres backend The mirror of userdb V15, registered with goose as a Go migration rather than SQL: the conversion validates every value against its own definition and re-encodes it as typed JSON, and one legacy quality string becomes two rows — neither is expressible in SQL without duplicating the manifest. The rules stay in internal/settingsmigrate, so the two backends cannot disagree. RunTx, so the whole backfill lands in goose's transaction. The down migration empties the canonical tables; the legacy ones are never touched by the up, which is what keeps the cutover reversible until the follow-up migration drops the superseded columns. preferred_metadata_language is read here and only here — the column exists in this schema and not in SQLite's, so this is the sole source for catalog.metadata_language. Verified against a real Postgres: the full goose chain runs, 1080p-high decomposes to ("1080p", 10000), values land as typed jsonb rather than strings (jsonb_typeof reports number), rejects carry a queryable jsonb identity, and the composite profile foreign key refuses a row naming a profile that does not exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical settings API The routes that make the typed storage reachable. Until now the manifest, the resolver and the migration all existed with nothing able to call them. GET /settings/contract serves the public manifest behind an ETag — clients vendor a pinned copy and generate bindings from it, so the common request asks "still the same contract?" rather than transferring it. Its capabilities sibling reports revision and supported scopes for feature detection instead of version sniffing. /settings/values/{key} reads, writes and clears an explicit value at one named scope, which is what a reset affordance needs: "did I set this here" is a different question from "what applies", and the old endpoint could only answer a blurred version of both. Scope comes from the query while profile and device come from session headers, so one profile cannot address another's settings by naming it. /settings/values/effective resolves any number of keys in one request, with the resolution ladder and the source of each answer reported so a client can offer "reset this device's override" against the exact row holding it. Asking for no keys returns every remote setting, which is what a settings screen wants. Writes are idempotent when a client sends X-Silo-Mutation-Id: a retry after a dropped response replays the receipt, and reusing an id with different content is a conflict rather than a silent overwrite of the wrong thing. Three things the string-only endpoint could not do, each pinned by a test: an unknown key is refused rather than stored in the extension bag, values are checked against their declared type and range, and a write to a scope the definition does not allow is rejected. Registered before the catch-all /{key} routes, which would otherwise swallow "contract" and "values" as setting names. The legacy endpoints stay live for now; deleting them is the next commit, once their consumers move. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): generate typed bindings for all four languages One generator rather than one per repo. The point of the contract is that four codebases agree on keys, types, scopes and defaults, and four independently written generators would be four chances to disagree. Go and TypeScript land in this repo; Kotlin and Swift are written into the sibling client checkouts, skipped with a note when they are not present so a server-only developer can still run it. Output is sorted by key so an unrelated manifest edit does not produce spurious diffs. The Kotlin output is the interesting one: it generates the DeviceSettings allowlist Android maintained by hand, plus the BOOLEAN_KEYS/INT_KEYS/ DOUBLE_KEYS classification it kept as a *second* hand-maintained table that had to agree with the first. Both are manifest questions now, so the whole class of "wrote a local key to the server" and "flushed a value the store could not parse" bugs stops being possible by construction. The TypeScript output carries the full definition table — labels, controls, enum members, bounds — so web/src/lib/settingsManifest.ts can be deleted rather than kept in sync: it declared 17 definitions against the contract's 49, with its own two-scope model that does not match the contract's five. make verify-settings-bindings fails when the committed output disagrees with the manifest, wired into CI, so a manifest change cannot merge leaving every client reading stale keys. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(web): add the two-axis quality picker and typed settings hooks Quality becomes one picker over two stored values. The server holds a resolution cap and a bandwidth cap independently, which is what the player has always sent on the wire — useTranscodeQuality.ts has decomposed 1080p-high into {resolution, bitrate} for as long as it has existed. Presets live in the client rather than the contract so retuning what "High" means is a one-line edit here instead of a contract change four codebases have to agree on, and an older server keeps working because it only ever sees the two axes it already understands. A combination no preset covers still gets a truthful label rather than a picker showing the wrong entry: reachable by setting the axes separately through the API, or from a legacy value whose bitrate is off this ladder. Choosing an uncapped preset clears the bitrate rather than storing a sentinel, so "no cap" stays the absence of a value at every layer. Adds hooks over the canonical API alongside the legacy ones rather than replacing them wholesale — a key that is not in the manifest cannot be expressed, because SettingKey is generated from it, and the default for an unset value comes from the generated table rather than a literal at the call site. That last part is what stops the flip-off bug the Apple client carries a hand-written guard for. A test asserts every preset composes values the contract actually accepts, so a preset naming a resolution outside the enum or a bitrate outside the declared bounds fails here rather than 400ing when a user picks it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): resolve catalog playback preferences through the contract catalog/detail.go held the two hardest ladders in the codebase: subtitles resolved across four levels by hand, audio across three, each partially overriding the last through Has* flags. Both now call the canonical resolver, so the precedence lives in the manifest and this file cannot disagree with the contract about which override wins. Adding a scope is a manifest change rather than another branch here. The subtitle track signature stays on its specialized table — it identifies a concrete track rather than expressing a preference, so it is not a setting. Resolution keeps the memoization the old lookups had: the audio resolver still reads once per profile and once per library rather than once per file, which is what kept a many-track audiobook detail page fast. The test that guards it now counts resolver reads instead of GetProfile calls, since the guarantee is about scaling with file count rather than about which method does the reading. Four tests seeded the profile column directly. That column is a migration source now, not a read path, so they seed the canonical value instead — they were passing against storage nothing reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): close the unknown-key extension bag keyUsesUserScope returned true for any key the registry did not know, so a client could invent a production setting unilaterally and the server stored it as an unvalidated string. That is how six ui.* settings and five orphan keys reached production untyped, and it is the root enabler the design names. An unknown key is no longer a user setting, so the legacy write path rejects it and the canonical API — which validates every value against its own definition — is the only way to store something new. jellycompat's DisplayPreferences blobs ride the same table under synthetic keys and keep working: they are that subsystem's storage rather than user settings, and they move to dedicated storage in the follow-up rather than being dropped here. Also repoints the DisplayPreferences seed at the canonical resolver. Resolved at profile scope with no device on purpose — Jellyfin clients do not carry Silo's device identity, so a device override leaking into the seed would hand one device's settings to every Jellyfin client on the account. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): enforce viewer quality caps through resolver constraints constraintsFor was the unwired half of the preferences-versus-restrictions seam: it returned nil, so a profile capped at 1080p by policy still resolved its stored 2160p preference at face value through the effective endpoint. The settings routes are mounted inside RequireViewerAccess, so the resolved access scope is already on the request context. Scope.MaxPlaybackQuality holds a literal member of the contract's quality enum ("1080p"/"2160p"), which is exactly what the manifest binds playback.preferred_quality's ceiling to under policy_input "max_playback_quality" — so the wiring is a direct map with no translation table. An empty value means the policy sets no cap, expressed by returning nil so the resolver leaves the preference alone. catalog.metadata_language deliberately stays unconstrained: the manifest notes record that the allowlist draft was circular (the policy input it would bind to is populated from the very preference it would narrow). The handler test covers both halves of the seam: a 2160p preference under a 1080p cap resolves to the cap with constrained:true/ceiling and the authored value reported in stored_value, the stored row itself is not rewritten, and an uncapped viewer gets the preference unchanged with no constraint noise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): publish user_settings change events Add a user_settings realtime channel so clients learn when a setting changed on another device without polling. The channel is modeled on user_state: non-admin subscribable, per-user addressed envelopes, null snapshot. SettingValuesHandler gains an EventsHub and publishes user_settings.changed after every successful PUT and DELETE on /settings/values/{key}. The payload carries only key, scope and profile_id — never the value. Admins receive every user's user-scoped events, so a value in the payload would leak private settings to admins; interested clients re-fetch over the scoped REST API instead. The payload is always non-empty because an empty Data falls back to a null snapshot in the hub. A nil hub (tests) skips publishing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): sweep expired mutation receipts daily Setting-mutation idempotency receipts were written with an expires_at that nothing enforced, so the table grew forever. Add a hidden daily system task (05:00) that walks every login account, opens its user store, and calls DeleteExpiredSettingMutations. A user whose store fails to open or sweep is logged and skipped so one broken store cannot stall retention for everyone else; the delete is idempotent, so the next run repairs anything missed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(settings): resolve metadata language canonically in access and policy Repoint the last legacy column readers onto canonical contract resolution (settings cutover task A4a): - access.Resolver and policy.ViewerResolver now resolve catalog.metadata_language through settingsresolve (profile scope -> contract default) via a shared access.PreferredMetadataLanguage helper, instead of reading user_profiles.preferred_metadata_language. Resolution is deliberately unconstrained: the policy input this preference feeds is the one a constraint would have to reference, which is circular — see the key's manifest notes. - playback start now resolves playback.audio_language canonically for the profile default instead of reading user_profiles.language, matching the catalog detail path. Series and library override handling is unchanged. - items.go needed no change: it already consumes the resolver-produced scope.PreferredMetadataLanguage. The legacy columns keep their values but are no longer read on these paths; a profile with only a column value now resolves to the contract default, and a stored canonical value wins. Tests pin both directions in access, policy (including scope parity, where the column is now a decoy), and the playback handler. Read cost is one batched store read per resolution, same as the profile-row read it replaces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(jellycompat): give DisplayPreferences its own table The Jellyfin DisplayPreferences blobs rode the legacy user_settings key/value table under synthetic jellycompat:* keys, which forced the legacy settings API to carry a prefix carve-out in its otherwise-closed unknown-key gate. They are the compat subsystem's storage, not user settings: the contract neither validates nor resolves them. Move them to a dedicated jellycompat_displayprefs table in both backends, keyed by (prefs id, client) per user, with the blob stored as opaque text served back byte-for-byte (deliberately not jsonb, which would re-serialize it). The data-copy migrations — per-user SQLite V16 and a paired SQL + Go goose migration for Postgres — are transactional and harmless to re-run, and both drive their key parsing and row classification from the new internal/jellycompat/displayprefs package so the backends cannot diverge, following the internal/settingsmigrate precedent. A jellycompat:* row that does not parse as a DisplayPrefs key (only ever writable through the removed carve-out) is recorded in user_setting_migration_rejects rather than silently deleted. With the last non-settings tenant gone, the jellycompatSettingPrefix carve-out is deleted: the legacy settings endpoints now refuse jellycompat:* keys like any other unknown key and never surface them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): serve admin user-settings through the canonical API Replace the ten string-registry /admin/users/{id}/settings* and device-settings* routes with the canonical contract surface: one list of every explicit value the target user has stored across all scopes, and set/delete at an explicit scope named in the query string. The admin handlers live on SettingValuesHandler and share the session routes' implementation rather than duplicating it — the same key/scope parsing, identity validation, contract scope allowance, value normalization and mutation-receipt idempotency, factored into keyedScopeFromRequest/completeIdentity and setValueAt/deleteValueAt. The only admin-specific parts are the target user coming from the path, profile and device ids coming from the query (an admin holds no session claim to the user being inspected, so its named profile is checked to exist), and change events attributed to the target user so their clients refresh. The list is a new UserStore read, ListAllSettingValues, implemented in both backends and pinned by the shared storetest conformance suite: the admin surface wants the stored truth (which overrides exist, for a per-row reset affordance), which no resolution-shaped read answers. The ten removed routes are recorded in the pre-lock removals table in docs/architecture/v1-scope.md per the v1 API rules; the web admin device-overrides page moves onto the new surface in the Phase B rewrite inside this same unmerged PR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(settings): add the cross-platform conformance fixture and its Go and web runners contracts/settings/v1/conformance.json is the spec's named drift gate: 21 hand-authored cases of {keys, stored rows, context, constraints, expected effective value + source}, every one executable against the shipped manifest. They pin the semantics most likely to drift across four resolver implementations: the full resolution ladder (series > library > device > profile > default), an absent identity dropping its scopes, foreign-identity rows never resolving, ceiling caps that report the authored value with constrained:true, the ordered-enum sentinels (auto below every cap, original above), null-on-a-nullable-numeric meaning unbounded and being brought down by a ceiling but ignored by a floor, allowlist falling back to the first allowed member rather than the (possibly forbidden) default, and playback.subtitle_appearance resolving device > profile only with the sparse device object replacing, not merging. Cases may inject a constraint binding onto a copy of a real definition so constraint kinds no shipped definition carries stay testable. The Go runner (internal/settingsresolve/conformance_test.go) resolves each case through the real resolver against the embedded manifest. The web runner (web/src/lib/settingsConformance.test.ts) runs the same cases through a new client-side resolver, web/src/lib/settingsResolve.ts, which mirrors the server's semantics; the TypeScript bindings now carry each definition's ordered flag and constrained_by binding so that resolver derives constraint behavior from the contract instead of hardcoding it. Both runners reject unknown fixture fields — schema drift in the fixture itself is drift — and both refuse a fixture authored against a different manifest revision. The fixture travels with the bindings: make settings-bindings vendors the copy the web runner reads, and make verify-settings-bindings fails CI when that copy goes stale. The Kotlin and Swift copies land together with their runners in the client repos, which will pick their own test-resource paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settings): review pass over the phase A stack Fixes the eight adversarially-confirmed defects the review of the unpushed phase A stack (40e0f77a..1f2c7fe4) found, each with a test that fails without its fix. Writers left behind by the language cutover (high). |
||
|
rxwatcherandQuick
|
ad122c38c2 | fix(catalog): allocate collision-free subtitle indexes | ||
|
rxwatcherandQuick
|
bfa5baec86 |
fix(catalog): give external subtitles their combined-space indexes
buildVersionSubtitleTracks never set Index on external subtitle entries, so every external sub in a version's subtitle_tracks serialized the zero value. Any file with two external subs (or one external plus an embedded stream index 0) published duplicate indexes; clients keying subtitle rows on index crashed (Compose LazyColumn duplicate key) or resolved selections against the wrong track. Externals now carry their combined-space ordinal (externals occupy 0..n-1 in the playback selection space — the same identity session subtitle_urls and ResolveSubtitlePolicyV3 use); embedded entries keep their ffprobe stream indexes unchanged. |
||
|
Quick
|
824d7bdf43 | fix(catalog): preserve normalized TMDB ownership | ||
|
|
9f4f9a3bd8 |
fix(catalog): match TMDB ids stored in TMDB's own id-slug form
Attaching a TMDB id compared the stored value byte-for-byte, so a
media_items row holding "1931-disney-s-adventures-of-the-gummi-bears"
conflicted with the bare "1931" being attached and the presence-lookup
backfill failed:
WARN requests: failed to backfill tmdb id from presence lookup
error="media item tmdb id conflict:
got \"1931-disney-s-adventures-of-the-gummi-bears\", want \"1931\""
That form is not corrupt data — it is how TMDB writes its own URLs
(/tv/1931-disney-s-adventures-of-the-gummi-bears) — and enough of it has
reached the catalog to matter. The visible cost is that a title already
in the library never gets linked, so clients keep offering to request
something you own.
The numeric prefix is the identity, so compare on that. Values with no
leading digits, or digits not followed by a separator, are not TMDB ids
and still conflict as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Db4dSxN9tH8yN7uUP549tK
|
||
|
|
7ab393fc3a |
fix(jellycompat): resolve item duration probed-first with runtime fallback (#493)
* fix(jellycompat): resolve item duration probed-first with runtime fallback Jellyfin-protocol clients received no runtime at all for items whose catalog runtime is 0. RunTimeTicks is omitempty, so a zero value is dropped from the JSON entirely rather than sent as 0, and strict clients (Infuse) abandon playback on those items. On the production deployment 5,245 movies have media_items.runtime = 0 while 5,239 of them have a correct probed media_files.duration. Resolve duration at read time the way /api/v1 already does: probed file duration first, catalog runtime as the fallback. The item row is deliberately not backfilled — one item can have several versions of different lengths, so per-file data does not belong there. - scanner: FirstDurationsByContentIDs / FirstDurationsByEpisodeIDs, batched lookups using the same "first live file with duration > 0, ordered by id" rule as the v1 API's contentDurationSeconds. The episode_id IS NULL guard on the content-id query is load-bearing: every episode file carries its series' content_id, so without it a series row would report an episode's duration. - catalog: optional batchDurationFetcher extension on DetailService, following the existing extraFileFetcher pattern so test fakes need no changes. Nil-receiver safe and fail-soft — a failed lookup logs and degrades to the catalog runtime rather than failing the page. - jellycompat: DurationSeconds on upstreamListItem/upstreamEpisode, a shared runtimeTicks resolver, and fillListItemDurations wired into the nine page producers. Fixes the three sites that had no fallback (itemFromList, episodeFromUpstream, HandleSearchHints); the detail and PlaybackInfo paths were already correct. This is additive within the v1 rules — it populates a field that was previously omitted. No field is renamed, removed, retyped, or repurposed. * fix(jellycompat): avoid duplicate duration lookups --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
383973ec22 |
feat(metadata): improve match accuracy and localized titles (#461)
* feat(metadata): improve match accuracy and localized titles * fix(metadata): address matching review findings * test(catalog): align empty alias snapshot scope --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
QuickandGitHub
|
2bc6ebb013 |
Merge pull request #460 from Silo-Server/agent/repair-anchored-identity-links
fix(scanner): repair historical provider-anchored merges |
||
|
QuickandGitHub
|
5e2dd91d6b |
Merge pull request #456 from Silo-Server/feat/admin-settings-contract
fix(admin): enforce settings contracts end to end |
||
|
Quick104
|
e625d574e3 | fix(scanner): repair historical provider-anchored merges | ||
|
Quick104andClaude Fable 5
|
6da6cf7a71 |
fix(search): opt episode documents out of embedder vectors explicitly
A userProvided Meilisearch embedder rejects any document that omits _vectors entirely, failing the whole indexing task once episode documents enter a rebuild batch. Episode docs now carry the documented `_vectors.<embedder>: null` opt-out, and attachDocumentVectors runs the per-document pass even for episode-only batches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104
|
3c56ed606c | fix(admin): enforce settings contracts end to end | ||
|
Quick104
|
2e45a9c015 |
Merge remote-tracking branch 'origin/main' into pr-397-devmerge
# Conflicts: # internal/catalog/item_repo.go |
||
|
|
b96e359b4e |
feat(catalog): typo-tolerant Postgres search fallback (did-you-mean) (#386)
* feat(catalog): typo-tolerant search for the Postgres (non-Meilisearch) path
## What this does (plain language)
When someone searches the library and misspells a title — "intersteller",
"godfathr", "jurasic" — the Postgres-backed search used to return nothing,
because it only did exact full-text matching. This adds a "did you mean"
fallback: when the normal search finds little or nothing, we run a second,
typo-tolerant lookup and surface the closest titles.
This only affects deployments that search via Postgres (the fallback path).
Meilisearch already does its own typo tolerance and is left untouched.
## Why not just make the main query fuzzy
The obvious approach — OR a trigram similarity match into the main search — is a
performance trap. The trigram operator is "lossy", so Postgres re-checks every
near-miss candidate by rebuilding three title search-vectors per row. On a real
library that turned routine searches into multi-second queries.
Measured on a 175k-title dev database:
- exact full-text only: ~60 ms
- fuzzy OR'd into the main query: ~217 ms (and far worse on prod-sized data)
## How it works
The fuzzy arm is a completely separate query (buildFuzzySearchSQL). It matches
only on the trigram-indexed title_normalized column and ranks only by
similarity() on that same column — it never touches the title search-vectors, so
it pays no per-row rebuild. It runs only when the exact search is "sparse" (fewer
than 5 hits) and the query is long enough for the trigram index to help (>= 4
characters), so the common case stays on the fast exact path. It is wired into
SearchPage (not just the thin Search wrapper) so the catalog search provider
benefits too.
## Measured on the live 183k-title catalog (read-only EXPLAIN ANALYZE)
- exact query for a typo: ~0.8 ms (0 hits -> triggers the fallback)
- fuzzy fallback query: ~5-27 ms, always via the trigram index, with no
search-vector rebuild
- "intersteller" -> Interstellar (similarity 0.63)
- "godfathr" -> GodFather (0.58), The Godfather
- "breakin" (134 exact hits) -> fuzzy correctly does NOT fire
Shared scope predicates (type / library / access / manga-exclusion) are extracted
into appendSearchScopeFilters so the exact and fuzzy queries filter identically.
Adapted from the earlier feat/search-fuzzy-fallback prototype onto main's current
SearchPage / includeTotal architecture.
* refactor(catalog): correct fuzzy-search pagination and parse the query once
Follow-up to the fuzzy fallback, from an adversarial code review. Two things: a
pagination correctness fix and a small performance/readability cleanup. Both were
validated against the live 183k-title catalog.
## The pagination bug (plain language)
Fuzzy results are shown after the exact results, as one combined list. The first
version stitched that list together with page-offset math, and got the math wrong
past the first page:
- the reported result count grew as you paged (page 1 said "31 results",
page 2 said "33");
- titles shown on page 1 could reappear on page 2;
- paging far past the end still ran the (pointless) fuzzy query every time;
- a tiny page size (e.g. an autocomplete asking for 3) could hide the fuzzy
results behind a page the client was told did not exist.
## The fix
Because the fuzzy fallback only runs when exact results are sparse (< 5) and the
fuzzy part is capped at 50, the whole combined list is tiny. So instead of
fragile per-page offset math, we now fetch that small combined list once and take
the requested slice in memory. Every page is then correct by construction: stable
total, no repeats, no wasted work past the end.
Before -> after, typo search "intersteller" (21 results, page size 5):
- total reported on page 2: 31 then 33 (drifting) -> 21 (stable)
- repeated titles across pages: yes -> none
- request past the end (offset 500): 2-3 DB queries -> 0 extra queries
- autocomplete (page size 1): fuzzy hidden -> paginates correctly
Cursor-style callers (that don't ask for a total) can't locate the boundary
between the two blocks on a later page, so they now get the fuzzy results as a
single terminal first page — no misleading "more results" flag.
## The cleanup
The raw query string was being parsed three times per search (once for the
eligibility check, once in each SQL builder). It is now parsed once in SearchPage
and passed down; the shared search-text derivation is extracted so the two
builders can't disagree; and the normalized form the eligibility gate needs is
precomputed at parse time. ("Performance first", per the repo guidelines.)
Also considered and rejected: excluding exact hits from the fuzzy query with a
NOT(full-text) clause instead of by id. It reintroduced the search-vector rebuild
the whole design avoids — measured ~51 ms vs ~20 ms on the worst case — so
id-based exclusion stayed.
Known limitation: the fuzzy path re-reads the small exact block in a second
query, so a title written in the sub-millisecond gap between the two reads could
be missed until the next search. Harmless and inherent to a multi-query design.
* fix(catalog): close fuzzy-search library-scope leak and restore small-limit cursor recall
Addresses two findings from the PR #386 review bots.
## Library-scope leak (Codex P1)
The search scope helper shared by the FTS query and the trigram fuzzy fallback
filtered libraries with `JOIN media_item_libraries mil` +
`NOT (mil.media_folder_id = ANY($disabled))`. An item linked to BOTH a disabled
and a non-disabled library fans out to two joined rows; the non-disabled row
satisfies the deny check, GROUP BY collapses the item back, and it surfaces in
search results despite the disabled library. Because the new fuzzy fallback
reuses this helper, typo searches could leak disabled-library items too.
appendSearchScopeFilters now delegates to the leak-safe
appendLibraryAccessConditions (access_filter.go), which emits item-scoped
EXISTS/NOT EXISTS subqueries — the same form GetByIDs/EnsureAccessible already
use — and needs no membership JOIN. The disabled-only path keeps its
argument-free positive-membership EXISTS so orphan items don't slip through a
vacuous NOT EXISTS. The scored CTEs keep GROUP BY (now required only for the
MAX() ranking aggregates). New regression test pins the EXISTS/NOT EXISTS shape
and the absence of a JOIN for both the FTS and fuzzy builders.
## Small-limit cursor recall (Codex P2)
In cursor mode (include_total=false) the FTS probe fetched only limit+1 rows.
For a tiny caller limit (e.g. an autocomplete asking for 2) with a few incidental
exact hits, that made ftsHasMore true, so the block never looked "sparse" and the
typo fallback never fired — and subsequent offsets are barred from triggering it,
so the fuzzy results were unreachable entirely.
SearchPage now floors the cursor-mode probe at fuzzyFallbackThreshold rows, and
execSearchBlock returns the pre-trim row count so sparsity is judged as
`fetched < threshold` independent of the caller's page size. The returned page is
still trimmed to limit with correct hasMore. Exact mode is unchanged (it judges
sparsity by the page-independent window count).
* fix(catalog): harden fuzzy-search fallback per adversarial review
Addresses the confirmed findings from a deep review of the fuzzy-search
fallback:
- Cursor mode now enters the fallback only when the whole sparse FTS
block fits the caller's page, so the terminal fuzzy page can never
hide exact matches the plain hasMore path would have surfaced
(jellycompat clients with EnableTotalRecordCount=false lost matches).
- execSearchBlock takes a querier and returns its untrimmed rows;
SearchPage hands the already-fetched block to the fallback instead of
re-running an identical FTS query on every sparse search.
- Fuzzy truncation is detected with LIMIT cap+1 instead of a
COUNT(*) OVER () window count that only fed a debug log; truncated
exact-mode responses now report total_exact=false rather than
presenting the cap as an exact count.
- The fuzzy query runs in a transaction pinning
pg_trgm.similarity_threshold via SET LOCAL, so match quality cannot
drift with cluster configuration.
- When the FTS block has real hits, fuzzy augmentation demands
similarity >= 0.45 so correctly-spelled sparse queries only gain
near-identical titles instead of base-threshold trigram noise.
- filterCatalogSearchItems no longer erases fuzzy matches on the
filtered/sorted/prefix resolver path: a typo token is never a
substring of the titles it matched, which left typo search returning
zero results there while the plain search box showed matches.
- The cursor probe floor applies only when the fallback can fire;
cursor fuzzy fetches no more rows than the terminal page can serve.
- slog.Debug -> slog.DebugContext (sloglint); reuse
contentIDsFromMediaItems instead of a duplicate helper; document the
title-only fuzzy scope.
Verified against the dev deployment: stable totals across pages, no
duplicates, small-limit cursor recall restored, filtered-path typo
search working, ~160ms typo-path latency.
Part of PR #386.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(catalog): reach long titles via strict word similarity in fuzzy search
Full-string trigram similarity is diluted by every extra trigram a long
title contributes, so a typo of one word could never reach titles like
"Avengers: Endgame" ("avegners" scores ~0.38 against "avengers" but far
below threshold against the full title). Swap the fuzzy predicate from %
to <<% (strict_word_similarity), which scores the query against the best
word-boundary extent of the title. At equal thresholds <<% is a strict
superset of %, and the existing gin_trgm_ops index serves both — no
migration needed.
The SET LOCAL pin moves to pg_trgm.strict_word_similarity_threshold and
is load-bearing: the 0.6 server default would reject ordinary one-edit
typos outright.
Ranking is strict word similarity first with whole-title similarity()
as tie-break, so near-identical short titles ("The Avengers") sort above
long titles that merely contain the matched word.
The 0.45 augmentation floor deliberately stays on whole-title
similarity(): word similarity rates embedded prefix words far too high
("coral" scores 0.5 against "coraline"), which dev testing showed would
flood a correctly-spelled sparse query with 27 noise rows. Zero-hit
(true typo) queries skip the floor, so the new long-title recall applies
where it matters.
Dev-verified: "avegners" now returns The Avengers first, then Avengers
Grimm / Avengers: Endgame; "coraline" still returns exactly its 4 real
titles; cursor small-limit recall, filtered-path typo search, pagination
stability, and ~160ms typo-path latency all unchanged.
Part of PR #386.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
91e1164090 |
feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
1664c60425 |
fix(metadata): publish artwork revisions atomically (#399)
* fix(metadata): publish artwork revisions atomically * fix(metadata): harden artwork revision cleanup * fix(metadata): address artwork revision review findings - restore image applies for all media_items types and reject unsupported target/image combinations with 400 before uploading; episodes coerce to stills and the web dialog no longer offers image tabs episodes can't use - add WHEN clauses to displacement triggers and hoist to_jsonb so bulk catalog upserts that assign unchanged artwork columns skip the trigger - make artworkkey the single variant-ladder owner: imagecache derives its widths from it and triggers store image_type instead of hardcoded variant arrays, expanded by the collector at deletion time - sweep dormant registry rows periodically so references lost through untriggered surfaces degrade to slow cleanup instead of leaking - park just-published revisions dormant, keep dormant rows dormant on re-cache, and batch the GC reference pre-check per run - heal rows re-referencing a just-deleted revision via reconciler-style resets after the deletion commits - share a per-URL image-loaded hook across DetailHero, ItemCard, SectionItemCard, GlobalSearch, and CollectionPosterCard - deduplicate Cache/CacheBytes finalization and drop unused VariantPaths plumbing Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): cast reused timestamp parameter in revision upsert Postgres cannot deduce one type for $3 used both as a plain value and inside a CASE arm; the dev deploy surfaced it as SQLSTATE 42P08 on every publication. Cast both uses and cover the arm/park/track upserts with database-backed tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): address artwork revision review comments - keep a durable heal path: deletion marks deleted_at instead of removing the registry row, so a failed post-delete heal retries with backoff and broken references never park; trackers clear the marker on re-upload - never treat bare existence as an immutable-content match; backends without content verification rewrite the object - exercise revisioned cover keys in scanner/enrichment fakes, compare the tracked manifest exactly, and honor cancellation in the blocking test deleter Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
8fc054c15d |
fix(scanner): never purge files under unreachable library roots (#372)
* fix(scanner): never purge files under unreachable library roots An unreachable root is not a removed root. When one root of a multi-root library dies (unmounted share, dead drive) while another root still has files, the whole-library empty-root guard does not fire — the surviving root produced files — so the scan marks everything under the dead root missing_since (desired: hides it from browse/playback) and then, with the default scanner.empty_trash_after_scan=true + 24h file_removal_grace, the next scan after the grace hard-deletes every row under the dead root. A week-long drive outage silently destroys the root's entire catalog state: probe data, intro/credits markers, file hashes. Worse, membership reconciliation immediately purges media_items whose only files lived on the dead root, cascading user collections (library_collection_items has ON DELETE CASCADE) and deleting cached artwork. This change makes "temporarily offline" survivable: - Probe each configured root at scan start (os.Stat + IsDir + ReadDir, factored into the new internal/rootcheck package and shared with the admin mount-check endpoint). Unreachable roots are skipped by the walk but their scopes still reconcile, so files are still marked missing. - The trash sweep (DeleteMissingByFolder) now excludes rows whose path sits under an unreachable root, using the same exact-path + escaped prefix-LIKE matching as ListIDsOutsideRoots (a sibling root that merely shares a string prefix is never protected). With all roots reachable the emitted SQL is unchanged. - Membership removal still happens — browse/home hide items via media_item_libraries, so removal is what keeps a dead-root-only title out of the catalog — but the orphan media_items purge exempts items whose files sit under an unreachable root. Their metadata, artwork, and collection links survive; when the root returns, the upsert clears missing_since and syncPresentLibraryState re-inserts the membership, restoring the item with zero re-probing or re-matching. - The folder surfaces scan_warning_code='dead_root' with a message naming the unreachable roots; a fully healthy scan or a successful mount check clears it, mirroring empty_root. The admin UI shows a badge and banner. - Deliberate deletion is untouched: removing a path from the library config still purges via ListIDsOutsideRoots, files under reachable roots keep the exact 24h-grace purge, the empty-root guard and the autoscan dead-mount guard are unchanged. The audiobook/podcast/ebook reconcile paths share the same folder-wide sweep and orphan purge, so they get the same guard. Covered by tests: an end-to-end two-root scan (root dies -> rows survive a zero-grace sweep and warning is set; root returns -> rows resurrect with their original ids and the warning clears; deleting a file under a reachable root still purges), repo-level sweep-protection and sibling-prefix tests, orphan-purge exemption, and rootcheck unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scanner): probe uncompacted roots and take dead-root path on full outage Review follow-ups: (1) probe every configured path instead of the compacted traversal roots, so a nested child mount that dies under a reachable parent is still protected from the sweep; (2) when every configured root is unreachable, bypass the empty-root confirm flow (without consuming the one-time cleanup allowance), mark files missing, and raise dead_root instead of empty_root; (3) dead_root warning banner no longer shows empty-root confirm-deletion guidance as its fallback hint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(scanner): simplify dead-root protection plumbing - extract pathscope.CoverageClauses as the single builder for the exact-path + escaped prefix-LIKE root predicate; scanner's rootCoverageClauses delegates to it and catalog's excludeOrphansUnderProtectedPrefixes reuses it instead of hand-rolling the same clause loop - extract Scanner.sweepMissingAndReconcile to replace the identical trash-sweep + membership-reconcile + S3-image-cleanup block that was triplicated across the audiobook, ebook, and podcast scans (callers keep their flavor-specific log lines so messages stay constant) - add unreachableConfiguredRoots helper for the repeated probeUnreachableRoots(ctx, folder.ID, cleanScanRoots(folder.Paths)) expression in scanPaths and ScanFile - drop the unread Path field from rootcheck.Result - move the dead/empty-root warning text constants in AdminLibraries.tsx out of the middle of the import block Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): close dead-root protection gaps found in review Remediates the confirmed findings from the deep review of this PR: - Scoped audiobook scans (autoscan file events, subtree scans) ran the folder-wide sweep while probing only the scoped clone's Paths, so a healthy-subtree event could hard-delete a dead sibling root's rows. sweepMissingAndReconcile now reloads the folder's configured roots from the DB and probes them uncompacted, which also protects nested child mounts in the audiobook/ebook/podcast reconcilers. - A lost mount that leaves an empty, stat-able mountpoint probed as reachable and kept the historical purge timeline. A reachable root that is a literally empty directory while cataloged rows remain under it is now treated as suspect: rows are only marked missing, the sweep and orphan purge exempt it, dead_root is raised, and the mount-check endpoint reports it (additive suspect_empty field) instead of clearing the warning. Arming the one-time empty-cleanup allowance completes the deletion, including in the mixed case where other roots are healthy. Roots that still have directory entries keep the historical grace-then-purge path. - Confirmed empty cleanup (allow_empty_cleanup_once) no longer force-deletes rows under probe-dead roots: an outage is not a confirmation, so a dead sibling root's catalog survives a confirmed cleanout of a reachable empty root. - Root probes are now bounded (rootcheck.ProbeWithTimeout, 5s): a hung network mount degrades into the protected unreachable path with a probe_timeout error code instead of stalling every scan of the folder indefinitely. - Documented the cross-library limitation of the orphan-purge exemption next to the query it applies to. All behavior is pinned by new DB-backed tests (suspect-empty protection + confirmed completion, confirmed-cleanup dead-root survival, scoped/nested-root sweep protection, suspect-root query, probe timeout). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): address dead-root review findings --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
f377be9d6e |
feat(catalog): include per-episode overlay summaries on season episode listings (#364)
* feat(catalog): include per-episode overlay summaries on season episode listings Episode cards already render user-configurable overlay badges everywhere except the series/season detail pages: the season-episodes endpoint never included overlay_summary, so SeasonEpisodeGrid and EpisodeRow had nothing to render. Server: add overlay_summary (omitempty) to episodeResponse and populate it in buildEpisodeResponses via overlays.BuildSummary over the episode's already-loaded media files, access-filtered with FilterMediaFilesByAccess to match the browse/sections paths. No extra queries; the files were already batch-fetched for the files[] payload. Additive-only API change. Web: extend EpisodeListItem with overlay_summary, add overlayDataFromEpisodeListItem (shared extract helper), and render CardOverlays (variant="wide") on the episode stills in SeasonEpisodeGrid and EpisodeRow using useOverlayPrefs, matching the ContinueWatchingCard pattern. The hook already honors the admin kill switch and per-user prefs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(catalog): apply library restrictions in FilterMediaFilesByAccess without a quality ceiling FilterMediaFilesByAccess short-circuited whenever MaxPlaybackQuality was empty, skipping the AllowedLibraryIDs/DisabledLibraryIDs checks that FileAllowedByAccess enforces. For viewers with library restrictions but no quality ceiling, callers (episode listings, browse/section overlay summaries, item detail versions) received files from restricted libraries. Short-circuit only when no access criteria are set at all. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Suspense
|
e487253ad3 |
fix(search): refresh federation capability cache
Retry federated episode searches after the capability cache expires, clear the downgrade after a successful probe, and exercise lean ranking for episode and mixed catalog branches. |
||
|
Suspense
|
808883cf8f |
feat(search): add scalable TV episode search
Add mixed PostgreSQL and Meilisearch episode search with access-safe hydration, incremental indexing, and regression coverage for explicit all-media searches.\n\nPart of #396 |
||
|
QuickandGitHub
|
02b6648995 | feat(notifications): harden push relay credentials (#373) | ||
|
|
c1c110e3d2 |
feat(playback): improve web subtitles and track selection (#362)
* fix(player): keep text subtitles in sync across copy-mode restarts and sparse cue windows - Rebase already-loaded cues in place when streamOriginSeconds changes (copy-mode session restart) instead of leaving them offset by the delta. - Stop inferring end-of-input from where a window's cues stop; only the known media duration marks EOF, so a dialogue gap no longer silently ends prefetching for the rest of playback. - Anchor the first window fetch to the intended start position (resume target or pending seek) while the element still reports currentTime=0, and reset coverage on forward seeks past the fetched window. * fix(playback): align encoded transcode start with the declared segment boundary A mid-segment start (resume, seek restart, audio switch) spawned ffmpeg at the raw seek position while labeling its first segment with the grid number, whose synthetic-manifest start is up to one segment earlier. hls.js aligns the first fragment's content to that declared position, shifting the whole session's timeline late by seek mod segment_duration (0-2s): subtitles trail dialogue by a constant per-session offset and progress/resume positions drift by the same amount. Snap the ffmpeg start position down to the segment boundary for encoded sessions so declared and produced timelines match exactly; the player still seeks to the precise requested position. Copy-mode sessions serve ffmpeg's real manifest and keep the raw seek. * fix(api): forward http.Flusher through response-writer middleware wrappers Streamed subtitle extracts (and any progressive response) flush per chunk via an http.Flusher assertion, but none of the status-capturing middleware wrappers implemented Flush, so the assertion failed and cues sat in Go's response buffer until ffmpeg finished. On large remuxes where a 600s window takes 20s+ to demux, captions appeared only when the whole window completed instead of within the first seconds. Give every wrapper a Flush() (satisfies plain assertions, including chi's Compress) and Unwrap() (satisfies http.ResponseController). The jellycompat image-proxy tag rewriter flushes only in passthrough mode since it buffers JSON bodies for rewriting. Regression test asserts the API chain forwards Flush end to end. * feat(player): PGS subtitles honor size, position, and background settings Port the tvOS/iOS bitmap-cue styling to the web player. libpgs now decodes in worker mode but draws to a hidden source canvas on the main thread; a compositor detects cue regions from the frame's alpha channel and re-places them on a visible overlay canvas per the shared subtitle appearance settings: size scale (with the 0.85 authored-size compensation), vertical position preset (dialogue-band cues only — floating signs keep authored placement, matching the Apple implementation), and the background box. Font family, text color, and outline are baked into the source pixels and remain inapplicable. * fix(player): anchor PGS position presets to the text overlay's reference frame The initial port used silo-apple's 30/1080 bottom margin and video-relative lower-third/top anchors; the web text overlay anchors to a 16:9 reference frame with 7%/18% offsets that extends into the letterbox for wide content. Use the same anchors so PGS dialogue lands exactly where SRT text does. * feat(player): size PGS cues to match the text subtitle line height Replace the authored-size ladder (0.85 × font-size ratio) with per-cue text-line matching: the region detector reports the tallest text line inside each cue, and the compositor scales the cue so one line of bitmap text renders at the same pixel height as the SRT overlay's font at the current preset. Authored size differences between discs no longer leak through; upscaling is capped at 2.5× to keep small bitmaps from going blurry. * feat(subtitles): opt-in windowed PGS extraction to cut mid-file load latency PGS extracts always demuxed the source from byte 0, so starting a large remux mid-file meant minutes before the first bitmap cue. The web player now opts in to windowed extraction (?windowed=1&position=&duration=) and re-points libpgs at a fresh window on seeks and near coverage end; ffmpeg input-side -ss with -copyts keeps absolute source timestamps. Without the explicit opt-in the endpoint behaves byte-identically, so Apple/Android and other single-fetch consumers are unaffected. ASS remains unconditionally non-windowed (its header only exists at offset 0). * feat(playback): cache extracted PGS subtitle tracks Every selection of an embedded PGS track re-ran a full ffmpeg extract that demuxes the entire source file from byte 0 — minutes for a large remux — and responses were Cache-Control: no-store, so repeat selections, re-watches, and multiple viewers all paid full price. Add a disk cache for full-track .sup extracts under <transcode_dir>/subtitle-cache, created lazily: - Keyed by source path hash + subtitle stream ordinal + source mtime+size (encoded in the filename), so a replaced source file implicitly invalidates its entries; the source is stat'ed on every lookup. - Cache miss: ffmpeg stdout is teed to the response (first viewer still streams progressively, first-byte latency unchanged) and into a temp file that is fsynced and atomically renamed into the cache on clean ffmpeg exit. Any error — ffmpeg failure, client disconnect, tee write failure, or the source changing mid-extract — discards the temp file, so a partial entry is never served. - Cache hit: served via http.ServeContent (Range support, Content-Length, Last-Modified from the source mtime) with a revalidatable Cache-Control instead of no-store. - Concurrent requests for the same in-flight track run their own uncached extract (mutex + in-flight key set) rather than blocking on another client's connection. - Scan-on-commit LRU eviction under a 2 GiB cap (recency tracked by bumping entry mtime on hit; atime is unreliable under relatime), plus sweep of crash-orphaned .part temp files. - Windowed PGS requests (?windowed=) bypass the cache in both directions: their output covers only a slice of the track. Both the integrated API handler and the standalone proxy subtitle path share the same playback.SubtitleCache.ServeSUPExtract helper. VTT (already windowed and fast) and ASS (small) stay uncached. No API surface change. AI-use disclosure: implemented with Claude Code. * fix(playback): check Close error returns in subtitle cache paths Silence errcheck on the cache-hit defer and the test's simulated disk-full Close. AI-use disclosure: implemented with Claude Code. * feat(playback): warm PGS cache in background and window from cached track Windowed PGS requests bypassed the cache entirely, so every window fetch re-demuxed the multi-GB original file. Now a windowed miss kicks off a detached background warm (full-track extract into the cache, at most 2 concurrent server-wide, coalesced with client-driven fills), and once the entry exists windowed extracts read the 15-80MB cached .sup instead — seeks and re-enables become near-instant after the first load. Verified empirically that ffmpeg preserves absolute PTS when windowing a sup input. * feat(player): hold playback while PGS subtitle cues load When a PGS track is enabled (or a seek lands outside the fetched window), extraction takes seconds and dialogue could play unsubtitled. The player now pauses until the renderer's parsed data covers the playhead — tracked via libpgs' parsed-timestamp watermark, the exact predicate it renders by — showing a "Loading subtitles…" indicator after 500ms. User play/pause always wins over the hold, a 20s safety timeout prevents stranding playback, and background prefetch never pauses. If future libpgs versions reshape the observed internals the hook degrades to the old play-through behavior. * perf(player): shrink uncached PGS window to 600s Draining a windowed extract from the source reads the full interleaved container across the window (~1GB per 100s of remux on measured hardware); a 3600s window cost ~12GB of reads per fetch while cold. Once the server cache is warm a window costs milliseconds regardless of size, so smaller windows only add trivially cheap re-fetches. * feat(subtitles): burn in PGS/bitmap subtitles for the web player The web player rendered PGS client-side via libpgs, which required extracting the .sup track — a cold ffmpeg demux that took seconds even windowed, since c:s copy still reads the whole interleaved container up to the playhead. Every other server (Plex, Jellyfin default, Emby) burns image subtitles into the video instead, and that is the only path with no per-seek extraction cost. Selecting a bitmap subtitle (PGS/DVD/DVB) now restarts the transcode with subtitle_burn_in at the current aligned position, reusing the same restart machinery as an audio/quality switch so the segment-boundary timeline alignment holds. The server composites the decoded subtitle onto the video with an overlay filter_complex graph (libass's subtitles= filter is text-only); overlay runs at native resolution before any target scaling, and hardware pipelines round-trip through CPU like the text path. Burn-in forces a video encode, so copy-video recipes are upgraded to h264 both client- and server-side. Text subtitles keep the instant, styled, client-side path. The .sup streaming endpoints, cache, and windowing are retained for the Apple client, which renders PGS natively. The now-dead web PGS stack (usePGSSubtitles, pgsPlacement, libpgs dep) is removed. Tradeoff: bitmap subtitles no longer honor web appearance settings (baked into the video) and toggling one restarts the transcode (~1-2s buffering), matching Plex behavior. * fix(player): rebuild text subtitle track when turning off PGS burn-in Selecting an SRT track that turned off bitmap burn-in restarted the transcode, and the client TextTrack built in that same moment was orphaned when the <video> element reloaded, so the subtitles never rendered (and a seek could not recover the dead track). Rebuild the text track once the new stream settles, gated on the burn-in-off transition so quality/audio switches and copy-mode seek restarts keep their subtitles without a needless re-extract. * fix(player): render web subtitles behind the control HUD The text subtitle overlay sat at z-20, above the controls layer (z-10), so cues painted over the bottom HUD and cluttered the control bar. Drop it to z-[5] — above the video, below the controls — so the HUD paints over the cues while it is visible. When controls are hidden the whole controls layer is opacity-0, so cues remain fully visible. * feat(player): lift web subtitles above the control bar while it's visible Rather than hiding bottom-anchored cues behind the HUD, raise them just above the control bar (measured height + a small gap) whenever the bar is visible in the foreground player, then settle them back when it hides. The bar is a roughly fixed pixel height while the cue offset scales with the player, so the bar is measured via ResizeObserver rather than hardcoded. Top-anchored cues never collide with the bottom HUD, so they stay put. z-[5] is retained as a safety so any residual overlap tucks behind the bar. * fix(player): coalesce same-tick transcode restarts into one dispatch Starting playback with a persisted bitmap subtitle fired transcode/start twice within milliseconds: the auto-start effect dispatched before subtitle auto-selection restored the burn-in, whose effect then forced a second start. The first request was already on the wire (no abort signal was passed to fetch), so the server spawned an ffmpeg only to kill it for the second start — visible in production as an ffmpeg exit error ~1ms after every such session start, and slowing time to first frame. Defer the network dispatch by one macrotask so back-to-back restart calls in a tick collapse into a single request carrying the final parameters; state updates stay synchronous. Pass the abort signal into playerFetch so a superseded in-flight request is actually cancelled, and drop any deferred dispatch on unmount so a stray transcode/start cannot land after the session's exit DELETE. * fix(catalog): resolve effective subtitle defaults for movie item details Movie pre-play subtitle selectors were missing the effective defaults (including per-item overrides saved from a previous play) that episodes and watch payloads already resolve. Extract applyToItemDetail/ applyToWatchDetail helpers and apply defaults for movies in buildMediaItemDetail. The SubtitlesPopover now also eagerly loads downloaded subtitles when the saved preference points at one so the closed trigger's Auto summary reflects the override. * feat(player): scale subtitle font size with the rendered video Replace fixed rem font sizes with px values defined at a 720px 16:9 reference height, scaled proportionally with the actually-rendered video (object-fit: contain) so subtitles keep the same relative size as the window grows or shrinks, with a 12px legibility floor. Rename useSubtitlePositionStyle to useSubtitleLayout, returning both the position style and the font scale, and add unit tests for the appearance helpers. * fix(player): satisfy strict index checks in transcode quality test * feat(player): let the pre-play Auto option clear the saved subtitle override A manual in-player subtitle selection persists as an 'always' override for that movie/series, but nothing in the UI could undo it — auto selection stayed pinned to the chosen track forever. Choosing 'Auto' in the pre-play subtitles popover now also deletes the stored preference (movie content ID / episode series ID) and invalidates item details so profile-level auto selection applies again. * feat(player): persist pre-play subtitle selections as the item override Choosing a track (or Off) in the pre-play subtitles popover only lived in component state: it applied to that playback session but vanished on returning to the detail page. Persist it through PUT /subtitle-prefs — the same 'always'/'off' override a manual in-player selection saves — keyed by movie content ID or episode series ID, and invalidate item details so the effective defaults reflect it immediately. * feat(ui): show the saved subtitle override and richer pre-play pill summaries A stored per-item override displayed as 'Auto: <language>', hiding both that an override exists and which track it is. The pre-play subtitle pill now shows the resolved track directly (name with (SDH)/(Forced) markers plus format, skipping markers the name already carries), the matching list row gets the checkmark instead of Auto, and the Auto row reads 'Reset to profile defaults'. Subtitle, audio, edition, and version pill summaries also truncate much later (max-w-44/sm:max-w-64). * fix(player): recover text subtitles from stream reloads and failed window fetches Three failure modes could silently freeze or stop web text subtitles: - A stream restart (seek-triggered transcode restart, quality/audio switch) reloads the <video> element and can orphan the programmatic TextTrack — cuechange stops firing and the last cue freezes on screen. Only the PGS-burn-in-off transition rebuilt the track. Now every settled stream URL change bumps the generation, and the rebuild carries loaded cues (converted back to source time) and window coverage over so it costs no refetch. - The sliding-window fetcher committed windowEnd before the fetch ran, so a failed or hung window counted as covered and was never retried — subtitles silently stopped for up to 10 minutes. Coverage now commits only after the window streams in fully; failures leave the range uncovered and retry after a 5s backoff. - A hung extraction (one fetch in flight at a time, no deadline) blocked every future window for the session. Reads now arm a 30s stall timer that aborts a response which stops delivering chunks; slow-but- progressing streams keep resetting it. Diagnosed from a session where ffmpeg took 69s to stream one subtitle window and a transcode restart landed mid-fetch, freezing the active cue. * fix(playback): keep subtitle selections stable across file changes * fix(web): clarify subtitle labels and positioning * fix(player): hide HUD when pointer leaves * fix(web): tidy subtitle track badges * fix(playback): remap audio tracks across file versions * fix(web): tidy audio track labels * docs(playback): clarify bitmap subtitle appearance * fix(playback): preserve selection state on restart * fix(http): preserve response state across flushes * fix(web): preserve pending quality for burn-in * fix(playback): preserve subtitle inventory identity --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
8b9149a789 |
feat(catalog): remove smart collection item limit cap (#340)
* feat(catalog): remove smart collection item limit cap MaxSmartCollectionItemLimit silently clamped every smart collection to 500 items, and an unset limit defaulted to 100. On servers with large libraries this truncates collections with no indication to the admin — a per-language smart collection (original_language is fr) has 10,625 real matches and showed under 1% of them. Remove the cap and make an unset limit resolve the full result set: the default becomes a large sentinel so every consumer's int plumbing (SQL LIMIT) keeps working. An explicit positive limit in the query definition is still honored as-is — membership size is an admin decision, and every read path that serves collection items either paginates in SQL or scales with the actual collection size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(jellycompat): page smart collection browse in SQL The Jellyfin BoxSet children path resolved a smart (live-query) collection's entire membership and sliced one page locally. With the item cap removed (smart collection membership is now uncapped, an admin decision), a large collection browsed from a Jellyfin client materialized tens of thousands of rows per request — latency and memory scaling with collection size. For the default browse (no explicit SortBy), page the smart query directly in SQL via QueryExecutor.PreviewPage(offset, limit), hydrating only the requested page. Per-request work is now proportional to page size, not collection size. TotalRecordCount still reports true membership (the count query caps at the query's item limit), and past-the-end pages preserve the real total. Shared query prep is extracted into prepareSmartCollectionQuery, used by both the new paged path and the existing full-resolution path (still used by the explicit-sort allowlist branch, whose output is already SQL-paginated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> |
||
|
|
4d99597966 |
feat(watchlist): honor provider list order by default (#352)
The watchlist catalog page silently returned the stored list order while its sort dropdown claimed "Date Added", and mirroring a provider's list order (e.g. MDBList) was off by default, so synced watchlists appeared in first-sync-time order with no way to tell what was happening. - Web: the watchlist source now uses the same source-order sentinel as collections — the dropdown shows "List Order" as the default, and an explicit "Date Added" pick sends sort=added_at instead of being stripped (previously indistinguishable from the default). - Server: on personal lists (watchlist/favorites) an explicit added_at sort now takes the source-order path, where added_at means "date added to the list"; the query executor path sorted by the library's created_at instead. History keeps the executor path since its ID loading ignores the sort. - watchsync: new connections default sync_watchlist_order_enabled to true, with a migration flipping existing rows to match. Providers without the provides_watchlist_order capability ignore the flag at sync time. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e96a8a0cf8 |
feat(search): binary-quantized embedder vectors — optional, default-on for fresh installs (#351)
* feat(search): binary quantization setting for the Meilisearch embedder
New server setting catalog.search.meilisearch.binary_quantized
(default false) threads into the embedder index settings
("binaryQuantized": true) and into the schema-version hash, so flipping
it closes the sync gate and mandates a rebuild in both directions —
Meilisearch cannot de/re-quantize an index in place.
With 3072-dimensional embeddings this cuts vector storage ~32x
(≈12KB → 384B per document), keeping the whole vector store in page
cache: rebuilds and hybrid queries get sharply cheaper. Hybrid search
(keyword + semantic) cushions the small relevance cost of sign-only
vectors.
The hash token is appended only when the flag is set, so indexes built
before this change keep their schema version while it stays off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(search): binary quantization default-on for fresh installs + admin toggle
- Migration seeds catalog.search.meilisearch.binary_quantized=true only
when no active catalog search index exists. Existing deployments stay
unset (= off): flipping quantization changes the index schema-version
identity, which closes the incremental-sync gate until a full rebuild
runs — that must never happen implicitly on upgrade. Fresh installs
have no index yet, so their first rebuild simply starts quantized.
- Search settings page gains the toggle with an explicit
"requires a full index rebuild" warning, a status row, and settings-
search keywords.
Prod benchmark (607.9k docs, 3072-dim vectors, N=10 medians, replicated):
hybrid 0.5 unchanged (7.5ms float vs 8.0ms quantized, within ±2ms
keyword-control jitter); pure semantic 9ms → 4ms; on-disk index 18G →
8.3G; rebuild duration unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(search): address review feedback on binary-quantized embedder
- Add catalog.search.meilisearch.binary_quantized to the restart-required
registry. The provider freezes BinaryQuantized into MeilisearchProviderConfig
at construction, so without this a toggle-then-rebuild in the same process
builds a quantized schema while the live provider still compares against the
old value and falls back until restart (Codex P2).
- Validate binary_quantized in HandleUpdateSetting, mirroring semantic_enabled.
A raw API write of a non-bool previously persisted unnormalized, then failed
CatalogSearchSettingsFromMap on load and silently reverted the entire search
config to Postgres defaults.
- Gate the binary_quantized token in the schema-version identity on
semanticEnabled: with semantic off the index has no embedders, so the flag
has no on-index effect and must not force a pointless rebuild. Stays
byte-identical to a pre-flag index. Covered by a new test.
- Clarify the seed migration comment (guard is "no active index", which also
covers Meilisearch-configured-but-never-indexed deployments) and the UI hint
(~30x smaller raw vectors, index roughly halves; only applies with semantic).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
3e340552a8 |
fix(search): raise Meilisearch task-wait timeout from 5m to 2h (#347)
Semantic (vector-embedded) index batches can take Meilisearch many minutes on modest hardware, and Meilisearch auto-batches consecutive queued document tasks — with rebuild_task_queue_depth tasks in flight the oldest task's completion covers the whole fused unit. WaitTask's 5-minute default deadline killed four prod rebuilds mid-batch (2026-06-29 x2, 2026-07-08 x2) with "context deadline exceeded", each time leaving incremental sync blocked on a stale schema version and an unbounded event backlog (1.6M+ events). The wait still exits immediately when the task reaches a terminal state (succeeded/failed/canceled); the deadline only bounds tasks genuinely still processing, where giving up guarantees rebuild failure and gains nothing. Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
a2ef26bece |
perf: root-cause fixes for endpoints still slow after #292 (NextUp, series badges, resume tail, subtitle fonts) (#350)
* docs(plans): root-cause analysis for endpoints still slow after PR #292 Five endpoint groups stayed slow after the home/Continue Watching/Latest latency work shipped: Resume (110s p95), NextUp (17s p95), Latest (17s), /Items, and the home sections routes. The caps and caches from PR #292 are live in the deployed binary; they bounded how many rows the loops touch but not what each underlying query costs. Documents the four confirmed root causes (4.3M stale completed-with-position progress rows + missing resume index, unbounded next-up anchor scan, per-episode series rollup fanout, two index-starved history/scanner paths) with live EXPLAIN ANALYZE measurements and the fix plan implemented by the follow-up commits. AI-use disclosure: analysis and doc produced with AI (Claude) assistance. * perf(catalog): bound the global next-up anchor scan to recent completions The completed_episodes CTE in buildListNextUpQuery derived per-series anchors from the profile's ENTIRE completed history — DISTINCT ON over 233k rows joined to episodes for the worst bulk-import profile, then a per-series LATERAL that scans every episode of a fully-watched series before yielding nothing. 648 slow executions in a 19h window, 44.7s worst; this drove /Shows/NextUp (17.1s p95) and the next-up injection on the native home sections aggregate. Global queries now derive anchors from the profile's nextUpAnchorMaxRows (500) most recent completed rows — an ordered index walk on idx_uwp_profile_completed, with the hidden-items exclusion and date cutoff applied inside the bounded scan so hidden/old rows never consume the anchor budget. A next-up rail surfaces ~24 series; the 500 most recent completions cover every series that can realistically rank on it. Series-scoped calls (the show-detail tile) keep the unbounded shape: they must anchor on the series' last completed episode no matter how long ago it was watched, and are naturally bounded by one series. Measured on the live worst-case profile with the exact generated SQL: 44.7s worst / ~2.6s avg before; 10ms after (together with the one-time stale-resume-point data repair applied directly to the deployment DB — see docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md). AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat,userstore): aggregate series watch-state rollup in SQL The series Played/UnplayedItemCount badge on list rails (per-library Latest, library browse, search results) and series detail pages was computed by materializing EVERY episode of every series on the page (episodeRepo.ListBySeriesIDs) and then batching per-episode progress+history lookups in 500-id chunks. A 50-series page of an episode-heavy library (Sports) expanded to 32,467 episode rows and ~65 sequential queries — measured 17-18s per /Items/Latest request, and PR #292's cached Latest fast path pays it on every response for series libraries. The same fanout made /Items?searchTerm=... slow whenever the result set was mostly series (Meilisearch itself answers in milliseconds). New optional store capability userstore.SeriesEpisodeRollupStore, implemented by PostgresUserStore as one GROUP BY e.series_id aggregate with semantics identical to the chunked path (episode availability via episode_libraries, hidden-items visibility on progress rows, completed-history fold, in-progress = not watched with position > 0 — verified value-for-value against the old semantics on a real 1,586-episode series). enrichSeriesListUserData and enrichDetailUserData use it when present; SQLite-backed stores and rollup query failures keep the existing chunked path as fallback. catalog.SeasonUserDataFromCounts pins the counts-to-DTO mapping to EpisodeRollupUserData. Measured on the live worst-case profile against the real 50-series Sports Latest page: ~17s of chunked round-trips before, 119ms in one query after. Part of docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md. AI-use disclosure: implemented with AI (Claude) assistance. * perf(catalog): bound superseded-episode completed walk to recent history The Resume / Continue Watching superseded-episode filter loaded a profile's *entire* completed history into memory on every request that contained an in-progress episode: CompletedProgressSnapshots paged user_watch_progress WHERE completed=TRUE with no upper bound. The 2026-07-06 slow-query comparison showed this surviving as a 60-116s Resume tail even after the in-progress index landed live, because the 4.3M zeroed Plex-import rows are still completed=TRUE and were re-walked every load. A completed episode can only supersede an in-progress one it was finished more recently than (the query gates on done_progress.updated_at > ip_progress.updated_at), so only completed rows newer than the oldest in-progress entry can matter. Compute that cutoff in SupersededEpisodeProgressIDs and pass it to CompletedProgressSnapshots, which — since the completed listing is ordered updated_at DESC — stops paging as soon as it crosses the cutoff. Import-heavy profiles whose back-catalogue predates their current in-progress items now stop on the first page instead of paging hundreds of thousands of irrelevant rows. Correctness is unchanged: no relevant superseding row is excluded. * perf(catalog): hard-cap superseded-episode completed walk at 5 pages The updated_at cutoff added in the previous commit bounds the completed walk on the relevance axis, but a very old in-progress entry sitting behind a large volume of newer completions could still page deep. Add a 5-page (2,500-row) hard backstop on top of the cutoff: normal profiles still stop on page one via the cutoff, and only the adversarial tail hits the cap. When it engages the tail of the completed set goes unscanned, so a superseded episode could momentarily survive on Continue Watching — we log a warning when that happens (with profile_id + rows scanned) rather than mis-filter silently, and it self-corrects once the stale in-progress entry ages out of the scanned window. * perf(playback): extract subtitle fonts in a single ffmpeg pass Embedded ASS/SSA font extraction spawned one ffmpeg process per font attachment, each re-opening the (usually CephFS-backed) media file. Anime releases carry 15-47 fonts, so the per-spawn file-open cost dominated and pushed GET /api/v1/stream/{sid}/subtitles/{track}/fonts to a 17-60 s plateau (p95 ~33 s in the live logs). Collapse the N spawns into one ffmpeg invocation that dumps every attachment to a temp dir (-dump_attachment:idx path ... -i file -map 0:t? -c copy), then read the files back. The file is opened once instead of N times, taking p95 from ~30 s to ~1-2 s with no change to output. Safety is preserved. The 32-attachment / 32 MiB caps still apply: attachment size is stat'd before read so an over-limit font never enters memory, and a watchdog polls the dump dir and kills ffmpeg if its on-disk output crosses the cap -- restoring the hard bound the old pipe-per-attachment reader enforced by killing at maxBytes+1, so a container with oversized "font" attachments can't fill the disk. Part of the slow-endpoint follow-up; see slow-query-analysis/subtitle-fonts-extraction-findings.md. * fix(review): report enforced font-byte cap; correct doc subtitle scope Address PR #350 review: - dumpFontAttachments reported the maxSubtitleFontBytes package constant in both over-limit errors instead of the maxBytes argument the caller passed, so the message misstated the enforced bound whenever a different cap was in effect (as the tests use). Interpolate maxBytes in both messages. - The root-cause plan claimed subtitle extraction was 'out of scope' while the branch actually optimizes /subtitles/{track}/fonts. Scope the out-of-scope note to subtitle *track* conversion and record the fonts single-pass work as deliverable 5. |
||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
11704a1701 |
feat(sections): fix broken home section templates and add six new ones (#332)
* feat(sections): fix broken home section templates and add six new ones Fixes templates that silently produced nothing: - award_winners: hide from gallery (resolver is a stub until award data exists); saved sections keep resolving - seasonal_themed: christmas/st_patricks/thanksgiving get an interim title-keyword resolver, and multi-theme selection skips themes without an executable query so a data-less theme can no longer black out the section during its own window (previously killed the section all of December) - taste_match: empty genre now auto-picks the profile's strongest taste cluster (fallback: server top genre); the default preset was permanently empty - because_you_watched: honor the recipe's anchor_item_id key (fetcher only read legacy source_item_id, so pinning an anchor did nothing) - editorial_spotlight: reject subject_type=franchise (validated but could never resolve); fix drawer misrepresenting pinned presets as auto-rotate - admin_curated_list: add a catalog-search item picker so Editor's Picks is actually addable; block saving an empty list; hide admin_only recipes from profile-facing galleries - discovery fetchers (hidden_gems, forgotten_favorites, critically_acclaimed): honor single/multi library scope, intersected with viewer access; implement hidden_gems max_play_count New templates: returning_shows (new season of shows you've watched), genre_roulette (rotating top-genre spotlight with title override), anniversaries (milestone release anniversaries this month), short_watches (well-rated movies under a runtime cap), family_movie_night seasonal theme (Fri/Sat evenings), and a "New in 4K" format_showcase preset via a new sort=recent param. Adds a blanket test asserting every visible gallery preset's defaults pass its own recipe validation — the gap that let taste_match and Editor's Picks ship broken. New SQL shapes validated with EXPLAIN against the dev database. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sections): address PR #332 review findings Codex review: - returning_shows: the new-season file check now applies the effective library scope (section scope ∩ viewer-allowed, minus disabled) to media_files.media_folder_id, so an episode file that only exists in an out-of-scope folder can no longer surface the series - buildLibraryScope: replaced the media_item_libraries row join with EXISTS / NOT EXISTS semi-joins. An item in several in-scope libraries now yields exactly one row in the non-GROUP BY rails (short_watches, anniversaries, seasonal keyword, format_showcase, new_to_library, ...), and the disabled-library check is item-level, closing the join-row leak where membership in an allowed library masked membership in a disabled one. Deny-only mode keeps the positive-membership guard, mirroring catalog's appendDiscoveryLibraryScope. CodeRabbit review: - recommendations reader: a taste cluster whose cached items are entirely filtered out now falls through to the next cluster / global fallback instead of returning an empty row - genre_roulette: multi-library scopes get distinct rotation seeds - returning_shows: reject negative lookback_days at validation - shared oneOf() enum validator replaces per-recipe switch duplication - SeasonalTitleOverride usable-filter contract covered by a direct test - web NumberParamField: integer-only guard + step=1 (backend fields are Go ints; fractional values failed unmarshalling at save) - curated list picker: search failures show an error instead of a misleading "No matches."; pre-existing item_ids hydrate display titles via the watch-detail endpoint instead of rendering raw ids Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e140bd9424 |
feat(metadata,scanner): trailers and extras for movies and series (#322)
* feat(metadata,scanner): trailers and extras for movies and series Remote provider videos (TMDB trailers/teasers/featurettes/...) are fetched through the unified match/refresh pipeline into the new item_videos table, filtered per-library via media_folders.trailer_kinds, merged across providers with site/provider dedup, and lockable via FieldVideos. The movie scanner stops discarding supplemental directories (Trailers/, Featurettes/, Behind The Scenes/, ...) and classifies them — plus Jellyfin-style filename suffixes (-trailer, -behindthescenes, ...) and series-root supplemental dirs — into the new media_extras entity backed by ordinary media_files rows (extra_id ownership, content_id/episode_id NULL so existing version/matching queries stay structurally blind to extras). Series Extras/SxxExx season-0 mapping is unchanged. Extras are playable watch targets via a GetWatchDetail fallback tier (episodes precedent), with contentid.ForLocal minting stable ids. API: ItemDetail gains additive videos/extras arrays (single + batch parity); library settings expose trailer_kinds. jellycompat now populates RemoteTrailers, LocalTrailerCount/SpecialFeatureCount, and serves real /LocalTrailers + /SpecialFeatures items playable through PlaybackInfo. Requires silo-plugin-sdk v0.9.0 (VideoRecord) before go.mod can bump; builds locally via go.work against the SDK feat/metadata-videos branch. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): trailers and extras sections, library trailer-kinds setting TrailersSection (YouTube thumbnails + youtube-nocookie modal) and ExtrasSection (plays extras through the standard watch controller) on movie and series detail pages; admin library form gains a trailer-kinds allow-list synced with the server default (all provider kinds), now also honored on library create. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): scan extra_id in scanMediaFiles; review cleanups scanMediaFiles (the plural row scanner behind GetByContentID/GetByFolder/ GetByExtraID and 20+ other queries) was missing the scan destination for the new extra_id column, which would have failed every media-file read at runtime with a column/destination count mismatch. Also: extend the batch equivalence test to seed item_videos/media_extras so the new videos/extras prefetch wiring is actually proven; drop the one-off pgxRows interface for the repo-wide pgx.Rows convention; reuse formatClock instead of a third duration formatter in ExtrasSection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): bump silo-plugin-sdk to v0.9.0 for VideoRecord Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(matching): exclude extras files from match queues and bulk content linking Dev verification caught extras media_files rows (content_id NULL by design) being swept into the movie/series match queues and the root-claim bulk relink: a '-featurette' suffix extra was matched onto its parent as a version, and a Trailers/ file minted a spurious local skeleton item that shadowed the extra's watch target. Add 'extra_id IS NULL' to the queue eligibility conditions, root/group claim relinks, observed-root content assignment, and the admin unmatched-files listing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): authorize local extras files through their parent item Dev verification: playback/start (and the shared MediaFileAuthorizer used by markers/subtitles/ebook reader) resolved file ownership only via episode_id/content_id, so extras files (extra_id only) 404ed. Add an ExtraLookup tier that resolves media_extras and gates on the parent item's access, mirroring the episode->series pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): resolve local extras through GetItemDetail for compat playback jellycompat PlaybackInfo (and any per-item consumer resolving arbitrary content ids) goes through GetItemDetail, which lacked the extras tier that GetWatchDetail has — so Jellyfin clients got zero MediaSources for extras. Add buildExtraItemDetail (minimal detail + ordinary playback surface, parent-gated access) as the fourth resolution tier, and map the extra type to Jellyfin's Video kind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): allow youtube-nocookie embeds in CSP; trailer modal a11y The frontend CSP's frame-src blocked the trailer modal's youtube-nocookie.com iframe (found on dev verification). Also add the missing sr-only DialogDescription and drop the redundant allowFullScreen attribute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address PR review findings for trailers/extras - Extras watch/item detail no longer stamp SeriesID/SeriesTitle for movie-owned extras (players key episodic post-roll flows off series_id); series-owned extras keep them (Codex). - processExtraFiles resolves the parent and upserts media_extras before the unchanged fast-path, and the fast-path now also compares mtime, so rematched parents / reclassified kinds / same-size replacements converge (Codex + CodeRabbit). - media_files upsert clears content/episode linkage atomically when extra_id is set (ownership mutual exclusion in one statement); the now-redundant MarkFileAsExtra helper is removed (CodeRabbit). - ScanFile's extras branch runs syncPresentLibraryState + reconcileLibraryMemberships so converting a primary file to an extra cleans stale library membership immediately (CodeRabbit). - media_extras migration adds the media_files FK as NOT VALID + VALIDATE to avoid a full-scan exclusive lock on large tables (CodeRabbit). - trailer_kinds input is trimmed/lowercased/deduped and unknown values are dropped instead of silently widening the allow-list to 'other' (CodeRabbit). - Extras authorization branches match the episode branch's posture: unconfigured lookup is a config error, nil extra is a 404 (CodeRabbit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0fb5afe479 |
feat(matching): split wrongly merged versions with watch-state reattribution; anchor group keys on provider tags (#319)
* feat(matching): split wrongly merged versions, reattribute watch state, anchor group keys on provider tags
Wrong merges (two titles normalizing to the same title+year key) stacked
different films as fake "versions" of one item with no in-app repair, and
explicit {tmdb-…}/[imdb-…] folder tags could not prevent it because the
content-group key ignored provider IDs entirely. Merges also silently
orphaned all per-user watch state.
- Anchor group keys on structured provider tags: same tag always groups,
different tags can never merge; untagged files keep title+year keys.
- media_identity_overrides: path-scoped (root/file) forced identities applied
during group inference, so admin splits survive rescans.
- internal/catalog/reattribute: shared user-state mover — exact moves for
file-linked rows, evidence-based user_watch_history classification via the
playback session log, newest-wins progress conflicts; wired into
rebindItemToExistingItem to stop merge orphaning (with S/E episode mapping).
- POST /admin/items/{id}/split (dry-run = full transaction + rollback, so
previews are exact), POST /admin/items/{id}/merge, GET /admin/items/{id}/files.
- Web admin: Split Versions dialog (files by folder → candidate search →
preview → split), Resolve link from ambiguous-roots diagnostics.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(reattribute): classify history before moving session log; cover managed downloads and series-scoped preferences
Review findings on #319, all reproduced against a migrated scratch database:
- moveFileSubset re-pointed playback_history_admin before the history
evidence query ran, erasing exactly the evidence proving a profile's plays
were all on moved files — their history stayed behind as ambiguous.
History classification now runs first; the pre-fix code demonstrably fails
TestRun_HistoryEvidenceClassification.
- Managed offline downloads (downloads.content_id/episode_id) were not
remapped on split or merge, stranding rows on the old id. Now moved per
file on splits and swept per id pair on merges/episode re-anchoring.
- Series merges left user_audio_preferences, user_subtitle_preferences,
user_series_playback_preferences (series_id-keyed) and the denormalized
user_home_item_dismissals.series_id behind. All four now move, mirroring
the provider-merge remap.
All five reattribute DB tests now verified green against PostgreSQL, with
new coverage for managed downloads, subtitle preferences, and dismissal
series ids.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
QuickandClaude Fable 5
|
28340ad6f5 |
feat(watchlist): hide fully-watched series instead of removing them
Removing a series from the watchlist on full watch stranded it once new episodes aired: nothing ever re-added it. Split the behavior by type: - watchlist.Maintainer now auto-removes only fully-watched movies (still propagating removals to connected providers). - Series stay on the watchlist; the new catalog.WatchlistVisibility filter hides series whose available episodes are all completed on the display surfaces (sections rail, catalog watchlist source, GET /watchlist). A newly added episode makes the series reappear on the next fetch, and nothing is synced upstream since the entry never leaves the list. Sync, recommendations, notifications, and the watchlist check endpoint intentionally keep seeing the full list. The filter honors the existing per-profile remove-watched preference and uses batch lookups only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
69dde62fcd |
feat(sections): configurable sort for watchlist/favorites sections
Watchlist and favorites sections keep their stored list order by default (provider-synced positions first, then newest-added). An optional sort/order config now supports title, release date, IMDb rating, and date-added-to-list ordering, applied consistently on the home rail and the catalog "see all" page. added_at is resolved from the list entries via a shared OrderPersonalListIDs helper since it is distinct from the catalog's library added_at. The section editor gains a Sort dropdown. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
42602b7896 |
feat(policy): access groups + embedded OPA policy engine with decision audit log (#282)
* docs(policy): add OPA policy engine design spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(deps): add OPA v1.18.2 SDK for the policy engine Pulls github.com/open-policy-agent/opa v1.18.2 (policy engine core for the upcoming internal/policy subsystem) and the transitive upgrades go mod tidy applied (otel 1.44, grpc 1.81.1, prometheus/common 0.67.5). Full build verified. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add OPA engine core, vendor scope policy, and parity suite New internal/policy package (dead code — nothing wires into request paths yet): prepared-query Engine with 25ms eval timeout and fail-closed decode, typed PDP.ResolveViewerScope, go:embed vendor bundle, capabilities lockdown for future admin-authored Rego, and vendor scope.rego reproducing access.Resolver.Resolve (library intersection, disabled-library handling, quality/rating ceilings) with a narrowing-only silo_custom.scope.override extension hook. Parity proven by 1368 dual-execution subtests against the real access.Resolver, including the nil-vs-empty AllowedLibraryIDs battery and quality/rating variation; rank tables are test-pinned to internal/access. Rego unit tests run via opa/v1/tester inside go test. Bench: ~106µs/op per scope decision incl. input marshaling. Also restores the OPA requirement to go.mod (the earlier deps commit ran go mod tidy before any import existed, so tidy dropped it). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed, corrected (quality.allowed raw-file-rank divergence), and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy document store, foundation schema, and compile-check policy_foundation migration: policy_documents (one enabled doc per domain via partial unique index — two enabled docs would define override twice and conflict at eval), immutable policy_document_versions, single-row policy_generation counter, and the partitioned policy_decisions log table (daily range partitions, no FK, denial partial index). PolicyStore: transactional version numbering (FOR UPDATE), activation that verifies compiled_ok and bumps the generation in the same tx, enable/disable with typed ErrDomainAlreadyEnabled, and a delete guard for documents with an active version. CompileCheck sandboxes admin Rego: locked capabilities (no http.send/net.*/opa.runtime), enforced silo_custom.<domain> package path, vendor+stub layering, 2s budget, structured row/col errors. Engine gains NewEngineWithCustom / NewEngineFromStore with WARN-and-skip for invalid custom rows. DB-backed tests verified against a migrated Postgres (concurrent version numbering, atomic generation bumps, activation guards). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (domain constants extracted). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy System lifecycle with hot reload and cross-node invalidation policy.System owns one long-lived Engine and reloads it in place when policy documents change: EventPolicyChanged on the existing ChannelAdmin bus (new cache event constant) plus a 60s generation-poll fallback for Redis-less deployments, with a generation-consistent snapshot read. Vendor compile failure is startup-fatal; store/custom failures degrade to vendor-only and the poll loop heals them; runtime reload failures keep the last known-good engine. NotifyChanged gives the future admin handlers synchronous local reload + cross-node publish. Wiring: constructed in integrated/api modes only, PolicySystem field on api.Dependencies (unused by routes yet), policy.eval_timeout_ms setting (hot-reloaded via configWatcher.OnChange; default 25ms). Verified by a full server boot smoke and DB-backed convergence tests (event + poll paths, degraded boot, last-known-good). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add async decision logging with sampling, retention, and query repo DecisionLogger batch-inserts each node's policy decisions straight to the partitioned policy_decisions table via a non-blocking buffered channel (drop-and-count on overflow — logging never adds latency to or fails a decision). Scope decisions sample 1-in-N (default 50, setting policy.decision_log_scope_sample_rate); denials and eval errors always log; input/result JSON samples only at policy.decision_log_verbosity= verbose. Cursor-paginated DecisionRepository backs the upcoming admin log viewer. Retention via partman (daily partitions) and a PolicyDecisionLogCleanupTask honoring policy.decision_log_retention_days (default 14). PDP emits entries per evaluation; the System owns the logger lifecycle and settings hot-reload. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (removed an unused, unsynchronized PDP setter). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): add admin policy management API and capability endpoint /api/v1/policy/capability (authenticated feature detection) plus the acting-admin /api/v1/admin/policy surface: vendor Rego viewer, document CRUD with the one-enabled-per-domain conflict mapped to 409, immutable version creation (compile-checked; failed versions persist as audit history with structured row/col errors and can never activate), activate/rollback with synchronous reload + cross-node invalidation via System.NotifyChanged, stateless validate, throwaway-bundle simulate (never touches the live engine, never logs decisions), and cursor-paginated decision-log queries. Routes mount only when the policy system is wired, keeping proxy/transcode modes untouched. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (seeded the FK'd test user; replaced an unchecked fmt.Sscanf with strconv). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add /admin/policy workspace with Rego editor, simulate, and decision log New Policy admin page (System nav group): documents list with one-enabled-per-domain conflict handling, CodeMirror 6 Rego editor (hand-rolled StreamLanguage mode) with server compile issues rendered as inline lint diagnostics, explicit Save-version vs Activate flow with confirm, read-only vendor module viewer, simulate panel with seeded example inputs, version history with rollback, and a cursor-paginated decision-log browser. Capability-gated via /policy/capability. Adds the three decision-log settings to Log Retention. First code-editor dependency in web/ (@uiw/react-codemirror + @codemirror/*), decided in the design spec. Implementation drafted by Codex (GPT-5.5) via codex exec; verified here (lint, format:check, tsc --noEmit, vitest policy suites). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for viewer scope resolution policy.ViewerResolver implements the ViewerResolver interface backed by PDP.ResolveViewerScope and replaces access.Resolver at all five construction sites: router viewer middleware, notifications scopes, the reconciler, jellycompat's scope filter, and the ABS resolver (which now accepts a pre-built resolver, preserving its PIN-at-login semantics). PIN/profile-token verification and disabled-library loading are extracted into shared exported helpers used by both implementations, so the legacy resolver stays compiled as the parity reference with identical behavior. The adapter lives in internal/policy (which already depends on internal/access transitively) — direct typed PDP calls, no new import cycle. Sites without a policy system (proxy modes, bare test routers) keep the legacy resolver until the cleanup phase. Verified: full test suite green (jellycompat TestBeginWebOperation* and one playback GPU test are pre-existing failures, confirmed identical on main), 1368-case parity suite, dedicated ViewerResolver parity/PIN/ nil-vs-empty/fail-closed tests, and a full server boot smoke. Implementation drafted by Codex (GPT-5.5) via codex exec; a first-pass reflection-based adapter was rejected and reworked into the typed in-policy adapter; reviewed line-by-line and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for acting-admin and permission gates vendor/permission.rego reproduces the acting-admin rule (admin role + primary-profile-or-none), HasEffectivePermission semantics for marker_edit, and the metadata-curation rule including the subtle admin-past-refused-bypass case that requires the explicitly ASSIGNED permission. Policy-backed middleware in policy_gates.go keeps all Go-side lookups (declared-profile primary check, item->library resolution, the 404-on-unknown-item path) and preserves the legacy status/body taxonomy exactly — proven by dual-execution middleware tests that run every scenario through both implementations and assert byte-equal responses. Permission decisions always log (allowed flag populated); simulate and the capability endpoint gain the permission domain automatically via the domain registry. Router swaps behind single constructor choice points with the legacy gates retained for policy-less wiring. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for download and playback admission decisions vendor/action.rego decides download eligibility (downloads enabled + user allowed), download-transcode eligibility (transcode enabled + user allowed + artifacts available), and playback admission (stream/transcode counts vs limits, zero = unlimited), with a tightening-only silo_custom.action override that can also clamp a quality ceiling (never widen — merged via quality.min). Go keeps everything stateful: config loading, preset-ladder enumeration, and live session counting. Downloads consult an optional ActionDecider (nil = legacy logic) mapped back to the existing sentinel errors and capability response. Playback gains a minimal AdmissionDecider hook at the exact point of the legacy limit comparison: counts snapshot under the session mutex, PDP evaluated OUTSIDE the lock, then revalidated under lock before insert (retry on count drift) — no admission ever decided on stale counts and no eval under the mutex. Deny reasons map to the legacy ErrTooManyStreams / ErrTooManyTranscodes sentinels, pinned by tests. Parity: combination tables driven against the real PresetsFor / ensureTranscodeAllowed / SessionLimits math; full suite green (known pre-existing jellycompat flakes only). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (locking design verified line-by-line) and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): satisfy tsc -b strict return typing in the Rego stream tokenizer The production build (tsc -b) rejects assigning CodeMirror's string | void next() result to string | undefined; tsc --noEmit did not catch it. Restructured the string-literal loop. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): clearer error when a decision is undefined for partial input Vendor policies index required input fields directly, so a hand-written simulate payload missing fields yields an undefined decision. Surface that as 'decision X is undefined for this input (missing required input fields?)' instead of 'empty result' — found while exercising the simulate API against a live server. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(web): set changeOrigin automatically when the API proxy target is remote Remote dev backends sit behind vhost-routing proxies that reject a localhost Host header; local targets keep the existing pass-through behavior. Enables pointing the Vite dev server at a hosted backend via VITE_API_PROXY_TARGET in web/.env.local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): redesign the policy workspace around the decision pipeline The first-pass UI was structurally generic: a five-column document table squeezed beside the editor, three equal-weight action buttons with hidden preconditions, raw version IDs, and jargon copy — nothing taught the model. The page now teaches it: - A pipeline strip states the mental model up front: Silo decides the baseline -> your overrides narrow it -> every decision is logged. Tabs renamed to Overrides / Baseline / Decision Log (ids stay stable for bookmarked URLs). - The document table becomes one card per domain (Library visibility / Admin & permissions / Downloads & playback) with plain-language descriptions, example rules, status pills (Live vN / Draft / Disabled), inline creation, and the enable kill-switch in place. - Selecting an override drills into a full-width editor with a visible lifecycle rail (Draft -> Validated -> Saved -> Live) and one contextual primary action per step; the unedited live source shows no actions until edited. Version comments appear only at the save step. - Simulate is reframed as 'Test before going live' with a human verdict chip (Allowed / Denied — reason / ceiling summary) above the raw JSON; internal generation counters no longer surface. - History uses 'Make live' with plain go-live copy; authors read 'User N'; the baseline tab explains that upgrades never touch overrides. Hand-written redesign (no Codex); verified via vitest, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): present the policy baseline as readable rules, not raw Rego The Baseline tab dumped five Rego modules into read-only editors. It now leads with what the rules actually do: one card per domain with plain-language statements of the shipped behavior and a note on what an override may change, plus content-rating and playback-quality tier ladders parsed live from the lib module sources (so the tiers shown are the ones the server enforces, not a hardcoded copy). The Rego source stays one click away behind a per-module accordion and remains the stated source of truth; unrecognized modules fall back to source-only. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(policy): add access-groups design addendum Groups with permission toggles become the everyday admin surface; the Rego editor is demoted behind policy.editor_enabled (default off). Restriction-only composition: group grants are an upper bound, per-user settings tighten further — same rule as the existing account/profile merge, one layer up. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): add access groups — group defaults with restriction-only composition New access_groups table + users.access_group_id (one group per user, NULL = today's behavior). Group grants are an upper bound composed with the user's own settings by strictest-wins rules — library intersection, MinQuality, AND'd booleans, strictest positive stream/transcode limits, permission-mask intersection, and a requests toggle gating CreateRequest. The merge happens in Go (access.ApplyGroupPolicy / EffectivePolicyForUser) before policy inputs are built, so vendor Rego, the parity suites, and the decision log are untouched; every enforcement surface (viewer scope in both resolvers, permission gates, downloads, playback admission, requests) consumes the effective policy and fails closed on provider errors. Changing a group's quality ceiling bumps its members' access_policy_revision, mirroring the per-user rule. Additive admin API: /admin/access-groups CRUD with member counts; PUT /admin/users/{id} + user DTOs gain access_group_id. Also demotes the Rego editor: policy.editor_enabled (default off, hot-reloaded) drives the capability endpoint's editor_available and 403-gates editor endpoints while the engine and decision logging keep running. Design: docs/superpowers/specs/2026-07-02-access-groups-design.md. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (composition core + fail-closed call-site audit) and verified here. DB-backed group-store tests pending local Postgres recovery. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add Access Groups admin page and gate the policy editor New /admin/access-groups: a card grid summarizing each group (member count + key restrictions), drilling into an editor that reuses the same LibraryAccessSelector and quality presets as the user editor, with toggles for downloads/transcoded-downloads/requests, concurrent-stream and transcode limits, and a permissions mask (all-assignable by default, narrowable to specific permissions). Delete warns how many members fall back to the built-in defaults. Copy states the composition rule up front: a group grants the most a member can do; their own restrictions still apply on top. The user editor gains a Group picker and read-only row; the Policy nav entry is now hidden unless the capability reports the editor enabled. Plumbing (types, hooks, user-editor picker, nav gating) drafted by Codex (GPT-5.5); the Groups page hand-built. Verified: 25 tests across the touched suites, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): seed a Default Group and auto-assign newly created users Adds access_groups.is_default with a partial unique index (one default at most — the profiles is_primary pattern) and seeds a permissive 'Default Group' whose ceiling is a no-op, so assignment never changes anyone's effective access until an admin edits it. The seed is guarded against pre-existing defaults and name collisions; the Down migration only removes the row if it is still untouched. Assignment happens at the single INSERT INTO users choke point (UserRepository.Create): when no explicit group is given, access_group_id is filled by a scalar subquery on the default flag — NULL when no default exists. Every creation path (setup, signup, invites, OAuth, admin create) is covered by construction. Setting a new default via the API atomically clears the previous one in the same transaction. Deleting or unsetting the default is legal: new users then start with no group, which is pre-feature behavior. Implementation drafted by Codex (GPT-5.5); migration guards and the choke-point subquery reviewed line-by-line here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): surface the default access group Cards show a Default badge; the group editor gains a 'Default for new users' toggle (with copy noting existing users are never moved); the delete dialog warns when removing the default that new accounts will start with no group. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): ship the Default Group with house-rule ceilings Seed values per product decision: 5 concurrent streams, 5 transcodes, transcoded downloads off, and a permission mask of marker_edit only (metadata curation excluded). Plain downloads and requests stay on. The Down guard matches the new values so it still only removes an untouched seed row. Only newly created users are affected; existing users are never assigned. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): retire per-user defaults — the Default Group is the sole default policy Removes both legacy 'user defaults' mechanisms now that the seeded Default Group owns new-user policy: - users.max_streams / max_transcodes column defaults drop from 6/2 to 0 (= unrestricted at the user layer), so group ceilings apply to new signups/invites/OAuth users instead of fighting stale per-user numbers. Existing rows keep their stored values — nobody is silently uncapped on upgrade. - The dead defaults.max_playback_quality / defaults.max_profiles settings validation goes away with its only writer (the User Defaults dialog, removed on the web side). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): replace the User Defaults dialog with group-governed creation The Users page's 'User Defaults' dialog (defaults.* server settings) duplicated what access groups now do properly, and its values were only ever form prefill — no backend path applied them. The button now links to Access Groups, and the create-user form seeds unrestricted user-layer values (0 streams/transcodes, any quality, downloads allowed) so the member's group governs; per-user fields remain for tightening individual users. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): migrate existing non-admin users into the Default Group Existing users join the seeded Default Group on upgrade so one policy source governs the whole instance. Their per-user limits still holding the retired 6/2 column defaults are normalized to 0 in the same statement so the group's ceilings actually apply; deliberately customized values are preserved. Admin accounts stay ungrouped — scope/action decisions are role-blind, so grouping an admin would cap the server owner on upgrade. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): keep admins out of the Default Group and treat group moves as policy changes New-user creation now mirrors the migration's admin exclusion: the default access group is only auto-assigned to non-admin roles, so a fresh server owner no longer inherits the starter group's transcode denial and stream caps. Changing a user's access group now bumps access_policy_revision (the group carries permissions, quality, and limits, exactly like the per-user fields that already bump it) and triggers admin session revocation when the group actually changes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): enforce marker_edit through the PDP on marker write routes The Rego permission policy owned marker_edit but no Go caller ever consulted it: PUT/DELETE /markers went through a handler-local check that short-circuited admins and read only the user's own permissions, so group permission masks and custom policy overrides were ignored. Marker writes are now gated by router middleware like the other permission surfaces: a PDP-backed RequireMarkerEdit that evaluates the group-merged effective permissions (plus the legacy variant for proxy/test wiring without a policy system). The handler-local check and its user loader are gone. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert device/quality policy facts and honor the quality ceiling The download_transcode action check hard-coded an empty device ID and never asserted the requested quality, and no caller consumed ActionDecision.QualityCeiling — custom download policies keyed on those inputs were silently ineffective. Resolve now threads the request's device ID and requested quality into the action input, and a returned quality ceiling downscales the prepared transcode target (the ceiling applies to what is served, matching the serve-time rule in serveDownloadBytes). FileQuality and the content-rating pair stay intentionally empty for downloads — documented on downloadActionInput: those ceilings are enforced against the served artifact by the scope-derived access filter, and asserting the source's quality would wrongly deny capped transcodes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(access): align the default-group seed assertions with the migration The DB test still asserted the earlier no-op seed (transcode allowed, unlimited streams/transcodes, null permissions); the shipped migration seeds transcode denied, 5/5 limits, and marker_edit-only permissions, so the test failed on any database with the migration applied. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): lock the Rego sandbox by builtin purity and bound compile work Exclude every nondeterministic builtin from the admin sandbox instead of denylisting names, so OPA upgrades cannot silently expose impure builtins while pure helpers like net.cidr_contains stay usable. Apply the same capabilities to the runtime engine, cap concurrent compile checks, and reject oversized sources before they reach the uncancelable compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): require literal booleans in vendor override and input checks Bare object.get truthiness treated any non-false value as satisfied, so a malformed override 'allowed' value could fail to tighten a base grant and hand-crafted simulate input could flip flag predicates. Compare against literal true so anything else denies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): surface decision log cleanup failures to the task manager CleanupDecisionLogsOnce now returns the first error alongside the deleted count so a broken partition manager or DB outage marks the scheduled task failed instead of reporting 100% success while policy_decisions grows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): log admission decider errors before failing closed A policy-evaluation failure was silently mapped to the too-many-streams denial, making an engine outage indistinguishable from a real limit hit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): nil-guard the downloads user and restore the ABS legacy resolver effectiveDownloadUser dereferenced policy state before its nil-user check, and the ABS handler lost viewer-scoped filtering entirely when the policy system was unavailable because no legacy access.NewResolver fallback was wired like the other resolver paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): address admin policy review feedback - invalidate the version query by version_number, the key usePolicyVersion actually caches under - keep the goPrevious cursor-stack updater pure (Strict Mode double-invoke) - make version history rows keyboard-selectable like the document list - clamp download_transcode_allowed when downloads are disabled so groups cannot save a contradictory record Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): cap policy endpoint request bodies at 1 MiB The policy write endpoints (create document/version, set enabled, validate, simulate) decoded JSON bodies without a size limit, so an oversized payload buffered fully in memory before CompileCheck's 256 KiB source cap could reject it. Route all five through a shared decodePolicyRequest helper that wraps the body in http.MaxBytesReader and returns 413 with the repo's standard too_large error shape. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(access): forbid deleting or demoting the default access group Deleting the default group (or unsetting its is_default flag) left the server with no default: new non-admin users were then created ungrouped with max_streams/max_transcodes of 0 — unlimited — because the legacy per-user column defaults were retired in favor of the group's ceilings. The store now rejects both operations with ErrDefaultGroupRequired (mapped to 409); promoting another group remains the supported way to move the default, and atomically clears the previous one. The admin UI disables the delete button and the default toggle on the default group and explains the promote-another-group flow. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(web): keep unsaved policy drafts when a newer version activates elsewhere The editor state was keyed on the active version's id/sha, so a background refetch after another admin (or another tab) activated a version remounted the editor and silently discarded the dirty draft. PolicyEditorPanel now pins the seed it is editing against and only adopts an incoming seed when nothing can be lost: the editor is clean, the draft already equals the incoming source (the same-admin activate flow), or the selection moved to a different document. Otherwise the pinned editor stays mounted and an inline notice offers an explicit "Load live version" action. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(policy): fail reloads on invalid custom sources and surface degraded/apply state A stored custom source that stops compiling used to be silently skipped on reload: the bundle widened to vendor-only for that domain while the generation reported fully applied. Reload is now strict — a bad enabled source fails the reload and the last known-good engine keeps serving. Boot keeps its vendor fallback for availability, but skips are recorded on the engine and exposed (with store-outage reasons) through System.DegradedState and additive degraded fields on GET /policy/capability. Activate/SetEnabled re-run CompileCheck instead of trusting the stored compiled_ok flag. Mutation endpoints also no longer conflate persistence with live apply: activation/enable responses carry additive applied/failed_step/ loaded_generation fields and return 202 when the store change persisted but the local reload failed. Addresses review findings C1, C2, and the degraded-signal gap (6.1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): type deny reasons across the contract and enforce profile_verified Deny handling used to branch on exact free-text reason strings in three Go consumers, and playback reported ANY unrecognized reason — including custom override free text and engine failures — as a stream-limit error. Decisions now carry a stable reason_code (custom overrides always get custom_denial); downloads, the metadata-curation gate, and playback admission switch on codes, with a new ErrPlaybackNotAllowed -> 403 playback_not_allowed mapping for non-limit denials. Rego tests pin every vendor code. The scope contract's tighten-only profile_verified output was also emitted but never consumed; a policy revocation now surfaces as ErrProfileUnverified (403 profile_unverified) instead of silently proceeding. Addresses review findings 6.2 and C4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): close the dual-library disabled-scope bypass in direct item authorization EnsureAccessible, EnsureAccessibleIDs, and FilterAccessibleContentIDs gated library access with allow/deny predicates over a single joined media_item_libraries row, so an item linked to BOTH a passing library and a disabled one satisfied the disabled check via the passing row — a direct-ID bypass of disabled-library scope on the detail, media-file, playback, and download paths. All library access predicates now share one helper (libraryAccessConditions) emitting independent EXISTS / NOT EXISTS subqueries, the semantics GetByIDsWithAccess already used, including the orphan-item membership guard for disabled-only scopes. SQL-shape tests pin every builder and a DB-gated regression test covers the dual-library item end to end. Addresses review finding C3 (plus the same shape in buildFilterAccessibleContentIDsSQL, which the review did not flag). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): serialize quota check and row creation under a per-user advisory lock The concurrent-download quota was check-then-insert with nothing serializing the pair: parallel creates could all observe free quota before any row existed, bypassing the cap and stacking artifact encode jobs. All four check->insert spans (ephemeral original, artifact-backed, series batch, managed batch) now run inside Repository.WithUserQuotaLock — a pg_advisory_xact_lock keyed by user, so the serialization holds across nodes. The artifact path keeps the limiter-before-Ensure ordering (a rejected request must not leave an encode job behind) by holding the lock across Ensure. Managed-entry replacement stays quota-exempt and lock-free. A DB-gated barrier test races 8 creates against a cap of 1. Addresses review finding C5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert served quality at create time for original and remux downloads Direct-original and remux downloads serve the source resolution unchanged, but create-time policy checks left file_quality empty — an over-ceiling source registered a row serveDownloadBytes could never satisfy. Resolve now runs a final download action check with FileQuality populated on those two paths (capped transcodes keep the ceiling-on-artifact behavior), a custom override ceiling below the served resolution denies, and quality_ceiling_exceeded maps to ErrQualityUnavailable. The ActionInput contract now documents exactly when file_quality and the rating facts are supplied so custom policy authors are not misled. Addresses review finding C6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): guard activation against slow overrides and make eval timeouts observable A custom scope override that exceeds the 25ms eval budget compiled fine, activated fine, and then converted to 500s on every authenticated request — server-wide lockout authored in the admin editor. Activation and enable now run GuardEvalCost: the candidate source is evaluated on a throwaway engine against a canned representative input under the live budget, and a source that cannot complete is rejected 422 with ErrPolicySlowEval before it goes live. Runtime timeouts keep failing closed but now carry a distinct ErrPolicyEvalTimeout sentinel, an Error log, and a per-engine counter exposed as eval_timeouts on GET /policy/capability so intermittent near-budget policies are attributable. Addresses review finding C7. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt remediation files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
8fa40e928b |
feat(sections): wire watchlist/favorites rails with type and library filters
The dedicated watchlist/favorites section types were never dispatched by the section fetcher, so their home rails always resolved empty. Resolve them from the profile's user store, preserving stored order, and honor the section config's filter_type / filter_library_ids so admins can build e.g. separate "Movies Watchlist" and "TV Shows Watchlist" rails. Apply the same filters on the catalog "see all" path, and expose Media Type + Libraries pickers in the section editor for these types. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f7954f226e |
fix(catalog): resolve episode-scoped history to episode-level items (#307)
The History page's "Episodes" media filter was structurally empty: both history read paths collapse episode watch events into their series display item, and the resulting series ids were then filtered against type='episode' — guaranteed zero results. The episode media scope now resolves history at episode granularity: - loadPersonalSourceIDs keeps each watched episode's own id (deduped to the most recent watch, most-recent-first) instead of collapsing to the series; ListFilters and SearchFacet share the loader, so facets on the episode-scoped history view work too. - fetchAccessibleItemsByID gains an episode-scope path through the episode catalog query executor, since episode rows never hydrate from media_items via the browse repository. Every other scope keeps the existing series-collapse behavior. Fixes #299 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
193a2905b2 |
feat(collections): back collections with user-authored Trakt lists (#286)
* feat(collections): back collections with user-authored Trakt lists Collections could sync only Trakt's built-in trending/popular/recommended feeds; a server admin could not populate a collection from a specific user's Trakt list (e.g. a curated 'Saw in timeline order' list) (#214). - trakt.Client.GetUserList fetches /users/{user}/lists/{slug}/items in list order, mixing movies and shows and skipping non-title entries. - New 'trakt_list' collection source mode: catalog.ParseTraktListURL accepts a trakt.tv list URL (or bare user/slug), and syncTraktListCollection reuses the preset pipeline's matching/ordering via an extracted completeTraktEntrySync helper. Public lists need no access token. - Trakt import handler accepts list_url as an alternative to preset; the admin collection editor's Trakt form gains a Source toggle (discovery feed vs user list) with a list-URL input. Additive-only: new source mode + optional request field; preset path unchanged. Fixes #214 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(collections): round-trip trakt_list through the edit form, unlock mixed libraries, validate list host Three review fixes for user-list-backed collections: - The admin edit form now detects mode "trakt_list", shows an editable list URL (mirroring the create form) and saves the source back as trakt_list with list_url preserved — previously any edit silently rewrote the collection into a trakt_preset Trending Movies feed. - Library eligibility in list mode is mixed (movies + shows) instead of inheriting the hidden media-type default of movie, since Trakt lists mix both and entries match by their own type. - ParseTraktListURL only accepts trakt.tv / www.trakt.tv hosts, so a list-shaped URL on another domain fails fast with the format error instead of a confusing later sync failure. source_config now carries list_url alongside the legacy url key (additive); sync reads list_url, then url, then source_url. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
430224a1b9 |
perf: cut home-screen, Continue Watching, and Latest latency; cache shared home rails (#292)
* perf(jellycompat,sections): bound resume scan, batch leaf detail progress, widen section concurrency Three low-risk fixes from the section-fetch performance investigation (docs/superpowers/plans/2026-07-03-section-fetch-performance.md): - jellycompat: bound loadProgressPage at resumeScanMaxRows=300 so a single request never pages through more than that many in-progress rows. The cap is unconditional: it also covers the sparse-visible-set case (a heavy watcher whose recent rows are mostly dismissed/superseded, or a Series/Season-only request that matches no leaf in-progress row), where the page never fills and the loop would otherwise scan the entire history — previously an O(history) scan reaching tens of seconds. In the common case the loop exits far earlier, so the cap only bounds the pathological worst case; 300 leaves ample headroom to fill a ~20-item Continue Watching page. Beyond the cap the reported total is a clamped lower bound. Covered by TestLoadProgressPage_BoundsScanForSparseVisibleSet. - jellycompat: batch the leaf-item (movie/episode) progress lookup in GetItemDetailsByIDs via ListProgressWithCompletedHistory instead of a per-item GetProgressWithCompletedHistory (~100 sequential queries for a 50-item detail page). Series keep the per-item episode-rollup path (they own no progress row). Output is unchanged; a batch-lookup failure is now logged rather than silently dropping played state for the whole page. - sections: raise fetchAllMaxConcurrency 4 -> 6 to cut FetchAll wave count for large home layouts, staying within the default 20-conn pool. Part of the home/continue-watching latency work. AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat): keep Latest browse on the cross-library fast path under isPlayed /Items/Latest with isPlayed=false is the highest-frequency compat browse (~10.8k calls/day). The played overlay can't be pushed into SQL, so browse over-fetches and filters locally. The cross-library recently_added fast path (BrowseRecentlyAddedAcrossLibraries: one ~1ms index walk per library) was gated on Offset==0, so a heavy watcher who had already seen the newest items needed a 2nd chunk and fell through to BrowsePage — a whole-catalog MIN(first_seen_at) + GROUP BY HashAggregate over ~147k movies measured at ~755ms per call (0.8-1.6s observed end-to-end). Fetch the entire over-fetch budget (maxScannedRows) in a single merged fast-path walk instead of paging into BrowsePage, so the loop fills from one call. The clamp caveat (MaxLimit=1000 leaves a fall-through only for requestedLimit>200, off the Latest hot path) is documented inline. Part of the home/browse latency work. AI-use disclosure: implemented with AI (Claude) assistance. * fix(jellycompat): scope resume scan cap to resume path and bound the fast-path loop Addresses PR #292 review feedback: - Codex (P2): the resumeScanMaxRows cap was applied unconditionally in the general loop, which also paginates the completed (watched-items) list. Gate it on resumeFiltered so the completed path keeps exact TotalRecordCount and deep StartIndex pagination. Covered by TestLoadProgressPage_CompletedScanNotCapped. - CodeRabbit (Critical): the earlier raw-offset fast-path loop — the default Continue Watching shape and the sections-fallback route — had the same unbounded-scan bug and was not covered by the cap (the existing test forces EnableTotalRecordCount=true, routing around it). Bound it with the same resumeScanMaxRows guard. Covered by TestLoadProgressPage_BoundsFastPathScanForSparseVisibleSet. - CodeRabbit (Minor): tag the doc's fenced example blocks as text to satisfy markdownlint MD040. AI-use disclosure: implemented with AI (Claude) assistance. * perf(sections): cache shared user-agnostic home rails per access scope Home-screen rails that are identical for everyone who can see the same libraries (recently added, recently released, genre, trending on server, most watched, new to library, critically acclaimed, award winners, format showcase, seasonal, mood, trending discover, admin-curated lists, and library collections) were rebuilt from Postgres once per request, per user. Only the overlay on top of each row (watched flags, play position, presigned poster URLs) is actually per-user. Insert a process-global resolved-list cache at the FetchOne choke point in internal/sections. Each cacheable row is built once per access scope, held with a 15m TTL, and refreshed in the background 3m before expiry; singleflight collapses cold-miss stampedes into a single build. The per-user overlay still runs fresh in buildSectionsResponse, so no profile state is ever shared. Random and per-user rows (continue watching, next up, recommendations, hidden gems, forgotten favorites, activity feed, user collections) bypass the cache. The access-scope key captures every access boundary the fetch path enforces -- section identity (type + id + config hash) + item limit + accessible and disabled libraries + max content rating + excluded media types + name prefix + allowed-content-id allowlist -- and nothing per-user, so entries are safely shared. Empty membership is never cached (avoids freezing a transiently empty rail); background refreshes are bounded by a timeout. Scale (analytical, derived from the cache behavior -- not a measured latency): for the user-agnostic rows, Postgres section-query volume collapses from O(rows x concurrent requests) to O(rows x distinct access scopes) per 15m refresh window, because most users share a handful of access scopes. Illustrative -- 40 cacheable rows on a home screen, 1000 concurrent users falling into ~5 distinct access scopes: - before: ~40 x 1000 = ~40,000 section queries per wave of home loads - after: ~40 x 5 = ~200 builds per 15m window (plus one background refresh per row per scope), i.e. a warm home load runs zero section queries for these rows. That is a ~99% reduction in shared section-query load at that concurrency; the win grows with concurrency and shrinks as access-scope diversity rises. Design/plan doc added under docs/superpowers/plans/. * perf(jellycompat): serve per-library Latest via the cached recently-added section A jellyfin-compat per-library /Items/Latest rail is the same user-agnostic list as the native "recently added" library rail -- both order by mil.first_seen_at DESC. It was rebuilt on every request through directContentService.BrowseItems, missing the resolved-list cache entirely. Route per-library Latest for movies and series libraries through the native section fetch instead, so it reuses the shared cache. HandleLatest resolves the library's type once, and for a movies/series library builds a synthetic SectionRecentlyAdded with the same type + config + limit + access scope the native rail uses and calls FetchOne; the per-user overlay (favorites, progress, episode targets, presign) is extracted into buildLatestItemDTOs and shared by both the native and BrowseItems paths, so no overlay logic is duplicated. Cached *models.MediaItem values are read-only -- LocalizeItemModels deep-copies before any presign mutation. To let the two surfaces share one entry, resolvedListCacheKey no longer includes the arbitrary section ID: every cacheable section type derives its membership from type + config + limit + scope, never from its own ID (audited all 14 cacheable types plus the library-collection path; the sole s.ID read lives in the non-cacheable user-collection branch). A native recently-added rail and the compat Latest for the same library + scope now collapse to ONE cache entry, built once and reused. Access-scope isolation is unchanged -- the removed ID never carried access information, and every access boundary (libraries, rating cap, excluded types, content allow-list, name prefix) still keys the entry. Guardrails: the native path is restricted to movies and series libraries; every other library type (ebook, music, manga, mixed) is ignored and keeps its exact BrowseItems behavior -- important because an unfiltered recently-added fetch would otherwise surface non-video items to Jellyfin clients that only expect video. Deeper pages, played-filter and backdrop-required requests, a client asking for a type other than the library's own, and any FetchOne error also fall back to BrowseItems. Chosen over an alternative that gave the synthetic section a deterministic ID (which kept two separate cache entries): both returned identical data with similar complexity, so the shared-entry design won. * fix(sections,jellycompat): post-review fixes for the shared-list cache and Latest path Consolidates fixes from the branch's adversarial review and PR #292 review comments into one commit: - Latest fast path: fall back to BrowseItems when a request carries a genre, name-prefix, or person filter (the synthetic recently-added section cannot express these, so serving it unfiltered would return a wrong, broader set). Eligibility is decided by latestFastPathEligible and covered by a test. - Clamp the /Items/Latest page size to compatBrowseMaxLimit before building the section, matching the BrowseItems fallback, so a large client Limit can't drive an oversized recently-added fetch or explode the shared cache key with unbounded ItemLimit values. - Evict expired entries from the process-global resolvedListCache: resolvedListSet sweeps expired keys at most once per minute, bounding the map to scopes seen within one TTL window. Covered by TestResolvedListCacheEvictsExpiredEntries. - Log a short digest of the cache key (resolvedListLogKey) instead of the raw key in the background-refresh panic/error paths, since the key embeds user-controlled access-scope fields such as NamePrefix. Skipped review comments (verified already fixed or stale against current code): the resume fast-path scan bound and watched-items cap (04d2e795) and the docs fence-language tags (already addressed). Build, vet, and go test -race pass for internal/sections and internal/jellycompat. * perf(plugins): cache plugin installations in-memory, invalidated on lifecycle change ## Problem Every poster/image on a warm home rail re-read plugin_installations from Postgres to answer "is this plugin enabled?" and to acquire the plugin client (Source A: metadata chain buildProviders enabled-check; Source B: ensureClient -> loadInstallation). Plugin-resolved image URLs are never URL-cached, so the plugin source and the DB read behind it fired again on every identical warm request; 100% of images in the target library are plugin-backed. ## Solution - Guarded in-memory installation cache (map[int]*Installation + RWMutex) in plugins.Service. loadInstallation reads through it; the requireEnabled gate stays after the cache read so ErrInstallationDisabled semantics are unchanged. invalidateInstallationCache clears it and is self-registered as a lifecycle hook, so Service.OnLifecycleChange wipes it on install/enable/disable/update/ uninstall. - A generation counter closes an invalidate-vs-repopulate race: captured before installations.GetByID and re-checked under the write lock, so a row fetched before a lifecycle mutation is never written into a freshly cleared cache (would otherwise resurrect a just-disabled plugin). - Route the metadata chain enabled-check through the same cache via a structural InstallationEnabledChecker interface (nil-safe: falls back to the pool query when no checker is injected), wired in cmd/silo/main.go. ## Post-review fix (auto-update reliability blocker) AutoUpdateService mutated installations (new InstallPath, old dir deleted) on the default auto update policy without firing OnLifecycleChange, leaving the cache stale and breaking plugins with "stored plugin manifest mismatch" until restart. It now takes an onChange callback wired to Service.OnLifecycleChange and fires it once per Check run that mutated a row. ## Verification go build/vet, go test ./internal/plugins/... ./internal/metadata/... (-race). Tests: cache hit/invalidation, racing-invalidation guard, IsInstallationEnabled, auto-update fires onChange. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(jellycompat): batch per-item presign, and enrich series on the cached Latest path ## Problem List rails presigned each item's poster/backdrop/logo/still image individually (~160 singular resolver calls for a 40-item page where 4 batched calls suffice), and ItemsHandler carried a near-verbatim duplicate of the batch presigner. ## Solution (batching) Promote the batch presigner to a shared package-level presignCompatListItems (presign_list.go) with a generic collectImagePaths[T]; convert the per-item loops (cached home/Latest rail, favorites, batch loaders, userdata favorites) to one batched PresignImageURLsWithExpiry per image type per page; batch the season/episode collections; delete the three duplicate presign helpers. URL output is unchanged (verified byte-for-byte). ## Post-review fix (series Latest data-parity regression) The native cached Latest fast path built items via compatListItemsFromModels + buildLatestItemDTOs and never ran the series watch-state rollup, so a series library's Latest lost Played / UnplayedItemCount and page 1 disagreed with the BrowseItems fallback. enrichSeriesUserData is promoted to the ContentService interface and called on the native path (reused, not duplicated). ## Verification go build/vet, go test ./internal/jellycompat/... ./internal/catalog/... Tests: bounded presign invocation counts + per-item URL mapping; series rollup populated on the native Latest path. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(sections): gate personalized rails out of the shared cache; widen refresh lead ## Problem 1. The shared home-rail cache whitelisted custom_filter/genre sections by TYPE alone, but those route through fetchFiltered -> ParseQueryDefinition and can carry personalized (per-profile) rules/sorts (watched, favorited, in_watchlist, in_progress, last_watched; sorts progress/date_viewed/plays). Their membership is per-profile yet the cache key excludes userID/profileID, so a personalized rail built for one profile was served to others in the same access scope for up to 15m -- a cross-profile watchlist/watch-state leak. 2. The background-refresh lead was tuned so steady traffic is served a warm entry from a longer soft window. ## Solution - Add QueryDefinition.IsPersonalized() (reusing the existing QueryFieldRequiresProfile/QuerySortRequiresProfile helpers). isCacheableSectionType now parses the section QueryDefinition and refuses to cache custom_filter/genre when personalized; non-personalized definitions stay cacheable. Seasonal/mood/trending build their definitions server-side and stay unconditionally cacheable. - resolvedListRefreshLead 3m -> 10m (soft threshold builtAt+5min instead of builtAt+12min). ## Verification go build/vet, go test ./internal/sections/... ./internal/catalog/... (-race). Test: personalized custom_filter/genre not cacheable; non-personalized are. ## AI-use disclosure Implemented with AI assistance (Claude). * fix(sections,metadata): post-review fixes for shared cache and plugin chain staleness Addresses three review findings on PR #292: - sections: canonicalize section config JSON before hashing so configs differing only in whitespace/field order share a cache entry (native + jellycompat rail sharing). Added TestHashSectionConfigCanonicalizes. - metadata: invalidate the resolved-chain cache on plugin lifecycle changes; the installation-enabled check already reads the invalidated plugin cache, but resolveChainCached could serve a stale provider chain for up to chainCacheTTL after a provider's availability changed. - jellycompat: move ctx to the first parameter of presignCompatListItems for consistency with the other presign helpers. Skipped the episode-image presign batching nitpick: the resolver already dedupes+singleflights, so it is a Minor perf-only item not worth the two-pass refactor risk in this pass. * fix(sections,jellycompat): harden shared rail cache and Latest fast path per review Addresses the eight findings from the deep review of this PR: - Detach the blocking cold-miss rebuild from the singleflight leader's request context (context.WithoutCancel + the shared 30s build timeout) so one client disconnect no longer fails every collapsed waiter and leaves the entry uncached. - Stop client-controlled values minting unbounded cache entries: the compat Latest fast path now always fetches a fixed 100-row budget and slices to the requested limit (one entry per scope+library instead of one per Limit value), and an unrecognized MaxOfficialRating string disqualifies the fast path instead of entering the global cache key. - Add release_date to the sections item projection/scan so movies served via the Latest fast path keep PremiereDate (Jellyfin default-set field) in parity with the BrowseItems fallback. - Fall back to per-item progress lookups when the batched leaf progress query fails, restoring one-item-at-a-time degradation instead of blanking played state for the whole page. - Derive cache eligibility from a single source of truth: fetchSection and isCacheableSectionType now share the userAgnosticSectionFetcher table, whose no-userID/profileID signature makes a fetcher drop out of the cacheable set at compile time if it ever gains per-profile inputs. - Decide Latest fast-path eligibility off the actual browse params the fallback would receive, so any filter later added to buildBrowseParams automatically disqualifies the cached path; share one compatDefaultBrowseLimit constant between both paths. - Extract AccessFilter.WriteAccessScopeCacheKey as the shared, security- critical serializer for all access-scoped caches (resolved-list, editorial candidates, audiobook groups); the editorial key now captures ExcludedMediaTypes, which its loaders already applied in SQL. - Strip leaked agent-transcript markup from the section-fetch plan doc. go build ./..., go vet, gofmt clean; go test -race on internal/sections, internal/catalog, internal/jellycompat passes (TestBeginWebOperation* failures are the known pre-existing flakes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ce09d2b6fc |
fix(catalog): harden Meilisearch search integration (#291)
Findings from a full review of the Meilisearch implementation: - Give the indexer its own 2m HTTP timeout instead of reusing the 800ms search-path timeout_ms, so large document uploads to a non-loopback Meilisearch stop timing out. - Delete superseded indexes after a rebuild (previous active + leftover <prefix>_rebuild_* partials); every rebuild previously leaked a full copy of the catalog on the Meilisearch instance. - Cache the index state row + pending count for 3s on the search hot path (was two Postgres round trips per search request); a failed search invalidates the cache immediately. - Swap the active-index pointer before marking outbox events processed so a crash between the two replays events instead of losing them. - End pagination only on a short page; estimatedTotalHits is an estimate and could truncate results. - Latch the startup-resolved provider process-wide so package-level enqueue helpers stop querying server_settings in write transactions. - Surface dead-lettered outbox events in the admin status + web UI. - Remove unwired provider config knobs, dedupe the manga-chapter exclusion predicate, split the vector cache onto its own mutex, real rebuild progress percentages, and expand client/coalesce test coverage. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c0f209a936 |
feat(catalog): Latest Episodes sort — order series by newest episode file (#283)
* feat(catalog): Latest Episodes sort — order series by newest episode file Adds a latest_episode_added sort so users can see which shows received new episodes. Today's recently-added surfaces reflect when the SERIES was first added: linking a new episode file never bumps the series' media_item_libraries.first_seen_at (ON CONFLICT DO NOTHING), so a long-running show with a fresh episode sorts as stale (#202). - New denorm media_items.latest_episode_added_at (migration + backfill + partial series index), mirroring the last_air_date_at precedent. Source of truth is episode_libraries.first_seen_at; the three insert paths (UpdateEpisodeLink, BulkLinkEpisodesBySeries, scanner folder restore) bump the parent series atomically in the same statement, monotonically via GREATEST, and only for genuinely new links. - Sort registered in both frameworks: querySortDefs (sections + smart collections + /v1/catalog pick it up automatically via QuerySortFieldSet) and the browse buildOrderByPlan path. - Jellyfin compat: SortBy=DateLastContentAdded now maps to the new sort instead of silently collapsing to series creation date — Jellyfin clients already send this for the TV "Latest" shelf, so they get the correct behavior with no client changes. DatePlayed keeps its old created_at mapping instead of piggybacking. - Web sort picker gains "Latest Episode Added" (series scope). Additive-only per v1 API rules: new sort value, no field/status changes. Part of #202 Fixes #202 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): include latest_episode_added in the api QuerySort field union The picker-side QuerySortField gained the value but the api-layer QuerySort union did not, breaking the production tsc build. Part of #202 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): recompute latest_episode_added_at when episode memberships are removed The denorm was only ever bumped upward (GREATEST) at insert time, but UpdateEpisodeLink also deletes the old episode's library membership on re-link, and reconciliation/path-prefix clears remove memberships too — leaving a stale timestamp that kept the series sorting as recently updated. All removal paths now run in a transaction and finish with a shared full MAX() recompute (catalog.RecomputeSeriesLatestEpisodeAdded) that also resets to NULL when no memberships remain, mirroring the last_air_date_at maintenance pattern. Sequential statements are load-bearing here: data-modifying CTEs are invisible to reads in the same statement, which also silently no-op'd the old path-prefix membership delete. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(jellycompat): keep DateLastContentAdded scoped to series-only requests mapSortBy runs for every /Items browse, so the latest_episode_added mapping leaked into movie and untyped requests where the column is always NULL, destroying the previous created_at ordering. The sort now falls back to created_at unless IncludeItemTypes is exactly Series. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
f983c54e87 |
fix(metadata): re-fetch titles/overviews when a library's metadata language changes (#278)
* fix(metadata): re-fetch titles/overviews when a library's metadata language changes An item's default_metadata_language was stamped once at first match and never updated: the canonical-language pin in mergeAndPersist routed any refresh in a different language into the localization tables, the upserts' COALESCE kept the old stamp forever, quick-mode library refresh skipped complete items entirely, and changing the language in HandleUpdateLibrary triggered nothing. Items stayed in the old language no matter how often the admin refreshed (#211). Four coupled changes: - ProcessRequest.AdoptLanguage: folder-scoped manual refreshes adopt the library's language as the item's new canonical language when it differs from the stamp, rewriting the base row instead of localizing to the side. Only ModeManualRefresh adopts — scheduled refreshes merge fill-empty and would restamp without rewriting the text. - Upsert language pins inverted (media_items, seasons, episodes): prefer the incoming non-empty default_metadata_language over the existing stamp. All existing callers send the unchanged stamp or empty, so behavior is unchanged outside adoption; the restamp is atomic with the canonical write. - Quick-mode refresh lister now includes complete items whose stamp differs from the library's configured language. - HandleUpdateLibrary enqueues a quick library metadata refresh when the metadata language changes, mirroring the paths-change rescan trigger. Fixes #211 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): gate language adoption on field locks and library agreement Two hardening fixes for AdoptLanguage from review of #278: - Locked fields defeat the restamp: title/overview merges honor per-field locks, but the restamp was unconditional. An item with both language- bearing fields locked kept its old-language text yet got stamped the new language, so the quick-refresh mismatch predicate never flagged it again. mergeAndPersist now skips adoption when both name and overview are locked, falling back to the non-adopting behavior: the stamp stays put, isCanonicalWrite goes false, and the fetch routes to the localization tables exactly like a non-adopting refresh in that language does today. One locked field still adopts — the other is actually rewritten. - Multi-library flip-flop: an item in libraries with different metadata languages had its canonical base row rewritten to whichever library refreshed last, oscillating forever. Process now requires every library containing the item (media_item_libraries) to resolve to the adoption target before setting AdoptLanguage, via the existing GetDistinctMetadataLanguagesForItem (which applies the same empty→en default as resolveFolderLanguage). Disagreement or a lookup failure keeps the current stamp — stable beats flip-flopping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
cf0db385f3 |
Add Apple push notifications support (#255)
* Add push notifications support * fix(notifications): address push notification review findings - Gate the capability endpoint's apple_push availability on the admin delivery toggle, matching web push: Available now means setup will actually deliver. - Reject direct admin writes to push_relay_deployment_id/api_key; the relay issues them as a pair during registration and a lone write desyncs them (and poisons the next rotation request). - Purge a device's registrations under other profiles when it re-registers, so a profile switch on a shared device stops the old profile's pushes (attempts cascade); adds a DB-backed test. - Extract the shared channelDispatcher core + retry sweep and rebuild the webhook/web push/Apple push dispatchers on it instead of keeping three copies of the worker-pool/retry loop. - Deduplicate relay URL validation (admin setting + register flow) and the push outbox attempt-building loops behind shared helpers. - Cap free-text decline reasons in notification display bodies. - Fix TestHandleApplePushDisplayDB expectations to match the shared display copy (test previously failed under SILO_TEST_DATABASE_URL). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(notifications): route push relay URL writes through registration only Direct writes to notifications.push_relay_url via the admin settings endpoint bypassed the relay registration flow, letting the stored URL drift out of sync with the deployment id / API key pair the relay minted for it. Reject the URL alongside the deployment id and API key in the settings handler; POST /admin/notifications/push/relay/register remains the only path that persists all three together. The admin UI's Relay URL field now edits local draft state and is applied by the Register/Rotate action instead of the settings save. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
652b5d1e9f |
perf(jellycompat): fast-paths for slow Jellyfin-compat queries (#247)
* perf(jellycompat): fast-paths for slow Jellyfin-compat queries
Rebuilds the jellycompat slow-query fast-paths cleanly on main, without
the hub-sections virtual-library feature (which is intentionally absent
on this branch). Each path is bounded so behavior stays predictable
under load.
Fast-paths:
- Episode catalog: add the load-bearing `si.type = 'series'` hydration
guard plus a matching entry-scan series-parent guard, so the page and
count scans stay aligned with hydration (no under-filled pages, no
inflated TotalRecordCount from podcast/audiobook episode rows).
- /Shows/{id}/Episodes AdjacentTo: serve a bounded prev/self/next
neighbor window with series-membership validation and a cross-series
id guard, falling back to the full listing.
- Resume: serve Continue Watching through the capped native
continue-watching fetcher (next-up suppressed, in-progress only)
instead of loadProgressPage's unbounded scan. Wires only the
sections read-time fetcher (StoreProvider) into ItemsHandler; no
hub-section/virtual-library exposure.
- Played-history: push the type + library filter down into SQL.
- recently_added: fan a no-parentId browse out across libraries with an
app-side dedup merge, clamped to BrowseFilters.MaxLimit, and exclude
compat-hidden (audiobook/podcast) libraries from the fan-out.
- Detail fields: batch per-item GetItemDetail in the Latest/browse
detail-field paths via GetItemDetailsByIDs, with per-item fallback.
- Search: term-length and result-count guards for aux search.
* perf(jellycompat): clamp Resume fast-path fetch to maxResumeItems
A large client-supplied StartIndex on /UserItems/Resume inflated
loadResumeViaSections' fetchLimit (pageSize + StartIndex) and fed it
straight into ResolvedSection.ItemLimit, letting the continue-watching
fetcher scan up to its continueProgressMaxScanned hard cap (1000 rows)
instead of stopping near maxResumeItems. Clamp fetchLimit to
maxResumeItems and short-circuit to an empty page when StartIndex is at
or beyond the cap (no visible row can land there), keeping the capped
fetcher's scan bounded and predictable under load.
Addresses CodeRabbit review finding on PR #247.
|
||
|
CoffeeKnyteandGitHub
|
4fb8f6a711 |
fix(metadata): merge duplicate people on refresh instead of looping on 23505 (#250)
* fix(metadata): merge duplicate people on refresh instead of looping on 23505 The background person-refresh worker re-selected the same people every cycle. When a refresh resolved an external id (tmdb_id/imdb_id) already held by another people row, the UPDATE violated a partial-unique index and raised SQLSTATE 23505; the tx rolled back so updated_at never advanced and FindCandidates re-qualified the row forever. The underlying cause is two people rows for the same human created from credits ingested with disjoint id sets (one tmdb-only, one imdb-only) that BatchFindOrCreate never reconciled. PersonRepository.Update now reconciles the collision instead of failing: - The common no-collision path stays a single plain transaction (no added cost). - On a 23505, updateResolvingConflicts runs the whole reconciliation in ONE transaction, retrying the write via savepoints so it commits atomically: it locks both rows FOR UPDATE in id order, then either merges the partner into the survivor (repoint item_people skipping duplicate credits, fold the partner's ids/fields onto the survivor, delete the partner) or, when the rows are not confidently the same human, drops just the conflicting id. - canMergePeople requires compatible ids AND matching names, so a provider that hands the same id to two genuinely different people cannot trigger a destructive delete; that case falls to the non-destructive drop. - A row merged away concurrently surfaces as pgx.ErrNoRows, which the refresh service maps to ErrPersonNotFound. Existing stuck rows self-heal: they are still re-selected each cycle and now merge (or drop) instead of looping, draining the warning population to zero. Adds unit tests for the merge-decision logic (guard, compatibility, field-folding, constraint mapping). AI-use disclosure: implemented and adversarially reviewed with AI assistance. * fix(metadata): preserve survivor's existing id when declining a person merge The non-mergeable branch of resolveExternalIDConflict blanked the conflicting external-id field before retrying the write. Because execPersonUpdate is a full-row UPDATE, the retry persisted an empty string and returned success, silently dropping a previously-valid provider id (e.g. the admin PATCH path that mutates an existing id into a colliding value) instead of leaving the row unchanged as the 23505 did. Restore the locked survivor's currently-persisted value for the field so the retried write is a no-op on that column: it commits without looping, without deleting a possibly-distinct person, and without blanking an id the survivor already held. Writing a row's own current value back can never violate the unique index, so the field will not re-trigger the conflict. The refresh-worker path is unchanged (survivor value is empty). Convert clearExternalIDField into a general setExternalIDField setter and extend its unit test to cover set-to-value and set-to-empty. |
||
|
CoffeeKnyteandGitHub
|
1dbcf2cd9a |
fix(jellycompat): guard aux search paths and index-back person search (#252)
* fix(jellycompat): guard aux search paths and index-back person search
Short, recursive type-ahead terms (e.g. a single "a") against the
PostgreSQL people index and the in-memory collection/box-set filter
produced multi-second scans that pegged the server when a client fired
one search per keystroke. Every jellycompat search path except the
Meilisearch-backed /Items media search now rejects a provided SearchTerm
shorter than 3 runes without touching any backend, and caps results at
20 regardless of the client-requested Limit:
- /Persons (PostgreSQL people scan)
- /Search/Hints (catalog search)
- /Items BoxSet (in-memory collection filter)
Shared policy + helpers live in search_guard.go (auxSearchMinTermLen=2,
i.e. reject 1-2 runes / allow 3+, and auxSearchMaxResults=20) with unit
coverage. The 3-rune floor is deliberate: 3 runes is the point where a
pg_trgm trigram index becomes usable, so the gate lines up with the index
and still lets legitimate short titles/names ("300", "Saw", 3-letter
actors) and 3-char type-ahead hints through.
PersonRepository.Search filters with `name ILIKE '%'||$1||'%'` rather
than `LOWER(name) LIKE '%'||LOWER($1)||'%'`. The old expression filtered
on LOWER(name), which the trigram GIN index idx_people_name_trgm (built
on name) could not serve, so every search fell back to an ordered index
scan on idx_people_name that walked the whole table for rare terms
(~300-400ms on the 889k-row production people table). ILIKE on name lets
pg_trgm serve rare 3+ char terms from the trigram index, while the planner
still picks the ordered btree scan with early termination for common
terms. ILIKE is case-insensitive, so behavior is preserved (including the
pre-existing treatment of % and _ in the term as LIKE wildcards).
Verified on production silo-postgres (889,302 people), parameterized query
under both custom and generic plan cache modes:
rare 3-char 'qzx': 304ms -> 1-8ms (trgm bitmap index)
rare 4-char 'zzzz': 329ms -> 1-2ms (trgm bitmap index)
common 3-char 'ann': 15ms -> 7-74ms (btree early-stop / trgm bitmap)
New worst case across all terms is ~83ms (generic-plan common 3-char).
* fix(jellycompat): stop gating meilisearch hints, clamp box-set search
Two review follow-ups on the aux-search guards:
- HandleSearchHints is served by the catalog (Meilisearch-backed) search
provider, which already bounds and short-term-handles its own results.
Gating it with auxSearchTermTooShort contradicted the guard's own
"non-Meilisearch paths only" policy and hid valid 1-2 char titles
("Up", "It") from global type-ahead. Drop the too-short gate there;
keep the empty-term check and the result clamp.
- handleBoxSetsList gated short terms but passed query.limit straight to
slicePage, which treats <=0 as no cap, so a box-set *search* was
uncapped unlike Persons/Hints. Clamp the limit when a search term is
present; empty-term browse keeps its client paging window.
|