2dedf3ff26e488dce37d5ef81d7cdce2d9ec80e5
41
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
dc4b9a0909 |
feat(settings): add the cross-platform settings contract and its manifest (#479)
* docs(settings): define the cross-platform settings contract Turns the audit in #376 into a decision-complete design for how user settings work across the server, bundled web client, Apple clients, and Android clients. Today there are three partial contracts - the server registry, the web client's own manifest, and independently owned key constants in each native client - and they have measurably drifted. The root enabler is that keyUsesUserScope returns true for any unregistered key, so a client can invent a production setting unilaterally and the server stores it as an unvalidated string. The design decides: Ownership. Every production user-facing setting needs a server-owned manifest entry, even when the value is stored only on one client. The single exception is private local.<client>.* diagnostics, bounded by five conditions. Types and scopes. Native JSON values instead of strings. Five remote scopes plus client_local, and each definition declares its own resolution order rather than inheriting a global precedence. Preferences versus restrictions. internal/policy already resolves max_playback_quality and metadata-language limits over the same controls this contract resolves preferences for. Definitions declare constrained_by, the effective response reports the permitted value alongside the user's stored one, and a mutation exceeding a restriction is stored rather than rejected - a capped 4K preference should take effect the day the cap lifts, not be destroyed by it. Compatibility. Widening a scope, adding an enum member, or widening a range is additive and revision-tagged; narrowing anything needs a new key. introduced_in is a manifest revision attached to individual enum members and scopes, not just whole definitions, so a newer client never offers a choice an older server will reject. Rollout. One coordinated breaking release, with no compatibility shim, projection, or client fallback. After the cutover no future setting requires coordination. No settings version check goes in the authenticated middleware and nothing returns 426: deleting the old routes already produces the break, and a gate would be more code in four repos for the same outcome while permanently coupling every endpoint to one subsystem's versioning. Scope placement. Appearance and date/time move from account to profile scope. Account scope was an artifact of pre-profile storage; leaving it there means a household shares one theme and text size, and any non-child profile can restyle everyone else. Read path. Batched context resolution, index requirements, a session-snapshot rule, and a no-regression benchmark gating storage consolidation - profile_series resolution is per-item, so a season view would otherwise issue one request per episode. Verified against the current server, Apple, and Android implementations. Two findings shape it: the unknown-key extension bag is real, and v1 scope reads NOT LOCKED, so removing the legacy surface needs no amendment if it lands before lock. Related to #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical settings contract manifest First implementation step for the cross-platform settings contract (#376). Adds the artifact everything else depends on: the manifest, its JSON Schema, the object value schemas, and a Go loader that validates the whole thing at load time. No routes, no storage, no behavior change — nothing reads this yet. contracts/settings/v1/ holds the artifact at a stable path because clients vendor it and generate bindings from it. The embed directive has to sit beside it (go:embed cannot reach outside its own directory), so that directory is a tiny Go package containing nothing else; loading and validation live in internal/settingscontract. 38 definitions: 35 remote, 3 contract-known client_local. That covers every key the legacy registry accepts, every unregistered key the extension bag was silently accepting from the web client, every unregistered device key Android writes, and the profile preference columns that become settings. Registering the previously-unregistered keys is where the drift shows up, and the manifest records each case in a notes field: - ui_theme, ui_text_scale, ui_text_weight, ui_high_contrast, ui_custom_theme_vars, and ui_custom_css reached the server only because keyUsesUserScope returns true for any unregistered key. They are now typed, renamed to the dotted convention every other key uses, and moved to profile scope per the design. - player.match_frame_rate and player.sleep_timer_default_minutes are written by Android against a server that does not register them, so every write and reset is currently rejected. Registered. - player.next_up_prompt_seconds is Android's alias for playback.next_up_prompt_seconds and does not become a definition; the test matrix pins it as a migration alias. - player.playback_speed is capped at 3.0, matching the server rather than Android's 4.0. - subtitle_appearance becomes playback.subtitle_appearance. Every other canonical key carries a domain prefix, and preserving accidental key names is an explicit non-goal of the design. Validation is deliberately stricter than the schema can express. Beyond shape, it enforces that a resolution order ends in "default", that it only resolves scopes the definition allows, and — the one most likely to bite — that every writable scope is actually read, so a setting cannot accept writes at a scope it will never honor. Defaults are validated against their own value schema, so a default that violates its own range or enum fails at load. Revision tags are checked to never run ahead of the manifest revision, which is what makes revision-aware client filtering trustworthy. Ceiling and floor policy constraints are rejected on unordered types, where capping would silently do nothing; playback.preferred_quality's enum is therefore ordered ascending. ValidateValue is the single validation path, so the mutation endpoint, the migration, and the manifest's own default checks cannot diverge later. Numbers decode through json.Number so an integer setting rejects 30.5 rather than truncating, and object values validate against their referenced JSON Schema instead of accepting arbitrary JSON the way validateJSONSetting does today. Canonicalization implements RFC 8785 over the value domain the contract uses: sorted keys, no insignificant whitespace, ECMAScript number formatting. The digest is the ETag, and PublicBytes strips maintainer notes so the served manifest never carries internal commentary. Promotes santhosh-tekuri/jsonschema/v6 from indirect to direct. Verification: 124 tests pass across 16 cases; golangci-lint clean; make verify-local-paths passes. Two failures in internal/api/handlers (TestRemoveJellyfinCompatWebDisablesWebSetting, the playback v3 seek recovery test) reproduce unchanged on main and are unrelated. Part of #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): give ui.theme a device override Theme joins text scale, text weight, and high contrast as a profile default with an optional per-device override, resolving profile_device -> profile -> default. The right theme is partly a function of the screen and the room — a light theme on a phone in daylight, a dark one on a TV at night — which is the same reasoning the other three appearance keys already used. All four appearance settings now cascade consistently, which also means one rule to explain in the UI rather than "these three follow the device, that one does not". ui.custom_theme_vars and ui.custom_css stay profile-wide. They are authored styling rather than a contextual preference, so a profile's custom tokens still apply on top of whichever theme a device resolves to. Recorded in the definition notes because it is a visible consequence: vars tuned against a dark theme will sit on top of a light one if a device overrides the theme. Widening those to profile_device later is an additive revision bump if it turns out to matter. Part of #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(web): tag local appearance caches with their owning account The theme, text scale, text weight, high contrast, custom theme variable and custom CSS caches in localStorage were untagged, so on a shared browser a second account inherited the first account's appearance: with no server value of its own, every fallback resolved to whatever the previous account had stored, and the leftover `silo-theme` key also suppressed the admin-configured default theme for the new account. DateTimeFormatProvider already solved this by stamping its cache with the authenticated user id and refusing another account's values. Extract that mechanism into `createOwnedCache` in utils/storage.ts (where key namespacing lives) and put all three groups behind it, so appearance and custom theme get the same protection instead of a third copy of the rule. - Each group carries its own owner stamp. A shared stamp would be unsafe: the groups are written by hooks nested inside each other, and effects run inner-first, so whichever hook stamped first would vouch for the other's still-stale values. - A null owner (auth bootstrapping, or signed out) still trusts the cache, which keeps the warm start and the login screen's last look. - An unstamped cache is not trusted once an account is known, so existing users take a one-time appearance reset on first load rather than a chance of seeing someone else's settings. - When a foreign cache is detected the values are dropped and the empty cache is handed to the new account, so a later single save cannot re-trust the rest of the previous account's state. Owner is the user id because /settings is user-scoped server side; it lives in one helper (`appearanceCacheOwner`) so it can be widened if appearance moves to profile scope. `shouldLoadApiTheme` is gone: it had become a synonym for `appearanceCacheOwner(...) !== null` with no callers left. Part of #376 AI-use disclosure: implemented with Claude Code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): make the settings contract enforceable and fix the appearance cache The contract manifest landed as a document nothing checked. This makes it a mechanism, and fixes the one defect in the change set that hurt users on merge rather than at cutover. Web appearance cache. useTheme cleared the cache for any account whose stamp did not match and never repopulated it — the only writers were the four user-action setters — so every upgrading user lost their warm start on every load, not once, and x-large-text and high-contrast users lost theirs too. The owner-stamp protocol is replaced with per-account key namespacing (`silo-theme:7`): a foreign value is absent rather than present-and-distrusted, so nothing has to be deleted, the first account keeps its warm start, and there is no shared stamp for a second tab, a stale debounce timer, or an out-of-order effect to race on. Widening ownership to profile scope, which this manifest requires, is now a change to appearanceCacheOwner alone. Adds the API-to-cache mirror useTheme was missing, cancels pending debounced writes across an account change, and re-seeds provider state during render so no frame paints the previous account's look. Canonicalization. writeCanonical used json.Marshal, which HTML-escapes < > and &, and canonicalNumber used Go's 'g' format — both diverge from RFC 8785, so the first label containing an ampersand or bound below 1e-4 would have forked the server's ETag from every conforming client. Output is now byte-identical to ECMAScript String() across the edge cases, verified against node. The ETag also covers the value schemas, which decide what the server accepts and previously could change while the tag stood still. All four derived representations are memoized; a conditional GET no longer costs a full parse and re-serialize. Validation. strictUnmarshal's decoder.More() answered false for a stray ] or }, so `true]` validated as a boolean. Enum matching compared fmt.Sprintf tokens, so the string "3" satisfied an integer member. Declared steps were never enforced. The language pattern rejected tags both mobile platforms emit unprompted (en_US, ca-ES-valencia, ar-EG-u-nu-latn) and never normalized case, so en-US and en-us were two rows for one preference; NormalizeValue now canonicalizes on the shared path. Manifest. show_forced_subtitles defaulted false where the server column is NOT NULL DEFAULT true, which would have turned forced subtitles off for every profile that never touched it. preferred_quality declared 13 members where the planner speaks 6 and collapses the rest to auto. metadata_language's allowlist was bound to the very column it migrates from. subtitle-appearance pinned fontFamily to three families while Apple stores any installed system font. Registers five user-facing settings the clients already ship, and corrects three notes that described Android behaviour that was not true. Enforcement. The package had no non-test callers, so MustLoad never ran; it now loads and logs at startup. The inventory test compared the manifest against a hand-copied map and could not see the drift it named; it now iterates settingsRegistry and checks defaults too — both verified to fail on injected drift. Adds .github/workflows/ci.yml, the repo's first CI that runs go test, go vet, gofmt, and the frontend suite. Known pre-existing failures are named individually in the Makefile so everything else stays gated and the list can only shrink. Part of #135 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): align the sleep timer default and range with the shipped client Android is the only client that implements this setting. It clamps to 0..240 and defaults to 30. The manifest said 0..480 with a default of 0, so a manifest-driven UI would have offered durations no client can store, and every user who never opened the picker would have had the preset silently turned off at cutover. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: give the new workflow the deps it actually needs The first run exposed two gaps in the workflow itself. go build ./... fails without libvips headers, because h2non/bimg binds libvips through cgo and pkg-config; the Dockerfile installs the same package. And pnpm/action-setup resolves its version from package.json, but there is no package.json at the repo root — the packageManager field lives in web/package.json, and a job's defaults.run.working-directory does not apply to an action's inputs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(web): stop the diagnostics download test depending on the Node version new Response(blob) reads the body through blob.stream(), which jsdom's Blob does not implement on Node 22 — the version the Dockerfile builds with. The test passed locally on Node 24 and threw "object.stream is not a function" in CI. Nothing in it asserts on the body, only that the object URL and filename reach the anchor, so a string body is equivalent and works on both. Surfaced by the CI workflow added in this branch, which is the first thing in this repo to run the frontend suite anywhere but a developer's machine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(build): copy the settings contract into the container build context Both Dockerfiles copy cmd/, internal/, migrations/ and web/embed.go, but the manifest lives in contracts/settings/v1 — an embedded Go package that sits outside internal/ because clients vendor those files. The image build therefore fails with "no required module provides package .../contracts/settings/v1". Caught deploying to the dev box. Nothing had built an image since the manifest landed: the Docker workflow only runs on pushes to main and workflow_dispatch, and CI's go build runs against a full checkout, so neither gate covers the container context. This would have broken the published image on merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): enforce the language-tag and step constraints the manifest declares A sweep of all 43 manifest definitions against the running server (160 checks: declared default, both boundaries, and deliberate violations for each remote key) found two places where the live registry accepts what the contract forbids. Both are fixed by calling the contract's own validators rather than adding a second implementation. playback.audio_language was checked as "32 characters or fewer", so the server stored "!!!" for a field the manifest declares as language_tag — a value track matching would then silently never match. It now requires a well-formed tag via settingscontract.NormalizeLanguageTag. The empty string is still accepted: the string-only endpoint has no way to send null, and both Android and web send "" to clear the choice, so rejecting it would break clearing the preference. player.playback_speed declared step 0.05 and nothing enforced it, so 0.26 was stored — a value no client's stepper can represent and that every client would silently snap on the next write. settingscontract.StepAligned is now exported and used by both the contract validator and the registry, so there is one definition of "on step" rather than two that can drift. This gives the contract its first production consumer beyond the startup load, which is the direction Phase 2 continues in. Also fixes a genuinely flaky test that the new CI gate would have hit intermittently: TestRemoveJellyfinCompatWebDisablesWebSetting used t.TempDir as the install root, but the endpoint returns 202 and its goroutine keeps writing there after the test body returns, so cleanup tripped "directory not empty" roughly one run in four. Confirmed pre-existing and unrelated to settings; the suite now passes six consecutive full-package runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): keep widened numeric bounds resolvable at older revisions A bound was one scalar plus the revision that introduced it, which discards the value it replaced. Widening a maximum from 240 to 480 at revision 3 left a revision-3 client with no correct answer against a revision-1 server: honoring 480 offers values that server rejects, and filtering the tagged bound out leaves the setting unbounded. Since clients are specified to filter their pinned contract against the server's advertised revision, the bound has to carry what it used to be. Bounds now hold their full history, oldest first, and AtRevision hands back the limit a given peer actually enforces. A bound nobody has widened still serializes as a bare number, so the manifest reads the same and untouched entries do not churn the ETag. Validation gains the rules the representation makes checkable: a maximum may only grow and a minimum may only shrink, history is strictly ordered, later entries must say when they arrived, and the first entry cannot predate the definition. That last rule is the lower bound allowed_scopes already enforced; the same gap is closed for enum members, which could previously claim to predate the definition containing them. Reported by Codex review on #479. * fix(settings): accept the partial subtitle appearance objects already stored The schema required all nine properties, but the current API accepts and round-trips sparse objects — settings_device_test.go stores {"fontSize":"xxlarge"} and reads it back — and the web client has always merged whatever it gets over DEFAULT_SUBTITLE_APPEARANCE. Requiring the full object would have made the cutover migration quarantine preferences users really set, or block on them. Every property is now optional and a stored value is documented as a sparse override merged over the definition's complete default. An empty object is still rejected: an override that overrides nothing is the same state as no override, which the contract represents as unset. Cross-scope resolution is deliberately unchanged. A device override still replaces the profile's object rather than merging into it, because a device override means "draw subtitles this way on this screen", not "amend the profile" — and that is what the server does today. Reported by Codex review on #479. * fix(jellycompat): scan the parent directory when a sidecar changes Autoscan matched scantrigger rejections by comparing RequestError.Message against literal strings. One of those messages became "Unsupported media file extension for library type" and the copy in handlers_autoscan.go did not, so the comparison silently stopped matching. The effect is user-visible: a Jellyfin client posting a change for Movie.nfo or poster.jpg gets a 400 and the batch is abandoned, when the sidecar should have resolved to a scan of the directory containing it. Three tests covered exactly this and had been excluded rather than read. RequestError now carries a Reason the caller can switch on. Message stays prose for the client reading the response — it is meant to be reworded, and nothing should break when it is. Also makes two tests honest about asynchronous work. The Jellyfin Web teardown deleted its install root while the operation goroutine was still writing to it, where a late write recreates a path RemoveAll already walked past; it now waits for the operation's terminal state, which required exporting CurrentWebOperation. And the direct-play If-Range test pinned size and mtime so ctime was the only remaining validator, then read it back inside a single coarse-clock tick — it failed about 85% of the time on main for a reason unrelated to what it tests, and now rewrites until the stamp moves. With those fixed, GOTEST_KNOWN_FAILURES is empty and gone: make test-go runs the whole Go suite. The one test that cannot pass yet — TestHandleReplanPlaybackV3SeekFailureRecoveryNeverChangesMediaVersion, which has failed since the commit that introduced it and describes unimplemented v3 planner behavior — carries a t.Skip explaining that where the test is, rather than a regex in the Makefile. Reported by CodeRabbit review on #479. * fix(settings): reject JSON the decoder would otherwise rewrite Two cases where encoding/json accepts input by quietly changing it, which is the one thing a contract promising byte-identical agreement between peers cannot tolerate. Duplicate object properties. jsonschema.UnmarshalJSON keeps the last occurrence, so {"fontSize":"small","fontSize":"large"} validated and stored "large". Which one wins is a property of the parser, not of the contract: a client generated against a different JSON library can disagree about what it just sent, and the canonical form cannot represent the duplicate at all. Lone surrogates. An unpaired \ud800 became U+FFFD and canonicalization reported success, so the server would issue canonical bytes and an ETag for an artifact a conforming implementation must refuse — RFC 8785 requires terminating here. Substitution also means the value read back is not the value written. Both checks run before the decode that would hide them, on the shared decodeJSON path that the manifest, its public projection and every value schema go through, and again on the object branch of ValidateValue, which uses a different decoder. Reported by Codex review on #479. * ci: gate Go lint on the lines a branch changes AGENTS.md told contributors CI ran the same checks as `make lint`, and the Go job ran only gofmt and vet. A change failing the documented Go lint gate passed all three jobs. Running the linter as-is is not an option: the tree has ~296 findings today, which is why this half of `make lint` was never enforced. Blocking every PR on a cleanup nobody has scheduled gets the gate deleted again, so CI runs with --new-from-merge-base and only the lines a branch touches have to be clean. The count can then only fall. golangci-lint is built from source at a pinned version rather than downloaded. A released binary refuses to run against a Go newer than the one it was built with, and go.mod here tracks Go closely enough that the current release already fails that way on 1.26.4. .golangci.yml declared version 2 while still using v1's issues.exclude-rules key. Current golangci-lint ignores it, so the "allow repeated strings and unchecked cleanup errors in tests" exclusions silently did not apply — 16 findings in test files that the config says to skip. Moved to linters.exclusions, which `golangci-lint config verify` accepts. The four lines this surfaced in scantrigger are fixed rather than excluded: its repeated status codes and messages are now named constants, so one condition cannot end up worded two ways. Also drops the workflow token to contents:read and stops persisting credentials in the three checkouts, neither of which any job needs. Reported by CodeRabbit and Codex review on #479. * docs(v1): record the settings removal as a pre-lock exception The design removes the legacy /api/v1/settings routes and the profile DTO preference fields, while AGENTS.md states /api/v1 is additive-only and removals go through Deprecation/Sunset. Read together those contradict. They do not actually conflict: v1-scope.md scopes the additive-only rule to "when the scope locks", and the scope is still open, so a removal taken now is in scope and there is no amendment process to invoke yet. But that reasoning lived only in the settings design, where nobody checking the API policy would find it. v1-scope.md now carries a pre-lock removals table naming what goes and why waiting is worse, and states the deadline the argument depends on: a removal listed there must ship before lock or fall back to Deprecation/Sunset. AGENTS.md points at the table and says to treat an unlisted removal as a mistake. Reported by CodeRabbit review on #479. * fix(settings): clear the remaining review findings Small, unrelated except that each was raised on #479. compileObjectSchemas parsed every non-directory file under schemas/ as a JSON Schema, so a stray editor backup or .DS_Store would panic the server at startup through MustLoad. schema_ref can only name a .json file; anything else is skipped. cmd/silo used MustLoad while the ETag check beside it and every other startup failure use log.Fatalf. It now fails the same way, so a bad contract prints an error instead of a stack trace. TestRegistryDefaultsMatchTheContract called scalarDefault before handling null, and scalarDefault rejects null as non-scalar — so the subtest skipped and the comparison after it was unreachable. A nullable contract default could disagree with a non-empty registry default and nothing failed. Confirmed by injecting that drift, which now reports it. The three appearance providers each adapted the auth context to AppearanceAuth with identical code, putting the shape of auth back in three places that widening cache ownership would have to find. useAppearanceCacheOwner now does it once. useTheme.test.ts cleared storage.KEYS between cases, but appearanceCache writes namespaced keys and an owner pointer that are not in that list, so both survived and the suite was order-dependent. It clears the store, as storage.test.ts already did. The abs_smart_collection_store comment is reworded rather than given back its SQL quotes: gofmt folds a pair of apostrophes in a doc comment into a typographic quote, which is how it became one in the first place. Reported by CodeRabbit review on #479. * feat(settings): add canonical typed storage for the settings contract The cross-platform settings contract needs one typed store behind it before a resolver, routes or a migration can exist. This adds that storage to both user-store backends and holds them to identical behavior. PostgreSQL gets user_setting_values with the scope CHECK constraints, the five partial unique indexes that enforce one explicit value per identity, and the covering indexes the one-query read path needs, plus user_setting_mutations for mutation_id idempotency and the inert user_setting_migration_rejects audit table. The per-user SQLite store gets the same shape minus user_id, since that database is already user-scoped. The UserStore interface grows the typed operations: read one explicit value at one scope, collect every candidate row for a resolution request in a single query, upsert with a revision increment, unset, and the idempotency receipt operations. The resolution read deliberately returns unranked candidates so the resolver can rank in Go — one query per request, never one per scope, which the pgx query-count test pins. Delete behavior is application-enforced. Neither backend can inherit it from constraints: the SQLite store declares no foreign keys, and library, series and device columns are not FK targets in Postgres either. Profile deletion cascades to profile-anchored values while account scope survives, forgetting a device clears its profile_device values alongside the legacy overrides, and the library/series purges remove only what is scoped to that entity. The shared conformance suite covers all of it, including the set-versus-unset distinction for false, 0, "" and null, so a divergence between the two backends fails a test rather than reaching a client. Part of #376 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(settings): pin the settings-value schema constraints in both backends Completes the storage track. The conformance suite exercises the store API, which validates identities in Go before any SQL runs — so nothing noticed whether the CHECK constraints and partial unique indexes actually existed. The one-time migration writes these rows in bulk without going through the per-request path, so the schema is the only thing guarding it. Adds constraint tests to both backends covering every scope's column requirements, rejection of an unknown scope, a profile that does not exist, non-JSON values, and each of the five partial unique indexes. Also clears the lint the storage commit did not get to: sql.ErrNoRows and pgx.ErrNoRows compared with == rather than errors.Is (which fails on a wrapped error), an unchecked rows.Close, and repeated fixture literals in the shared suite now named so a backend that confuses two scope columns fails on the assertion rather than on a typo. * fix(settings): close the review findings in the validator and the theme cache Four defects the existing tests did not reach. The web theme resolver compared the server's value against the appearance cache and fell back when they agreed, but the mirroring effect writes the server's value into that same cache — so the comparison held on the first render and stopped holding on the second, reverting an explicitly chosen theme to the default. The server's value is this account's own stored choice, so it now simply wins. The regression test re-renders rather than asserting on the first paint, which is why the original one passed. golangci-lint's exclusions.paths is a path regex, not a directory list, so a bare `web` also excluded internal/jellycompat/web_component.go, internal/webhooksync/, internal/notifications/webhook*.go and eleven other non-test files that were being linted before. Anchored. json.Number is a string kind, so `"1.5"` unmarshalled into it happily and Float64 parsed the quoted digits: a numeric setting validated as a JSON string and NormalizeValue stored the quoted form into jsonb. Rejected. The lone-surrogate check ran only on the object branch, so a lone surrogate in ui.custom_css decoded to U+FFFD on SQLite and was refused outright by Postgres jsonb — the two backends disagreeing about whether the same value could be stored. Hoisted to cover every type. The strict language-tag validation this branch added is correct, but it rejects what the shipped Android client sends; the companion fix is silo-android 4aeb78b4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(auth): stop TestJWT_TamperedToken passing a valid signature The test overwrote the last character of the signature with "X". An HMAC-SHA256 signature is 32 bytes, so its base64url encoding is 43 characters and the final one carries only four significant bits — U, V, W and X all decode to the same trailing byte. Roughly one token in sixteen was therefore left byte-identical and validly signed, and the test failed because ValidateToken correctly accepted it. Measured at 3098/50000 (6.2%) over distinct signatures; it just failed the Go job on this branch for reasons unrelated to the branch. Flipping a character in the middle of the signature is 0/50000. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): reject raw invalid UTF-8, not just escaped surrogates The previous commit hoisted the lone-surrogate check to cover every value type, but that only closes the escaped path. A raw 0xff byte inside a quoted string — what an HTTP body carries when a client encodes text in the wrong charset — is not an escape, so the surrogate scan never sees it, while encoding/json still substitutes U+FFFD and reports success. NormalizeValue then stores the original bytes, which SQLite's json_valid accepts and Postgres jsonb refuses: the same backend divergence, reached the other way. Found by the Codex review bot on the previous commit's own diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): size the library page state bound to what the web client writes ui.library_page_state's `search` was bounded at 256 characters. The web client serializes an advanced library view as URLSearchParams, encoding each filter rule as three groups[i][rules][j][field|op|value] keys — measured at 216 characters for one rule, 518 for three, 820 for five. The current endpoint validates this key by checking only that it parses, so those oversized values are already stored in production. Typing them at the declared bound would have failed the migration for anyone who had saved a view with more than one filter rule, and rejected the equivalent write afterwards. Raised to 4096, which clears ten rules with room to spare while staying a real bound. The test pins it against the key shapes libraryPageSearchParams.ts actually emits rather than a round number. Reported by the Codex review bot; the lengths above were measured by calling serializeLibraryPageSearchParams, not estimated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): split quality into two axes and register the orphan keys Two manifest changes the cutover needs. **Quality becomes resolution + bitrate.** The legacy ladder values (1080p-high, 720p-medium, 1080p-8, 420p, 328p) were never a third dimension — they are a bitrate spelled into the resolution string. The web player already decomposes them: useTranscodeQuality.ts defines 1080p-high as {resolution: 1080p, bitrate: 10000} and sends the two separately, so the compound form never reached the wire. Downloads went further and kept only a bitrate ladder. So playback.preferred_quality keeps the six clean resolutions and playback.max_bitrate_kbps becomes the second axis, nullable because "uncapped" is a real answer and a numeric sentinel would need widening every time hardware improves. Clients compose their own presets from the pair, which means retuning what "High" means is a client release rather than a contract break. Migration decomposes each legacy value losslessly, so none of them lands in the rejects table. **The five extension-bag keys are now definitions.** card_overlays, next_up_mode, sidebar_pins, disabled_library_ids and library_order reached the server only through the unknown-key path, stored as unvalidated strings. Two of them the server reads back — next_up_mode decides home section assembly and card_overlays falls back to an admin default — so they cannot be demoted to client-local. Registering them is what lets the extension bag close. Adds three schemas for their shapes and a test that exercises every schema_ref against a real value: each of these is nullable with a null default, so the existing default-validation test returns at the null branch without ever compiling the reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical resolution engine One answer to "what is this setting, for this profile, on this device, for this content". Before this, each caller carried its own ladder: catalog/detail.go resolved subtitles across four levels by hand and audio across three, handlers/settings.go had a two-level device/user resolution with a lazy write-back inside a GET, and jellycompat read profile columns directly. Those disagreed about precedence, which is the drift the contract exists to remove. Resolution is one batched read regardless of how many keys, libraries, or series are in play — ranking happens in Go against each definition's declared resolution_order. Five sequential index lookups per key per item is the implementation the design rejects, and a season view is exactly where it would have shown up. An absent identity drops its scope rather than erroring, so one code path serves an identified client, an anonymous jellycompat seed, and a batch spanning many series. Rows for a foreign profile, device, library or series are ignored even though the batched read returns them. Constraints narrow without destroying: a capped 4K preference resolves to the cap, reports itself constrained, and keeps the authored value so it takes effect the day the cap lifts. Two cases needed care — null on a nullable numeric means unbounded, so a ceiling must cap it rather than rank it equal and let the value that most needs capping slip past; and an allowlist falls back to a permitted member rather than the definition's default, which may itself be outside the list. Adds ValueSchema.CompareValues to the contract package, since ordering values is what makes a ceiling or floor mean anything and value semantics belong with the schema that declares them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the one-time migration planner The conversion rules from legacy settings storage to canonical values, as ordinary Go rather than twice in two SQL dialects. Both backends read their own rows, hand them to Plan, and write what comes back — so the decisions are testable without a database and SQLite and Postgres cannot drift apart in what they decide. The rules that needed care, each pinned by a test: Column defaults are not choices. quality_preference is NOT NULL DEFAULT '1080p' while the contract defaults to auto, so migrating the column unconditionally would pin every profile in the install to 1080p having never chosen it — and that stored value would then outrank the contract default forever. Same for language 'en', subtitle_mode 'auto', and show_forced_subtitles true. The empty string is unset, not a value. The legacy string API had no way to send null, so both Android and web spell "clear my choice" as "". Storing that would make a cleared setting outrank the default. Legacy quality decomposes rather than rejects. Every compound value maps to a resolution and a bitrate from the ladder in useTranscodeQuality.ts, so nothing lands in the rejects table. Account rows fan out to every profile, which is the account-to-profile move the contract makes for appearance and search scope: a household that shared one theme each end up owning theirs. Legacy strings become typed JSON — "true" to true, "30" to 30 — or every generated binding would fail to decode what the migration wrote. Nullability differs per backend, so profile columns arrive as pointers and the caller resolves "chose the default" versus "never written" when it reads. jellycompat's DisplayPreferences blobs ride the same table under synthetic keys and are left alone; they are that subsystem's storage. Everything that cannot convert is recorded with a reason rather than dropped, and a final test asserts every planned row would be accepted by the mutation endpoint's own validation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): run the one-time migration on the SQLite backend Wires the planner to real storage as userdb migration V15. V14 created the tables; this fills them. It runs inside runMigrations' existing transaction, so a database either comes out fully migrated or untouched — a partial migration is the one state neither the operator's backup nor a rollback covers. Pinned by a test that rolls back and asserts nothing was left behind. Two things the wiring had to get right that the planner could not see: Reject identities are JSON. Postgres declares that column jsonb NOT NULL and SQLite guards it with a json_valid CHECK, so the free-form "profile=p1 device=d1" the planner emitted would have failed to insert — on exactly the rows the table exists to record. They are structured documents now, which is also queryable. Subtitle and audio preferences are two tables keyed the same way, so they merge into one per-series record before planning. Converting them independently would have produced two rows racing for the same identity. Every legacy read tolerates a missing table, since this runs against databases created at any schema version, and preferred_metadata_language is deliberately absent: that column exists only in the Postgres schema. Tested end to end against a real database rather than only through the planner — the rows land, satisfy the scope CHECK and the partial unique indexes, and hold valid JSON. Also covers the empty-install case and asserts a second run fails rather than silently doubling every value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): run the one-time migration on the Postgres backend The mirror of userdb V15, registered with goose as a Go migration rather than SQL: the conversion validates every value against its own definition and re-encodes it as typed JSON, and one legacy quality string becomes two rows — neither is expressible in SQL without duplicating the manifest. The rules stay in internal/settingsmigrate, so the two backends cannot disagree. RunTx, so the whole backfill lands in goose's transaction. The down migration empties the canonical tables; the legacy ones are never touched by the up, which is what keeps the cutover reversible until the follow-up migration drops the superseded columns. preferred_metadata_language is read here and only here — the column exists in this schema and not in SQLite's, so this is the sole source for catalog.metadata_language. Verified against a real Postgres: the full goose chain runs, 1080p-high decomposes to ("1080p", 10000), values land as typed jsonb rather than strings (jsonb_typeof reports number), rejects carry a queryable jsonb identity, and the composite profile foreign key refuses a row naming a profile that does not exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical settings API The routes that make the typed storage reachable. Until now the manifest, the resolver and the migration all existed with nothing able to call them. GET /settings/contract serves the public manifest behind an ETag — clients vendor a pinned copy and generate bindings from it, so the common request asks "still the same contract?" rather than transferring it. Its capabilities sibling reports revision and supported scopes for feature detection instead of version sniffing. /settings/values/{key} reads, writes and clears an explicit value at one named scope, which is what a reset affordance needs: "did I set this here" is a different question from "what applies", and the old endpoint could only answer a blurred version of both. Scope comes from the query while profile and device come from session headers, so one profile cannot address another's settings by naming it. /settings/values/effective resolves any number of keys in one request, with the resolution ladder and the source of each answer reported so a client can offer "reset this device's override" against the exact row holding it. Asking for no keys returns every remote setting, which is what a settings screen wants. Writes are idempotent when a client sends X-Silo-Mutation-Id: a retry after a dropped response replays the receipt, and reusing an id with different content is a conflict rather than a silent overwrite of the wrong thing. Three things the string-only endpoint could not do, each pinned by a test: an unknown key is refused rather than stored in the extension bag, values are checked against their declared type and range, and a write to a scope the definition does not allow is rejected. Registered before the catch-all /{key} routes, which would otherwise swallow "contract" and "values" as setting names. The legacy endpoints stay live for now; deleting them is the next commit, once their consumers move. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): generate typed bindings for all four languages One generator rather than one per repo. The point of the contract is that four codebases agree on keys, types, scopes and defaults, and four independently written generators would be four chances to disagree. Go and TypeScript land in this repo; Kotlin and Swift are written into the sibling client checkouts, skipped with a note when they are not present so a server-only developer can still run it. Output is sorted by key so an unrelated manifest edit does not produce spurious diffs. The Kotlin output is the interesting one: it generates the DeviceSettings allowlist Android maintained by hand, plus the BOOLEAN_KEYS/INT_KEYS/ DOUBLE_KEYS classification it kept as a *second* hand-maintained table that had to agree with the first. Both are manifest questions now, so the whole class of "wrote a local key to the server" and "flushed a value the store could not parse" bugs stops being possible by construction. The TypeScript output carries the full definition table — labels, controls, enum members, bounds — so web/src/lib/settingsManifest.ts can be deleted rather than kept in sync: it declared 17 definitions against the contract's 49, with its own two-scope model that does not match the contract's five. make verify-settings-bindings fails when the committed output disagrees with the manifest, wired into CI, so a manifest change cannot merge leaving every client reading stale keys. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(web): add the two-axis quality picker and typed settings hooks Quality becomes one picker over two stored values. The server holds a resolution cap and a bandwidth cap independently, which is what the player has always sent on the wire — useTranscodeQuality.ts has decomposed 1080p-high into {resolution, bitrate} for as long as it has existed. Presets live in the client rather than the contract so retuning what "High" means is a one-line edit here instead of a contract change four codebases have to agree on, and an older server keeps working because it only ever sees the two axes it already understands. A combination no preset covers still gets a truthful label rather than a picker showing the wrong entry: reachable by setting the axes separately through the API, or from a legacy value whose bitrate is off this ladder. Choosing an uncapped preset clears the bitrate rather than storing a sentinel, so "no cap" stays the absence of a value at every layer. Adds hooks over the canonical API alongside the legacy ones rather than replacing them wholesale — a key that is not in the manifest cannot be expressed, because SettingKey is generated from it, and the default for an unset value comes from the generated table rather than a literal at the call site. That last part is what stops the flip-off bug the Apple client carries a hand-written guard for. A test asserts every preset composes values the contract actually accepts, so a preset naming a resolution outside the enum or a bitrate outside the declared bounds fails here rather than 400ing when a user picks it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): resolve catalog playback preferences through the contract catalog/detail.go held the two hardest ladders in the codebase: subtitles resolved across four levels by hand, audio across three, each partially overriding the last through Has* flags. Both now call the canonical resolver, so the precedence lives in the manifest and this file cannot disagree with the contract about which override wins. Adding a scope is a manifest change rather than another branch here. The subtitle track signature stays on its specialized table — it identifies a concrete track rather than expressing a preference, so it is not a setting. Resolution keeps the memoization the old lookups had: the audio resolver still reads once per profile and once per library rather than once per file, which is what kept a many-track audiobook detail page fast. The test that guards it now counts resolver reads instead of GetProfile calls, since the guarantee is about scaling with file count rather than about which method does the reading. Four tests seeded the profile column directly. That column is a migration source now, not a read path, so they seed the canonical value instead — they were passing against storage nothing reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): close the unknown-key extension bag keyUsesUserScope returned true for any key the registry did not know, so a client could invent a production setting unilaterally and the server stored it as an unvalidated string. That is how six ui.* settings and five orphan keys reached production untyped, and it is the root enabler the design names. An unknown key is no longer a user setting, so the legacy write path rejects it and the canonical API — which validates every value against its own definition — is the only way to store something new. jellycompat's DisplayPreferences blobs ride the same table under synthetic keys and keep working: they are that subsystem's storage rather than user settings, and they move to dedicated storage in the follow-up rather than being dropped here. Also repoints the DisplayPreferences seed at the canonical resolver. Resolved at profile scope with no device on purpose — Jellyfin clients do not carry Silo's device identity, so a device override leaking into the seed would hand one device's settings to every Jellyfin client on the account. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): enforce viewer quality caps through resolver constraints constraintsFor was the unwired half of the preferences-versus-restrictions seam: it returned nil, so a profile capped at 1080p by policy still resolved its stored 2160p preference at face value through the effective endpoint. The settings routes are mounted inside RequireViewerAccess, so the resolved access scope is already on the request context. Scope.MaxPlaybackQuality holds a literal member of the contract's quality enum ("1080p"/"2160p"), which is exactly what the manifest binds playback.preferred_quality's ceiling to under policy_input "max_playback_quality" — so the wiring is a direct map with no translation table. An empty value means the policy sets no cap, expressed by returning nil so the resolver leaves the preference alone. catalog.metadata_language deliberately stays unconstrained: the manifest notes record that the allowlist draft was circular (the policy input it would bind to is populated from the very preference it would narrow). The handler test covers both halves of the seam: a 2160p preference under a 1080p cap resolves to the cap with constrained:true/ceiling and the authored value reported in stored_value, the stored row itself is not rewritten, and an uncapped viewer gets the preference unchanged with no constraint noise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): publish user_settings change events Add a user_settings realtime channel so clients learn when a setting changed on another device without polling. The channel is modeled on user_state: non-admin subscribable, per-user addressed envelopes, null snapshot. SettingValuesHandler gains an EventsHub and publishes user_settings.changed after every successful PUT and DELETE on /settings/values/{key}. The payload carries only key, scope and profile_id — never the value. Admins receive every user's user-scoped events, so a value in the payload would leak private settings to admins; interested clients re-fetch over the scoped REST API instead. The payload is always non-empty because an empty Data falls back to a null snapshot in the hub. A nil hub (tests) skips publishing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): sweep expired mutation receipts daily Setting-mutation idempotency receipts were written with an expires_at that nothing enforced, so the table grew forever. Add a hidden daily system task (05:00) that walks every login account, opens its user store, and calls DeleteExpiredSettingMutations. A user whose store fails to open or sweep is logged and skipped so one broken store cannot stall retention for everyone else; the delete is idempotent, so the next run repairs anything missed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(settings): resolve metadata language canonically in access and policy Repoint the last legacy column readers onto canonical contract resolution (settings cutover task A4a): - access.Resolver and policy.ViewerResolver now resolve catalog.metadata_language through settingsresolve (profile scope -> contract default) via a shared access.PreferredMetadataLanguage helper, instead of reading user_profiles.preferred_metadata_language. Resolution is deliberately unconstrained: the policy input this preference feeds is the one a constraint would have to reference, which is circular — see the key's manifest notes. - playback start now resolves playback.audio_language canonically for the profile default instead of reading user_profiles.language, matching the catalog detail path. Series and library override handling is unchanged. - items.go needed no change: it already consumes the resolver-produced scope.PreferredMetadataLanguage. The legacy columns keep their values but are no longer read on these paths; a profile with only a column value now resolves to the contract default, and a stored canonical value wins. Tests pin both directions in access, policy (including scope parity, where the column is now a decoy), and the playback handler. Read cost is one batched store read per resolution, same as the profile-row read it replaces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(jellycompat): give DisplayPreferences its own table The Jellyfin DisplayPreferences blobs rode the legacy user_settings key/value table under synthetic jellycompat:* keys, which forced the legacy settings API to carry a prefix carve-out in its otherwise-closed unknown-key gate. They are the compat subsystem's storage, not user settings: the contract neither validates nor resolves them. Move them to a dedicated jellycompat_displayprefs table in both backends, keyed by (prefs id, client) per user, with the blob stored as opaque text served back byte-for-byte (deliberately not jsonb, which would re-serialize it). The data-copy migrations — per-user SQLite V16 and a paired SQL + Go goose migration for Postgres — are transactional and harmless to re-run, and both drive their key parsing and row classification from the new internal/jellycompat/displayprefs package so the backends cannot diverge, following the internal/settingsmigrate precedent. A jellycompat:* row that does not parse as a DisplayPrefs key (only ever writable through the removed carve-out) is recorded in user_setting_migration_rejects rather than silently deleted. With the last non-settings tenant gone, the jellycompatSettingPrefix carve-out is deleted: the legacy settings endpoints now refuse jellycompat:* keys like any other unknown key and never surface them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): serve admin user-settings through the canonical API Replace the ten string-registry /admin/users/{id}/settings* and device-settings* routes with the canonical contract surface: one list of every explicit value the target user has stored across all scopes, and set/delete at an explicit scope named in the query string. The admin handlers live on SettingValuesHandler and share the session routes' implementation rather than duplicating it — the same key/scope parsing, identity validation, contract scope allowance, value normalization and mutation-receipt idempotency, factored into keyedScopeFromRequest/completeIdentity and setValueAt/deleteValueAt. The only admin-specific parts are the target user coming from the path, profile and device ids coming from the query (an admin holds no session claim to the user being inspected, so its named profile is checked to exist), and change events attributed to the target user so their clients refresh. The list is a new UserStore read, ListAllSettingValues, implemented in both backends and pinned by the shared storetest conformance suite: the admin surface wants the stored truth (which overrides exist, for a per-row reset affordance), which no resolution-shaped read answers. The ten removed routes are recorded in the pre-lock removals table in docs/architecture/v1-scope.md per the v1 API rules; the web admin device-overrides page moves onto the new surface in the Phase B rewrite inside this same unmerged PR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(settings): add the cross-platform conformance fixture and its Go and web runners contracts/settings/v1/conformance.json is the spec's named drift gate: 21 hand-authored cases of {keys, stored rows, context, constraints, expected effective value + source}, every one executable against the shipped manifest. They pin the semantics most likely to drift across four resolver implementations: the full resolution ladder (series > library > device > profile > default), an absent identity dropping its scopes, foreign-identity rows never resolving, ceiling caps that report the authored value with constrained:true, the ordered-enum sentinels (auto below every cap, original above), null-on-a-nullable-numeric meaning unbounded and being brought down by a ceiling but ignored by a floor, allowlist falling back to the first allowed member rather than the (possibly forbidden) default, and playback.subtitle_appearance resolving device > profile only with the sparse device object replacing, not merging. Cases may inject a constraint binding onto a copy of a real definition so constraint kinds no shipped definition carries stay testable. The Go runner (internal/settingsresolve/conformance_test.go) resolves each case through the real resolver against the embedded manifest. The web runner (web/src/lib/settingsConformance.test.ts) runs the same cases through a new client-side resolver, web/src/lib/settingsResolve.ts, which mirrors the server's semantics; the TypeScript bindings now carry each definition's ordered flag and constrained_by binding so that resolver derives constraint behavior from the contract instead of hardcoding it. Both runners reject unknown fixture fields — schema drift in the fixture itself is drift — and both refuse a fixture authored against a different manifest revision. The fixture travels with the bindings: make settings-bindings vendors the copy the web runner reads, and make verify-settings-bindings fails CI when that copy goes stale. The Kotlin and Swift copies land together with their runners in the client repos, which will pick their own test-resource paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settings): review pass over the phase A stack Fixes the eight adversarially-confirmed defects the review of the unpushed phase A stack (40e0f77a..1f2c7fe4) found, each with a test that fails without its fix. Writers left behind by the language cutover (high). |
||
|
Quick
|
a7601028d9 | fix(playback): fail closed on inconclusive PPS scans | ||
|
CoffeeKnyteandQuick
|
9281ae61d9 |
fix(playback): transcode H.264 sources with conflicting in-band PPS
H.264 streams that redefine the same pic_parameter_set_id in-band with different content cannot be safely stream-copied into an avc1/fMP4 HLS segment: the avcC advertises a single parameter set, so VideoToolbox (Safari/Chrome on macOS) decodes with the wrong PPS and desyncs mid-GOP, surfacing as PIPELINE_ERROR_DECODE / kVTVideoDecoderBadDataErr (-12909). At playback start, a bitstream scan (DetectMultiplePPSH264) runs for H.264 files on the probe-ensure path, grouping in-band PPS by id and flagging any id carrying more than one distinct definition. The result is a runtime-only flag (VideoTrack.MultiplePPS, json:"-") memoized per process — never written to the database, no schema change, recomputed on the first play after a restart. The v3 planner and legacy resolver disqualify a copy-unsafe source from the video stream-copy / remux ladder, routing it to a real transcode. Direct play of the original container is left intact: decoders that reparse in-band parameter sets (ExoPlayer, VLC, native) handle the source fine. Part of #135. |
||
|
CoffeeKnyteandQuick
|
7907e0c28c |
fix(playback): composite bitmap subtitle burn-in on the GPU for QSV/VAAPI
Bitmap subtitle burn-in (PGS/VOBSUB/DVB) ran the whole video through a GPU->CPU->GPU roundtrip: every decoded frame was hwdownload'd to system memory, the subtitle bitmap composited with the software overlay filter, then hwupload'd back for the encoder. On a 1080p source that pins the encode below realtime (~0.68x measured), so the client can never build a buffer and rides the produced-head edge indefinitely; the same roundtrip also intermittently crashes the QSV buffer path with SIGBUS. Composite on the GPU instead via overlay_vaapi for QSV and VAAPI: the decoded video never leaves its VAAPI surface and only the small, low-frequency subtitle bitmap is uploaded. Measured ~7x realtime and crash-free on the same file. The libass text path is unchanged (it must stay on CPU), and NVENC/CPU keep the software overlay because overlay_cuda is unverified on the bundled ffmpeg. Adds QSV and NVENC bitmap burn-in tests and updates the VAAPI test to the GPU graph. |
||
|
|
c52ca7dd7a |
feat(admin): identify compat sessions and Android devices in the live session view (#495)
* feat(admin): identify Android devices by model in live session view Android clients that send a bare default User-Agent (e.g. "Dalvik/2.1.0 (Linux; U; Android 11; AFTKRT Build/RS8180.3729N)") showed up as "Dalvik" in the admin live-session view, which tells an operator nothing about the device. Parse the model code out of the UA (the token between the last ';' and "Build/") and map the Amazon Fire TV family and NVIDIA Shield to product names. Unknown but parseable models fall back to "Android · <MODEL>" instead of "Dalvik"; multi-word models like "Pixel 7" are preserved whole. This is display-only: the session still stores the raw model code in its user agent, and no response field or contract changes. * feat(admin): mark Jellyfin-compat sessions with the JF pill by origin The admin "JF" pill was derived at read time by substring-matching a token list against the client name / user agent. A real Jellyfin client that authenticates through the compat surface but sends a bare User-Agent and no MediaBrowser client name (e.g. a Fire TV app) got no pill, even though it plainly came through the Jellyfin API. Stamp compat origin as immutable identity at session creation and carry it through to the admin view: - ClientInfo.IsCompat is set true in the jellycompat auth path; newSession copies it onto Session.IsJellyfinCompat. - The flag rides the durable RecipeCard (next to the client metadata that already exists so the pill survives reconstruction) and is restored in ReconstructSession, so a server restart keeps the pill. - buildLiveSessionSync -> worker.SessionSync -> a new compat_origin column on playback_sessions_sync (added migration); the reconciler upserts, reloads, and compares it so origin changes still publish and unchanged rows do not churn. - The handler ORs the stored origin with the existing name/UA heuristic, which stays as a fallback for rows written before this column existed. is_jellyfin_client keeps the same name and type on the wire; it is only sourced more accurately. * fix(admin): correct Android device labels --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
8dea4b9056 |
fix(playback): stop a broken Dolby Vision RPU from hanging playback (#485)
* fix(playback): stop a broken Dolby Vision RPU from hanging playback
A Profile 7 source whose RPU ffmpeg cannot parse took the whole session
down. The dovi_rpu bitstream filter does not fail cleanly — it rejects
every packet while ffmpeg runs on, so one observed session emitted
376,316 stderr lines before the process was killed, no manifest was ever
built, and the client got a 503 after ~10 seconds that it showed as an
endless spinner:
[dovi_rpu] Failed to read unit 1 (type 39).
[vost#0:0/copy] Error applying bitstream filters to a packet:
Invalid data found ... Invalid SEI message: payload_size too large
Whether the strip works is a property of the file, not of ffmpeg, so
SupportsDoviRPUFilter cannot answer it — but asking ffmpeg to strip two
seconds to the null muxer can, in about a second. Profile 7 sources are
probed once each on the start path and the result is cached; a source
that fails is copied without the filter, which leaves the base layer and
plays. Refusing to play at all does not.
The probe reads stderr rather than trusting the exit code: ffmpeg treats
a per-packet bitstream-filter error as non-fatal and exits 0, which is
exactly how a stream that could never start reached a live session.
Cache is keyed on path plus size so a replaced file is re-probed, and
bounded like the letterbox cache. A nil probe keeps stripping, since most
Profile 7 sources need it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Db4dSxN9tH8yN7uUP549tK
* chore(playback): use InfoContext in the rpu probe
* fix(playback): decide the DV RPU strip in the plan, not at the transport
The probe answered the right question in the wrong place. Suppressing the
bitstream filter as the transport was being built left the plan still
promising DynamicRange "hdr10", Claims.Video.HDR10 and the
dolby_vision_metadata_removed / hdr10_base_layer_preserved validated
claims, and left session.RemuxDVMode persisted as strip_to_hdr10 — so the
client was told it was getting clean HDR10 while receiving Profile 7 with
dangling RPUs, and every path that re-derives the filter from the durable
session put the hanging filter straight back:
- HandleStartTranscode (quality change, seek, burn-in restart), which
also re-derives it from file.PrimaryDVProfile() == 7 with no plan at all
- the remote audio-switch restart, from updatedSession.RemuxDVMode
- the progressive-remux transport, which plan_v3 evaluates *before* the
HLS branches, so for many clients the broken file never reached the
probe in the first place
The verdict is now a planner input alongside the transformation
registries: the registries answer whether the executor carries the
transformation, this answers whether the file does. A source that fails it
is never planned onto a strip route, so the plan's claims, RemuxDVMode and
every restart derived from them agree with what the pipeline can produce.
With no tone-map recipe in this tree an HDR10-only client has no route
left, so it gets a dv_conversion_unsupported terminal naming the real
cause rather than a generic HDR message; a client that can run its own DV
transformation still gets that route, with a degradation warning
explaining why the server route was dropped.
The two paths that bypass the plan entirely are gated at the executor:
legacy/auto remux neutralizes the profile exactly as it already does for a
missing dovi_rpu filter, and the explicit v3 strip recipe fails loudly
rather than emit dangling RPUs under an HDR10 claim.
The probe itself:
- Tri-state verdict. Only a stderr-confirmed rejection is cached. A
timeout, a cancelled request or an ffmpeg that will not start is
inconclusive: the strip is kept and nothing is written, so one client
disconnecting can no longer disable the strip for a file permanently.
- Singleflight, so concurrent or retried starts of a title share one run.
- Timeout cut to 6s, inside the budget a client waits on the manifest,
and off the session lifecycle lock now that it runs at planning time.
- Keyed on size and mtime as well as path and binary, so a file replaced
in place with the same length is re-probed.
- stderr capture bounded at 64 KiB; the markers are in the first lines
and a rejecting filter emits a pair per frame.
- The head-only coverage is stated rather than asserted: this catches a
source that rejects from the first access unit, not one that breaks an
hour in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(playback): keep the RPU probe alive when its caller leaves
The probe rode the caller's context, so a leader whose client disconnected
returned dvRPUUnknown — and CanStrip then handed that fail-open to every
follower already blocked on the shared call, even though their own requests
were still alive. One client walking away was enough to give a live session
the strip the source cannot survive, which is the hang this whole change
exists to prevent. The work was also thrown away, so the next start paid for
the probe again.
Run it under context.WithoutCancel instead. dvRPUProbeTimeout still bounds
it, so nothing is left running; a verdict reached after the leader has gone
is still correct and still worth caching. Follower behaviour is unchanged:
a follower whose own request is cancelled still leaves immediately rather
than waiting on someone else's probe.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
|
||
|
|
02203d9e40 |
fix(playback): trust the server's media runtime end to end (#482)
* fix(scanner): reject durations that imply an impossible bitrate The duration-plausibility rule only rejected videos of 10 seconds or less, so a feature film that probed as 61 seconds passed untouched and persisted. Clients then had nothing trustworthy to anchor on: Android's grow-only duration ratchet has no floor to hold when the catalog value is wrong, so the playback engine's growing-HLS-window duration won and a 90-minute movie displayed as ~1 minute. Size and duration together pin an implied bitrate, which separates the two cases the absolute floor conflates. A genuine short clip has an ordinary bitrate; a 100 GB file claiming 61 seconds implies ~13 Gbps. The ceiling sits far above any real medium, so legitimate content cannot trip it — and unlike the absolute floor, it does not false-positive on a genuine high-bitrate short. Also bump the repair-rule revision marker so rows judged by the previous, weaker rule are re-checked once under this one. Without that bump an improved rule never reaches the rows it was written for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(playback): publish source runtime in v3 plans and stop faking the copy seek window Two defects with one root: a v3 plan described where playback sits without ever stating how long the media is. Add source.duration_seconds. It is the file's full runtime, never `total - source_start` and never adjusted by timeline_offset_seconds, and it is omitted rather than null when unknown — clients that coerce null to a numeric default would read it as zero, the exact value this field exists to stop them inventing. It is set in SourceDescriptorFromFileV3, the single place every delivery already flows through, so direct play, progressive remux, HLS remux and HLS transcode all carry it. Until now the v3 plan omitted duration entirely, so clients fell back to the playback engine. On an HLS copy remux the server intentionally serves FFmpeg's still-growing playlist, so the engine reports the length produced so far. With no server-supplied runtime to anchor on, a feature film played back as a couple of minutes. The legacy protocol already answered this correctly via fileDurationSeconds; this restores parity. Separately, the copy branch published seek_window_end_seconds as the media runtime. That made the window look *complete*, which clients read as proof that any target inside it is locally seekable, so they native-seek past the produced head of a growing playlist instead of asking for a reanchor. Leave the end open: an incomplete window plus can_seek_anywhere=false routes every seek through the server, which is what legacy did before v3 added the bound. Advertise plan_source_duration_v1 so a client can distinguish "this server does not populate the field" from "this server knows the runtime is genuinely unknown" — without it, both look like an absent field and a client cannot tell whether its own catalog fallback is still required. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(web): pair the exit position with the media runtime, not the element duration The player's exit state converts its position to media time but took the duration from the video element, which is player-local. On a remux or transcode stream the element only covers the window produced so far, so the two values live in different coordinate systems. Resuming a movie 50 minutes in makes that concrete: the exit position is ~3060s of media time while the element reports ~120s. The progress cache then evaluates `position >= duration`, marks the item completed, latches the watched badge, and — because completion clears the resume point — resets position to 0. Exiting a resumed movie destroyed the resume point and claimed it had been watched. The server's runtime is authoritative and already expressed in media time, so prefer it and fall back to the element only when no server value exists. The rule moves into mediaTimeline.ts next to the coordinate conversions it depends on, which is also what makes it testable — VideoPlayer itself has no test harness. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
ee31a1f0e2 |
feat(playback): formalize resumable direct streams and stall observability (#464)
* feat(playback): formalize resumable direct streams and stall observability Implements #443: strong stat-based ETag + If-Range on original-file direct play (via http.ServeContent), stream-end outcome classification in RollingDeadlineWriter (stalled_reap vs client_gone vs completed) with a structured log event and Prometheus counters, the direct_stream_resume_v1 protocol-v3 capability, and a contract doc. Progressive remux is explicitly excluded from the resume contract. Code written by OpenAI Codex CLI (gpt-5.6-sol) from a Claude-authored spec; reviewed and verified by Claude. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden direct stream resume contract * test(playback): cover resume platform contracts --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104
|
4b3362d32e | fix(playback): expose source color range to native clients | ||
|
|
845b96e703 |
fix(playback): preserve remux copy on seek (#422)
* fix(playback): preserve remux copy on seek * fix(playback): harden remux replacement transactions --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
8044eb84dd |
feat(activity): refine play-method tags and add a Jellyfin-client pill (#387)
* feat(activity): refine play-method tags and add a Jellyfin-client pill
Two related tagging improvements to the admin activity views, squashed:
Split audio transcodes into their own tag. The Play Method summary and
Server Activity popover bucketed every session by its raw play_method,
lumping real video transcodes together with video-copy HLS repackages and
having no separate tag for audio-only transcodes. Classify each session by
the per-stream decisions the backend already reports:
- video re-encoded -> "transcode" (yellow)
- only audio re-encoded -> "audio" (red)
- streams only repackaged -> "remux" (blue, incl. video-copy HLS)
- nothing touched -> "direct" (green)
ordered direct -> remux -> transcode -> audio across the distribution bar,
legend, method filter/sort, the per-row badge, and the Server Activity
stream counts.
Add a Jellyfin-client "JF" pill. Sessions from a Jellyfin-ecosystem client
(Jellyfin Web, Findroid, Swiftfin, Infuse, etc.) get a purple "JF" pill
next to the play-method tag. Detection is UI-only: isJellyfinSession()
positively matches client_name (set from the Jellyfin MediaBrowser auth
header) and then the raw user agent against the known Jellyfin client
tokens, mirroring the server's client-labeling list. The pill is orthogonal
to the method classification — a session can be both "transcode" and JF.
Pure UI/presentation change; no backend behavior changes.
AI-use disclosure: implemented with AI assistance (Claude Code).
* fix(web): cache-control on SPA shell so deploys bust stale UI
The frontend handler served index.html with no cache directives, leaving
freshness to browser/CDN heuristics. A stale index.html at a CDN edge kept
serving old content-hashed bundles, so a client-side hard refresh couldn't
recover — one browser would show the new UI while another showed the old.
Apply the standard SPA cache policy:
- index.html (and SPA-route fallbacks): no-cache + a truncated-SHA-256
ETag, so the shell is cached but revalidated on every load and answers
an unchanged request with a cheap 304.
- /assets/* (Vite content-hashed bundles): public, max-age=31536000,
immutable — cached indefinitely; a new build changes the filename hash,
which busts them automatically.
- other stable-named bundled files (sw.js, icons, fonts): no-cache, so a
changed service worker or icon can't stay stuck in a cache.
Caching is preserved (no no-store anywhere); only the tiny HTML shell is
revalidated, which is what busts a stale UI on deploy.
* fix(activity): compute the method bucket server-side and unify every session surface
Review follow-ups for the play-method tags (PR #387):
- The server now emits effective_play_method (additive field) from the same
per-stream decisions that drive the badges, so all consumers — web, realtime
popover, and the Android/Apple admin views later — agree on the bucket
instead of each client re-reducing raw play_method. Rows with an unknown
play_method (stale rows from older nodes) stay unbucketed rather than being
misreported as audio transcodes off the bare transcode_audio flag; the web
fallback classifier mirrors that and reports "unknown".
- Jellyfin-ecosystem detection moved server-side as is_jellyfin_client, owned
next to the client-labeling rules so the two lists cannot drift; the web
token list is gone. Adds kodi/mpv/delfin/finamp, which reach Silo only
through the Jellyfin compat surface.
- The dashboard stream cards, stats session table, and household streams panel
now use the same classification as the activity page and popover — they
previously showed contradictory tags for the same live session.
- One shared method->label/color table in adminActivityPresentation.ts
replaces the four independent copies (METHOD_META + three switches); the
method column sort now uses the shared cost-order comparator instead of
alphabetical; dead "copy"/"hls" order entries removed and the reachable
"unknown" bucket is styled.
* fix(server): make SPA revalidation RFC-compliant and stop rebuilding the shell per request
Review follow-ups for the SPA cache policy (PR #387):
- Stable-URL bundled files (sw.js, icons, vendor bundles) now carry a content
ETag. The embedded FS has no modtimes, so http.FileServer emits no validator
of its own — no-cache alone forced a full re-download of multi-megabyte
vendor trees on every use because there was nothing to revalidate against.
- Shell and favicon conditional requests go through http.ServeContent, which
implements RFC 9110 If-None-Match semantics (weak comparison, ETag lists).
The previous exact string compare never matched once a fronting proxy
compressed the response and weakened the ETag to W/"...", silently killing
the 304 path in the most common deployment topology.
- The rendered shell (index read + branding render + SHA-256) is cached per
branding snapshot via the new Snapshot.RenderKey instead of being rebuilt on
every request — the 304 revalidation that no-cache makes the common case now
costs two header writes. The misnamed weakContentETag (it emits a strong
validator) is renamed contentETag.
* fix(activity): show the JF pill on every session surface, not just the mobile row
Review comments on PR #387: the JF pill only rendered inside Admin
Activity's sm:hidden mobile row, so the desktop table — and the other
session surfaces that now share the method classification — never
identified Jellyfin-compat sessions.
Extract the pill into a shared JellyfinSessionPill component (renders
nothing for native sessions) and drop it into the Admin Activity desktop
client line, the dashboard stream cards, the household streams panel,
and the stats active-session table.
* fix(playback): sync real encode decisions and client identity for compat transcodes
Review comments on PR #387:
- Jellyfin HLS sessions that copy video and re-encode only audio synced as
full video transcodes: ensureUpstreamPlayback resets transcodeAudio for the
transcode transport method, and the TargetCodecVideo "copy" decision lived
only in TranscodeOpts. A new SessionManager.SetTranscodeStreamDetails
mirrors the actual decisions onto the upstream session when the transcode
starts (local and remote-node paths, via an optional interface so test
fakes are unaffected), so these sessions now bucket as "audio"/"remux".
- Transcode recipe cards now record TranscodeAudio derived from the opts
(only an explicit "copy" leaves audio untouched — empty runs ffmpeg's aac
default), so a session rebuilt after a restart keeps the same bucket.
- Recipe cards carry client name/version/user-agent, and reconstruction
restores them, so the admin client label and the JF pill survive server
restarts; the compat fallback card populates them from the live
MediaBrowser request. Deliberately not projected into stream-token claims,
where a user agent would bloat every stream URL.
* feat(api): capability endpoint for the live-session activity fields
Review comment on PR #387: effective_play_method and is_jellyfin_client are
omitempty, so an independently deployed client cannot distinguish an older
server from a supported one reporting an unknown method or a non-Jellyfin
session. GET /admin/sessions/capabilities advertises both fields plus the
closed bucket vocabulary, following the additive capability-endpoint rule
(same pattern as /collections/capabilities).
* fix(playback): treat empty target audio codec as an AAC re-encode in live state
ffmpeg defaults an empty target audio codec to AAC (appendAudioArgs), and the
new recipe logic already records that as an audio transcode — but the live
native path computed transcodeAudio=false for an empty codec, so the running
stream reported remux until a restart flipped it to audio. Extract the
predicate into playback.TranscodesAudio, share it across the live path, the
recipe card, and the compat mirror, and make appendAudioArgs case-insensitive
so the ffmpeg switch agrees with the predicate for any spelling.
Part of #387 review follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(jellycompat): re-sync sessions after recording compat encode decisions
ensureUpstreamPlayback flushes the session (compat_start) before
ensureTranscodeSession / startRemoteTranscode record the actual codec
decisions, and that later mutation triggered no sync — so the admin view
showed a video-copy stream as a full video transcode until the periodic
reconciler ran. Trigger syncSessionsNow after the details are recorded
successfully; the helper is shared, so both the local and remote-node
paths are covered.
Part of #387 review follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
CoffeeKnyteandGitHub
|
b3722dac58 |
fix(playback): open the listener before sweeping stale transcode dirs (#413)
* fix(playback): run orphaned-transcode cleanup in the background at startup The native and Jellyfin-compat routers swept stale per-session transcode dirs synchronously during NewRouter, before the listener bound. On a slow network filesystem this blocked startup for 80+s (64 leftover dirs on the last deploy), so restart-reconnect clients were turned away and the health check reported the server unhealthy the whole time. Move both sweeps into a background goroutine (StartBackgroundOrphanCleanup) so the listener comes up immediately and the cleanup runs concurrently. The delete logic is unchanged: same active-session snapshot and MaxTokenTTL age-sparing, only later. A package-level mutex serializes concurrent sweeps of the shared transcode root so the two background sweeps can't race on os.RemoveAll. Part of #412 * fix(transcode): background the node boot-time transcode-dir sweep A dedicated transcode node swept leftover transcode dirs synchronously in NewServer, before startStandaloneServer bound its listener. On a slow network filesystem that delete blocked the node from coming online at boot, the same startup-stall class as the main server. Move the sweep into the shared StartBackgroundOrphanCleanup goroutine so the node's listener binds immediately. Backgrounding required an age guard: the sweep previously ran as a full wipe (minAge=0) with an empty active-set, which was only safe because it completed before any request could arrive. Run concurrently that would race a token-carried reconstruct writing into TranscodeDir/<sessionID>, deleting segments a fresh ffmpeg is producing. Passing MaxTokenTTL spares any dir younger than the max token lifetime — exactly the ones a still-valid reconnect could reconstruct — while dirs older than any surviving token (never reconstructable) are still reclaimed. Part of #412 * feat(playback): reclaim orphaned transcode dirs periodically, not just at boot The orphaned-transcode sweep only ran at startup on both the central server and transcode nodes, so it only ever reclaimed dirs left by an ungraceful prior shutdown. During a long uptime the in-memory session reapers delete the dirs of sessions they still track, but a dir whose owning session was dropped without its RemoveAll succeeding becomes an "untracked orphan" with no runtime GC — on a box that runs for weeks these accumulate until the next restart. Add StartPeriodicOrphanCleanup: an immediate background sweep followed by an hourly re-run bound to a lifecycle context. Wire it on all three surfaces — native API and Jellyfin-compat (via deps.AppContext) and the transcode node (via a new Server.StartOrphanSweeper(appCtx), replacing its boot-only sweep). When no context is supplied (tests) it degrades to a single boot-time sweep so no ticker goroutine outlives the caller. The sweep stays age-guarded at MaxTokenTTL, so nothing reconstructable is ever reaped. Because the node sweep now runs during live traffic, it snapshots the live job set (Server.activeSessionIDs) and spares those dirs by id rather than by age alone — a long-lived session that only re-serves already-written segments stops advancing its dir mtime, which age could otherwise misclassify. In integrated mode the native and compat sweeps share one TranscodeDir but each snapshots only its own manager's live set; the resulting cross-manager reap of a >24h idle dir is bounded (rebuilds from token/recipe) and documented at both call sites. Part of #412 |
||
|
Jeremy SmithandGitHub
|
6ece120f77 |
fix(playback): stop orphan cleanup deleting the subtitle cache (#379)
The orphaned session sweep in CleanupOrphanedTranscodeDirs treats every subdirectory of the transcode dir as a dead session, but the subtitle cache lives in there too. Cache hits only bump the entry file mtimes, never the directory's, so an active cache that had not seen a new extraction in 24 hours looked stale and got wiped in one go. That forces a full ffmpeg demux on the next PGS subtitle selection, which is the exact cost the cache exists to avoid, and the transcode node's boot wipe removed it unconditionally. Skip the cache directory in the sweep. It validates entries on lookup and runs its own LRU eviction, so leaving it alone is safe. |
||
|
Quick104andClaude Fable 5
|
18283c2c9b |
fix(playback): HLS-safe audio policy and surround-preserving transcodes
Two audio fixes on the V3 planner and transcode pipeline: - Copied DTS in an HLS route drags Media3's audio clock (device stall corrections, ~0.3x pacing, frozen position reports). DTS/TrueHD/PCM are not HLS-native codecs regardless of the client's progressive decode claims, so HLS remux routes now convert them to AAC. Validated on the Shield: the copy-remux fallback went from ~0.3x pacing to exactly real time. - Transcodes no longer hard-downmix to stereo: multichannel sources keep 5.1 through the AAC re-encode (384k, -ac 6), plumbed through TranscodeOpts, the planner result, and the transcode-node protocol (new optional target_audio_channels field, ignored by older nodes). Also logs one "playback plan decided" line per V3 start (decision reason, delivery, play method, DV profile, quality inputs) so route selection is reconstructible from server logs alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
075e217477 |
feat(playback): plan v3 routes from pooled node capabilities (#408)
* feat(playback): plan v3 routes from pooled node capabilities Protocol v3 planning previously gated every server transformation on the API host's local ffmpeg probe, so deployments whose toolchain lives on transcode nodes (libx264/aac/dovi_rpu on nodes, minimal binary locally) received conversion terminals before transport preparation ever consulted the selected node's capabilities. Planning now draws on two registries split by executor pool: - Registry stays the local probe and keeps gating progressive remux routes, which execute in this process and can never offload. - HLSRegistry widens availability for HLS deliveries with the pooled transcode nodes' advertised transformations (name and recipe version pinned to the local specs), fetched concurrently under a short planning deadline through the existing TTL cache. Failures are now negatively cached so an unreachable node costs one timeout per window rather than one per start. The remux family picks the executor per branch: a recipe needing transformations only nodes carry skips the progressive remux and ships the same recipe on the HLS remux delivery instead. The local-fallback path in prepareTransportV3 now validates plans against the local registry's advertised set — mirroring the per-node validation — and returns the existing retryable transcode_node_capability_unavailable terminal when no executor can run the recipe, instead of spawning an ffmpeg that would fail at runtime. Deferred from PR #398 review (comment 3579105380). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden union capability planning from review Addresses all four review findings on the capability-union feature: - Select capability-matching nodes: plans carrying server transformations now restrict node selection to nodes whose advertised capabilities validate against the plan (nodepool.PlanSessionWith with a set-lookup predicate), so heterogeneous pools cannot load-balance a recipe onto a node that would reject it while a capable sibling exists. Transformation-free plans keep pure load-based selection. - Split the capability cache by consumer: planning honors negatively cached fetch failures (one timeout per window), while the transport path fetches through them — a memoized 3s planning deadline must not reject an already-selected node that the 10s transport budget could still validate. - Gate node-widened availability on the HLS engine: a progressive-only client that needs audio conversion keeps its specific retryable audio_conversion_unsupported terminal instead of falling through to a non-retryable adaptation_unavailable for routes it can never run; the DV strip union flag is gated identically. - Make HLSRegistry a lazy, memoized producer: the planner only builds the widened registry when a route decision depends on node capabilities, so direct-play and other source-preserving starts never wait on node capability fetches (or their dead-node deadlines). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
854d07cf8f |
feat(playback): add protocol v3 planning and recovery (#398)
* docs(playback): plan protocol v3 server implementation * docs(playback): incorporate protocol v3 review * feat(playback): implement protocol v3 server * fix(playback): persist empty route diagnostics * feat(playback): harden protocol v3 HDR routing * feat(playback): complete protocol v3 client contract * fix(playback): harden protocol v3 recovery * fix(playback): restore dovi_rpu strip filter for DV remuxes The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg; the probe failed closed on every deployment, disabling the new validated DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1 remux path from main. Restore dovi_rpu across the probe, remux and HLS copy arguments, and the recipe-card constant. Also from review: validate the remux DV mode for every profile (garbage modes on non-P7 sources silently no-opped), reject preserve mode for P7 outright (a base-layer-only remux cannot preserve dual-layer DV), tag dvhe sample entries only for the explicit v3 preserve recipe so legacy web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the token-frozen DV mode in the proxy remux path instead of legacy-auto. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): correct v3 planner policy and contract validation Review fixes to the v3 planner and wire contracts: - Bar Profile 7 sources from the non-strip progressive remux route: a base-layer-only remux can never deliver native dual-layer DV, so the planner no longer emits plans claiming validated Dolby Vision while the executed remux drops the enhancement layer. - Accept the device-quirks feature flag from either capability location, matching every other dual-location feature check. - Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with a degradation warning instead of leaving them unplayable under v3. - Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and wire the previously dead Metered signal into conservative auto rungs. - Degrade to the validated source-quality route instead of a terminal when only an implicit quality reduction demanded an unsupported transcode; explicit user-selected rungs keep terminal behavior. - Bound inner capability lists and strings; compare attempt keys exactly instead of case-folded; make ParseTrackIDV3 strict about canonical numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising burn-in for unknown subtitle codecs; probe every h264 encoder rather than requiring libx264; normalize the file-level bitrate fallback. - Evaluate subtitle renderability against the engine each candidate route executes on, not always media3_direct. - Pin the with-quirks attempt-key preimage arity in the cross-language fixture so the Kotlin client stays in lockstep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 control-plane reliability Review fixes to the v3 session, store, and handler layer: - Bound concurrent replans with a slot semaphore: each replan pins a pooled connection for its advisory lock while issuing further store queries from the same pool, so an unbounded recovery storm could turn every connection into a lock holder and deadlock the server. - Make CompleteReplan a real compare-and-swap (base-revision predicate, ErrReplanSupersededV3) and map BeginReplan insert races to a replay instead of a raw unique violation. - Fingerprint start requests (request_digest column): an attempt ID reused with different input is now a 409-style conflict rather than a silent replay, and both replay paths check session liveness so dead sessions surface as retryable terminals. - Pre-delete expired attempt rows on SaveAttempt so a retry during the cleanup window cannot wedge on an unreachable conflict. - Align the in-memory store's semantics with Postgres and add DB-backed planstore tests (SILO_TEST_DATABASE_URL), including a regression test inserting every route-event name against the real CHECK constraint. - Session manager: v3 route-set updates own RemuxDVMode outright so a replan onto an SDR source clears a stale strip mode; replacement reservations survive unrelated legacy stream updates; replacement admission excludes the replaced session explicitly instead of decrementing totals it may no longer be part of; the admission CAS loop is bounded and decider errors are logged. - Map transient store failures to 500s instead of terminal 404/403s; authorize route events via identity-only projections after the rate limiter; keep sanitized diagnostics deterministic. - Merge the server-computed durable plan key into replan exclusions so unreproducible client history cannot re-select the failed route. - Remap tracks only when the effective edition changes (a same-file replan no longer switches audio to a lookalike track) and remap ID-only subtitle selections on edition fallback. - Cache the v3/shadow feature flags for five seconds instead of one settings SELECT per playback request; stop remote transports best-effort when the start call times out; carry dvm/tid claims and the transport-scoped job identity through the legacy audio-change re-mint; index playback_route_events(received_at) for the retention delete; run store maintenance for DB-less deployments too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): reap idle node jobs and gate WebVTT conversion - Add an idle reaper to the transcode node: a job untouched by manifest or segment requests for ten minutes is closed and unregistered. After a v3 replan retires a transport ID, a stale in-flight stream token could resurrect the old job via reconstruct and encode to end-of-file for nobody; jobs waiting on readiness count registration as access and are never reaped mid-wait, and reaping keeps the recipe so a still-valid token reconstructs on the next hit. - Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with 415 before headers are written instead of spawning an ffmpeg command that always fails mid-response, and make the extract-format override fall back to source-driven mapping for bitmap codecs. - Drain error bodies on non-202 node responses so the HTTP transport can reuse connections. - Pin the transcode-dir cleanup separator-boundary semantics with a regression test (a session ID sharing another's prefix must not retain foreign directories). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): close v3 planner policy gaps from review - Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has no rung below 480p/1500kbps, so lower caps were silently exceeded even though the cap is documented as a hard delivery ceiling. - Treat video-only media as audio-compatible instead of forcing an AAC conversion (or an audio_conversion_unsupported terminal) onto a file with no audio stream. Tracks whose codec failed to probe keep the gate. - Only promise a bitmap subtitle sidecar for embedded PGS with an engine that renders embedded bitmap: external/downloaded bitmap and embedded DVD/DVB published artifact URLs that always failed at fetch. They now fall through to burn-in or its terminal. - Accept client_video_transformations_v1 from either client_features or the nested context when validating client-executor transformations, matching the planner's dual-source reads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): probe and execute DV remuxes with one ffmpeg binary The v3 transformation registry probed the configured playback.ffmpeg_path while progressive remux execution resolved the process-global discovery path, so a deployment where only one binary carries dovi_rpu could plan a server_dv7_to_hdr10 route and then fail it at stream time. Resolution now goes through a shared ResolveFFmpegPath (configured path first, discovery fallback — the same rule the transcode pipeline already used), the dovi_rpu probe is cached per binary path, and the stream handler and proxy worker pass their configured path into ServeRemuxWithDVMode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 replan identity and control-plane limits - Seed failure-replan track selections from the durable current plan before overlaying the request: after an alternate-version fallback the normalized request still carries requested-edition track IDs, so a replan omitting unchanged tracks was rejected as a track/file mismatch. - Remap ID-only audio selections across edition changes (parse the ID to an index like the subtitle remap already does) instead of leaving a stale file-bound ID to fail validation. - Release the node planner reservation when a prepared remote transport rolls back after the node accepted the job; repeated failed starts could otherwise pin max-job/bandwidth budgets for the full reservation age. - Size the replan semaphore below the PostgreSQL pool via a store capacity advisor: with max_connections at or below the fixed bound, advisory-lock holders could starve the inner store queries they need to finish. - Contain shadow-planner panics with a recover boundary; it runs on a bare goroutine where an escaped panic kills the process for what is telemetry-only work. Document why the memory store's session lock is deliberately a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): serialize node job teardown against reconstructs - Look up and touch manifest/segment sessions in one critical section so the idle reaper cannot unregister a job between the lookup and its liveness refresh. - Re-validate each reap candidate under the per-session lifecycle lock before closing it: Close removes the output directory, and without the lock it could race a token reconstruct and wipe the segments the fresh ffmpeg is writing. - Take the lifecycle lock in handleStop so a stop racing a RequireReady start's readiness wait blocks until registration and tears the job down, instead of 404ing and orphaning the ffmpeg until the reaper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
28c6ddc237 |
feat(playback): add per-user transcoding controls (#375)
* feat(playback): add per-user transcoding controls * fix(playback): enforce forced video transcode permission * chore: address transcode control review feedback * fix(playback): recheck transcode permission on audio switch --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
c1c110e3d2 |
feat(playback): improve web subtitles and track selection (#362)
* fix(player): keep text subtitles in sync across copy-mode restarts and sparse cue windows - Rebase already-loaded cues in place when streamOriginSeconds changes (copy-mode session restart) instead of leaving them offset by the delta. - Stop inferring end-of-input from where a window's cues stop; only the known media duration marks EOF, so a dialogue gap no longer silently ends prefetching for the rest of playback. - Anchor the first window fetch to the intended start position (resume target or pending seek) while the element still reports currentTime=0, and reset coverage on forward seeks past the fetched window. * fix(playback): align encoded transcode start with the declared segment boundary A mid-segment start (resume, seek restart, audio switch) spawned ffmpeg at the raw seek position while labeling its first segment with the grid number, whose synthetic-manifest start is up to one segment earlier. hls.js aligns the first fragment's content to that declared position, shifting the whole session's timeline late by seek mod segment_duration (0-2s): subtitles trail dialogue by a constant per-session offset and progress/resume positions drift by the same amount. Snap the ffmpeg start position down to the segment boundary for encoded sessions so declared and produced timelines match exactly; the player still seeks to the precise requested position. Copy-mode sessions serve ffmpeg's real manifest and keep the raw seek. * fix(api): forward http.Flusher through response-writer middleware wrappers Streamed subtitle extracts (and any progressive response) flush per chunk via an http.Flusher assertion, but none of the status-capturing middleware wrappers implemented Flush, so the assertion failed and cues sat in Go's response buffer until ffmpeg finished. On large remuxes where a 600s window takes 20s+ to demux, captions appeared only when the whole window completed instead of within the first seconds. Give every wrapper a Flush() (satisfies plain assertions, including chi's Compress) and Unwrap() (satisfies http.ResponseController). The jellycompat image-proxy tag rewriter flushes only in passthrough mode since it buffers JSON bodies for rewriting. Regression test asserts the API chain forwards Flush end to end. * feat(player): PGS subtitles honor size, position, and background settings Port the tvOS/iOS bitmap-cue styling to the web player. libpgs now decodes in worker mode but draws to a hidden source canvas on the main thread; a compositor detects cue regions from the frame's alpha channel and re-places them on a visible overlay canvas per the shared subtitle appearance settings: size scale (with the 0.85 authored-size compensation), vertical position preset (dialogue-band cues only — floating signs keep authored placement, matching the Apple implementation), and the background box. Font family, text color, and outline are baked into the source pixels and remain inapplicable. * fix(player): anchor PGS position presets to the text overlay's reference frame The initial port used silo-apple's 30/1080 bottom margin and video-relative lower-third/top anchors; the web text overlay anchors to a 16:9 reference frame with 7%/18% offsets that extends into the letterbox for wide content. Use the same anchors so PGS dialogue lands exactly where SRT text does. * feat(player): size PGS cues to match the text subtitle line height Replace the authored-size ladder (0.85 × font-size ratio) with per-cue text-line matching: the region detector reports the tallest text line inside each cue, and the compositor scales the cue so one line of bitmap text renders at the same pixel height as the SRT overlay's font at the current preset. Authored size differences between discs no longer leak through; upscaling is capped at 2.5× to keep small bitmaps from going blurry. * feat(subtitles): opt-in windowed PGS extraction to cut mid-file load latency PGS extracts always demuxed the source from byte 0, so starting a large remux mid-file meant minutes before the first bitmap cue. The web player now opts in to windowed extraction (?windowed=1&position=&duration=) and re-points libpgs at a fresh window on seeks and near coverage end; ffmpeg input-side -ss with -copyts keeps absolute source timestamps. Without the explicit opt-in the endpoint behaves byte-identically, so Apple/Android and other single-fetch consumers are unaffected. ASS remains unconditionally non-windowed (its header only exists at offset 0). * feat(playback): cache extracted PGS subtitle tracks Every selection of an embedded PGS track re-ran a full ffmpeg extract that demuxes the entire source file from byte 0 — minutes for a large remux — and responses were Cache-Control: no-store, so repeat selections, re-watches, and multiple viewers all paid full price. Add a disk cache for full-track .sup extracts under <transcode_dir>/subtitle-cache, created lazily: - Keyed by source path hash + subtitle stream ordinal + source mtime+size (encoded in the filename), so a replaced source file implicitly invalidates its entries; the source is stat'ed on every lookup. - Cache miss: ffmpeg stdout is teed to the response (first viewer still streams progressively, first-byte latency unchanged) and into a temp file that is fsynced and atomically renamed into the cache on clean ffmpeg exit. Any error — ffmpeg failure, client disconnect, tee write failure, or the source changing mid-extract — discards the temp file, so a partial entry is never served. - Cache hit: served via http.ServeContent (Range support, Content-Length, Last-Modified from the source mtime) with a revalidatable Cache-Control instead of no-store. - Concurrent requests for the same in-flight track run their own uncached extract (mutex + in-flight key set) rather than blocking on another client's connection. - Scan-on-commit LRU eviction under a 2 GiB cap (recency tracked by bumping entry mtime on hit; atime is unreliable under relatime), plus sweep of crash-orphaned .part temp files. - Windowed PGS requests (?windowed=) bypass the cache in both directions: their output covers only a slice of the track. Both the integrated API handler and the standalone proxy subtitle path share the same playback.SubtitleCache.ServeSUPExtract helper. VTT (already windowed and fast) and ASS (small) stay uncached. No API surface change. AI-use disclosure: implemented with Claude Code. * fix(playback): check Close error returns in subtitle cache paths Silence errcheck on the cache-hit defer and the test's simulated disk-full Close. AI-use disclosure: implemented with Claude Code. * feat(playback): warm PGS cache in background and window from cached track Windowed PGS requests bypassed the cache entirely, so every window fetch re-demuxed the multi-GB original file. Now a windowed miss kicks off a detached background warm (full-track extract into the cache, at most 2 concurrent server-wide, coalesced with client-driven fills), and once the entry exists windowed extracts read the 15-80MB cached .sup instead — seeks and re-enables become near-instant after the first load. Verified empirically that ffmpeg preserves absolute PTS when windowing a sup input. * feat(player): hold playback while PGS subtitle cues load When a PGS track is enabled (or a seek lands outside the fetched window), extraction takes seconds and dialogue could play unsubtitled. The player now pauses until the renderer's parsed data covers the playhead — tracked via libpgs' parsed-timestamp watermark, the exact predicate it renders by — showing a "Loading subtitles…" indicator after 500ms. User play/pause always wins over the hold, a 20s safety timeout prevents stranding playback, and background prefetch never pauses. If future libpgs versions reshape the observed internals the hook degrades to the old play-through behavior. * perf(player): shrink uncached PGS window to 600s Draining a windowed extract from the source reads the full interleaved container across the window (~1GB per 100s of remux on measured hardware); a 3600s window cost ~12GB of reads per fetch while cold. Once the server cache is warm a window costs milliseconds regardless of size, so smaller windows only add trivially cheap re-fetches. * feat(subtitles): burn in PGS/bitmap subtitles for the web player The web player rendered PGS client-side via libpgs, which required extracting the .sup track — a cold ffmpeg demux that took seconds even windowed, since c:s copy still reads the whole interleaved container up to the playhead. Every other server (Plex, Jellyfin default, Emby) burns image subtitles into the video instead, and that is the only path with no per-seek extraction cost. Selecting a bitmap subtitle (PGS/DVD/DVB) now restarts the transcode with subtitle_burn_in at the current aligned position, reusing the same restart machinery as an audio/quality switch so the segment-boundary timeline alignment holds. The server composites the decoded subtitle onto the video with an overlay filter_complex graph (libass's subtitles= filter is text-only); overlay runs at native resolution before any target scaling, and hardware pipelines round-trip through CPU like the text path. Burn-in forces a video encode, so copy-video recipes are upgraded to h264 both client- and server-side. Text subtitles keep the instant, styled, client-side path. The .sup streaming endpoints, cache, and windowing are retained for the Apple client, which renders PGS natively. The now-dead web PGS stack (usePGSSubtitles, pgsPlacement, libpgs dep) is removed. Tradeoff: bitmap subtitles no longer honor web appearance settings (baked into the video) and toggling one restarts the transcode (~1-2s buffering), matching Plex behavior. * fix(player): rebuild text subtitle track when turning off PGS burn-in Selecting an SRT track that turned off bitmap burn-in restarted the transcode, and the client TextTrack built in that same moment was orphaned when the <video> element reloaded, so the subtitles never rendered (and a seek could not recover the dead track). Rebuild the text track once the new stream settles, gated on the burn-in-off transition so quality/audio switches and copy-mode seek restarts keep their subtitles without a needless re-extract. * fix(player): render web subtitles behind the control HUD The text subtitle overlay sat at z-20, above the controls layer (z-10), so cues painted over the bottom HUD and cluttered the control bar. Drop it to z-[5] — above the video, below the controls — so the HUD paints over the cues while it is visible. When controls are hidden the whole controls layer is opacity-0, so cues remain fully visible. * feat(player): lift web subtitles above the control bar while it's visible Rather than hiding bottom-anchored cues behind the HUD, raise them just above the control bar (measured height + a small gap) whenever the bar is visible in the foreground player, then settle them back when it hides. The bar is a roughly fixed pixel height while the cue offset scales with the player, so the bar is measured via ResizeObserver rather than hardcoded. Top-anchored cues never collide with the bottom HUD, so they stay put. z-[5] is retained as a safety so any residual overlap tucks behind the bar. * fix(player): coalesce same-tick transcode restarts into one dispatch Starting playback with a persisted bitmap subtitle fired transcode/start twice within milliseconds: the auto-start effect dispatched before subtitle auto-selection restored the burn-in, whose effect then forced a second start. The first request was already on the wire (no abort signal was passed to fetch), so the server spawned an ffmpeg only to kill it for the second start — visible in production as an ffmpeg exit error ~1ms after every such session start, and slowing time to first frame. Defer the network dispatch by one macrotask so back-to-back restart calls in a tick collapse into a single request carrying the final parameters; state updates stay synchronous. Pass the abort signal into playerFetch so a superseded in-flight request is actually cancelled, and drop any deferred dispatch on unmount so a stray transcode/start cannot land after the session's exit DELETE. * fix(catalog): resolve effective subtitle defaults for movie item details Movie pre-play subtitle selectors were missing the effective defaults (including per-item overrides saved from a previous play) that episodes and watch payloads already resolve. Extract applyToItemDetail/ applyToWatchDetail helpers and apply defaults for movies in buildMediaItemDetail. The SubtitlesPopover now also eagerly loads downloaded subtitles when the saved preference points at one so the closed trigger's Auto summary reflects the override. * feat(player): scale subtitle font size with the rendered video Replace fixed rem font sizes with px values defined at a 720px 16:9 reference height, scaled proportionally with the actually-rendered video (object-fit: contain) so subtitles keep the same relative size as the window grows or shrinks, with a 12px legibility floor. Rename useSubtitlePositionStyle to useSubtitleLayout, returning both the position style and the font scale, and add unit tests for the appearance helpers. * fix(player): satisfy strict index checks in transcode quality test * feat(player): let the pre-play Auto option clear the saved subtitle override A manual in-player subtitle selection persists as an 'always' override for that movie/series, but nothing in the UI could undo it — auto selection stayed pinned to the chosen track forever. Choosing 'Auto' in the pre-play subtitles popover now also deletes the stored preference (movie content ID / episode series ID) and invalidates item details so profile-level auto selection applies again. * feat(player): persist pre-play subtitle selections as the item override Choosing a track (or Off) in the pre-play subtitles popover only lived in component state: it applied to that playback session but vanished on returning to the detail page. Persist it through PUT /subtitle-prefs — the same 'always'/'off' override a manual in-player selection saves — keyed by movie content ID or episode series ID, and invalidate item details so the effective defaults reflect it immediately. * feat(ui): show the saved subtitle override and richer pre-play pill summaries A stored per-item override displayed as 'Auto: <language>', hiding both that an override exists and which track it is. The pre-play subtitle pill now shows the resolved track directly (name with (SDH)/(Forced) markers plus format, skipping markers the name already carries), the matching list row gets the checkmark instead of Auto, and the Auto row reads 'Reset to profile defaults'. Subtitle, audio, edition, and version pill summaries also truncate much later (max-w-44/sm:max-w-64). * fix(player): recover text subtitles from stream reloads and failed window fetches Three failure modes could silently freeze or stop web text subtitles: - A stream restart (seek-triggered transcode restart, quality/audio switch) reloads the <video> element and can orphan the programmatic TextTrack — cuechange stops firing and the last cue freezes on screen. Only the PGS-burn-in-off transition rebuilt the track. Now every settled stream URL change bumps the generation, and the rebuild carries loaded cues (converted back to source time) and window coverage over so it costs no refetch. - The sliding-window fetcher committed windowEnd before the fetch ran, so a failed or hung window counted as covered and was never retried — subtitles silently stopped for up to 10 minutes. Coverage now commits only after the window streams in fully; failures leave the range uncovered and retry after a 5s backoff. - A hung extraction (one fetch in flight at a time, no deadline) blocked every future window for the session. Reads now arm a 30s stall timer that aborts a response which stops delivering chunks; slow-but- progressing streams keep resetting it. Diagnosed from a session where ffmpeg took 69s to stream one subtitle window and a transcode restart landed mid-fetch, freezing the active cue. * fix(playback): keep subtitle selections stable across file changes * fix(web): clarify subtitle labels and positioning * fix(player): hide HUD when pointer leaves * fix(web): tidy subtitle track badges * fix(playback): remap audio tracks across file versions * fix(web): tidy audio track labels * docs(playback): clarify bitmap subtitle appearance * fix(playback): preserve selection state on restart * fix(http): preserve response state across flushes * fix(web): preserve pending quality for burn-in * fix(playback): preserve subtitle inventory identity --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
b7292a9473 |
fix(streaming): stop killing healthy streams at the server WriteTimeout (#361)
The main API server's WriteTimeout (120s) is an absolute deadline from request start, so every streaming response still being written at T+120s was cut mid-body with a clean close. Clients saw multi-GB direct streams truncate every two minutes; the Apple client's cursor-resume reconnect absorbed most kills silently, but one landing during backpressure or a demuxer resync exhausted its retry budget and forced a full player teardown (visible stop + historical audio desync seeding). Fix: internal/httpstream.RollingDeadlineWriter pushes the connection's write deadline forward with progress via http.ResponseController — a response that keeps moving lives indefinitely, a stalled one is still reaped within the window (180s default, SILO_STREAM_WRITE_STALL_TIMEOUT to override). ReadFrom delegates in bounded slices so http.ServeContent keeps its sendfile fast path. Wired into direct play, remux, downloads, the transcode-node proxy, and ebook serving; the server-level 120s guard stays for every other route. The metrics and request-logger response writers now implement Unwrap — without it http.ResponseController cannot traverse to the connection and SetWriteDeadline fails, silently disabling the fix (exactly what the first dev deploy showed). A middleware-chain integration test locks the whole path down against future wrappers missing Unwrap. Validated on dev: 200s/512MB direct and 300s/768MB via CDN sustained range-GETs (previously dying at 120s), zero duration_ms=120000 stream entries since deploy. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
a2ef26bece |
perf: root-cause fixes for endpoints still slow after #292 (NextUp, series badges, resume tail, subtitle fonts) (#350)
* docs(plans): root-cause analysis for endpoints still slow after PR #292 Five endpoint groups stayed slow after the home/Continue Watching/Latest latency work shipped: Resume (110s p95), NextUp (17s p95), Latest (17s), /Items, and the home sections routes. The caps and caches from PR #292 are live in the deployed binary; they bounded how many rows the loops touch but not what each underlying query costs. Documents the four confirmed root causes (4.3M stale completed-with-position progress rows + missing resume index, unbounded next-up anchor scan, per-episode series rollup fanout, two index-starved history/scanner paths) with live EXPLAIN ANALYZE measurements and the fix plan implemented by the follow-up commits. AI-use disclosure: analysis and doc produced with AI (Claude) assistance. * perf(catalog): bound the global next-up anchor scan to recent completions The completed_episodes CTE in buildListNextUpQuery derived per-series anchors from the profile's ENTIRE completed history — DISTINCT ON over 233k rows joined to episodes for the worst bulk-import profile, then a per-series LATERAL that scans every episode of a fully-watched series before yielding nothing. 648 slow executions in a 19h window, 44.7s worst; this drove /Shows/NextUp (17.1s p95) and the next-up injection on the native home sections aggregate. Global queries now derive anchors from the profile's nextUpAnchorMaxRows (500) most recent completed rows — an ordered index walk on idx_uwp_profile_completed, with the hidden-items exclusion and date cutoff applied inside the bounded scan so hidden/old rows never consume the anchor budget. A next-up rail surfaces ~24 series; the 500 most recent completions cover every series that can realistically rank on it. Series-scoped calls (the show-detail tile) keep the unbounded shape: they must anchor on the series' last completed episode no matter how long ago it was watched, and are naturally bounded by one series. Measured on the live worst-case profile with the exact generated SQL: 44.7s worst / ~2.6s avg before; 10ms after (together with the one-time stale-resume-point data repair applied directly to the deployment DB — see docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md). AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat,userstore): aggregate series watch-state rollup in SQL The series Played/UnplayedItemCount badge on list rails (per-library Latest, library browse, search results) and series detail pages was computed by materializing EVERY episode of every series on the page (episodeRepo.ListBySeriesIDs) and then batching per-episode progress+history lookups in 500-id chunks. A 50-series page of an episode-heavy library (Sports) expanded to 32,467 episode rows and ~65 sequential queries — measured 17-18s per /Items/Latest request, and PR #292's cached Latest fast path pays it on every response for series libraries. The same fanout made /Items?searchTerm=... slow whenever the result set was mostly series (Meilisearch itself answers in milliseconds). New optional store capability userstore.SeriesEpisodeRollupStore, implemented by PostgresUserStore as one GROUP BY e.series_id aggregate with semantics identical to the chunked path (episode availability via episode_libraries, hidden-items visibility on progress rows, completed-history fold, in-progress = not watched with position > 0 — verified value-for-value against the old semantics on a real 1,586-episode series). enrichSeriesListUserData and enrichDetailUserData use it when present; SQLite-backed stores and rollup query failures keep the existing chunked path as fallback. catalog.SeasonUserDataFromCounts pins the counts-to-DTO mapping to EpisodeRollupUserData. Measured on the live worst-case profile against the real 50-series Sports Latest page: ~17s of chunked round-trips before, 119ms in one query after. Part of docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md. AI-use disclosure: implemented with AI (Claude) assistance. * perf(catalog): bound superseded-episode completed walk to recent history The Resume / Continue Watching superseded-episode filter loaded a profile's *entire* completed history into memory on every request that contained an in-progress episode: CompletedProgressSnapshots paged user_watch_progress WHERE completed=TRUE with no upper bound. The 2026-07-06 slow-query comparison showed this surviving as a 60-116s Resume tail even after the in-progress index landed live, because the 4.3M zeroed Plex-import rows are still completed=TRUE and were re-walked every load. A completed episode can only supersede an in-progress one it was finished more recently than (the query gates on done_progress.updated_at > ip_progress.updated_at), so only completed rows newer than the oldest in-progress entry can matter. Compute that cutoff in SupersededEpisodeProgressIDs and pass it to CompletedProgressSnapshots, which — since the completed listing is ordered updated_at DESC — stops paging as soon as it crosses the cutoff. Import-heavy profiles whose back-catalogue predates their current in-progress items now stop on the first page instead of paging hundreds of thousands of irrelevant rows. Correctness is unchanged: no relevant superseding row is excluded. * perf(catalog): hard-cap superseded-episode completed walk at 5 pages The updated_at cutoff added in the previous commit bounds the completed walk on the relevance axis, but a very old in-progress entry sitting behind a large volume of newer completions could still page deep. Add a 5-page (2,500-row) hard backstop on top of the cutoff: normal profiles still stop on page one via the cutoff, and only the adversarial tail hits the cap. When it engages the tail of the completed set goes unscanned, so a superseded episode could momentarily survive on Continue Watching — we log a warning when that happens (with profile_id + rows scanned) rather than mis-filter silently, and it self-corrects once the stale in-progress entry ages out of the scanned window. * perf(playback): extract subtitle fonts in a single ffmpeg pass Embedded ASS/SSA font extraction spawned one ffmpeg process per font attachment, each re-opening the (usually CephFS-backed) media file. Anime releases carry 15-47 fonts, so the per-spawn file-open cost dominated and pushed GET /api/v1/stream/{sid}/subtitles/{track}/fonts to a 17-60 s plateau (p95 ~33 s in the live logs). Collapse the N spawns into one ffmpeg invocation that dumps every attachment to a temp dir (-dump_attachment:idx path ... -i file -map 0:t? -c copy), then read the files back. The file is opened once instead of N times, taking p95 from ~30 s to ~1-2 s with no change to output. Safety is preserved. The 32-attachment / 32 MiB caps still apply: attachment size is stat'd before read so an over-limit font never enters memory, and a watchdog polls the dump dir and kills ffmpeg if its on-disk output crosses the cap -- restoring the hard bound the old pipe-per-attachment reader enforced by killing at maxBytes+1, so a container with oversized "font" attachments can't fill the disk. Part of the slow-endpoint follow-up; see slow-query-analysis/subtitle-fonts-extraction-findings.md. * fix(review): report enforced font-byte cap; correct doc subtitle scope Address PR #350 review: - dumpFontAttachments reported the maxSubtitleFontBytes package constant in both over-limit errors instead of the maxBytes argument the caller passed, so the message misstated the enforced bound whenever a different cap was in effect (as the tests use). Interpolate maxBytes in both messages. - The root-cause plan claimed subtitle extraction was 'out of scope' while the branch actually optimizes /subtitles/{track}/fonts. Scope the out-of-scope note to subtitle *track* conversion and record the fonts single-pass work as deliverable 5. |
||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
4e03f4b807 |
fix(playback): last-write-wins progress and DV P7 RPU strip on remux (#334)
* fix(playback): last-write-wins progress and DV P7 RPU strip on remux Progress: UpdateProgress (the live playback-session path) clamped position_seconds to GREATEST(new, old), so a deliberate backward seek could never persist — "rewind and stop" resumed at the stale later position on every client. Position is now last-write-wins, matching the /sync/progress path that was always unconditional. The completed latch and rewatch re-entry semantics are unchanged. Remux: profile 7 Dolby Vision remuxes drop the enhancement-layer track (-map 0:v:0 keeps only the base layer) but previously left the dangling dual-layer RPUs on the BL — broken metadata that a DV-honoring display can mis-render. Remuxes of P7 files now strip DV RPUs via the dovi_rpu bitstream filter, yielding a clean HDR10 stream (the same fallback presentation the Apple client's P7 HDR10 toggle produces). Profile 8 RPUs are kept: the BL is self-contained and DV clients render it. Adds MediaFile.PrimaryDVProfile() and threads the profile through ServeRemux callers; the proxy path (no track metadata in claims) keeps prior behavior. True P7->8.1 DV conversion needs dovi_tool alongside FFmpeg (the dovi_rpu bsf only strips/recompresses); the remux plumbing now carries the DV profile so that can slot in later. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(userdb): apply last-write-wins progress to the SQLite backend too Review follow-up (P2): the LWW change only covered pgstore; the SQLite userdb UpdateProgress kept the MAX clamp, so rewind-and-stop still resumed at the stale later position for sqlite-backed installs. The conflict clause now matches Postgres (position last-write-wins, completed latch and rewatch re-entry unchanged), with a backward-seek regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden LWW progress and DV RPU strip from review - Probe ffmpeg for the dovi_rpu bitstream filter once per process and fall back to a no-strip remux (the pre-existing behavior) when it is missing: on pre-7.1 ffmpeg the unknown filter aborted the process, turning every Dolby Vision profile 7 remux into a hard playback failure. - Skip zero-position heartbeats in persistProgress, mirroring the stop path and the jellycompat report path. Under last-write-wins an early zero heartbeat (e.g. before the client seeks to its resume point) would wipe the stored resume position; GREATEST previously masked this. - Carry the DV profile in stream token claims (dvp, omitempty) so standalone proxy nodes strip profile 7 RPUs the same way integrated mode does. Old tokens decode as 0 and keep prior behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
42602b7896 |
feat(policy): access groups + embedded OPA policy engine with decision audit log (#282)
* docs(policy): add OPA policy engine design spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(deps): add OPA v1.18.2 SDK for the policy engine Pulls github.com/open-policy-agent/opa v1.18.2 (policy engine core for the upcoming internal/policy subsystem) and the transitive upgrades go mod tidy applied (otel 1.44, grpc 1.81.1, prometheus/common 0.67.5). Full build verified. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add OPA engine core, vendor scope policy, and parity suite New internal/policy package (dead code — nothing wires into request paths yet): prepared-query Engine with 25ms eval timeout and fail-closed decode, typed PDP.ResolveViewerScope, go:embed vendor bundle, capabilities lockdown for future admin-authored Rego, and vendor scope.rego reproducing access.Resolver.Resolve (library intersection, disabled-library handling, quality/rating ceilings) with a narrowing-only silo_custom.scope.override extension hook. Parity proven by 1368 dual-execution subtests against the real access.Resolver, including the nil-vs-empty AllowedLibraryIDs battery and quality/rating variation; rank tables are test-pinned to internal/access. Rego unit tests run via opa/v1/tester inside go test. Bench: ~106µs/op per scope decision incl. input marshaling. Also restores the OPA requirement to go.mod (the earlier deps commit ran go mod tidy before any import existed, so tidy dropped it). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed, corrected (quality.allowed raw-file-rank divergence), and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy document store, foundation schema, and compile-check policy_foundation migration: policy_documents (one enabled doc per domain via partial unique index — two enabled docs would define override twice and conflict at eval), immutable policy_document_versions, single-row policy_generation counter, and the partitioned policy_decisions log table (daily range partitions, no FK, denial partial index). PolicyStore: transactional version numbering (FOR UPDATE), activation that verifies compiled_ok and bumps the generation in the same tx, enable/disable with typed ErrDomainAlreadyEnabled, and a delete guard for documents with an active version. CompileCheck sandboxes admin Rego: locked capabilities (no http.send/net.*/opa.runtime), enforced silo_custom.<domain> package path, vendor+stub layering, 2s budget, structured row/col errors. Engine gains NewEngineWithCustom / NewEngineFromStore with WARN-and-skip for invalid custom rows. DB-backed tests verified against a migrated Postgres (concurrent version numbering, atomic generation bumps, activation guards). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (domain constants extracted). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy System lifecycle with hot reload and cross-node invalidation policy.System owns one long-lived Engine and reloads it in place when policy documents change: EventPolicyChanged on the existing ChannelAdmin bus (new cache event constant) plus a 60s generation-poll fallback for Redis-less deployments, with a generation-consistent snapshot read. Vendor compile failure is startup-fatal; store/custom failures degrade to vendor-only and the poll loop heals them; runtime reload failures keep the last known-good engine. NotifyChanged gives the future admin handlers synchronous local reload + cross-node publish. Wiring: constructed in integrated/api modes only, PolicySystem field on api.Dependencies (unused by routes yet), policy.eval_timeout_ms setting (hot-reloaded via configWatcher.OnChange; default 25ms). Verified by a full server boot smoke and DB-backed convergence tests (event + poll paths, degraded boot, last-known-good). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add async decision logging with sampling, retention, and query repo DecisionLogger batch-inserts each node's policy decisions straight to the partitioned policy_decisions table via a non-blocking buffered channel (drop-and-count on overflow — logging never adds latency to or fails a decision). Scope decisions sample 1-in-N (default 50, setting policy.decision_log_scope_sample_rate); denials and eval errors always log; input/result JSON samples only at policy.decision_log_verbosity= verbose. Cursor-paginated DecisionRepository backs the upcoming admin log viewer. Retention via partman (daily partitions) and a PolicyDecisionLogCleanupTask honoring policy.decision_log_retention_days (default 14). PDP emits entries per evaluation; the System owns the logger lifecycle and settings hot-reload. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (removed an unused, unsynchronized PDP setter). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): add admin policy management API and capability endpoint /api/v1/policy/capability (authenticated feature detection) plus the acting-admin /api/v1/admin/policy surface: vendor Rego viewer, document CRUD with the one-enabled-per-domain conflict mapped to 409, immutable version creation (compile-checked; failed versions persist as audit history with structured row/col errors and can never activate), activate/rollback with synchronous reload + cross-node invalidation via System.NotifyChanged, stateless validate, throwaway-bundle simulate (never touches the live engine, never logs decisions), and cursor-paginated decision-log queries. Routes mount only when the policy system is wired, keeping proxy/transcode modes untouched. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (seeded the FK'd test user; replaced an unchecked fmt.Sscanf with strconv). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add /admin/policy workspace with Rego editor, simulate, and decision log New Policy admin page (System nav group): documents list with one-enabled-per-domain conflict handling, CodeMirror 6 Rego editor (hand-rolled StreamLanguage mode) with server compile issues rendered as inline lint diagnostics, explicit Save-version vs Activate flow with confirm, read-only vendor module viewer, simulate panel with seeded example inputs, version history with rollback, and a cursor-paginated decision-log browser. Capability-gated via /policy/capability. Adds the three decision-log settings to Log Retention. First code-editor dependency in web/ (@uiw/react-codemirror + @codemirror/*), decided in the design spec. Implementation drafted by Codex (GPT-5.5) via codex exec; verified here (lint, format:check, tsc --noEmit, vitest policy suites). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for viewer scope resolution policy.ViewerResolver implements the ViewerResolver interface backed by PDP.ResolveViewerScope and replaces access.Resolver at all five construction sites: router viewer middleware, notifications scopes, the reconciler, jellycompat's scope filter, and the ABS resolver (which now accepts a pre-built resolver, preserving its PIN-at-login semantics). PIN/profile-token verification and disabled-library loading are extracted into shared exported helpers used by both implementations, so the legacy resolver stays compiled as the parity reference with identical behavior. The adapter lives in internal/policy (which already depends on internal/access transitively) — direct typed PDP calls, no new import cycle. Sites without a policy system (proxy modes, bare test routers) keep the legacy resolver until the cleanup phase. Verified: full test suite green (jellycompat TestBeginWebOperation* and one playback GPU test are pre-existing failures, confirmed identical on main), 1368-case parity suite, dedicated ViewerResolver parity/PIN/ nil-vs-empty/fail-closed tests, and a full server boot smoke. Implementation drafted by Codex (GPT-5.5) via codex exec; a first-pass reflection-based adapter was rejected and reworked into the typed in-policy adapter; reviewed line-by-line and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for acting-admin and permission gates vendor/permission.rego reproduces the acting-admin rule (admin role + primary-profile-or-none), HasEffectivePermission semantics for marker_edit, and the metadata-curation rule including the subtle admin-past-refused-bypass case that requires the explicitly ASSIGNED permission. Policy-backed middleware in policy_gates.go keeps all Go-side lookups (declared-profile primary check, item->library resolution, the 404-on-unknown-item path) and preserves the legacy status/body taxonomy exactly — proven by dual-execution middleware tests that run every scenario through both implementations and assert byte-equal responses. Permission decisions always log (allowed flag populated); simulate and the capability endpoint gain the permission domain automatically via the domain registry. Router swaps behind single constructor choice points with the legacy gates retained for policy-less wiring. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for download and playback admission decisions vendor/action.rego decides download eligibility (downloads enabled + user allowed), download-transcode eligibility (transcode enabled + user allowed + artifacts available), and playback admission (stream/transcode counts vs limits, zero = unlimited), with a tightening-only silo_custom.action override that can also clamp a quality ceiling (never widen — merged via quality.min). Go keeps everything stateful: config loading, preset-ladder enumeration, and live session counting. Downloads consult an optional ActionDecider (nil = legacy logic) mapped back to the existing sentinel errors and capability response. Playback gains a minimal AdmissionDecider hook at the exact point of the legacy limit comparison: counts snapshot under the session mutex, PDP evaluated OUTSIDE the lock, then revalidated under lock before insert (retry on count drift) — no admission ever decided on stale counts and no eval under the mutex. Deny reasons map to the legacy ErrTooManyStreams / ErrTooManyTranscodes sentinels, pinned by tests. Parity: combination tables driven against the real PresetsFor / ensureTranscodeAllowed / SessionLimits math; full suite green (known pre-existing jellycompat flakes only). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (locking design verified line-by-line) and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): satisfy tsc -b strict return typing in the Rego stream tokenizer The production build (tsc -b) rejects assigning CodeMirror's string | void next() result to string | undefined; tsc --noEmit did not catch it. Restructured the string-literal loop. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): clearer error when a decision is undefined for partial input Vendor policies index required input fields directly, so a hand-written simulate payload missing fields yields an undefined decision. Surface that as 'decision X is undefined for this input (missing required input fields?)' instead of 'empty result' — found while exercising the simulate API against a live server. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(web): set changeOrigin automatically when the API proxy target is remote Remote dev backends sit behind vhost-routing proxies that reject a localhost Host header; local targets keep the existing pass-through behavior. Enables pointing the Vite dev server at a hosted backend via VITE_API_PROXY_TARGET in web/.env.local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): redesign the policy workspace around the decision pipeline The first-pass UI was structurally generic: a five-column document table squeezed beside the editor, three equal-weight action buttons with hidden preconditions, raw version IDs, and jargon copy — nothing taught the model. The page now teaches it: - A pipeline strip states the mental model up front: Silo decides the baseline -> your overrides narrow it -> every decision is logged. Tabs renamed to Overrides / Baseline / Decision Log (ids stay stable for bookmarked URLs). - The document table becomes one card per domain (Library visibility / Admin & permissions / Downloads & playback) with plain-language descriptions, example rules, status pills (Live vN / Draft / Disabled), inline creation, and the enable kill-switch in place. - Selecting an override drills into a full-width editor with a visible lifecycle rail (Draft -> Validated -> Saved -> Live) and one contextual primary action per step; the unedited live source shows no actions until edited. Version comments appear only at the save step. - Simulate is reframed as 'Test before going live' with a human verdict chip (Allowed / Denied — reason / ceiling summary) above the raw JSON; internal generation counters no longer surface. - History uses 'Make live' with plain go-live copy; authors read 'User N'; the baseline tab explains that upgrades never touch overrides. Hand-written redesign (no Codex); verified via vitest, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): present the policy baseline as readable rules, not raw Rego The Baseline tab dumped five Rego modules into read-only editors. It now leads with what the rules actually do: one card per domain with plain-language statements of the shipped behavior and a note on what an override may change, plus content-rating and playback-quality tier ladders parsed live from the lib module sources (so the tiers shown are the ones the server enforces, not a hardcoded copy). The Rego source stays one click away behind a per-module accordion and remains the stated source of truth; unrecognized modules fall back to source-only. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(policy): add access-groups design addendum Groups with permission toggles become the everyday admin surface; the Rego editor is demoted behind policy.editor_enabled (default off). Restriction-only composition: group grants are an upper bound, per-user settings tighten further — same rule as the existing account/profile merge, one layer up. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): add access groups — group defaults with restriction-only composition New access_groups table + users.access_group_id (one group per user, NULL = today's behavior). Group grants are an upper bound composed with the user's own settings by strictest-wins rules — library intersection, MinQuality, AND'd booleans, strictest positive stream/transcode limits, permission-mask intersection, and a requests toggle gating CreateRequest. The merge happens in Go (access.ApplyGroupPolicy / EffectivePolicyForUser) before policy inputs are built, so vendor Rego, the parity suites, and the decision log are untouched; every enforcement surface (viewer scope in both resolvers, permission gates, downloads, playback admission, requests) consumes the effective policy and fails closed on provider errors. Changing a group's quality ceiling bumps its members' access_policy_revision, mirroring the per-user rule. Additive admin API: /admin/access-groups CRUD with member counts; PUT /admin/users/{id} + user DTOs gain access_group_id. Also demotes the Rego editor: policy.editor_enabled (default off, hot-reloaded) drives the capability endpoint's editor_available and 403-gates editor endpoints while the engine and decision logging keep running. Design: docs/superpowers/specs/2026-07-02-access-groups-design.md. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (composition core + fail-closed call-site audit) and verified here. DB-backed group-store tests pending local Postgres recovery. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add Access Groups admin page and gate the policy editor New /admin/access-groups: a card grid summarizing each group (member count + key restrictions), drilling into an editor that reuses the same LibraryAccessSelector and quality presets as the user editor, with toggles for downloads/transcoded-downloads/requests, concurrent-stream and transcode limits, and a permissions mask (all-assignable by default, narrowable to specific permissions). Delete warns how many members fall back to the built-in defaults. Copy states the composition rule up front: a group grants the most a member can do; their own restrictions still apply on top. The user editor gains a Group picker and read-only row; the Policy nav entry is now hidden unless the capability reports the editor enabled. Plumbing (types, hooks, user-editor picker, nav gating) drafted by Codex (GPT-5.5); the Groups page hand-built. Verified: 25 tests across the touched suites, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): seed a Default Group and auto-assign newly created users Adds access_groups.is_default with a partial unique index (one default at most — the profiles is_primary pattern) and seeds a permissive 'Default Group' whose ceiling is a no-op, so assignment never changes anyone's effective access until an admin edits it. The seed is guarded against pre-existing defaults and name collisions; the Down migration only removes the row if it is still untouched. Assignment happens at the single INSERT INTO users choke point (UserRepository.Create): when no explicit group is given, access_group_id is filled by a scalar subquery on the default flag — NULL when no default exists. Every creation path (setup, signup, invites, OAuth, admin create) is covered by construction. Setting a new default via the API atomically clears the previous one in the same transaction. Deleting or unsetting the default is legal: new users then start with no group, which is pre-feature behavior. Implementation drafted by Codex (GPT-5.5); migration guards and the choke-point subquery reviewed line-by-line here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): surface the default access group Cards show a Default badge; the group editor gains a 'Default for new users' toggle (with copy noting existing users are never moved); the delete dialog warns when removing the default that new accounts will start with no group. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): ship the Default Group with house-rule ceilings Seed values per product decision: 5 concurrent streams, 5 transcodes, transcoded downloads off, and a permission mask of marker_edit only (metadata curation excluded). Plain downloads and requests stay on. The Down guard matches the new values so it still only removes an untouched seed row. Only newly created users are affected; existing users are never assigned. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): retire per-user defaults — the Default Group is the sole default policy Removes both legacy 'user defaults' mechanisms now that the seeded Default Group owns new-user policy: - users.max_streams / max_transcodes column defaults drop from 6/2 to 0 (= unrestricted at the user layer), so group ceilings apply to new signups/invites/OAuth users instead of fighting stale per-user numbers. Existing rows keep their stored values — nobody is silently uncapped on upgrade. - The dead defaults.max_playback_quality / defaults.max_profiles settings validation goes away with its only writer (the User Defaults dialog, removed on the web side). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): replace the User Defaults dialog with group-governed creation The Users page's 'User Defaults' dialog (defaults.* server settings) duplicated what access groups now do properly, and its values were only ever form prefill — no backend path applied them. The button now links to Access Groups, and the create-user form seeds unrestricted user-layer values (0 streams/transcodes, any quality, downloads allowed) so the member's group governs; per-user fields remain for tightening individual users. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): migrate existing non-admin users into the Default Group Existing users join the seeded Default Group on upgrade so one policy source governs the whole instance. Their per-user limits still holding the retired 6/2 column defaults are normalized to 0 in the same statement so the group's ceilings actually apply; deliberately customized values are preserved. Admin accounts stay ungrouped — scope/action decisions are role-blind, so grouping an admin would cap the server owner on upgrade. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): keep admins out of the Default Group and treat group moves as policy changes New-user creation now mirrors the migration's admin exclusion: the default access group is only auto-assigned to non-admin roles, so a fresh server owner no longer inherits the starter group's transcode denial and stream caps. Changing a user's access group now bumps access_policy_revision (the group carries permissions, quality, and limits, exactly like the per-user fields that already bump it) and triggers admin session revocation when the group actually changes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): enforce marker_edit through the PDP on marker write routes The Rego permission policy owned marker_edit but no Go caller ever consulted it: PUT/DELETE /markers went through a handler-local check that short-circuited admins and read only the user's own permissions, so group permission masks and custom policy overrides were ignored. Marker writes are now gated by router middleware like the other permission surfaces: a PDP-backed RequireMarkerEdit that evaluates the group-merged effective permissions (plus the legacy variant for proxy/test wiring without a policy system). The handler-local check and its user loader are gone. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert device/quality policy facts and honor the quality ceiling The download_transcode action check hard-coded an empty device ID and never asserted the requested quality, and no caller consumed ActionDecision.QualityCeiling — custom download policies keyed on those inputs were silently ineffective. Resolve now threads the request's device ID and requested quality into the action input, and a returned quality ceiling downscales the prepared transcode target (the ceiling applies to what is served, matching the serve-time rule in serveDownloadBytes). FileQuality and the content-rating pair stay intentionally empty for downloads — documented on downloadActionInput: those ceilings are enforced against the served artifact by the scope-derived access filter, and asserting the source's quality would wrongly deny capped transcodes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(access): align the default-group seed assertions with the migration The DB test still asserted the earlier no-op seed (transcode allowed, unlimited streams/transcodes, null permissions); the shipped migration seeds transcode denied, 5/5 limits, and marker_edit-only permissions, so the test failed on any database with the migration applied. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): lock the Rego sandbox by builtin purity and bound compile work Exclude every nondeterministic builtin from the admin sandbox instead of denylisting names, so OPA upgrades cannot silently expose impure builtins while pure helpers like net.cidr_contains stay usable. Apply the same capabilities to the runtime engine, cap concurrent compile checks, and reject oversized sources before they reach the uncancelable compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): require literal booleans in vendor override and input checks Bare object.get truthiness treated any non-false value as satisfied, so a malformed override 'allowed' value could fail to tighten a base grant and hand-crafted simulate input could flip flag predicates. Compare against literal true so anything else denies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): surface decision log cleanup failures to the task manager CleanupDecisionLogsOnce now returns the first error alongside the deleted count so a broken partition manager or DB outage marks the scheduled task failed instead of reporting 100% success while policy_decisions grows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): log admission decider errors before failing closed A policy-evaluation failure was silently mapped to the too-many-streams denial, making an engine outage indistinguishable from a real limit hit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): nil-guard the downloads user and restore the ABS legacy resolver effectiveDownloadUser dereferenced policy state before its nil-user check, and the ABS handler lost viewer-scoped filtering entirely when the policy system was unavailable because no legacy access.NewResolver fallback was wired like the other resolver paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): address admin policy review feedback - invalidate the version query by version_number, the key usePolicyVersion actually caches under - keep the goPrevious cursor-stack updater pure (Strict Mode double-invoke) - make version history rows keyboard-selectable like the document list - clamp download_transcode_allowed when downloads are disabled so groups cannot save a contradictory record Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): cap policy endpoint request bodies at 1 MiB The policy write endpoints (create document/version, set enabled, validate, simulate) decoded JSON bodies without a size limit, so an oversized payload buffered fully in memory before CompileCheck's 256 KiB source cap could reject it. Route all five through a shared decodePolicyRequest helper that wraps the body in http.MaxBytesReader and returns 413 with the repo's standard too_large error shape. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(access): forbid deleting or demoting the default access group Deleting the default group (or unsetting its is_default flag) left the server with no default: new non-admin users were then created ungrouped with max_streams/max_transcodes of 0 — unlimited — because the legacy per-user column defaults were retired in favor of the group's ceilings. The store now rejects both operations with ErrDefaultGroupRequired (mapped to 409); promoting another group remains the supported way to move the default, and atomically clears the previous one. The admin UI disables the delete button and the default toggle on the default group and explains the promote-another-group flow. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(web): keep unsaved policy drafts when a newer version activates elsewhere The editor state was keyed on the active version's id/sha, so a background refetch after another admin (or another tab) activated a version remounted the editor and silently discarded the dirty draft. PolicyEditorPanel now pins the seed it is editing against and only adopts an incoming seed when nothing can be lost: the editor is clean, the draft already equals the incoming source (the same-admin activate flow), or the selection moved to a different document. Otherwise the pinned editor stays mounted and an inline notice offers an explicit "Load live version" action. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(policy): fail reloads on invalid custom sources and surface degraded/apply state A stored custom source that stops compiling used to be silently skipped on reload: the bundle widened to vendor-only for that domain while the generation reported fully applied. Reload is now strict — a bad enabled source fails the reload and the last known-good engine keeps serving. Boot keeps its vendor fallback for availability, but skips are recorded on the engine and exposed (with store-outage reasons) through System.DegradedState and additive degraded fields on GET /policy/capability. Activate/SetEnabled re-run CompileCheck instead of trusting the stored compiled_ok flag. Mutation endpoints also no longer conflate persistence with live apply: activation/enable responses carry additive applied/failed_step/ loaded_generation fields and return 202 when the store change persisted but the local reload failed. Addresses review findings C1, C2, and the degraded-signal gap (6.1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): type deny reasons across the contract and enforce profile_verified Deny handling used to branch on exact free-text reason strings in three Go consumers, and playback reported ANY unrecognized reason — including custom override free text and engine failures — as a stream-limit error. Decisions now carry a stable reason_code (custom overrides always get custom_denial); downloads, the metadata-curation gate, and playback admission switch on codes, with a new ErrPlaybackNotAllowed -> 403 playback_not_allowed mapping for non-limit denials. Rego tests pin every vendor code. The scope contract's tighten-only profile_verified output was also emitted but never consumed; a policy revocation now surfaces as ErrProfileUnverified (403 profile_unverified) instead of silently proceeding. Addresses review findings 6.2 and C4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): close the dual-library disabled-scope bypass in direct item authorization EnsureAccessible, EnsureAccessibleIDs, and FilterAccessibleContentIDs gated library access with allow/deny predicates over a single joined media_item_libraries row, so an item linked to BOTH a passing library and a disabled one satisfied the disabled check via the passing row — a direct-ID bypass of disabled-library scope on the detail, media-file, playback, and download paths. All library access predicates now share one helper (libraryAccessConditions) emitting independent EXISTS / NOT EXISTS subqueries, the semantics GetByIDsWithAccess already used, including the orphan-item membership guard for disabled-only scopes. SQL-shape tests pin every builder and a DB-gated regression test covers the dual-library item end to end. Addresses review finding C3 (plus the same shape in buildFilterAccessibleContentIDsSQL, which the review did not flag). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): serialize quota check and row creation under a per-user advisory lock The concurrent-download quota was check-then-insert with nothing serializing the pair: parallel creates could all observe free quota before any row existed, bypassing the cap and stacking artifact encode jobs. All four check->insert spans (ephemeral original, artifact-backed, series batch, managed batch) now run inside Repository.WithUserQuotaLock — a pg_advisory_xact_lock keyed by user, so the serialization holds across nodes. The artifact path keeps the limiter-before-Ensure ordering (a rejected request must not leave an encode job behind) by holding the lock across Ensure. Managed-entry replacement stays quota-exempt and lock-free. A DB-gated barrier test races 8 creates against a cap of 1. Addresses review finding C5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert served quality at create time for original and remux downloads Direct-original and remux downloads serve the source resolution unchanged, but create-time policy checks left file_quality empty — an over-ceiling source registered a row serveDownloadBytes could never satisfy. Resolve now runs a final download action check with FileQuality populated on those two paths (capped transcodes keep the ceiling-on-artifact behavior), a custom override ceiling below the served resolution denies, and quality_ceiling_exceeded maps to ErrQualityUnavailable. The ActionInput contract now documents exactly when file_quality and the rating facts are supplied so custom policy authors are not misled. Addresses review finding C6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): guard activation against slow overrides and make eval timeouts observable A custom scope override that exceeds the 25ms eval budget compiled fine, activated fine, and then converted to 500s on every authenticated request — server-wide lockout authored in the admin editor. Activation and enable now run GuardEvalCost: the candidate source is evaluated on a throwaway engine against a canned representative input under the live budget, and a source that cannot complete is rejected 422 with ErrPolicySlowEval before it goes live. Runtime timeouts keep failing closed but now carry a distinct ErrPolicyEvalTimeout sentinel, an Error log, and a per-engine counter exposed as eval_timeouts on GET /policy/capability so intermittent near-budget policies are attributable. Addresses review finding C7. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt remediation files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
604bbf1a0f |
feat(playback): unified restart-resilient playback (native + jellycompat) (#174)
* feat(playback): unified restart-resilient playback via shared TranscodeManager Make direct, remux, and native HLS transcode sessions survive a server restart through one shared flow instead of per-method paths. A missing in-memory session becomes a reconstruct trigger, not a 404: the server rebuilds the session from a tiny durable recipe card plus the position the client re-supplies on its next request. - internal/playback/transcode_manager.go: shared TranscodeManager owning the transcodes map, recipe-card lifecycle, reconstruct single-flight + concurrency cap, LoadOrReconstructSession front door, ReconstructSession / ReconstructTranscode, and orphan cleanup. ~90% is logic moved out of the native handler (no behavior change), not new surface. - internal/playback/recipecard.go + recipecard_postgres.go: RecipeCard with a PlayMethod discriminator (direct/remux/transcode; empty decodes as transcode for back-compat) behind a swappable, nil-safe RecipeStore interface backed by transcode_recipes. - internal/playback/session.go: RegisterReconstructed inserts a rebuilt Session under its existing id (no UUID mint, no limit double-count, race-yielding). - internal/playback/transcode.go: CloseProcess keeps the output dir so a reconstruct winner keeps serving; Close removes it. - internal/api/handlers: drain the transcode lifecycle into the manager; wire reconstruct into the stream/segment serve paths; re-bind ownership to the live caller (refuse userID==0/mismatch); card-aware orphan cleanup. - migrations: add transcode_recipes (expires_at TTL, filter-on-read, indexed). Ownership stays two-factor: an authenticated caller AND a session.UserID that matches; the card stores no secrets and identity is re-resolved per request. Tests: recipe-card round-trip/legacy-decode/disabled-noop, RegisterReconstructed insert/race/concurrency, close-vs-close-process dir semantics, the LoadOrReconstructSession status matrix, and the reconstruct concurrency cap. AI-use: implemented with AI assistance (design, implementation, adversarial review). * feat(jellycompat): reconstruct transcodes across restart via shared manager Bring Jellyfin (jellycompat) HLS playback onto the same restart-resilient flow as the native path. Previously jellycompat owned a separate PlaybackHandler with a private transcodes map and a duplicated transcode lifecycle that never grew the reconstruct half, so an in-flight Jellyfin transcode died on restart and the next segment request 404'd. - Embed the shared playback.TranscodeManager and delete the duplicate lifecycle, so jellycompat gets reconstruct, the concurrency cap, the node-affinity rule, and the card lifecycle for free. - internal/jellycompat/playback_sessions_postgres.go: DurableCompatPlaybackStore, a write-through cache over jellycompat_playback_sessions behind the new CompatPlaybackStore interface (nil pool degrades to cache-only). This persists the load-bearing PlaySessionId -> UpstreamSessionID mapping (plus media sources, route item id, seek) so it survives a restart instead of vanishing with the map. - Write a recipe card on compat transcode start keyed by the upstream session id, using the native StreamAppUserID so the ownership re-bind matches; reconstruct the upstream session and the transcode seeked to the requested seg_NNNNN. - migrations: add jellycompat_playback_sessions (expires_at TTL + compat_token index, full PlaybackSession in data JSONB). Auth is mapped to the native user id before reconstruct so the same two-factor ownership check and userID==0/mismatch refusal apply unchanged. Tests: DB-gated (SILO_TEST_DATABASE_URL) durable-store round-trip proving a session written by one instance reloads in a fresh one (the restart case), plus a nil-pool cache-only path; existing handler tests updated to the manager. AI-use: implemented with AI assistance (design, implementation, adversarial review). * docs(playback): consolidate unified playback reconstruction design Replace the three overlapping playback docs (the native Postgres restart-resilience spec, the jellycompat plan, and the unification spec) with a single self-contained design at docs/superpowers/specs/unified-playback-reconstruct.md. The doc leads with the unified design — the one-idea reconstruct model, a strong visual flow of a restart mid-playback, the shared TranscodeManager + recipe card, the two swappable durable stores, security, the concurrency cap and node-affinity constraint, preconditions, and verification. The design history and rationale (reconstruct-not-rehydrate, phased delivery, Redis-vs-Postgres, token-as- descriptor, failure analysis) move to an appendix. It references no other md file. AI-use: written with AI assistance. * fix(playback): address review on restart-resilient playback Four fixes from PR review of the unified reconstruction work: - Rewrite the recipe card on audio-track change. HandleChangeAudioTrack only updated the in-memory session/transcode, so after a restart reconstruct resumed with the stale AudioTrackIndex/TranscodeAudio (and stale play method) from the start-time card. Re-save the card (direct/remux/transcode) with the switched state, mirroring the start-card pattern. - Guard nil TranscodeManager in LoadOrReconstructSession and ReconstructSession. StreamHandler.TM is documented optional (tests/minimal setups); a missing session previously panicked in recipeEnabled instead of returning SessionMissing. ReconstructTranscode already guarded nil; make the two siblings consistent. - Reject direct/remux cards in doReconstructTranscode before spawning ffmpeg, so a non-transcode card id can never enter the HLS reconstruction path. - Log a non-success status from the remote transcode-node DELETE in CloseTranscodeSession; a 401/404/500 was previously silent. AI-use: implemented with AI assistance. * fix(playback): harden restart-resilient compat sessions * feat(playback): token-carried reconstruction across restarts Build on the shared TranscodeManager (introduced earlier in this branch) so a playback session survives an API-server or transcode-node restart without the client re-negotiating, and retire the Postgres transcode_recipes store in favor of a recipe carried inside the signed stream token. - RecipeCard encodes the byte-affecting encode parameters and rides inside the stream token; LoadOrReconstructSession rebuilds the in-memory Session (and, for integrated transcodes, the ffmpeg process) on a cold miss, single-flighted per session and paced by a spawn semaphore. Removes recipecard_postgres.go and the 20260617233705_add_transcode_recipes migration. - transcodenode reconstructs a lost ffmpeg node-side from the forwarded token. - TR-lease: proxy/streamauth enforce a revocation deny-marker on every served segment, with a 500ms Redis timeout, a bounded per-session "allowed" cache (3s TTL, expiry-first graceful eviction), and a degraded-fail-open counter. Review hardening folded in: - Manifest/segment handlers do the in-memory session lookup first and only verify the stream token on a reconstruct miss (token HMAC was per-segment). - Copy-mode reconstruct never applies the encoded-only seg*dur seek, at spawn time or via the recovery path: RestartSeekTarget reports "unresolved" for a copy session whose manifest cannot yet map the segment, so the client retries instead of seeking to a fabricated source time. - Crash teardown is a compare-and-delete (CloseTranscodeSessionIf returns whether it matched); the crash closure tears down the playback session only when it matched, so a session reconstructed under the same id is not killed. - Reconstruct enforces the same per-user stream/transcode caps as a fresh start (RegisterReconstructedWithLimits), closing a token-replay slot bypass. AI-use disclosure: implemented with AI assistance (Claude Code), including a two-round multi-agent adversarial review whose findings drove the hardening. * feat(jellycompat): node-side transcode reconstruct via shared recipe store Make Jellyfin-compat playback sessions survive a server or transcode-node restart by reusing the shared TranscodeManager reconstruct path and a durable recipe store, on top of the durable compat session store added earlier in this branch. - Node-side transcode reconstruct goes through the shared recipe store; the recipe is persisted to the control-plane store (Redis) when a dedicated transcode node is used so the node can rebuild ffmpeg after its own restart. - Adopt the shared manager's API (3-arg OnFFmpegCrash carrying the dead session, guarded CloseTranscodeSessionIf, RegisterReconstructedWithLimits). Review hardening folded in: - Recipe lifecycle: noderecipe.Store gains Delete, called on deliberate teardown (stop, method-switch discard, node stop/force-reload) so a stopped session cannot be resurrected by a buffered request after a node restart; crash paths intentionally keep the recipe so a resume can reconstruct. - Crash closure tears down the upstream session only when the guarded transcode close matched, so a reconstructed successor is never left orphaned. - Copy-mode segment recovery surfaces a retryable not-found instead of a wrong-position restart, matching the native and node paths. - Durable Update is now a SELECT ... FOR UPDATE transaction, removing the lost-update clobber that could silently drop a transcode recipe. - Empty-token route resolution no longer falls back to an unbounded full-table scan; DB expiry filters bind the injected clock; the redundant re-Get is gone. AI-use disclosure: implemented with AI assistance (Claude Code), including a two-round multi-agent adversarial review whose findings drove the hardening. * docs(playback): consolidate restart-resilient playback design Replace the superpowers spec with a single architecture record describing the token-carried recipe card, the shared TranscodeManager reconstruct path for direct/remux/transcode, the jellycompat durable session + node recipe store, and the revocation-lease model with its fail-open tradeoff. AI-use disclosure: written with AI assistance (Claude Code). * docs(playback): correct jellycompat node-recipe rationale in comments The noderecipe / transcode-node / jellycompat comments justified the Redis recipe store with "a Jellyfin client cannot round-trip a token". The real reason: the node-hop token is server-minted and could carry the recipe, but the recipe is mutated in place under a stable session id (a /Sessions/Playing/Progress audio switch restarts ffmpeg without re-minting the client's token) and a third-party Jellyfin client cannot be driven to refresh a stale token, so the node must reconstruct from a server-authoritative, node-reachable store. Aligns the comments with docs/architecture/restart-resilient-playback.md §10. Comment-only; no behavior change. * refactor(playback): remove deny-lease revocation, defer to future PR The deny-lease stream-revocation mechanism (the internal/streamauth package, its silo:streamauth:<sid> Redis markers, the proxy Allowed() enforcement, and the admin Stop/Terminate deny write) only ever enforced on the offload-proxy topology and was a silent no-op on the integrated single box and the dedicated transcode node. Rather than ship a partial revocation feature that looks complete but isn't, remove it wholesale and defer a uniform cross-topology revocation design to a dedicated follow-up. Removed: internal/streamauth (package + tests); the LeaseDenier field, StreamLeaseDenier interface, and denyStreamLease helper in playback.go; the admin deny write; the router/main wiring; and the proxy verifyToken Allowed() gate. The unified-reconstruct core (recipe-token, LoadOrReconstructSession) is orthogonal and untouched. Known limitation (now on every topology): admin Terminate and user Stop tear down the live in-memory session and ffmpeg producer, but a still-valid stream token can reconstruct the session until its 24h TTL expires. No node-side byte-withholding ships in this PR. docs/architecture/restart-resilient-playback.md is updated to mark the revocation/deny-lease sections as deferred and to drop the overstated "instant revocation on admin kill" claim. * fix(playback): allow zero-caller bearer on transcode reconstruct The authless HLS transcode delivery routes (master.m3u8 / segment) treat the session UUID as the bearer credential, so a real request carries requestUserID == 0. The live serve path already allows this, but ReconstructSession hard-rejected a zero caller, so a request that worked before a restart became SessionMissing -> 404 after the in-memory session was gone, breaking the restart resilience these routes advertise. Match the live-path contract in LoadOrReconstructSession: allow a zero caller (UUID-as-bearer) and refuse only a non-zero caller that mismatches the card owner. The reconstructed session is bound to card.UserID either way. Adds TestReconstructSession_Ownership covering both cases. * fix(jellycompat): re-persist recipe on local audio switch A Jellyfin client switching audio on an integrated/local compat transcode restarted live ffmpeg with the new track but did not re-persist PlaybackSession.Recipe. The remote branch already re-persists via startRemoteTranscode -> persistTranscodeRecipe. After a central restart, reconstruct rebuilt ffmpeg from the stale Recipe.AudioTrackIndex, so the integrated session resumed on the original audio track. Persist the updated recipe (best-effort) after a successful Restart in the local branch, mirroring the remote branch, so the durable Recipe.AudioTrackIndex tracks live ffmpeg. Adds a regression test. * fix(playback): strip stream token from proxied transcode-node URL proxyToTranscodeNode appended the client's raw query string to the internal transcode-node URL and logged that URL on transport failure. When a remote transcode runs without a separate proxy node, that query carries ?st=<signed JWT> — a 24h bearer reconstruction descriptor exposing the media path and recipe claims — placing the token into internal requests and error logs. Strip the "st" param before building targetURL, preserving any other query params. The token is neither forwarded to the node nor present in the logged URL. Header-forwarding of the token (so the node can reconstruct) is a separate follow-up (#6). * fix(playback): fail open on transient limit-provider error in reconstruct During the reconstruct wave right after a restart (Postgres under peak load), a transient limit-provider DB error was collapsed into a hard 404, permanently stopping playback for a user within their limits. limitsForUser wrapped any provider error, RegisterReconstructedWithLimits propagated it, and ReconstructSession mapped every error to SessionMissing -> 404 - indistinguishable from a genuine over-cap rejection. Distinguish the two: tag provider errors with a new ErrLimitProviderUnavailable sentinel and, during reconstruct, fail OPEN on a provider error (admit via RegisterReconstructed + log a degraded warning) rather than refuse - mirroring the reliability-first fail-open-on-dependency-error philosophy. A genuine ErrTooManyStreams / ErrTooManyTranscodes over-cap still refuses. Adds tests for both the fail-open and still-refused paths. * fix(playback): forward stream token to transcode node as header The dedicated transcode node's reconstruct path reads the stream token only from the X-Silo-Stream-Token header, but proxyToTranscodeNode forwarded only the node-API bearer token (and #5 now strips st from the URL). So when the central API proxied to the node and the node self-restarted, it could not reconstruct from the recipe-complete native token -> 404. Capture st before stripping it from the URL, verify it at the API boundary (streamtoken.Verify + SessionID match, mirroring the node's own check), and forward it as X-Silo-Stream-Token. Best-effort: a missing/invalid token never blocks the live proxy, and the token is still kept out of the forwarded URL and logs. * fix(playback): restart node ffmpeg on native remote audio switch A native audio-track switch on an offloaded/remote transcode was a no-op at the node yet returned 200 with a fresh URL: HandleChangeAudioTrack restarted ffmpeg only when the API owned a LOCAL TranscodeSession, so for an offloaded transcode the node kept serving the OLD audio (the node consults the token only on a session miss). The replacement URL was also minted from identity- only claims, so a later node restart 404'd. For the offloaded transcode case (detected via session.TranscodeNodeURL), POST a fresh /transcode/start to the node with the new AudioTrackIndex (handleStart tears down and restarts ffmpeg) and mint the replacement proxy URL from a full RecipeCard so reconstruct survives a node restart. The encode recipe is derived from the durable session target fields plus the file, mirroring HandleStartTranscode. A concrete SegmentDuration (playback.DefaultSegmentDuration) is embedded rather than 0: the node's token completeness gate treats SegmentDuration<=0 as incomplete and falls back to a recipe store the native path never populates, which would 404 on a node restart - the exact resilience this path provides. A failed node POST now surfaces 502 rather than a false 200. Remux and non-offloaded (local) transcode paths keep their prior identity-claim URLs unchanged. Known limitation: Session does not persist the original SegmentDuration or SubtitleTrackIndex/SubtitleBurnIn, so a remote audio switch resets subtitle selection to none and assumes the default segment length; a client that started with a non-default segment length will resegment on switch. Making that state durable on the session is a follow-up. * docs(playback): scrub stale deny-lease/revalidator comments The deny-lease revocation mechanism and its "central revalidator" were removed earlier in this branch, but four comments still described them as live (transcode_manager.go, noderecipe/store.go, streamtoken/token.go, proxy/server.go). Reword them to match the shipped behavior: ownership claims are re-resolved at reconstruct, the noderecipe store shares Redis only with the node-session tracker, and a sub-TTL hard cut depends on a node-side revocation mechanism that is deferred to a future PR. * fix(jellycompat): surface durable playback-session write failures DurableCompatPlaybackStore.Update applied the in-memory mutation and then swallowed every Postgres commit-failure path, returning nil. Callers that promise restart resilience (persistTranscodeRecipe's recipe write, the upstream-session binds in streams.go) were told the session was durably persisted when only the cache held it, so a transient DB hiccup could leave the next restart reloading a stale row (wrong audio track) or 404ing. updateDB now returns the genuine DB round-trip error (begin/query/unmarshal/ marshal/exec/commit); Update propagates it while still applying the in-memory mutation so live state stays correct. A nil pool and a genuinely absent/expired row remain best-effort (return nil) — only real infrastructure failures propagate, so existing rollback paths fire exactly when durability is lost. Part of #174 * fix(playback): re-inject stream token into proxied transcode manifests API-proxied remote transcode manifests dropped the reconstruct token from their segment URLs, so playback died after a node or API restart. When a remote transcode has no separate proxy node, the client loads its manifest via the API-local path; proxyToTranscodeNode strips the signed token ("st") from the forwarded URL (keeping it off node URLs and logs, forwarded only as the X-Silo-Stream-Token header), and the node builds relative segment URIs from that token-less query. The segment URLs the client received carried no token, and the proxy only re-attached the header when an incoming segment request already had "st" — which it never did — so a restart made those segments non-reconstructable and they 404'd. proxyToTranscodeNode now rewrites the manifest body at the boundary: every segment and #EXT-X-MAP init URI gets the client-facing, API-verified token re-appended (new playback.AppendManifestQueryParam helper), so the client's later segment fetches carry "st" again and reconstruct after a restart. The token still never reaches the node URL or its logs. Only 200 .m3u8 responses are rewritten (Content-Length corrected); segments stream through untouched. Part of #174 * fix(playback): preserve subtitle/cadence recipe across offloaded audio switch Switching audio on a remote (offloaded) transcode with burned-in subtitles silently dropped them, and reset a non-default segment cadence. The offloaded audio-switch restart rebuilt the node start request from Session state, but Session/SessionStreamState retained no subtitle or segment-duration state (only the live local ts.Opts() and the RecipeCard did), so the branch hard-coded SubtitleTrackIndex:-1, SubtitleBurnIn:false and SegmentDuration:Default — signing that altered recipe into the replacement stream token. An audio switch then changed bytes beyond audio selection, and any later reconstruct kept the wrong no-subtitle/wrong-cadence recipe. Persist the byte-affecting recipe on the session: SubtitleTrackIndex, SubtitleBurnIn and SegmentDuration are added to Session/SessionStreamState, populated at start (finalizeTranscodeStart) and on post-restart reconstruct (ReconstructSession from the card), carried forward on every audio-switch state update, and read back when rebuilding the offloaded node request and its recipe card. The restart now reproduces the exact live stream. Also resolves the M-4b non-default segment_duration reset. Part of #174 * fix(playback): serialize transcode spawn paths with a per-session lock Reconstruct was single-flighted only against other reconstructs, so a restart-driven segment reconstruct racing a quality/seek/audio fresh start could spawn two ffmpeg processes writing the same output directory at once — segment corruption, partial-write closes, orphaned processes, and skewed active-job accounting. The atomic register-after-spawn (GetOrRegister / the reconstruct compare-on-register) prevented a map leak but not the concurrent disk writers, because the losing path had already spawned. The dedicated transcode node had the same split between handleStart and spawnReconstruct. Add a refcounted per-session lifecycle lock to both TranscodeManager and the node Server, held across "check existing -> spawn -> register": - reconstruct (doReconstructTranscode / spawnReconstruct) re-checks under the lock and yields to any live session instead of spawning a duplicate; - the native and jellycompat fresh-start paths take the lock around their spawn+register (the native path also closes any session a reconstruct rebuilt in the meantime so its fresh ffmpeg is the sole writer); - the node handleStart holds it across teardown+spawn+register. The refcount drops the map entry once no path holds/waits, keeping it bounded. GetOrRegisterTranscodeSession is removed — the lock supersedes it and keeping a register-after-spawn primitive would invite reintroducing the race. Part of #174 * fix(playback): serialize restart re-spawn under the session lifecycle lock TranscodeSession.Restart() releases s.mu across cancel -> wait-for-done -> re-exec and spawns ffmpeg into opts.OutputDir without holding the per-session lifecycle lock. LockSessionLifecycle's contract (fresh start, restart, reconstruct) requires restart to hold it too, but all five callers invoked Restart unlocked: native audio-switch and segment-recovery, compat audio-switch and segment-recovery, and the transcode-node segment-recovery. A restart racing another restart (audio-switch vs segment-recovery) or a fresh-start/reconstruct could land two ffmpeg processes writing the same segment directory -- mixed timelines, init.mp4/segment mismatch, and an orphaned-but-still-writing ffmpeg -- the exact concurrent-writer corruption the lifecycle lock exists to prevent. Add RestartSessionLocked (TranscodeManager) and restartSessionLocked (node Server) that hold LockSessionLifecycle only across the cancel->respawn transition, re-check that the handle is still the live mapped session under the lock, and return ErrSessionSuperseded rather than re-spawning a stale handle. Route all five call sites through them. The lock is released before callers wait on segments so recovery latency is unchanged. Tests: gating (restart blocks until the lifecycle lock frees, then spawns), concurrent-restart serialization, and superseded re-check on both the manager (covers native + compat) and node lock owners. --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
a4402d4621 |
fix(playback): stop seek-restart churn and premature paused-session reaping (#279)
* fix(playback): stop seek-restart churn and premature paused-session reaping Two latent defects (present since the initial migration) explain the cross-client freezes on resume-after-pause, intro-skip, and manual seek reported in #243: 1. Dueling transcode restarts. TranscodeSession.Restart had no concurrency guard, and SegmentRecoveryDecision told concurrent segment requests to restart again while a restart was already in flight (the restart window runs with running=false, so requests hit the restart-happy transcode_not_running branch before the Restarting check). Pipelined HLS segment requests therefore spawned restarts that kept killing the ffmpeg the player was waiting on — a 30s stall and another restart, surfacing as buffer-then-freeze. Restart is now single-flight and the recovery decision waits out an in-flight restart (WaitForSegment already polls politely through one). 2. Paused sessions were reaped after 2 minutes, killing the transcode with no revival path — every resume request then 404s. Clients that stop progress reporting while paused (tvOS; backgrounded web tabs) froze on play after a >5 minute pause. Paused grace is now 30 minutes in both the in-memory session manager and the DB session cleaner. Also unifies post-restart re-arming: the throttler and exit monitor are re-armed via a restart hook installed at session creation and fired by Restart itself, so the jellycompat seek path (which previously re-armed nothing) and the audio-switch path (which re-armed only the throttler) now behave identically to web segment recovery. Follow-up (not in this PR): lazily revive a reaped session's transcode on resume instead of 404ing, so even multi-hour pauses recover. Fixes #243 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(playback): scope the restart-hook comment to handler-created sessions The comment above SetRestartHook claimed the hook covers restarts from 'jellycompat seek', but jellycompat creates its own TranscodeSessions via playback.StartTranscode in a separate registry and calls Restart directly; those sessions never get the hook and never had throttler/exit-monitor wiring to re-arm. Reword the comment (and the matching test comment) to scope the guarantee to handler-created sessions. Also fix TestRestartInvokesRestartHook to resolve `true` via PATH instead of hardcoding /bin/true, which does not exist on macOS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
9cae868a27 |
feat(downloads): offline sync for mobile — downloads v2 (#258)
* feat(downloads): offline sync for mobile (downloads v2) Replace internal/download with a unified internal/downloads package and add fully-offline download + watch-sync support for mobile clients, across five independently-shippable phases: - Phase 0: reshape the downloads table and the /downloads contract to be device- and format-aware; add GET /downloads/capability; extend DownloadConfig (default-off keys); update the web download hooks/components in lockstep. This is the one approved pre-lock exception to the additive-only /api/v1 rule (the web app is the only consumer and is updated together). - Phase 1: managed device-library entries (create/list/PATCH/delete/serve), keyed on the X-Silo-Device-Id header. - Phase 2: offline playback manifest plus artwork/subtitle proxy endpoints that strip every presigned URL (inline thumbhashes + authenticated proxies). - Phase 3: prepare-to-file (remux + transcode-to-single-file) as a durable, leased artifact queue with startup recovery, hosted on the task manager; playback.PrepareFile emits one +faststart MP4. Adds the admin transcode toggle and per-artifact LRU cleanup. - Phase 4: offline progress reconciliation -- a clamped event_at LWW key plus a server-assigned synced_seq cursor on watch_progress; an optional clamped updated_at on POST /sync/progress and an opaque ?since= cursor on GET /progress (additive; existing callers unaffected). Security & reliability invariants, each with an acceptance test: 1. Server-owned sync ordering: ?since= delta delivery is driven only by the server-assigned synced_seq; the client clock is bounded (event_at, clamped to now+skew) and used only for last-write-wins on the caller's own profile. 2. Full profile+device authorization on every managed endpoint, with a per-profile content/library access re-check before serving any bytes/assets. 3. Durable artifact recovery: a transactionally-claimed (FOR UPDATE SKIP LOCKED), lease-heartbeat, attempt-counted queue with a startup sweep, so no crash strands a download in preparing and concurrent workers never double-encode. Migrations are timestamped Goose files: reshape downloads (device/format); download_artifacts (durable queue); watch_progress event_at/synced_seq. DB-backed acceptance tests skip without SILO_TEST_DATABASE_URL and run in CI; the invariant-1 progress test also runs against the real SQLite backend locally. Client repos (silo-android, silo-apple) consume the reshaped /downloads/* contract and the updated_at/?since= progress fields and require coordinated follow-up. Implements the maintainer-approved v1 capability proposal for offline sync (downloads v2). AI-use disclosure: implemented by Claude (Claude Code) from the approved design doc under docs/superpowers/specs, with human review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(downloads): series & season downloads + client-pull monitoring Build season downloads and a "monitor a series" capability on top of the downloads v2 (offline sync for mobile) work. Season downloads: - POST /downloads accepts season_number (with series:true) to download one season. CreateSeries/CreateSeason share one body via a listEpisodes closure and register managed entries under a shared batch_id (original-only). Episode files are resolved in a single batched query. Series monitoring (auto-download), client-driven: - New device-scoped download_subscriptions table with a Sonarr-style mode (all | future | latest_season | specific_seasons), a client-enforced delete_watched flag, and a max_storage_bytes cap. The server never deletes on-device files; retention and the hard cap are the client's, the server only soft-gates registration. - The client calls POST /downloads/subscriptions/sync on open / background refresh; the server registers the in-scope, not-yet-downloaded episodes (idempotent via the managed-entry unique index) and the device pulls them on its own schedule. No background worker and no dependency on the notifications subsystem. latest_season follows new seasons (>= subscribe-time season); future excludes the back catalog via air date. - Subscription CRUD + sync are profile+device authorized (device id from the X-Silo-Device-Id header only) with a per-request content-access re-check. The capability endpoint advertises season_download / series_monitoring / monitoring_modes. Also lands the downloads-v2 work already present in the tree: durable artifact (remux/transcode) preparation and offline watch-progress reconciliation, plus the design-spec updates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WIP: epitaxy pre-switch from feat/downloads-v2-offline-sync * test(downloads): fix deterministic ID collision in reconcile test Artifact IDs are time-sortable, so two artifacts created in the same moment share their first 8 chars; combined with a captured timestamp the two preparing-download IDs collided on downloads_pkey. Use the full artifact ID, which is unique per row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): support sqlite userdb backend for managed downloads With the sqlite userdb backend, profiles live only in per-user SQLite stores and public.user_profiles stays empty, so user_devices' profile FK made every managed create/subscription/offline-sync request fail with an FK violation. Drop the FK (shared Postgres tables must not FK profile tables — same rule as notifications) and replace the lost cascade with an app-level purge on profile deletion, wired through ProfileHandler for both backends. DB-backed regression tests cover the no-Postgres-profile-row path and the purge cascade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): dispatch encode kick asynchronously triggerDrain invoked the kick inline, and the kick (taskmanager RunTask) executes the encode task on the caller's goroutine — so a POST /api/v1/downloads with a bitrate quality blocked the HTTP request on the entire queue drain, ffmpeg encodes included, delaying the 202 by minutes on an idle queue. Dispatch the kick on a goroutine; the task manager already serializes concurrent runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): enforce per-user quota on the encode pipeline Two gaps let a user bypass MaxConcurrentPerUser entirely for prepared downloads: artifact-backed rows are created in 'preparing' (never 'queued'/'downloading'), which CountActiveByUser didn't count, and createArtifactDownload enqueued the encode job before limiter.Check, so even a 429-rejected request left a job the worker would transcode. Count 'preparing' as active and check the limiter before Ensure; managed replacements stay quota-exempt since they don't add a row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): protect ephemeral artifact links from LRU eviction HasActiveLink only counted managed (device_id IS NOT NULL) rows, so under a byte budget Cleanup could delete an artifact still referenced by a ready-but-unfetched ephemeral web download — permanently 404ing a row the API kept listing as ready (the artifact row is gone, so recovery can't re-queue it). Any non-terminal link now protects the artifact; only artifacts whose links are all cancelled/failed/revoked are evictable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): batch manifests skip bad entries instead of failing whole batch One deleted or access-filtered episode made GET /downloads/batches/{id}/manifests 404 for the entire season, so a client could no longer fetch manifests for the still-valid entries. Report unbuildable entries in a skipped[] array (revoked | not_found | error) alongside the delivered manifests, mirroring the create path's skip idiom. Also cut the batch cost: the shared series detail is resolved once per batch instead of once per episode, and buildSubtitles reuses the already-loaded media file instead of re-querying it per manifest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(migrations): wrap DO block in StatementBegin/End markers Under NO TRANSACTION goose splits statements on semicolons, so the dollar-quoted DO block failed every fresh install with 'unterminated dollar-quoted string' (SQLSTATE 42601). Already-applied databases are unaffected. Same fix is being applied to main; identical content merges cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): allow season 0 (Specials) in season downloads season_number was a plain int dispatched with '> 0', so requesting the Specials season was indistinguishable from omitting the field and silently broadened to a full-series download. Dispatch on pointer presence, treat 0 as the Specials season, and reject negatives with 400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): capability quality_presets is never JSON null PresetsFor returned a nil slice when downloads are disabled or the user lacks the permission, and Capability's []string{} initialization was immediately overwritten by it — so GET /downloads/capability serialized "quality_presets": null where the contract documents an array. Normalize at the source so every caller inherits the guarantee. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): subscription sync correctness + batched registration Three subscription fixes: - A paused subscription no longer syncs: PATCHing scope (or pausing and changing scope in one request) registered episodes for a monitor the user had just stopped, inconsistently with SyncSubscriptions' guard. - SubModeFuture compares calendar days (UTC): air_date is date-only, so the strict instant comparison permanently excluded episodes airing the same day the user subscribed; episodes with no air date now fall back to their ingest time instead of never registering. - Registration is one batched fetch (GetManagedEntriesByKeys) plus one batched INSERT ... ON CONFLICT DO NOTHING RETURNING (CreateManagedEntriesBatch) instead of a SELECT+INSERT per episode — a 300-episode series cost ~600 sequential round trips per request and every no-op sync re-walked the full set. RETURNING yields exactly the new rows, so the sync response's 'registered' count now honestly reports 0 in the steady state instead of the full in-scope count on every app open. The now-unused InsertManagedEntryIfAbsent is removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(userstore): stamp triggers own the event_at LWW key MarkProgressBatch (jellycompat series mark-played) advanced updated_at but never event_at, and both stamp triggers only defaulted event_at when NULL — so a queued offline event with a client time between the row's old event_at and the mark could win SetProgressIfNewer and resurrect a stale resume position that then re-synced to every device. Make the triggers authoritative instead of adding a tenth hand-written SET clause: whenever an UPDATE changes updated_at without explicitly changing event_at, the trigger advances the LWW key; writes that do set event_at (offline sync's clamped client event time) keep their value. Postgres gets a CREATE OR REPLACE migration; SQLite gets a v12 userdb migration that drops and reinstalls the trigger bodies (CREATE TRIGGER IF NOT EXISTS never replaces). Conformance tests cover both batch paths, the preserved-client-time invariant, and the v11→v12 upgrade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): lifecycle hygiene — squash migrations, dead status, stale-row sweeps Migrations: fold the 20260621 corrective migration back into the base Downloads V2 migrations (its columns/constraints already exist there) and fix the reshape Down, which re-added the narrow status CHECK without collapsing managed-lifecycle rows first — rollback aborted on any DB with preparing/ready/revoked rows; validated against a live row. Branch databases that applied the corrective migration need its version row removed: DELETE FROM goose_db_version WHERE version_id = 20260621020459. Code: drop the dead 'registered' status (nothing ever wrote it; the lifecycle is preparing -> ready; 'revoked' stays reserved for the planned admin revoke flow) along with unused KindDirect and ErrInvalidFormat. Sweeps: Cleanup now runs an age-based hygiene pass independent of the byte budget — cold terminally-failed artifacts (with .part leftovers), orphaned ready artifacts no download row references, and ephemeral web rows older than their convenience-record lifetime (also unpinning their artifacts and bounding GET /downloads growth). The byte budget remains the disk quota per the limits & restrictions design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): sync API doc with v2 fixes; HEAD on file route; Android handoff Document the contract changes from the review fixes: batch-manifest skipped[] shape, honest subscription 'registered' semantics, season 0 = Specials, always-array quality_presets, bytes_sent actual behavior, ephemeral 7-day retention, header-pairing requirement, progress-delta deletion caveat, and the ready/failed push event schema (new §9.4). Add an Android client handoff section (§11) mirroring the Apple one, register HEAD on /downloads/{id}/file for download stacks that probe before ranged GETs, and add season_number to the web create-request type. Flag the /direct-download session-token-in-URL tradeoff; a short-lived download-scoped URL is a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: consolidate download/progress helpers, prune dead code, gate sweeps Behavior-preserving consolidation from the Downloads V2 review: - appendVideoFilterArgs: one home for the burn-in/hwaccel -vf selection, shared by the HLS builder and the single-file prepare builder (the drift pattern that already bit tone-mapping once). - userstore.ResolveProgressState: one home for the min-resume/watched threshold rule, replacing five identical copies across both store backends and the offline-sync ingest. - Download file selection ranks resolutions via access.CompareQuality (adds 4320p, agrees with playback) instead of a private switch. - writeSubtitle uses the shared subtitles.SubtitleContentType mapping. - config.DefaultTranscodeDir replaces three '/tmp/silo-transcode' literals. - Read-side quality/revision defaulting helpers removed: insertArgs plus the NOT NULL/CHECK schema already guarantee the invariant. - Dead code removed: Repository.ListByUser, SubscriptionRepository. ListActiveBySeries, and the stale auto-register-worker comments (the design is client-pull; no worker exists). - Redundant left-prefix indexes dropped from the base migrations (their unique indexes serve the same prefixes). - recover()'s disk-presence sweep and the stale-row hygiene sweep run on startup then hourly instead of every 30s tick (both are O(cache size)). - gofmt/prettier fixes for pre-existing drift in handlers/playback.go and pages/Profiles.tsx. Deferred (noted for follow-ups): quality-ladder preset table collides with the drafted download limits & restrictions design, which specifies its own ladder helper; Download-literal construction consolidation and the managed-identity value object remain open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): draft download limits & restrictions design Design input for the follow-up v1 capability proposal (quality ceiling, batch size cap, per-user quantity/bandwidth overrides). Committed with downloads v2 because the remediation work explicitly defers the quality ladder refactor and revocation wiring to this spec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(progress): reject malformed updated_at; clamp negative progress inputs Review findings on #258: - A malformed (non-RFC3339) updated_at in POST /sync/progress previously parsed to the zero time, which clampEventAt treated as "now" — letting a stale offline event win LWW as a fresh server-time write. The item is now rejected with a per-item error instead. - ResolveProgressState now clamps negative position/duration before classification so no backend can persist negative progress through UpdateProgress/SetProgress. - The online-write event_at invariant test is table-driven over both SetProgress and UpdateProgress, which share the same contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): close review gaps — permission gates, file-access recheck, artifact-true manifests Review findings on #258: - UpdateSubscription now applies the same feature/DownloadAllowed gate as CreateSubscription and SyncSubscriptions; a PATCH could previously re-activate or widen a monitor and register managed rows after an admin disabled downloads or revoked the user. - Serving download bytes (managed and ephemeral) and /direct-download now mirror playback's per-file authorization via catalog.FileAllowedByAccess: library scope and the profile's max playback quality are re-checked at serve time, with artifact-backed rows checked against the artifact's resolution (a 720p transcode of a 4K source stays servable under a 1080p ceiling). - Offline manifests for remux/transcode entries now describe the prepared artifact (container, codecs, resolution, single selected audio track) instead of the catalog source file the client never receives. - ArtifactRepository.Requeue reports ErrNotFound when the row was concurrently swept; ArtifactManager.Ensure recreates the job in that case instead of linking downloads to a dead artifact id. - "No downloadable episodes" is a sentinel (mapped to 404 no_downloadable_episodes) rather than a bare error that surfaced as 500. - Subscription season_numbers are bounds-checked (0–9999) before the int32 narrowing in the repo could silently wrap them. - HandlePatchDownload reuses requireManaged instead of hand-rolling the same managed-identity checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Quick
|
227986094b | Propagate playback client metadata through session sync | ||
|
QuickandGitHub
|
b3198276f7 |
[codex] fix(subtitles): stream live transcribe_translate cues (#177)
* fix(subtitles): stream live transcribe_translate cues * fix(subtitles): harden live AI transcription |
||
|
|
fadd8ff456 |
feat(player): native PGS subtitle rendering via libpgs (#129)
* feat(playback): add IsPGS helper and sup streaming extract path PGS (Blu-ray bitmap) subtitle tracks can be copied losslessly into a .sup elementary stream for client-side rendering, so they no longer have to be burned in. streamExtractOutput maps PGS to (copy, sup), and the seek/-t windowing now skips PGS like ASS: both formats are fetched once and consumed whole by their client-side renderers. This also fixes a pre-existing truncation bug: the -t duration cap was applied unconditionally, cutting embedded ASS extracts off at the default 600s window even though the ASS client fetches the full track. Extract the ffmpeg argument construction into streamExtractArgs for testability, following the buildFFmpegArgs pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): expose PGS subtitle tracks as .sup stream URLs PGS tracks were filtered out of /playback/start subtitle_urls entirely, so the web player showed no subtitles for PGS-only files (#34). Include them with a .sup URL extension; DVD/DVB bitmap tracks stay hidden since they still have no non-burn-in delivery path. HandleSubtitle streams the full PGS track as application/octet-stream. The seek/duration window is forced to zero for sup: subtitleSeekPosition falls back to the session's last reported position even without a ?position= query, which would otherwise start the extract mid-file. The proxy-node subtitle handler gets the same sup branch, streaming ffmpeg output directly instead of buffering like its text paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(player): consolidate subtitle codec helpers into subtitleCodecs.ts Rename assSubtitles.ts to subtitleCodecs.ts — the module already labeled every codec, not just ASS — and add isPGSCodec/isBitmapCodec. Replace the duplicated BITMAP_CODECS set in SubtitleTranslateModal with the shared helper so codec lists live in one place. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(player): native PGS subtitle rendering via libpgs Render PGS subtitle tracks client-side instead of leaving them unavailable (#34). usePGSSubtitles mirrors the JASSUB hook: when a PGS track is active it lazy-loads libpgs, which fetches the .sup stream in a worker, decodes display sets progressively as bytes arrive, and draws them onto a canvas positioned over the video. The renderer looks up the display set at currentTime + timeOffset, so the HLS stream origin adds and the user-facing delay subtracts — a positive delay shows subtitles later, matching VTT semantics. Offset changes apply through the timeOffset setter without recreating the renderer; track switches, PiP detach, and unmount dispose it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(player): prefer text over bitmap tracks in subtitle auto-select With PGS tracks now listed, an earlier PGS track would win auto-select over a later same-language SRT/ASS track. Deprioritize bitmap codecs within the same source tier — text is lighter to render and styleable — while a PGS track still wins when it is the only language match, and forced-PGS auto-select now works. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a17529fd6f |
feat(config): live admin settings + truthful restart-required banner (#128)
* feat(nodeconfig): harden config watcher for integrated-mode use - RequestReload(): non-blocking, coalescing reload nudge that runs on the poll goroutine, so concurrent requests can never swap a stale snapshot over a newer one (unlike ForceReload from request handlers) - Skip OnChange callbacks when the reloaded config is deep-equal to the previous one, so the 60s poll doesn't fire rebuild/log callbacks on no-op reloads - Add RedisURL to BootstrapOverrides; previously a reload clobbered an env-provided Redis URL in the live config - Split reload into fetchSettings/applySettings and add unit tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(config): hot-reload config watcher in integrated mode Start nodeconfig.Watcher in integrated/api mode (previously only proxy/ transcode worker modes hot-reloaded). Expose the live config to the API and jellycompat routers via func-typed LiveConfig/OnConfigChange fields with nil fallbacks to the startup snapshot, and wire the admin settings update hook to RequestReload so same-process changes apply immediately even without Redis. No consumer reads the live config yet — conversions land separately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(admin): truthful restart-required banner for settings saves The settings UI showed 'restart required' after every save regardless of the key. The backend now classifies each key via a central registry (internal/config/restart_keys.go) and PUT /admin/settings/{key} reports restart_required per key; useSettingsForm only raises the banner when a saved key actually needs a restart (and keeps it raised until restart). The registry is conservative: every currently startup-frozen key is marked restart-required; subsequent hot-reload conversions shrink it. Settings read live from the settings repo (branding, overlays, markers, download.*, ...) default to no-restart. DownloadSettings/OverlaySettings drop their hardcoded restartRequired={false} special-casing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(logging): hot-reload server.log_level and server.log_quiet Share one slog.LevelVar across the handler chain and make logfilter.Handler's quiet-prefix list an atomic pointer shared with WithAttrs/WithGroup clones (New previously returned the inner handler unwrapped when the quiet list was empty, leaving nothing to update). The integrated-mode config watcher now applies both settings live; their keys leave the restart-required registry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(auth): hot-reload access/refresh token expiries JWTService stores expiries as atomics with a SetExpiries hook; all three instances (main API, ABS compat, jellycompat) re-apply them on config reload. Applies to newly issued tokens; outstanding tokens keep their original expiry. The JWT secret stays fixed for the process lifetime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(playback): read transcode config live at session start The playback and stream handlers pull ffmpeg path / hwaccel / transcode dir from the live config when starting a transcode or extracting subtitles, instead of values frozen at router construction. Each session snapshots the config once so its output dir and binary stay consistent. Also fixes a real bug: playback.hw_device was parsed into the config but never wired into the integrated-mode handler, so local transcodes always ran with an empty HWDevice while transcode nodes honored it. playback.transcode_dir leaves the restart-required registry (the handler is its only consumer); ffmpeg_path/hw_accel stay restart-required until scanner/chapterthumbs/audiobook consumers convert. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(jellycompat): read compat identity settings live per request System/Auth handlers take a config provider instead of the startup snapshot, so jellyfin_compat.public_url, .server_name, and .emulated_server_version apply without restart. server_id stays restart-required (generate-once, baked into the resource mapper), as do the session-store TTLs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scanner,metadata,mdblist): hot-reload worker pools and API key scanner.workers, matcher.workers/batch_size, metadata.cache_images, and mdblist.api_key convert to atomic fields with setters wired to the config watcher. Worker counts apply on the next scan/match cycle (the loops read them per cycle); the MDBList key applies to the next request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): hot-reload AI connection, models, toggles, and quotas The shared llm.Client holds its config behind an atomic pointer (UpdateConfig; each request snapshots once), and the subtitle/metadata AI services gain UpdateConfig plus setters on the translator (batching) and Whisper transcriber (ffmpeg path, chunk seconds). The router derives their configs from shared helpers used both at construction and in OnConfigChange callbacks, re-evaluating the chat-only-gateway transcribe guard on each reload and warning only when it newly fires. Everything on the AI Services page now applies without restart except ai.max_concurrent_jobs (fixed-capacity dispatch semaphore). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): wire transcode_enabled; remove dead playback/scanner knobs playback.transcode_enabled was parsed into the config but the resolver always received a hardcoded true — the admin toggle did nothing. It now reads the live config per playback start, so disabling transcodes applies without restart. Remove settings that were wired to nothing so 'save + restart' stops pretending: playback.allow_hevc_encoding (resolver field never assigned), playback.transcode_ahead_segments and playback.segment_duration (parsed, never consumed — segment duration is per-session from the client), scanner.file_removal_grace (DeleteMissing is never called). UI fields removed and the config struct fields pruned so they don't resurrect; YAML import still tolerates the legacy keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
39ba284c9d |
feat(ai): shared AI core — metadata translation, Whisper ASR, per-profile language, on-view translation (#127)
* docs: design + plan for shared AI core, metadata translation, Whisper ASR Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai): shared LLM client, segment translator, and job runner packages internal/ai/llm: OpenAI-compatible chat client moved out of subtitles/ai, plus /v1/audio/transcriptions (verbose_json) for the ASR work; one shared retry/backoff loop for both. internal/ai/translate: the batched indexed-JSON translation protocol generalized to text segments. internal/ai/jobrunner: dispatch/heartbeat/reaper/cancel lifecycle extracted behind a minimal store interface, with a semaphore shareable across job services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(subtitles): consume shared AI core LLMTranslator becomes a thin cue<->segment adapter over aitranslate; the service delegates dispatch/heartbeat/reaper/cancel to jobrunner; the local OpenAI client is gone in favor of internal/ai/llm. Behavior (prompts, wire protocol, job rows, recovery semantics) is unchanged. NewService now takes the dispatch semaphore so all AI job services can share one bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(config): shared ai.* settings, metadata translation job table, localization provenance columns ai.* connection keys (chat + optional separate ASR endpoint) load with a fallback to the legacy subtitle_ai.* rows — those are never renamed in SQL because encrypted values are GCM-bound to their setting key. New toggles: subtitle_ai.transcribe_enabled, metadata_ai.enabled. Migration adds metadata_translation_jobs, per-field provenance (provider|ai|manual) on the localization tables, and media_folders.auto_translate_metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): localization field provenance with provider/ai/manual precedence Provider upserts keep manual values and never blank a field with an empty incoming value; new UpsertAITranslation/UpsertAIOverview methods write AI fields only over empty or ai-sourced values (force adds provider, never manual) — all enforced in single-statement SQL. Serving now merges only non-empty localized fields onto the base item, since localization rows are legitimately partial (AI rows carry no titles/artwork). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(metadata): AI translation service, refresh auto-fallback, and admin API internal/metadata/translation: job service over the shared AI core that expands an item to its season/episode overviews, skips already-localized fields (zero model calls on repeat runs), batches paragraphs through the generic translator, and persists per batch with provenance-aware upserts. MetadataService gains an AutoTranslator seam invoked after each refresh for libraries with auto_translate_metadata. Admin endpoints under the metadata curation guard: enqueue, list (poll), cancel; plus a status probe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(subtitles): Whisper ASR transcribe and transcribe_translate jobs New WhisperTranscriber: one ffmpeg pass extracts the audio track to 10-min 16kHz mono WAV chunks (temp dir cleaned on every exit path), each chunk goes to the OpenAI-compatible /v1/audio/transcriptions endpoint (verbose_json, per-request timeout sized to 3x chunk duration), segment timestamps are offset and built into wrapped cues. Chunks process playhead-first and stream live to the requesting session. The transcript is stored as an ordinary downloaded subtitle (provider 'transcribed'); transcribe_translate chains the existing translator and stores the translated track as the job result. Enqueue accepts an optional kind; status reports transcribe_enabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): AI services settings, metadata translate action, library auto-translate, generate-from-audio New AI Services admin page hosts the shared endpoint config (reads fall back to legacy subtitle_ai.* values, writes target ai.*) and the three feature toggles; the AI card moves out of Subtitles settings. The metadata editor gains a Translate-with-AI panel with job polling and force/re-translate. The library form gains the auto-translate toggle (threaded through the libraries API). The player translate modal gains a From-audio mode that lists audio tracks and submits transcribe / transcribe_translate jobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt import grouping in router and translation tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): per-profile metadata language and viewer-triggered description translation user_profiles.preferred_metadata_language threads through the access scope into catalog serving: presentation language now resolves explicit param -> profile preference -> library metadata language (native API and jellycompat). ItemDetail gains pending_translation_language when the viewer's language is missing a localized overview. New metadata_ai.on_view setting (off|button| auto) gates POST /items/{id}/translate-description: any profile with item access may request its language, with in-flight dedup and a 15-minute failure cooldown so page views never hammer a broken endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): on-view description translation with per-profile metadata language Profile playback settings gain a Metadata language picker (library default inherit). Detail pages: when the server reports pending_translation_language and metadata_ai.on_view is 'auto', the description translates on view with a pulse animation until the refetched detail comes back localized (45s timeout); in 'button' mode a small Translate chip triggers the same flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): expose metadata_ai.on_view in AI Services settings The on-view translation mode had no UI control, so it could only ever be 'off' — viewers got neither the auto translation nor the fallback button. Adds the off/button/auto selector to the Features card, and the config loader now warns and falls back to 'off' on a bad row instead of refusing to start. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): clear configuration hint when the transcription endpoint is chat-only A blank Transcription base URL falls back to the chat endpoint; chat-only gateways reject the multipart upload with an opaque 400 that reads like a pipeline bug. 400/404/405 transcription failures now carry a hint to set a Whisper-compatible endpoint in AI Services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): wrap ASR cue text by rune count, not bytes Arabic/Cyrillic/Greek text is 2+ bytes per character in UTF-8, so byte-based wrapping broke lines at roughly half the intended visual width. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(web): steer transcription base URL hint away from chat-only gateways Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): block chat-only gateways for transcription, add endpoint presets llm.IsChatOnlyGateway (OpenRouter et al — no timestamped transcription API) is enforced in three layers: the settings API rejects ai.asr_base_url values pointing at one, the router disables ASR with a warning when the blank-URL fallback would land on one, and llm.Transcribe refuses outright. The AI Services page gains one-click transcription presets (Groq turbo/accurate, OpenAI, self-hosted speaches) plus the mirrored client-side check, and the settings API now also validates metadata_ai.on_view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): tighten ASR subtitle sync Three systematic timing-error sources addressed: cue offsets now use the segment muxer's exact per-chunk start times (segment_list CSV) instead of assuming index*chunk_seconds; the audio stream's start delay relative to the container timeline (common in TS remuxes) is probed via ffprobe and added to every cue; and the chunk length is now operator-tunable via subtitle_ai.asr_chunk_seconds (60-600s, default 600) since shorter chunks bound Whisper's within-chunk timestamp drift. Playhead-first ordering now pivots on real chunk starts, and a beyond-end playhead starts at the final chunk instead of restarting from zero. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): tolerate base URLs that already include the /v1 segment Providers like DeepInfra expose their OpenAI-compatible API under a base that contains the version segment (api.deepinfra.com/v1/openai); always appending /v1/... mangled those. endpointURL now appends bare paths when the base already carries /v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): prefer self-hosted transcription in presets and hints Preset order becomes self-hosted (recommended) -> Groq turbo -> Groq large-v3 -> OpenAI, and the settings hint plus the job-error hint lead with the self-hosted option. The self-hosted preset now fills the turbo CT2 model to match the recommended speaches setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): request VAD and word timestamps for ASR cue accuracy Without vad_filter, faster-whisper servers report wall-to-wall segment times: cues linger on screen through silence (verified up to 91s) and paragraph-length segments become single 400+ char cues. Request vad_filter=true (skipped for hosted providers that reject non-OpenAI fields and run VAD server-side) plus timestamp_granularities word+segment, and rebuild cues from word timings: split at speech pauses, sentence ends, text capacity, and a 7s max duration; cap word-less segments instead of trusting their reported end; stretch sub-second cues to a readable minimum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
5855cebe75 |
fix(audiobooks): complete playback and catalog parity (#96)
* fix(audiobooks): complete playback and catalog parity * fix(web): allow podcast continue targets * fix(audiobooks): use folder sidecar covers during scan * fix(audiobooks): read nullable poster paths during cover scan * feat(home): split continue listening sections * fix(catalog): coalesce nullable media artwork fields * feat(playback): surface audiobook sessions in admin activity * fix(audiobooks): address playback review feedback --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
978e1b4954 |
feat(playback): NVENC support for transcoding (#79)
* feat(playback): NVENC support for transcoding * fix(playback): probe nvenc before auto-selecting * fix(playback): use safe NVENC smoke probe dimensions --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
eb6024573e |
feat(audiobooks): make audiobook libraries first-class catalog items (#73)
* docs(audiobooks): design spec for plugin absorption Plan to absorb silo-plugin-audiobooks into silo-server as a first-party feature. Audiobooks land in silo's existing SPA; ABS clients connect directly. Hard constraints: reuse existing tables (media_items, media_files, user_watch_progress, user_playback_sessions, people, item_people, library_collections); only two new tables (abs_sessions, podcast_feeds) and at most one column add (media_libraries.kind); silo's main :8080 listener handles ABS Socket.io natively. Out of scope: audiobook requests flow, smart collections, share links, external recommender, custom metadata providers, separate audiobook SPA. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 1 (discovery + schema) First of six sub-plans for the absorption. Six tasks: a discovery audit that resolves the spec's Risk questions, four idempotent SQL migrations (abs_sessions, podcast_feeds, media_libraries.kind, audiobooks.enabled feature flag), and an empty-but-compiling internal/audiobooks package scaffolded into cmd/silo. Lands as a strict no-op for users (feature flag defaults to false). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): discovery findings for absorption sub-plan 1 Locks schema/code decisions for migrations 139-142 and downstream sub-plans. Resolves open Risk questions from the absorption design spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 139 add abs_sessions table Parallel of jellycompat_sessions for Audiobookshelf-compatible clients. Lets ABS mobile/desktop apps maintain a device-bound session that silo's audiobooks/abs handlers will validate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): match codebase conventions in migration 139 Lowercases type keywords in the abs_sessions CREATE TABLE body to match neighboring migrations, fixes the client_version column alignment, and replaces the misleading "parallel to jellycompat_sessions" header comment with a more accurate description of the table's role. Cosmetic only — the running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 140 add podcast_feeds table Side table on media_items for RSS-subscribed podcasts. Holds feed URL, ETag/Last-Modified for conditional fetches, last-refresh timestamp, and the per-feed refresh interval consumed by the upcoming podcastfeed.Refresher scheduled task. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): uppercase PRIMARY KEY in migration 140 Aligns with the codebase convention (type keywords lowercase, constraint keywords uppercase) established in migration 139's post-style-fix form. Cosmetic only — running schema is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(audiobooks): migration 141 no-op for media_folders.type Sub-plan 1 originally reserved migration 141 to add a 'kind' column to media_libraries discriminating audiobook/podcast libraries. Discovery audit (sub-plan 1 Task 1) found that the actual table is media_folders and it already has a type text NOT NULL column with no CHECK constraint or enum, so 'audiobooks' and 'podcasts' can be added as future values without DDL. Landing this migration as a documented no-op preserves the version numbering audit trail and pins the decision in git history. The matching down migration is also a no-op. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): migration 142 add audiobooks.enabled flag Server-settings row that gates the absorbed audiobooks feature. Defaults to 'false' so sub-plan 1 lands as a strict no-op; subsequent sub-plans branch on this flag and operators flip it to 'true' at cutover. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scaffold internal/audiobooks package Empty-but-compiling Service that reads the audiobooks.enabled feature flag from server_settings. Wired into cmd/silo so the package is referenced from the binary; no routes mounted, no scheduled tasks registered, no DB writes. Subsequent sub-plans hang scanner branches, ABS handlers, Socket.io, podcast refresher, and SPA pages off this Service. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(audiobooks): cosmetic cleanups in scaffolded package Two pre-emptive cleanups flagged by code review before sub-plan 2 copies the patterns: 1. Sort the internal/audiobooks import after internal/adminjob in cmd/silo/main.go (alphabetical). 2. Drop the redundant "audiobooks: " prefix from the Enabled() error wrap; matches how every other top-level service package (watchstate, scanqueue, metadata, etc.) formats errors. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 2 (scanner) Second of six sub-plans. 10 tasks: PersonKind constants for Author and Narrator, audio-extension recognizer, library-type helpers, a walkLogicalTree refactor (movieLibrary bool -> typed walkMode), chapter extraction via ffprobe, single-file and multi-file audiobook parsers, scanner write path producing media_items.type='audiobook', and a filesystem podcast parser (RSS deferred to sub-plan 5). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add Author and Narrator PersonKind constants Discovery audit confirmed item_people.kind is unconstrained smallint with values 1-6 in use. Reserve 7 = Author, 8 = Narrator for audiobook people-links written by the upcoming scanner branches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): add audio-extension recognizer for scanner Mirrors the existing videoExtensions/SupportsVideoFile pair. Used by upcoming audiobook and podcast scanner branches to filter directory walks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): library-type recognizers for scanner dispatch isAudiobookLibraryType and isPodcastLibraryType match singular and plural forms case-insensitively, mirroring isMovieLibraryType. Used by upcoming scanner walk branches (Task 4) that filter audio files into audiobook and podcast libraries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(scanner): replace movieLibrary bool with typed walkMode Lets walkLogicalTree dispatch on multiple library shapes (video, movie, audiobook, podcast) without proliferating boolean flags. Behavior for existing video and movie libraries is unchanged; audiobook and podcast modes will be consumed by the upcoming audiobook.go and podcast.go parsers in later tasks of this sub-plan. walkModeFor() derives the mode from a media_folders.type string; unknown types default to walkModeVideo to preserve prior behavior for any caller still passing a raw type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose ffprobe format tags on ProbeData The audiobook scanner needs format-level tags (title, artist, album, date) for media_items metadata; ffprobe already parses them in ffprobeFormat.Tags but ProbeData previously discarded them. Add FormatTags map[string]string to ProbeData, populate it in convertProbeData via a new normalizeFormatTags helper that lowercases keys and trims values. Adds a fixture audiobook .m4b with embedded chapters (Intro/Outro) and format tags, and a test that verifies ProbeFile() returns both correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): parser for single-file audiobook folders parseAudiobookFolder reads tags + chapters via the existing ProbeFile (now that Task 5 exposes FormatTags on ProbeData) and produces a parsedAudiobook struct. Title falls back from "title" tag to "album"; author from "artist" -> "album_artist" -> "composer"; series from "album" -> "series" -> "mvnm" (Movement Name, used by some MP4 tools). Year parsed from "date" or "year" tags, tolerating ISO dates and parenthesized forms. Single-file case only; multi-file folders (one audio file per chapter) return a placeholder error and arrive in Task 7. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): multi-file audiobook folder support Folders containing N audio files (one per chapter/part) get one parsedAudiobookFile per file; each file's chapter list is synthesized as a single chapter with title = filename stem. Title/author/series/ year come from the first file's tags. Also drops the duplicate pickFirstNonEmpty helper added in Task 6 in favor of the existing firstNonEmpty already in probe.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): scanner write path produces audiobook media_items ScanAudiobookFolder walks an audiobooks-typed media folder and treats each immediate subdirectory as one audiobook. For each parsed audiobook it upserts: - one media_items row with type='audiobook' - one media_files row per audio file (with chapters JSONB) - author/narrator links in item_people (kind=7, kind=8) Adds itemRepo and personRepo to the Scanner struct, wired from fileRepo.Pool() in NewScanner — no constructor signature change needed. ScanFolder dispatches to this path when folder.Type='audiobooks', bypassing the per-file movie/TV pipeline because audiobooks are folder-scoped entities. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): filesystem podcast scanner ScanPodcastFolder walks a podcasts-typed media folder, treating each subdirectory as a podcast show and each audio file inside as an episode. Writes media_items.type='podcast' + episodes rows + media_files rows. RSS-subscribed feeds (podcast_feeds table) arrive in sub-plan 5; this task covers filesystem-only ingestion. ScanFolder dispatches to this path when folder.Type='podcasts'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audiobooks): implementation plan sub-plan 5 (podcasts) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audiobooks): expose audiobooks/podcasts library types in admin UI Adds 'Audiobooks' and 'Podcasts' options to the library-type dropdown in the admin libraries page so operators can flag a folder as an audiobook or podcast library. Extends contentLevelsForType() so the admin UI's downstream filtering treats those types correctly (audiobook -> ['audiobook'], podcasts -> ['podcast', 'podcast_episode']). Backend scanner branches for these types were already wired in sub-plan 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 139_abs_sessions to 147 for origin/main merge origin/main adds 139_media_requests at the same number our local audiobook branch had used for abs_sessions. Renumber ours to 147 to free up 139 for the upstream migration. The schema_versions row is updated in lockstep on the running database so the migrator sees the abs_sessions migration as already applied at its new version. Migrations 140-146 (podcast feeds, media_folders kind noop, audiobook feature flag, abs playback sessions, podcast episode guid, audiobook series, audiobook title cleanup) stay where they are — they don't collide with anything on origin/main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(migrations): renumber 140_podcast_feeds to 157 for origin/main merge origin/main added 140_user_permissions at the same version this branch had used for podcast_feeds. Renumber ours to 157 (next free above the collections-unify migration at 156) so 140 is free for the upstream migration. schema_versions on the running database is updated in lockstep so the migrator sees podcast_feeds as already applied at its new version. Same pattern as |
||
|
RXWatcherandGitHub
|
266b4453ca |
fix(playback): avoid restarting active transcodes (#52)
* fix(playback): avoid restarting active transcodes * fix(playback): propagate transcode restart gating --------- Co-authored-by: RXWatcher <14085001+RXWatcher@users.noreply.github.com> |
||
|
|
0163df3683 |
[codex] Add IntroDB marker integration and dialogue-aware Chromaprint refinement (#57)
* docs(markers): design + implementation plans for multi-source markers & TheIntroDB contribution * fix(markers): TheIntroDB read-path correctness (TVDB, real confidence, best candidate) Honor TVDB ids in /media lookups (previously dropped — anime/TheTVDB-first libraries got no markers), decode and use the real per-segment confidence and submission_count instead of a hardcoded 0.9, and pick the most-submitted / highest-confidence candidate when several are returned. Adds httptest coverage for the introdb client and provider. Phase 1 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): multi-source dispatch, per-provider config, per-segment provenance Add marker_provider_config (per-provider fetch enable/priority + contribute gates, contribution off by default) and a cached ProviderConfigStore. Add Registry.FetchMerged: query all fetch-enabled providers concurrently and keep the best candidate per segment (submission_count, then confidence, then fetch priority), stamping each winning marker with its provider/algorithm. Thread per-segment provenance through MarkerUpdatePayload and scanner.MarkerUpdate (additive SegmentProvenance overrides) so a merged result writes correct per-segment provider/confidence/algorithm; the legacy shared columns keep a summary. The lazy-playback path now uses FetchMerged. With only TheIntroDB enabled, behavior is unchanged. Phase 2 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): TheIntroDB submission client, contribution audit, service engine Add a markers.Submitter capability and implement it on the introdb provider (POST /v3/submit, GET /v3/user/stats; key required, usage-limit aware, applies the null start/end conventions). Add the marker_contributions audit table and a value-hash-keyed ContributionStore for idempotency. Add ContributionService: resolves enabled submitter providers, gates eligibility (never re-submit online-sourced markers; auto runs require contribute_auto_local + scanner-intro above the per-provider confidence threshold), checks idempotency, submits, and records. Wired in main.go; no trigger yet (admin API and task follow). Phase 3 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): admin marker editing, contribution, and provider config endpoints Add the RequireAdmin marker API: GET/PUT /admin/files/{id}/markers (read with provenance; manual upsert where a segment object sets and null clears), DELETE .../markers/{segment}, POST .../contribute and GET .../contributions, plus GET/PUT /admin/markers/providers[/{provider}] and a .../validate key-check returning user stats. Manual writes go through the priority-gated UpsertMarkers (source=manual) and notify live sessions; a new FileRepository.ClearMarkers nulls a segment's columns. Validation mirrors the contribution rules. Phase 4 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(markers): daily auto-contribution task for local intro markers Add ContributeMarkersTask (daily 04:00, after local detection): when a provider has contribute_enabled + contribute_auto_local, page through episode files with a scanner intro marker at/above the provider's confidence threshold (new ContributionStore.CandidateLocalIntroFiles keyset query) and run them through ContributionService with Auto=true. No-op when no provider opts in; idempotent and resumable across runs. Phase 5 of docs/superpowers/plans/2026-06-06-marker-sources-and-contribution-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(intromarkers): refine chromaprint starts with dialogue cues * feat(markers): finish marker management backend * feat(web): add marker editing UI * feat(markers): use plugin marker providers * fix(markers): address PR review feedback * feat(player): show marker labels on seek hover * fix(markers): type nullable marker mutation params * feat(markers): audit marker edits and add permission --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
9f73ac6f1a |
feat(realtime): improve web UI reactivity and admin visibility (#48)
* fix(web): scope realtime user state events * feat(events): add canonical catalog event publishers * feat(events): publish canonical catalog events * refactor(web): centralize realtime events provider * feat(events): normalize user state event name * feat(web): patch item user state from realtime events * fix(web): refetch active catalog on realtime changes * fix(events): publish item changes during metadata enrichment * fix(web): improve dashboard and mutation reactivity * feat(admin): improve realtime session activity * feat(admin): refine playback admin surfaces * feat(admin): improve library task controls * fix(collections): position defaults progress below header * feat(library): surface matcher backlog * fix(admin): hide matcher backlog from server activity * chore(migrations): renumber branch migrations * feat(admin): show registered devices without overrides * feat(admin): improve scheduled task visibility * fix(realtime): tighten admin update handling * docs(admin): document library job id parsing * docs(library): explain mount check feedback timing * fix(library): guard metadata match queue handlers * fix(admin): avoid stale queued job cancellation * fix(settings): harden device registration and task timing * fix(jellycompat): fill large browse pages * perf(jellycompat): compress and batch list image work * feat(autoscan): pluggable scan-source autoscan category (Sonarr/Radarr) (#44) * docs: design spec for autoscan arr polling Periodic poller over autoscan-enabled Radarr/Sonarr instances (reusing request_integrations) that maps import paths to Silo media folders and enqueues targeted scans via the existing scantrigger + scanqueue. Lean single-service model: no cross-node fan-out guard or retry queue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan arr polling Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): settings and sources schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): core types Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): path rewrite helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): dedupe imported paths to parent folders Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): arr import-history client Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): settings + sources repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): redis scan-suppression seam Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): PollOnce poll cycle * feat(autoscan): poll task and wiring * feat(autoscan): admin API endpoints * feat(autoscan): admin API endpoints Adds ErrIntegrationNotFound sentinel (errors.Is) instead of string matching. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan types and hooks Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan admin tab * fix(autoscan): release suppression claim on enqueue failure; reconfigure trigger on interval change; skip source on key-resolution error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): update handler test for 3-arg NewAutoscanHandler * fix(autoscan): per-path suppression key, bounded poll window + overlap, boundary-safe rewrites, GREATEST cursor guard, async trigger, quiet unresolved-path skip, FK->404 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize Windows path separators, surface status errors, re-seed source editor on save Addresses minor code-review findings: Windows backslash paths now normalized before rewrite/dedupe; HandleStatus returns repository errors instead of 200; the per-source editor re-seeds from server data after a save. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan rewrite-sync from arr root folders Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan rewrite-sync Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): suffix-match rewrite suggester Add suggestRewrites / commonSuffixLen for Task 1 of the autoscan arr-polling feature. Pure function: matches arr root-folder paths to Silo media folder paths by longest common trailing segment count, adjusted for depth-delta so coincidental same-named segments at different structural levels don't inflate confidence. Categorises each arr root as Proposed, Ambiguous, Unmatched, or Covered by an existing PathRewrite rule. TDD: test file written first, verified failing, then implementation added. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): GetSource single-source lookup * feat(autoscan): arr root-folder client + Silo folder lister * feat(autoscan): Service.SuggestRewrites Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): rewrite-suggestions endpoint Add GET /autoscan/sources/{id}/rewrite-suggestions admin endpoint: extend the autoscanTriggerer interface with SuggestRewrites, wire SetRewriteResolvers in the router, and add handler + test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web): autoscan rewrite-suggestions types and hook * feat(web): autoscan sync-rewrites preview * fix(autoscan): normalize covered-rule paths, dedup roots/folders, skip no-op suggestions Addresses final-review edge cases: coveredBy normalizes the existing rewrite's From (so a stored Windows/dup-slash rule still covers a root); duplicate arr roots and duplicate Silo folder paths are de-duplicated; an arr path that already equals its Silo path is not proposed as a no-op rewrite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): vitest 4 compatible fetch spy in recipes.test (unblocks build after vitest 4.1.0 bump) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): non-null suggestion slices + move Sync into rewrites card - suggestRewrites initializes Proposed/Unmatched/Ambiguous/Covered to empty slices so the JSON response is [] not null — fixes the 'Something went wrong' crash when every root is already covered (frontend mapped over null). - Move the sync button into the Path rewrites card beside 'Add rewrite' and rename it 'Sync rewrites'; guard the proposed map with ?? []. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): long root-folder timeout + sync spinner + collapse rewrites on load - Root-folder fetch for sync uses a 2-min timeout: Radarr/Sonarr compute unmappedFolders by scanning all roots, so a large library's /rootfolder takes 20-30s+ and tripped arrclient's 30s default (Sonarr 502'd at exactly 30s). - Spin the sync icon + show 'Syncing…' while the request is in flight. - Path rewrites card starts collapsed on page load. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): rescan on Sonarr/Radarr file renames History polling previously only tracked downloadFolderImported events. A rename in Sonarr/Radarr (episodeFileRenamed / movieFileRenamed) moves a file without an import event, leaving the library folder stale until the next full scan. Extend the history client to also surface renamed paths: both the new path and the old sourcePath, since a rename can move a file between folders and both parents may need rescanning. Delete events are still skipped — upgrade-deletes are covered by the paired import, and standalone deletes carry no file path in arr history. Renames the interface method ImportedPaths -> ChangedPaths to reflect the broader scope. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): synchronize trigger test with detached PollOnce goroutine HandleTrigger dispatches PollOnce on a detached goroutine and responds 202 immediately. The test read trig.called straight after the handler returned, racing the goroutine (usually 'PollOnce was not invoked') and reading the field without synchronization (a data race under -race). Signal completion through a channel the fake sends on when PollOnce runs; the test waits on it (bounded) before asserting. The channel send happens-before the receive, so the subsequent read of called is race-free. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: design spec for autoscan as a pluggable scan-source category Reframes autoscan from a Requests-coupled, arr-only feature into a standalone Autoscan category. Change-detection providers become out-of-process plugins via a new additive scan_source.v1 capability (client-pull, opaque marker); Sonarr/Radarr is the first provider. Host keeps a provider-agnostic resolve/suppress/enqueue engine; all arr-specific logic (and path rewrites) move into the plugin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for scan_source.v1 SDK capability First of the per-repo plans from the autoscan-plugin-architecture spec. Adds the additive scan_source.v1 capability to silo-plugin-sdk (proto + codegen + capability allowlist + runtime wiring), TDD per task, tagged as v0.5.0 so the host and arr-plugin plans can build against it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plan for autoscan host backend (part 1 of 2) Backend for the standalone Autoscan category: scan_source.v1 plugin plumbing (pluginhost client + plugins.Service resolver), generalized engine driven by a provider seam, autoscan_connections + autoscan_sources schema (decoupled from Requests), connection resolution (own or Requests-linked), admin API. Depends on silo-plugin-sdk v0.5.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: implementation plans for autoscan arr plugin + host UI arr plugin: new installable scan_source.v1 plugin (history imports+renames, rewrites, Silo-native paths), structured like silo-plugin-tmdb; ports the arr-specific logic from the closed PR #43. host UI (part 2 of 2): standalone Autoscan admin category (connections, sources, settings) extracted out of Requests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(autoscan): replace silo-plugin-sdk with local scan_source.v1 checkout Temporary dev replace so the host backend can build against the unreleased scan_source.v1 capability (silo-plugin-sdk PR #2). Finalize to v0.5.0 once the SDK is tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pluginhost): scan_source.v1 capability client wrapper Adds ScanSourceClient struct, the Client.ScanSource() accessor (mirrors ScheduledTask pattern), and a PollChanges method. Also introduces client_test.go with capability-gate tests for both scheduled_task.v1 and scan_source.v1 using a lazy gRPC ClientConn. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test+fix(pluginhost): cover capability-id gate, dedicated scan_source timeout Adds a "wrong id returns error" subtest to both capability-gate tests so the capability-ID component is exercised independently of the type. Introduces DefaultScanSourceTimeout (2m) for PollChanges, which polls an external arr API that can be slow, instead of the generic 10s control timeout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(plugins): expose scan_source.v1 client resolver Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(migrations): autoscan v2 schema (connections + sources, no requests FK) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): v2 types and repository Replace the request_integrations-coupled model with the decoupled v2 schema (autoscan_settings + autoscan_connections + autoscan_sources). Connection CRUD, source upsert/list/get, and AdvanceMarker/RecordError for opaque marker bookkeeping. ErrIntegrationNotFound becomes ErrNotFound. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): resolve connections (own credentials or Requests-linked) ConnectionResolver turns a stored Connection into concrete credentials, reading a soft-linked Requests integration's live base URL/key when RequestIntegrationID is set, then resolving the api-key ref to plaintext. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): scan-source provider seam over the plugin resolver ScanSourceProvider lets the engine poll changed paths without a live plugin; pluginProvider adapts plugins.Service.ScanSourceClient in production. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): generic engine drives sources via scan_source provider Rewrite PollOnce to iterate enabled sources, resolve each connection, poll the provider for changed paths, and run the salvaged resolve→suppress→enqueue loop (uniqueParentDirs, (folder,path) suppression key, RequestError quiet-skip, release-claims-on-enqueue-fail) verbatim. Store the opaque next marker via AdvanceMarker only after a successful enqueue; RecordError + keep marker on provider failure. Tests reworked onto a fakeProvider/fakeStore with an added opaque-marker-verbatim assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): drop conflicting connection CHECK, add connection resolver tests - migration 172: remove the autoscan_connections_source_present CHECK. It conflicted with request_integration_id ON DELETE SET NULL: deleting a Requests integration that a linked-only connection (base_url NULL) points at would null the FK and trip the CHECK, blocking the delete. The intended behavior is for the connection to survive as an orphaned 'needs attention' row. Creation-time validity is now enforced at the application layer. Verified on a throwaway DB: full chain applies and the delete-cascade leaves an orphaned (both-null) connection. - connection.go: TrimSpace the api key ref + resolved secret before the empty-string checks, matching requests.resolveAPIKey parity. - connection_test.go: fake-based tests for ConnectionResolver.Resolve (own creds, linked, linked-missing error, lookup error, trim/fallback). - repository.go: bound RecordError's stored last_error to 2048 chars. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): autoscan v2 admin endpoints Rewrite the autoscan admin HTTP handler against the v2 model: settings, connection CRUD, source update, manual trigger (detached PollOnce), and status. Connection/source responses omit api_key_ref and resolved keys (has_api_key flag only); unknown connection/source ids map to 404 via autoscan.ErrNotFound. Retire the host-side rewrite-suggestions endpoint (now lives in the arr plugin). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): wire v2 service, routes, retire rewrite-suggestions Export PollChangesClient/ScanSourceResolver from the autoscan provider so the api package can declare a structurally-conformant plugin adapter (Go has no return-type covariance, so the adapter must name the interface as its return type). Add api.BuildAutoscanService with the requests-integration lookup and plugin scan-source adapters, shared by the router (manual trigger) and the background poll task. Re-wire router routes to the v2 connections/sources/settings/trigger/status surface and drop the rewrite-suggestions route. Update cmd/silo to build the v2 poll task, seeding its interval from Settings.DefaultPollIntervalSeconds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): enforce connection requires own URL or a Requests link Migration 172 dropped the DB CHECK that required an autoscan connection to carry either its own base_url or a request_integration_id, delegating that invariant to the application layer — but the enforcement was never added, so HandleCreateConnection/HandleUpdateConnection accepted both-NULL orphans that ConnectionResolver.Resolve would hand a plugin as an empty base URL. Add a shared validateConnectionInput helper (whitespace-only request_integration_id counts as absent) and reject both-empty payloads with HTTP 400 on both the create and update paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): deliver resolved connection to plugin PollChanges now populates PollChangesRequest.Connection with the resolved {base_url, api_key} instead of dropping the conn param on the floor. Drops the stale doc comment claiming the connection was delivered out-of-band at upsert time -- that mechanism never existed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): auto-discover sources from installed scan_source plugins Auto-discovery seeds a disabled, connection-less source row per installed scan_source.v1 capability before an operator binds a connection, so connection_id is now nullable end to end: - migration 172: connection_id drops NOT NULL (still ON DELETE RESTRICT) - Source.ConnectionID becomes *string; repository scans/writes it as nullable and adds idempotent EnsureSource (INSERT ... ON CONFLICT DO NOTHING) - new ScanSourceLister seam + Service.DiscoverSources, called at the start of PollOnce (errors logged, non-fatal); production adapter enumerates ListEnabled -> ListCapabilities filtered to scan_source.v1 - PollOnce skips an enabled source with no connection bound, recording 'no connection bound' so the UI can surface it - HandleUpdateSource rejects enabling a source with no effective connection (400); source DTOs expose connection_id as nullable - BuildAutoscanService / NewService thread the installation store at both wiring sites (router + poll task) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): honor per-source poll interval PollOnce now skips an enabled source that ran too recently: the floor is source.PollIntervalSeconds when set, else settings.DefaultPollIntervalSeconds. The global poll task fires at the default cadence, so this makes the per-source interval a 'poll at most every N seconds' floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): reconcile spec + arr-plugin plan with credential-in-request + auto-discovery The credential-delivery mechanism changed during execution: the host now passes resolved {base_url, api_key} in PollChangesRequest.connection each poll (not plugin runtime config). Also records source auto-discovery, nullable connection_id, and the per-source interval floor decided at the final integration review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan v2 types and query hooks Replace v1 autoscan types and hooks with v2 DTOs matching the backend handler (autoscan.go): settings, connection (with has_api_key, no raw key), source (installation_id/capability_id/connection_id), status. Add connections CRUD hooks, useAutoscanStatus, update sources hook to v2 input shape. Retain deprecated shims for AutoscanPathRewrite, AutoscanRewriteSuggestions, and useAutoscanRewriteSuggestions so AdminRequests.tsx continues to compile until Task 6 removes that tab. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan connections panel (reuse or own) Card+Table listing connections with "Reused from Requests" / "Own" badges. Add/edit dialog with two modes: reuse a Sonarr/Radarr Requests integration or enter own name/URL/API-key credentials. Delete with alert-dialog confirm. Never renders key material — only has_api_key is sent by the backend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan sources panel Table of auto-discovered scan sources (one row per installed scan_source plugin capability). Operator can bind a connection via inline Select (auto-saved on change), set a per-source poll interval (saved on blur), and toggle enabled. Shows a "Needs connection" badge for unbound sources; attempting to enable without a connection lets the backend 400 surface via the existing toast in useUpdateAutoscanSource.onError. Status column shows last_run_at relative time or last_error with icon. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): standalone Autoscan admin page Tabs page (Sources | Connections | Settings) mirroring AdminRequests header/layout. Settings tab exposes global enable switch, default poll interval, and debounce — all auto-saved on blur or toggle. "Run now" button calls useTriggerAutoscan and toasts "Autoscan triggered" on 202. Route and sidebar nav are intentionally deferred to Task 5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): route and sidebar nav for Autoscan category Add /admin/autoscan route pointing to AdminAutoscan and a matching "Autoscan" item in the Content group of the admin sidebar (with RefreshCw icon), so the new standalone page is reachable from the nav. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(web): move Autoscan out of Requests into its own category Remove the Autoscan tab, AutoscanTab/AutoscanSourceEditor component definitions, and AutoscanSettingsFormState from AdminRequests.tsx. Delete the Task-1 compatibility stubs: AutoscanPathRewrite and AutoscanRewriteSuggestions types from api/types.ts, and the useAutoscanRewriteSuggestions no-op shim from useAutoscan.ts. The build confirms zero dangling references. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): allow unbinding a source connection (full-state source update) Change the source-update input struct's connection_id from string to *string so the UI can send null to unbind, a UUID to bind, or omit (null) to clear. Remove the fall-back-to-existing logic; the handler now sets the source's ConnectionID directly from the input. The enable-guard fires when the resulting connection is nil regardless of cause. Frontend sends the complete triple (connection_id, enabled, poll_interval_seconds) on every mutation site; selecting "— No connection —" sends null for a real unbind. Adds aria-label to connection Select and interval Input for accessibility. Backend tests cover bind, unbind, unbind while enabled → 400, and enable without connection → 400. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(migrations): backfill autoscan v1 settings+connections instead of dropping Migration 172 unconditionally DROPped the shipped v1 autoscan_settings/ autoscan_sources (migration 171), losing an upgraded operator's enable flag, poll cadence, debounce, and arr server list — autoscan came back OFF. Rewrite 172 up to be non-destructive of what can be carried: rename the v1 tables aside, create the v2 schema, backfill settings (poll minutes -> seconds) and seed a reusable LINKED connection per distinct v1 source integration, then drop the renamed v1 tables. v2 sources are keyed on a plugin (installation_id, capability_id) that did not exist in v1, so they are left to runtime discovery; path rewrites move to plugin config and are intentionally not carried. Verified against a throwaway DB: after 171 + v1 seed data, applying 172 yields enabled=true, default_poll_interval_seconds=300, debounce_seconds=30, and one autoscan_connections row linked to the v1 integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): preserve api key on metadata-only connection edit UpdateConnection unconditionally wrote api_key_ref = nullable(c.APIKeyRef), so a metadata-only edit (the UI omits the key when left blank — "leave blank to keep existing") NULLed the stored key and broke the next poll. Mirror requests' UpdateIntegration: api_key_ref = CASE WHEN $5 = '' THEN api_key_ref ELSE $5 END, passing the raw trimmed string so a blank incoming ref keeps the existing value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): skip orphaned sources + add source delete endpoint An enabled source whose scan_source plugin was uninstalled/disabled kept its autoscan_sources row, which errored every poll cycle, and there was no way to remove it. DiscoverSources now returns the set of currently-discovered (installation_id, capability_id) pairs; PollOnce skips any enabled source not in that set quietly (no RecordError), stopping the per-cycle error spam for orphans. A nil set (no lister / discovery failed) disables pruning so a transient discovery failure does not silence live sources. Adds DELETE /admin/autoscan/sources/{id} -> HandleDeleteSource -> repo.DeleteSource so an operator can clear orphans (unknown id -> 404). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reject reused connection when Requests integration is disabled RequestIntegrationLookup.Get returned a linked integration's base_url/api_key even when the integration was disabled or had a blank base_url (the v1 poll gate `WHERE ri.enabled = true` was dropped in v2). Now Get surfaces a disabled or unconfigured linked integration as an error, which the engine turns into a logged skip / RecordError instead of polling an unusable target. The gating is extracted into a pure checkRequestIntegrationUsable helper so it is unit-testable without a DB-backed repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): reschedule poll task on settings change HandleUpdateSettings no longer rescheduled the poll task (the v1 triggerUpdater / UpdateTriggers wiring was dropped in v2), so a default_poll_interval_seconds change only applied after a restart. Re-add an optional triggerUpdater (taskmanager.UpdateTriggers) on AutoscanHandler, wired via SetTriggerUpdater from the router when a task manager is available. On a successful settings update the handler recomputes the interval trigger from default_poll_interval_seconds and calls UpdateTriggers("autoscan_poll", ...). The dependency is optional: a nil updater skips rescheduling so tests need no task manager, and a reschedule failure is non-fatal (the interval is persisted). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): disable enable toggle for unbound sources, add source delete + interval hint - Disable the Enable switch when a source has no effective bound connection (connection_id null and no pending edit selection), re-enabling once bound. - Add useDeleteAutoscanSource hook mirroring useDeleteAutoscanConnection pattern. - Add per-row delete button (Trash2 icon → AlertDialog confirm) to let operators remove orphaned/unwanted source rows. - Add interval floor helper text showing the global default poll interval so operators know values below it have no effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): consume source_paths from merged scan_source contract The merged plugin SDK renamed PollChangesResponse.changed_paths to source_paths and the plugin now returns RAW source-namespace paths. pluginProvider.PollChanges reads GetSourcePaths(); the host applies per-source path rewrites before resolving/enqueueing (separate commit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(migrations): add path_rewrites to autoscan_sources Add path_rewrites jsonb NOT NULL DEFAULT '[]' to the autoscan_sources CREATE in migration 172 (unreleased/branch-only, so amended in place). The host now owns per-source prefix rewrites. v1 path_rewrites cannot be backfilled (v2 sources key on a plugin installation/capability with no v1 mapping); documented that operators must re-enter rewrites post-upgrade. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-owned per-source path rewrites Rewrite ownership moved from the scan_source plugin to the host. The plugin returns raw source-namespace paths; the host now normalizes separators and applies the source's per-source prefix rewrites before dedupe/resolve/enqueue. - types: add PathRewrite{From,To} and Source.PathRewrites - rewrite: re-add applyRewrites/normalizeSeparators; apply the MOST-SPECIFIC (longest From) match, not first-match, so a broad rule can't shadow a nested one regardless of ordering - service.PollOnce: rewrite raw provider paths before resolveAndClaim - repository: marshal/unmarshal path_rewrites jsonb in UpsertSource and all source scans (EnsureSource discovery rows take the DB default []) - handlers: autoscanSourceInput/response + status DTO carry path_rewrites (full-state like connection_id); reject blank from/to with 400 - tests: rewrite unit tests, engine applies rewrites before enqueue, handler round-trips path_rewrites and 400s on a blank rewrite Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): discover installed scan_source plugins on sources-list view A scan_source plugin installed via the normal /admin/plugins flow must show up in the Autoscan component immediately, not only after a poll cycle (which runs only when autoscan is enabled). HandleListSources now runs discovery (seeding a disabled, connection-less source row per installed scan_source capability) before listing. Best-effort: discovery failure does not block listing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): per-source path rewrites editor + plugins-page install hint Add AutoscanPathRewrite type and path_rewrites fields to AutoscanSource/ AutoscanSourceInput. SourcesPanel gains an expandable rewrite editor per source row (from→to pairs, Add/Remove/Save) threaded into the full-state body so connection, interval, and rewrite changes always carry all fields. Adds a Plugins-page install hint in both the empty state and above the table for discoverability. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): host-owned path rewrites + install/discovery flow Reconcile the spec with the merged SDK decision (rewrites moved host-side; PollChangesResponse.source_paths carries raw provider paths). Document that scan-source plugins install via the normal /admin/plugins page and surface in Autoscan via discovery (run on poll cycles and on sources-list view). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: depend on merged silo-plugin-sdk via pseudo-version (drop local replace) PR #2 (scan_source.v1 + source_paths) is merged to silo-plugin-sdk main, so the host can resolve the canonical module at the merged commit (v0.4.1-0.20260603030807-807b07e785b2) instead of a local-path replace. The branch now builds off-machine (CI/Docker). Bump to a clean v0.5.0 once tagged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(migrations): single clean autoscan v2 migration (v1 never shipped) The v1 in-process autoscan (migration 171) was never released to origin/main, so no live system has v1 autoscan data to preserve. Collapse the v1-create + v2-rename/backfill/drop dance into one clean 171 that creates the v2 connections-based schema directly. Removes 172 entirely. The runner applies by version set-difference with no checksum validation, so the already-migrated test instance (171+172 recorded) skips both and is unaffected; fresh installs get the clean v2 schema in one step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): allow many sources per plugin + add-source enumeration Drop the one-source-per-(installation, capability) model. A single installed scan_source plugin capability can now back many sources, each bound to a different connection (e.g. one Sonarr plugin fronting four arr servers). - migration 171: remove the autoscan_sources UNIQUE(installation_id, capability_id) constraint; sources are operator-created, not auto-seeded. - repository: replace UpsertSource (relied on the unique conflict) with a plain CreateSource (fresh uuid) + a by-id UpdateSource; remove EnsureSource. - discovery: replace auto-seeding (DiscoverSources/RefreshDiscovered) with ListAvailableScanSources (the Add-source picker list, enriched with plugin id + display name) and an installedScanSources set used only for orphan-skip. - service: PollOnce stops seeding and instead fetches the installed-capability set for orphan detection; Store gains GetSource and drops EnsureSource. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): connection test endpoint (engine) Add Service.TestConnection / TestConnectionByID: resolve a connection (ad-hoc input or an existing stored connection) to concrete credentials and probe the arr GET /api/v3/system/status with a short timeout. A reachable/authorized target yields OK=true plus the reported version; an unreachable / 401 / non-200 target yields OK=false with a human-readable error (the probe failure is part of the result payload, never an error from the method itself). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): host-side rewrite suggester + admin API for new endpoints Port the path-rewrite suggester back host-side (it had moved into the plugin): suggestRewrites suffix-matches arr root folders against Silo media folders to propose path rewrites, reporting proposed / unmatched / ambiguous / covered. Service.SuggestRewrites resolves the source's bound connection, lists arr roots (GET /api/v3/rootfolder) and Silo folder paths, and runs the matcher; a source with no bound connection returns ErrNoConnection (400). Admin API (all admin-gated): - POST /admin/autoscan/sources create a source - GET /admin/autoscan/scan-source-plugins Add-source picker list - POST /admin/autoscan/connections/test probe a connection - GET /admin/autoscan/sources/{id}/rewrite-suggestions sync rewrites HandleListSources no longer auto-seeds; create validates the capability is currently installed and that enabling requires a connection. Wiring threads the arr root-folder/status client and the catalog folder lister through BuildAutoscanService; the lister now surfaces plugin id + display name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan hooks + types for sources, connection test, rewrites Add types and React Query hooks backing the autoscan admin UI batch: - AutoscanAvailableSource / useAvailableScanSources (scan-source plugins) - AutoscanSourceCreateInput / useCreateAutoscanSource (POST sources) - AutoscanConnectionTestResult / useTestAutoscanConnection (advisory test) - AutoscanRewriteSuggestions / useAutoscanRewriteSuggestions (on-demand) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): add-source dialog + sync-from-arr rewrites in SourcesPanel Add a "+ Add source" header action opening a dialog that creates a scan source from any installed scan-source plugin bound to an arr connection, so operators can add one source per connection (e.g. four arr instances). Empty state links to /admin/plugins when no plugins are installed. Add a "Sync from arr" button to each source's rewrite editor that fetches root-folder rewrite suggestions and renders a preview: checkbox-selectable Proposed rewrites plus collapsed Unmatched / Ambiguous / Already-mapped sections. "Apply selected" merges the checked rewrites (dedupe by `from`) and persists via the normal full-state source PUT. Sync is disabled until the source has a bound connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): test-connection button in autoscan ConnectionsPanel dialog Add an advisory "Test connection" button to the add/edit connection dialog. It probes the current dialog input — connection_id when editing, request_integration_id in reuse mode, or base_url/api_key_ref for own credentials — and renders the result inline: green "Connected (vX.Y)" on success, red error on failure. Never blocks save; stale results clear when credential fields change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): autoscan page polish + global enable toggle in header Surface a global Autoscan enable toggle and an enabled/disabled status badge next to the page title, alongside the existing "Run now" header action so primary controls are reachable without opening a tab. Remove the now-redundant enable switch from the Settings tab (it points at the header toggle instead). Tighten header layout for wrap on narrow widths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): right-align autoscan enable toggle + Run now in the page header Drop the redundant nested justify-between wrapper so the header actions sit directly under .page-header (space-between + bottom-align), matching the /admin/libraries header layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): hold poll marker when paths return but none resolve A freshly-enabled source whose path_rewrites aren't configured yet returns provider paths that resolve to zero library folders. PollOnce previously advanced the marker unconditionally on any successful poll, permanently skipping those imports. Now the marker advances only when there is nothing to do (zero paths) or at least one path resolved+enqueued; when paths come back but none resolve, the marker is held and an explaining error recorded so the operator can fix the rewrites and a later poll re-reads the same window. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): don't prune sources of disabled-but-installed plugins PluginScanSourceLister used the installation store's ListEnabled, so a temporarily-disabled plugin dropped out of the discovered set and PollOnce treated its sources as orphaned, skipping them with no last_error (silent vanish). Switch to List so only a fully-uninstalled plugin counts as orphaned; a disabled-but-installed plugin's sources are still attempted and surface a visible RecordError when the client fails to load. The Add-source picker shares the same all-installed set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): treat empty request_integration_id as no link ConnectionResolver.Resolve gated the linked-integration path on a non-nil RequestIntegrationID pointer, so a pointer-to-empty-string (from a both-NULL orphan or a stripped link) called requests.Get(""). Guard on a non-empty trimmed value so it falls back to the connection's own fields instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): align startup poll interval with reschedule computation Startup seeded the poll task by integer-dividing default_poll_interval_seconds by 60 (minutes), while HandleUpdateSettings reschedules with seconds*1000 ms; the two diverged for sub-minute and non-60-multiple intervals. NewAutoscanPollTask now takes the interval in milliseconds and main.go seeds it as seconds*1000, matching the reschedule path so both agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): normalize stored rewrite From at poll time applyRewrites matched the stored From after only TrimSpace/TrimSuffix, while suggest.go coveredBy normalizes via normalizePath (backslash->slash, collapse '//'). A Windows-style or dup-slash stored rewrite was thus reported 'covered' at suggest time yet never matched at poll time. applyRewrites now normalizes From through normalizePath so poll-time and suggest-time agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(web): don't corrupt source poll interval on enable/connection change Add a `parseInterval` helper that maps empty input to null (use global default), valid positive integers to the integer, and any other mid-edit-invalid value to the source's currently-persisted `poll_interval_seconds` — so toggling the enable switch or changing the connection cannot silently overwrite the interval with 0 or NaN. Wire the helper through `fullBody()` (the single source of truth for PUT payloads) and remove the two inline duplications in `handleConnectionChange` and `handleRewriteSave` that both previously used raw `Number()`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): make the source connection optional (provider-agnostic) A host connection is the credential/endpoint for server-based providers (Sonarr/Radarr); other scan_source providers (e.g. a CephFS/filesystem watcher that reads ceph.dir.r* xattrs) need none. PollOnce now polls connection-less sources, passing an empty ResolvedConnection the plugin may ignore; a plugin that requires credentials surfaces the error at poll time. Drops the enable-requires-connection 400s. Provider-specific config lives in the plugin's own global_config_schema, not a host connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): provider-agnostic autoscan copy + optional source connection Replace arr-hardcoded framing in AdminAutoscan, SourcesPanel, and ConnectionsPanel with neutral scan-source language. Remove the connection-required gate on the source enable toggle so connectionless providers (e.g. filesystem watchers) can be enabled; soften the badge from "Needs connection" to "No connection". Sync-from-server button remains gated on a bound connection (it needs a server to query). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: repo-relative paths in autoscan plans Replace local absolute filesystem paths (/opt/silo, sibling checkouts, /tmp/go/bin) in docs/superpowers/plans with repository-relative wording per CLAUDE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): rune-safe last_error truncation Truncate RecordError messages on a UTF-8 rune boundary so a byte-bounded cut can't split a multi-byte rune and store invalid UTF-8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): advance marker when resolved-but-suppressed (not unresolved) resolveAndClaim now reports resolvedAny (whether any path mapped to a Silo library folder, independent of suppression). PollOnce gates the "none matched a Silo library folder" hold+RecordError on !resolvedAny instead of len(targets)==0, so a poll whose paths resolved but were all debounced/suppressed advances the marker instead of being treated as a misconfiguration. Adds a regression test for the suppressed case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): normalize request_integration_id Trim whitespace and collapse empty-after-trim request_integration_id to nil on connection create and update, so a pointer-to-"" or " " is never persisted as a bogus Requests link. Also corrects a stale migration-172 comment to 171 (the collapsed migration number). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(autoscan): provider-agnostic poll-task copy Rename the poll task to "Autoscan poll" with a provider-agnostic description and progress message; drop Sonarr/Radarr/arr wording. Key() (autoscan_poll) is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(autoscan): fix typo in connectionless-source test name Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add scan source management * chore(deps): bump silo-plugin-sdk for structured scan source changes Pins silo-plugin-sdk to 0d78651, which adds source_config on PollChangesRequest plus the structured changes / ScanSourceChangeScope fields on PollChangesResponse that internal/autoscan/provider.go already consumes. Without this the branch fails to compile against the prior pin (807b07e). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): label scan sources by connection name in admin UI arr-plugin sources fan out one-per-connection under a single generic "arr" capability, so every row in the Sources and Activity panels rendered an identical "arr (plugin #N)" label. Lead with the bound connection name (Radarr/Sonarr/...) instead, demoting capability + plugin to a subtitle. Sources without a connection (e.g. cephfs) keep the capability fallback. Activity threads a source_id -> connection name lookup (built from the existing sources + connections queries) through the scan/poll tables the same way librariesByID is threaded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): spec for generic + operator-editable source labels Design for a shared label-resolution helper (operator label -> connection name -> manifest display_name -> capability_id) consumed by the Sources and Activity panels, plus an operator-editable per-source label backed by a new autoscan_sources.label column. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(autoscan): implementation plan for source labels Task-by-task TDD plan: migration 174 (label column), Go domain/repo/handler wiring with server-side normalization, shared frontend label helper, and Sources/Activity panel integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): migration for source label column * feat(autoscan): source label domain field + normalizer * feat(autoscan): persist source label in repository Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): accept, normalize, and return source label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): add label to source API types * feat(autoscan): shared source-label resolution helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): polish source-label helper per review Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(autoscan): label sources via shared helper + operator label input Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(autoscan): clarify source label naming per review * feat(autoscan): resolve activity source labels via shared helper Replace the sourceNames Map plumbing in ActivityPanel with SourceLabelLookups and delegate both name functions to resolveEventSourceName from @/lib/autoscanLabels, enabling the full label chain (operator label → connection name → manifest display_name → capability_id) for all Scan History and Poll log rows. * fix(autoscan): carry label on status source + guard poll label Final-review follow-ups: add the label field to the autoscanStatusSource response (and AutoscanStatusSource type) so the status view matches the source response per spec, and give pollSourceName a non-empty fallback for symmetry with scanSourceName. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(autoscan): resolve source aria-labels through the label chain Replace the legacy capability-only sourceLabel() helper with resolveSourceName() (operator label -> connection -> display_name -> capability). Row controls now announce the row's resolvedLabel (reflecting in-progress edits) and the delete dialog announces the resolved name, so screen readers hear "4K Movies" instead of "arr (plugin #4)". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(autoscan): paginate queue + history with a shared table pager Replace the card/table hybrid and 200-row "Load more" cap on the autoscan Activity panel with proper tables and real pagination. Backend: add offset + total-count to the scans/events list endpoints so history pages through the full set instead of a capped window. Extract shared event/scan WHERE-clause builders so list and count filter identically, and add CountEvents / CountAutoscanScans. Frontend: add a reusable TablePagination component (rows-per-page, "showing X-Y of Z", numbered window with ellipses, responsive) and reuse it for the server-paginated history (scans + polls) and the client-paginated live queue. Unify all three tables behind one DataTable shell so they read as one family. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> * fix(migrations): renumber PR 48 migrations * fix(migrations): tolerate stale device profile ids --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: fluxis <warmasterx555@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
QuickandGitHub
|
14b54cab0c | [codex] fix ASS subtitle font loading (#28) | ||
|
QuickandClaude Opus 4.8
|
e441d2d6e9 |
feat(subtitles): on-demand AI subtitle translation with live streaming
Add server-side AI subtitle translation backed by any OpenAI-compatible chat endpoint (OpenAI, Groq, a local Ollama/llama.cpp server). A viewer picks a source track and target language in the player; the server runs a bounded, resumable job pipeline that translates SRT/VTT cues in batches and streams them back over the realtime websocket so playback pauses, fills in cues near the playhead, and resumes. The finished track is persisted as an ordinary downloaded subtitle, so it reaches every client through the existing subtitle pipeline with no client changes. - Job lifecycle persisted in subtitle_ai_jobs (migration 168): enqueue with idempotency, bounded concurrency, progress/heartbeat, cancellation, and crash recovery. - New realtime events (subtitle_ready + subtitle_translation_*) with a per-session notifier; the player renders a synthetic "live" track fed by websocket cues. Timestamps never leave the server, so timing can't drift. - Admin settings card for endpoint / model / concurrency. Player + lifecycle hardening (from the code review of this feature): - Hand off from the live track to the persisted track on completion (selected by downloaded-subtitle id) and on the subtitle_ready broadcast, so the saved track survives a reload and a mid-stream socket drop. - Never persist the synthetic live-track sentinel index as a subtitle preference; restore the prior selection on failure; only auto-resume playback if the viewer was actually playing. - Resume promptly when the playhead is past the last cue; rebuild the live track on a new job; O(batch) live-cue ingestion instead of O(n^2). Reliability: - Root translation jobs in the application context so shutdown cancels them. - Heartbeat-based stale-job reaper (safe across multiple instances) replaces the table-wide startup reset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Silo Server Migration
|
04a7e8c82d |
Add online marker support for recap and preview segments
- Wire introdb marker fetching into playback and Jellyfin compat - Persist and expose recap/preview markers alongside intro and credits - Add new playback/profile settings for recap and preview behavior |
||
|
Silo Server Migration
|
c085b12fd1 | Initial Silo migration |