73488d1bfaf12c2ac2bc8a24ef7e04dbddfe06a7
88
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 QuickandGitHub
|
73488d1bfa |
feat(metadata): add original-language preferences (#526)
* feat(metadata): add original-language preferences * fix(settings): show metadata exceptions immediately * docs(settings): add metadata language screenshot * fix(settings): make language exceptions responsive * fix(settings): standardize language display names |
||
|
QuickandGitHub
|
d70b291bb8 | feat(settings): add shared language option catalogs (#521) | ||
|
|
dc4b9a0909 |
feat(settings): add the cross-platform settings contract and its manifest (#479)
* docs(settings): define the cross-platform settings contract Turns the audit in #376 into a decision-complete design for how user settings work across the server, bundled web client, Apple clients, and Android clients. Today there are three partial contracts - the server registry, the web client's own manifest, and independently owned key constants in each native client - and they have measurably drifted. The root enabler is that keyUsesUserScope returns true for any unregistered key, so a client can invent a production setting unilaterally and the server stores it as an unvalidated string. The design decides: Ownership. Every production user-facing setting needs a server-owned manifest entry, even when the value is stored only on one client. The single exception is private local.<client>.* diagnostics, bounded by five conditions. Types and scopes. Native JSON values instead of strings. Five remote scopes plus client_local, and each definition declares its own resolution order rather than inheriting a global precedence. Preferences versus restrictions. internal/policy already resolves max_playback_quality and metadata-language limits over the same controls this contract resolves preferences for. Definitions declare constrained_by, the effective response reports the permitted value alongside the user's stored one, and a mutation exceeding a restriction is stored rather than rejected - a capped 4K preference should take effect the day the cap lifts, not be destroyed by it. Compatibility. Widening a scope, adding an enum member, or widening a range is additive and revision-tagged; narrowing anything needs a new key. introduced_in is a manifest revision attached to individual enum members and scopes, not just whole definitions, so a newer client never offers a choice an older server will reject. Rollout. One coordinated breaking release, with no compatibility shim, projection, or client fallback. After the cutover no future setting requires coordination. No settings version check goes in the authenticated middleware and nothing returns 426: deleting the old routes already produces the break, and a gate would be more code in four repos for the same outcome while permanently coupling every endpoint to one subsystem's versioning. Scope placement. Appearance and date/time move from account to profile scope. Account scope was an artifact of pre-profile storage; leaving it there means a household shares one theme and text size, and any non-child profile can restyle everyone else. Read path. Batched context resolution, index requirements, a session-snapshot rule, and a no-regression benchmark gating storage consolidation - profile_series resolution is per-item, so a season view would otherwise issue one request per episode. Verified against the current server, Apple, and Android implementations. Two findings shape it: the unknown-key extension bag is real, and v1 scope reads NOT LOCKED, so removing the legacy surface needs no amendment if it lands before lock. Related to #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical settings contract manifest First implementation step for the cross-platform settings contract (#376). Adds the artifact everything else depends on: the manifest, its JSON Schema, the object value schemas, and a Go loader that validates the whole thing at load time. No routes, no storage, no behavior change — nothing reads this yet. contracts/settings/v1/ holds the artifact at a stable path because clients vendor it and generate bindings from it. The embed directive has to sit beside it (go:embed cannot reach outside its own directory), so that directory is a tiny Go package containing nothing else; loading and validation live in internal/settingscontract. 38 definitions: 35 remote, 3 contract-known client_local. That covers every key the legacy registry accepts, every unregistered key the extension bag was silently accepting from the web client, every unregistered device key Android writes, and the profile preference columns that become settings. Registering the previously-unregistered keys is where the drift shows up, and the manifest records each case in a notes field: - ui_theme, ui_text_scale, ui_text_weight, ui_high_contrast, ui_custom_theme_vars, and ui_custom_css reached the server only because keyUsesUserScope returns true for any unregistered key. They are now typed, renamed to the dotted convention every other key uses, and moved to profile scope per the design. - player.match_frame_rate and player.sleep_timer_default_minutes are written by Android against a server that does not register them, so every write and reset is currently rejected. Registered. - player.next_up_prompt_seconds is Android's alias for playback.next_up_prompt_seconds and does not become a definition; the test matrix pins it as a migration alias. - player.playback_speed is capped at 3.0, matching the server rather than Android's 4.0. - subtitle_appearance becomes playback.subtitle_appearance. Every other canonical key carries a domain prefix, and preserving accidental key names is an explicit non-goal of the design. Validation is deliberately stricter than the schema can express. Beyond shape, it enforces that a resolution order ends in "default", that it only resolves scopes the definition allows, and — the one most likely to bite — that every writable scope is actually read, so a setting cannot accept writes at a scope it will never honor. Defaults are validated against their own value schema, so a default that violates its own range or enum fails at load. Revision tags are checked to never run ahead of the manifest revision, which is what makes revision-aware client filtering trustworthy. Ceiling and floor policy constraints are rejected on unordered types, where capping would silently do nothing; playback.preferred_quality's enum is therefore ordered ascending. ValidateValue is the single validation path, so the mutation endpoint, the migration, and the manifest's own default checks cannot diverge later. Numbers decode through json.Number so an integer setting rejects 30.5 rather than truncating, and object values validate against their referenced JSON Schema instead of accepting arbitrary JSON the way validateJSONSetting does today. Canonicalization implements RFC 8785 over the value domain the contract uses: sorted keys, no insignificant whitespace, ECMAScript number formatting. The digest is the ETag, and PublicBytes strips maintainer notes so the served manifest never carries internal commentary. Promotes santhosh-tekuri/jsonschema/v6 from indirect to direct. Verification: 124 tests pass across 16 cases; golangci-lint clean; make verify-local-paths passes. Two failures in internal/api/handlers (TestRemoveJellyfinCompatWebDisablesWebSetting, the playback v3 seek recovery test) reproduce unchanged on main and are unrelated. Part of #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): give ui.theme a device override Theme joins text scale, text weight, and high contrast as a profile default with an optional per-device override, resolving profile_device -> profile -> default. The right theme is partly a function of the screen and the room — a light theme on a phone in daylight, a dark one on a TV at night — which is the same reasoning the other three appearance keys already used. All four appearance settings now cascade consistently, which also means one rule to explain in the UI rather than "these three follow the device, that one does not". ui.custom_theme_vars and ui.custom_css stay profile-wide. They are authored styling rather than a contextual preference, so a profile's custom tokens still apply on top of whichever theme a device resolves to. Recorded in the definition notes because it is a visible consequence: vars tuned against a dark theme will sit on top of a light one if a device overrides the theme. Widening those to profile_device later is an additive revision bump if it turns out to matter. Part of #376. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(web): tag local appearance caches with their owning account The theme, text scale, text weight, high contrast, custom theme variable and custom CSS caches in localStorage were untagged, so on a shared browser a second account inherited the first account's appearance: with no server value of its own, every fallback resolved to whatever the previous account had stored, and the leftover `silo-theme` key also suppressed the admin-configured default theme for the new account. DateTimeFormatProvider already solved this by stamping its cache with the authenticated user id and refusing another account's values. Extract that mechanism into `createOwnedCache` in utils/storage.ts (where key namespacing lives) and put all three groups behind it, so appearance and custom theme get the same protection instead of a third copy of the rule. - Each group carries its own owner stamp. A shared stamp would be unsafe: the groups are written by hooks nested inside each other, and effects run inner-first, so whichever hook stamped first would vouch for the other's still-stale values. - A null owner (auth bootstrapping, or signed out) still trusts the cache, which keeps the warm start and the login screen's last look. - An unstamped cache is not trusted once an account is known, so existing users take a one-time appearance reset on first load rather than a chance of seeing someone else's settings. - When a foreign cache is detected the values are dropped and the empty cache is handed to the new account, so a later single save cannot re-trust the rest of the previous account's state. Owner is the user id because /settings is user-scoped server side; it lives in one helper (`appearanceCacheOwner`) so it can be widened if appearance moves to profile scope. `shouldLoadApiTheme` is gone: it had become a synonym for `appearanceCacheOwner(...) !== null` with no callers left. Part of #376 AI-use disclosure: implemented with Claude Code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): make the settings contract enforceable and fix the appearance cache The contract manifest landed as a document nothing checked. This makes it a mechanism, and fixes the one defect in the change set that hurt users on merge rather than at cutover. Web appearance cache. useTheme cleared the cache for any account whose stamp did not match and never repopulated it — the only writers were the four user-action setters — so every upgrading user lost their warm start on every load, not once, and x-large-text and high-contrast users lost theirs too. The owner-stamp protocol is replaced with per-account key namespacing (`silo-theme:7`): a foreign value is absent rather than present-and-distrusted, so nothing has to be deleted, the first account keeps its warm start, and there is no shared stamp for a second tab, a stale debounce timer, or an out-of-order effect to race on. Widening ownership to profile scope, which this manifest requires, is now a change to appearanceCacheOwner alone. Adds the API-to-cache mirror useTheme was missing, cancels pending debounced writes across an account change, and re-seeds provider state during render so no frame paints the previous account's look. Canonicalization. writeCanonical used json.Marshal, which HTML-escapes < > and &, and canonicalNumber used Go's 'g' format — both diverge from RFC 8785, so the first label containing an ampersand or bound below 1e-4 would have forked the server's ETag from every conforming client. Output is now byte-identical to ECMAScript String() across the edge cases, verified against node. The ETag also covers the value schemas, which decide what the server accepts and previously could change while the tag stood still. All four derived representations are memoized; a conditional GET no longer costs a full parse and re-serialize. Validation. strictUnmarshal's decoder.More() answered false for a stray ] or }, so `true]` validated as a boolean. Enum matching compared fmt.Sprintf tokens, so the string "3" satisfied an integer member. Declared steps were never enforced. The language pattern rejected tags both mobile platforms emit unprompted (en_US, ca-ES-valencia, ar-EG-u-nu-latn) and never normalized case, so en-US and en-us were two rows for one preference; NormalizeValue now canonicalizes on the shared path. Manifest. show_forced_subtitles defaulted false where the server column is NOT NULL DEFAULT true, which would have turned forced subtitles off for every profile that never touched it. preferred_quality declared 13 members where the planner speaks 6 and collapses the rest to auto. metadata_language's allowlist was bound to the very column it migrates from. subtitle-appearance pinned fontFamily to three families while Apple stores any installed system font. Registers five user-facing settings the clients already ship, and corrects three notes that described Android behaviour that was not true. Enforcement. The package had no non-test callers, so MustLoad never ran; it now loads and logs at startup. The inventory test compared the manifest against a hand-copied map and could not see the drift it named; it now iterates settingsRegistry and checks defaults too — both verified to fail on injected drift. Adds .github/workflows/ci.yml, the repo's first CI that runs go test, go vet, gofmt, and the frontend suite. Known pre-existing failures are named individually in the Makefile so everything else stays gated and the list can only shrink. Part of #135 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): align the sleep timer default and range with the shipped client Android is the only client that implements this setting. It clamps to 0..240 and defaults to 30. The manifest said 0..480 with a default of 0, so a manifest-driven UI would have offered durations no client can store, and every user who never opened the picker would have had the preset silently turned off at cutover. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: give the new workflow the deps it actually needs The first run exposed two gaps in the workflow itself. go build ./... fails without libvips headers, because h2non/bimg binds libvips through cgo and pkg-config; the Dockerfile installs the same package. And pnpm/action-setup resolves its version from package.json, but there is no package.json at the repo root — the packageManager field lives in web/package.json, and a job's defaults.run.working-directory does not apply to an action's inputs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(web): stop the diagnostics download test depending on the Node version new Response(blob) reads the body through blob.stream(), which jsdom's Blob does not implement on Node 22 — the version the Dockerfile builds with. The test passed locally on Node 24 and threw "object.stream is not a function" in CI. Nothing in it asserts on the body, only that the object URL and filename reach the anchor, so a string body is equivalent and works on both. Surfaced by the CI workflow added in this branch, which is the first thing in this repo to run the frontend suite anywhere but a developer's machine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(build): copy the settings contract into the container build context Both Dockerfiles copy cmd/, internal/, migrations/ and web/embed.go, but the manifest lives in contracts/settings/v1 — an embedded Go package that sits outside internal/ because clients vendor those files. The image build therefore fails with "no required module provides package .../contracts/settings/v1". Caught deploying to the dev box. Nothing had built an image since the manifest landed: the Docker workflow only runs on pushes to main and workflow_dispatch, and CI's go build runs against a full checkout, so neither gate covers the container context. This would have broken the published image on merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): enforce the language-tag and step constraints the manifest declares A sweep of all 43 manifest definitions against the running server (160 checks: declared default, both boundaries, and deliberate violations for each remote key) found two places where the live registry accepts what the contract forbids. Both are fixed by calling the contract's own validators rather than adding a second implementation. playback.audio_language was checked as "32 characters or fewer", so the server stored "!!!" for a field the manifest declares as language_tag — a value track matching would then silently never match. It now requires a well-formed tag via settingscontract.NormalizeLanguageTag. The empty string is still accepted: the string-only endpoint has no way to send null, and both Android and web send "" to clear the choice, so rejecting it would break clearing the preference. player.playback_speed declared step 0.05 and nothing enforced it, so 0.26 was stored — a value no client's stepper can represent and that every client would silently snap on the next write. settingscontract.StepAligned is now exported and used by both the contract validator and the registry, so there is one definition of "on step" rather than two that can drift. This gives the contract its first production consumer beyond the startup load, which is the direction Phase 2 continues in. Also fixes a genuinely flaky test that the new CI gate would have hit intermittently: TestRemoveJellyfinCompatWebDisablesWebSetting used t.TempDir as the install root, but the endpoint returns 202 and its goroutine keeps writing there after the test body returns, so cleanup tripped "directory not empty" roughly one run in four. Confirmed pre-existing and unrelated to settings; the suite now passes six consecutive full-package runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): keep widened numeric bounds resolvable at older revisions A bound was one scalar plus the revision that introduced it, which discards the value it replaced. Widening a maximum from 240 to 480 at revision 3 left a revision-3 client with no correct answer against a revision-1 server: honoring 480 offers values that server rejects, and filtering the tagged bound out leaves the setting unbounded. Since clients are specified to filter their pinned contract against the server's advertised revision, the bound has to carry what it used to be. Bounds now hold their full history, oldest first, and AtRevision hands back the limit a given peer actually enforces. A bound nobody has widened still serializes as a bare number, so the manifest reads the same and untouched entries do not churn the ETag. Validation gains the rules the representation makes checkable: a maximum may only grow and a minimum may only shrink, history is strictly ordered, later entries must say when they arrived, and the first entry cannot predate the definition. That last rule is the lower bound allowed_scopes already enforced; the same gap is closed for enum members, which could previously claim to predate the definition containing them. Reported by Codex review on #479. * fix(settings): accept the partial subtitle appearance objects already stored The schema required all nine properties, but the current API accepts and round-trips sparse objects — settings_device_test.go stores {"fontSize":"xxlarge"} and reads it back — and the web client has always merged whatever it gets over DEFAULT_SUBTITLE_APPEARANCE. Requiring the full object would have made the cutover migration quarantine preferences users really set, or block on them. Every property is now optional and a stored value is documented as a sparse override merged over the definition's complete default. An empty object is still rejected: an override that overrides nothing is the same state as no override, which the contract represents as unset. Cross-scope resolution is deliberately unchanged. A device override still replaces the profile's object rather than merging into it, because a device override means "draw subtitles this way on this screen", not "amend the profile" — and that is what the server does today. Reported by Codex review on #479. * fix(jellycompat): scan the parent directory when a sidecar changes Autoscan matched scantrigger rejections by comparing RequestError.Message against literal strings. One of those messages became "Unsupported media file extension for library type" and the copy in handlers_autoscan.go did not, so the comparison silently stopped matching. The effect is user-visible: a Jellyfin client posting a change for Movie.nfo or poster.jpg gets a 400 and the batch is abandoned, when the sidecar should have resolved to a scan of the directory containing it. Three tests covered exactly this and had been excluded rather than read. RequestError now carries a Reason the caller can switch on. Message stays prose for the client reading the response — it is meant to be reworded, and nothing should break when it is. Also makes two tests honest about asynchronous work. The Jellyfin Web teardown deleted its install root while the operation goroutine was still writing to it, where a late write recreates a path RemoveAll already walked past; it now waits for the operation's terminal state, which required exporting CurrentWebOperation. And the direct-play If-Range test pinned size and mtime so ctime was the only remaining validator, then read it back inside a single coarse-clock tick — it failed about 85% of the time on main for a reason unrelated to what it tests, and now rewrites until the stamp moves. With those fixed, GOTEST_KNOWN_FAILURES is empty and gone: make test-go runs the whole Go suite. The one test that cannot pass yet — TestHandleReplanPlaybackV3SeekFailureRecoveryNeverChangesMediaVersion, which has failed since the commit that introduced it and describes unimplemented v3 planner behavior — carries a t.Skip explaining that where the test is, rather than a regex in the Makefile. Reported by CodeRabbit review on #479. * fix(settings): reject JSON the decoder would otherwise rewrite Two cases where encoding/json accepts input by quietly changing it, which is the one thing a contract promising byte-identical agreement between peers cannot tolerate. Duplicate object properties. jsonschema.UnmarshalJSON keeps the last occurrence, so {"fontSize":"small","fontSize":"large"} validated and stored "large". Which one wins is a property of the parser, not of the contract: a client generated against a different JSON library can disagree about what it just sent, and the canonical form cannot represent the duplicate at all. Lone surrogates. An unpaired \ud800 became U+FFFD and canonicalization reported success, so the server would issue canonical bytes and an ETag for an artifact a conforming implementation must refuse — RFC 8785 requires terminating here. Substitution also means the value read back is not the value written. Both checks run before the decode that would hide them, on the shared decodeJSON path that the manifest, its public projection and every value schema go through, and again on the object branch of ValidateValue, which uses a different decoder. Reported by Codex review on #479. * ci: gate Go lint on the lines a branch changes AGENTS.md told contributors CI ran the same checks as `make lint`, and the Go job ran only gofmt and vet. A change failing the documented Go lint gate passed all three jobs. Running the linter as-is is not an option: the tree has ~296 findings today, which is why this half of `make lint` was never enforced. Blocking every PR on a cleanup nobody has scheduled gets the gate deleted again, so CI runs with --new-from-merge-base and only the lines a branch touches have to be clean. The count can then only fall. golangci-lint is built from source at a pinned version rather than downloaded. A released binary refuses to run against a Go newer than the one it was built with, and go.mod here tracks Go closely enough that the current release already fails that way on 1.26.4. .golangci.yml declared version 2 while still using v1's issues.exclude-rules key. Current golangci-lint ignores it, so the "allow repeated strings and unchecked cleanup errors in tests" exclusions silently did not apply — 16 findings in test files that the config says to skip. Moved to linters.exclusions, which `golangci-lint config verify` accepts. The four lines this surfaced in scantrigger are fixed rather than excluded: its repeated status codes and messages are now named constants, so one condition cannot end up worded two ways. Also drops the workflow token to contents:read and stops persisting credentials in the three checkouts, neither of which any job needs. Reported by CodeRabbit and Codex review on #479. * docs(v1): record the settings removal as a pre-lock exception The design removes the legacy /api/v1/settings routes and the profile DTO preference fields, while AGENTS.md states /api/v1 is additive-only and removals go through Deprecation/Sunset. Read together those contradict. They do not actually conflict: v1-scope.md scopes the additive-only rule to "when the scope locks", and the scope is still open, so a removal taken now is in scope and there is no amendment process to invoke yet. But that reasoning lived only in the settings design, where nobody checking the API policy would find it. v1-scope.md now carries a pre-lock removals table naming what goes and why waiting is worse, and states the deadline the argument depends on: a removal listed there must ship before lock or fall back to Deprecation/Sunset. AGENTS.md points at the table and says to treat an unlisted removal as a mistake. Reported by CodeRabbit review on #479. * fix(settings): clear the remaining review findings Small, unrelated except that each was raised on #479. compileObjectSchemas parsed every non-directory file under schemas/ as a JSON Schema, so a stray editor backup or .DS_Store would panic the server at startup through MustLoad. schema_ref can only name a .json file; anything else is skipped. cmd/silo used MustLoad while the ETag check beside it and every other startup failure use log.Fatalf. It now fails the same way, so a bad contract prints an error instead of a stack trace. TestRegistryDefaultsMatchTheContract called scalarDefault before handling null, and scalarDefault rejects null as non-scalar — so the subtest skipped and the comparison after it was unreachable. A nullable contract default could disagree with a non-empty registry default and nothing failed. Confirmed by injecting that drift, which now reports it. The three appearance providers each adapted the auth context to AppearanceAuth with identical code, putting the shape of auth back in three places that widening cache ownership would have to find. useAppearanceCacheOwner now does it once. useTheme.test.ts cleared storage.KEYS between cases, but appearanceCache writes namespaced keys and an owner pointer that are not in that list, so both survived and the suite was order-dependent. It clears the store, as storage.test.ts already did. The abs_smart_collection_store comment is reworded rather than given back its SQL quotes: gofmt folds a pair of apostrophes in a doc comment into a typographic quote, which is how it became one in the first place. Reported by CodeRabbit review on #479. * feat(settings): add canonical typed storage for the settings contract The cross-platform settings contract needs one typed store behind it before a resolver, routes or a migration can exist. This adds that storage to both user-store backends and holds them to identical behavior. PostgreSQL gets user_setting_values with the scope CHECK constraints, the five partial unique indexes that enforce one explicit value per identity, and the covering indexes the one-query read path needs, plus user_setting_mutations for mutation_id idempotency and the inert user_setting_migration_rejects audit table. The per-user SQLite store gets the same shape minus user_id, since that database is already user-scoped. The UserStore interface grows the typed operations: read one explicit value at one scope, collect every candidate row for a resolution request in a single query, upsert with a revision increment, unset, and the idempotency receipt operations. The resolution read deliberately returns unranked candidates so the resolver can rank in Go — one query per request, never one per scope, which the pgx query-count test pins. Delete behavior is application-enforced. Neither backend can inherit it from constraints: the SQLite store declares no foreign keys, and library, series and device columns are not FK targets in Postgres either. Profile deletion cascades to profile-anchored values while account scope survives, forgetting a device clears its profile_device values alongside the legacy overrides, and the library/series purges remove only what is scoped to that entity. The shared conformance suite covers all of it, including the set-versus-unset distinction for false, 0, "" and null, so a divergence between the two backends fails a test rather than reaching a client. Part of #376 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(settings): pin the settings-value schema constraints in both backends Completes the storage track. The conformance suite exercises the store API, which validates identities in Go before any SQL runs — so nothing noticed whether the CHECK constraints and partial unique indexes actually existed. The one-time migration writes these rows in bulk without going through the per-request path, so the schema is the only thing guarding it. Adds constraint tests to both backends covering every scope's column requirements, rejection of an unknown scope, a profile that does not exist, non-JSON values, and each of the five partial unique indexes. Also clears the lint the storage commit did not get to: sql.ErrNoRows and pgx.ErrNoRows compared with == rather than errors.Is (which fails on a wrapped error), an unchecked rows.Close, and repeated fixture literals in the shared suite now named so a backend that confuses two scope columns fails on the assertion rather than on a typo. * fix(settings): close the review findings in the validator and the theme cache Four defects the existing tests did not reach. The web theme resolver compared the server's value against the appearance cache and fell back when they agreed, but the mirroring effect writes the server's value into that same cache — so the comparison held on the first render and stopped holding on the second, reverting an explicitly chosen theme to the default. The server's value is this account's own stored choice, so it now simply wins. The regression test re-renders rather than asserting on the first paint, which is why the original one passed. golangci-lint's exclusions.paths is a path regex, not a directory list, so a bare `web` also excluded internal/jellycompat/web_component.go, internal/webhooksync/, internal/notifications/webhook*.go and eleven other non-test files that were being linted before. Anchored. json.Number is a string kind, so `"1.5"` unmarshalled into it happily and Float64 parsed the quoted digits: a numeric setting validated as a JSON string and NormalizeValue stored the quoted form into jsonb. Rejected. The lone-surrogate check ran only on the object branch, so a lone surrogate in ui.custom_css decoded to U+FFFD on SQLite and was refused outright by Postgres jsonb — the two backends disagreeing about whether the same value could be stored. Hoisted to cover every type. The strict language-tag validation this branch added is correct, but it rejects what the shipped Android client sends; the companion fix is silo-android 4aeb78b4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(auth): stop TestJWT_TamperedToken passing a valid signature The test overwrote the last character of the signature with "X". An HMAC-SHA256 signature is 32 bytes, so its base64url encoding is 43 characters and the final one carries only four significant bits — U, V, W and X all decode to the same trailing byte. Roughly one token in sixteen was therefore left byte-identical and validly signed, and the test failed because ValidateToken correctly accepted it. Measured at 3098/50000 (6.2%) over distinct signatures; it just failed the Go job on this branch for reasons unrelated to the branch. Flipping a character in the middle of the signature is 0/50000. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): reject raw invalid UTF-8, not just escaped surrogates The previous commit hoisted the lone-surrogate check to cover every value type, but that only closes the escaped path. A raw 0xff byte inside a quoted string — what an HTTP body carries when a client encodes text in the wrong charset — is not an escape, so the surrogate scan never sees it, while encoding/json still substitutes U+FFFD and reports success. NormalizeValue then stores the original bytes, which SQLite's json_valid accepts and Postgres jsonb refuses: the same backend divergence, reached the other way. Found by the Codex review bot on the previous commit's own diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): size the library page state bound to what the web client writes ui.library_page_state's `search` was bounded at 256 characters. The web client serializes an advanced library view as URLSearchParams, encoding each filter rule as three groups[i][rules][j][field|op|value] keys — measured at 216 characters for one rule, 518 for three, 820 for five. The current endpoint validates this key by checking only that it parses, so those oversized values are already stored in production. Typing them at the declared bound would have failed the migration for anyone who had saved a view with more than one filter rule, and rejected the equivalent write afterwards. Raised to 4096, which clears ten rules with room to spare while staying a real bound. The test pins it against the key shapes libraryPageSearchParams.ts actually emits rather than a round number. Reported by the Codex review bot; the lengths above were measured by calling serializeLibraryPageSearchParams, not estimated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): split quality into two axes and register the orphan keys Two manifest changes the cutover needs. **Quality becomes resolution + bitrate.** The legacy ladder values (1080p-high, 720p-medium, 1080p-8, 420p, 328p) were never a third dimension — they are a bitrate spelled into the resolution string. The web player already decomposes them: useTranscodeQuality.ts defines 1080p-high as {resolution: 1080p, bitrate: 10000} and sends the two separately, so the compound form never reached the wire. Downloads went further and kept only a bitrate ladder. So playback.preferred_quality keeps the six clean resolutions and playback.max_bitrate_kbps becomes the second axis, nullable because "uncapped" is a real answer and a numeric sentinel would need widening every time hardware improves. Clients compose their own presets from the pair, which means retuning what "High" means is a client release rather than a contract break. Migration decomposes each legacy value losslessly, so none of them lands in the rejects table. **The five extension-bag keys are now definitions.** card_overlays, next_up_mode, sidebar_pins, disabled_library_ids and library_order reached the server only through the unknown-key path, stored as unvalidated strings. Two of them the server reads back — next_up_mode decides home section assembly and card_overlays falls back to an admin default — so they cannot be demoted to client-local. Registering them is what lets the extension bag close. Adds three schemas for their shapes and a test that exercises every schema_ref against a real value: each of these is nullable with a null default, so the existing default-validation test returns at the null branch without ever compiling the reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical resolution engine One answer to "what is this setting, for this profile, on this device, for this content". Before this, each caller carried its own ladder: catalog/detail.go resolved subtitles across four levels by hand and audio across three, handlers/settings.go had a two-level device/user resolution with a lazy write-back inside a GET, and jellycompat read profile columns directly. Those disagreed about precedence, which is the drift the contract exists to remove. Resolution is one batched read regardless of how many keys, libraries, or series are in play — ranking happens in Go against each definition's declared resolution_order. Five sequential index lookups per key per item is the implementation the design rejects, and a season view is exactly where it would have shown up. An absent identity drops its scope rather than erroring, so one code path serves an identified client, an anonymous jellycompat seed, and a batch spanning many series. Rows for a foreign profile, device, library or series are ignored even though the batched read returns them. Constraints narrow without destroying: a capped 4K preference resolves to the cap, reports itself constrained, and keeps the authored value so it takes effect the day the cap lifts. Two cases needed care — null on a nullable numeric means unbounded, so a ceiling must cap it rather than rank it equal and let the value that most needs capping slip past; and an allowlist falls back to a permitted member rather than the definition's default, which may itself be outside the list. Adds ValueSchema.CompareValues to the contract package, since ordering values is what makes a ceiling or floor mean anything and value semantics belong with the schema that declares them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the one-time migration planner The conversion rules from legacy settings storage to canonical values, as ordinary Go rather than twice in two SQL dialects. Both backends read their own rows, hand them to Plan, and write what comes back — so the decisions are testable without a database and SQLite and Postgres cannot drift apart in what they decide. The rules that needed care, each pinned by a test: Column defaults are not choices. quality_preference is NOT NULL DEFAULT '1080p' while the contract defaults to auto, so migrating the column unconditionally would pin every profile in the install to 1080p having never chosen it — and that stored value would then outrank the contract default forever. Same for language 'en', subtitle_mode 'auto', and show_forced_subtitles true. The empty string is unset, not a value. The legacy string API had no way to send null, so both Android and web spell "clear my choice" as "". Storing that would make a cleared setting outrank the default. Legacy quality decomposes rather than rejects. Every compound value maps to a resolution and a bitrate from the ladder in useTranscodeQuality.ts, so nothing lands in the rejects table. Account rows fan out to every profile, which is the account-to-profile move the contract makes for appearance and search scope: a household that shared one theme each end up owning theirs. Legacy strings become typed JSON — "true" to true, "30" to 30 — or every generated binding would fail to decode what the migration wrote. Nullability differs per backend, so profile columns arrive as pointers and the caller resolves "chose the default" versus "never written" when it reads. jellycompat's DisplayPreferences blobs ride the same table under synthetic keys and are left alone; they are that subsystem's storage. Everything that cannot convert is recorded with a reason rather than dropped, and a final test asserts every planned row would be accepted by the mutation endpoint's own validation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): run the one-time migration on the SQLite backend Wires the planner to real storage as userdb migration V15. V14 created the tables; this fills them. It runs inside runMigrations' existing transaction, so a database either comes out fully migrated or untouched — a partial migration is the one state neither the operator's backup nor a rollback covers. Pinned by a test that rolls back and asserts nothing was left behind. Two things the wiring had to get right that the planner could not see: Reject identities are JSON. Postgres declares that column jsonb NOT NULL and SQLite guards it with a json_valid CHECK, so the free-form "profile=p1 device=d1" the planner emitted would have failed to insert — on exactly the rows the table exists to record. They are structured documents now, which is also queryable. Subtitle and audio preferences are two tables keyed the same way, so they merge into one per-series record before planning. Converting them independently would have produced two rows racing for the same identity. Every legacy read tolerates a missing table, since this runs against databases created at any schema version, and preferred_metadata_language is deliberately absent: that column exists only in the Postgres schema. Tested end to end against a real database rather than only through the planner — the rows land, satisfy the scope CHECK and the partial unique indexes, and hold valid JSON. Also covers the empty-install case and asserts a second run fails rather than silently doubling every value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): run the one-time migration on the Postgres backend The mirror of userdb V15, registered with goose as a Go migration rather than SQL: the conversion validates every value against its own definition and re-encodes it as typed JSON, and one legacy quality string becomes two rows — neither is expressible in SQL without duplicating the manifest. The rules stay in internal/settingsmigrate, so the two backends cannot disagree. RunTx, so the whole backfill lands in goose's transaction. The down migration empties the canonical tables; the legacy ones are never touched by the up, which is what keeps the cutover reversible until the follow-up migration drops the superseded columns. preferred_metadata_language is read here and only here — the column exists in this schema and not in SQLite's, so this is the sole source for catalog.metadata_language. Verified against a real Postgres: the full goose chain runs, 1080p-high decomposes to ("1080p", 10000), values land as typed jsonb rather than strings (jsonb_typeof reports number), rejects carry a queryable jsonb identity, and the composite profile foreign key refuses a row naming a profile that does not exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): add the canonical settings API The routes that make the typed storage reachable. Until now the manifest, the resolver and the migration all existed with nothing able to call them. GET /settings/contract serves the public manifest behind an ETag — clients vendor a pinned copy and generate bindings from it, so the common request asks "still the same contract?" rather than transferring it. Its capabilities sibling reports revision and supported scopes for feature detection instead of version sniffing. /settings/values/{key} reads, writes and clears an explicit value at one named scope, which is what a reset affordance needs: "did I set this here" is a different question from "what applies", and the old endpoint could only answer a blurred version of both. Scope comes from the query while profile and device come from session headers, so one profile cannot address another's settings by naming it. /settings/values/effective resolves any number of keys in one request, with the resolution ladder and the source of each answer reported so a client can offer "reset this device's override" against the exact row holding it. Asking for no keys returns every remote setting, which is what a settings screen wants. Writes are idempotent when a client sends X-Silo-Mutation-Id: a retry after a dropped response replays the receipt, and reusing an id with different content is a conflict rather than a silent overwrite of the wrong thing. Three things the string-only endpoint could not do, each pinned by a test: an unknown key is refused rather than stored in the extension bag, values are checked against their declared type and range, and a write to a scope the definition does not allow is rejected. Registered before the catch-all /{key} routes, which would otherwise swallow "contract" and "values" as setting names. The legacy endpoints stay live for now; deleting them is the next commit, once their consumers move. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): generate typed bindings for all four languages One generator rather than one per repo. The point of the contract is that four codebases agree on keys, types, scopes and defaults, and four independently written generators would be four chances to disagree. Go and TypeScript land in this repo; Kotlin and Swift are written into the sibling client checkouts, skipped with a note when they are not present so a server-only developer can still run it. Output is sorted by key so an unrelated manifest edit does not produce spurious diffs. The Kotlin output is the interesting one: it generates the DeviceSettings allowlist Android maintained by hand, plus the BOOLEAN_KEYS/INT_KEYS/ DOUBLE_KEYS classification it kept as a *second* hand-maintained table that had to agree with the first. Both are manifest questions now, so the whole class of "wrote a local key to the server" and "flushed a value the store could not parse" bugs stops being possible by construction. The TypeScript output carries the full definition table — labels, controls, enum members, bounds — so web/src/lib/settingsManifest.ts can be deleted rather than kept in sync: it declared 17 definitions against the contract's 49, with its own two-scope model that does not match the contract's five. make verify-settings-bindings fails when the committed output disagrees with the manifest, wired into CI, so a manifest change cannot merge leaving every client reading stale keys. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(web): add the two-axis quality picker and typed settings hooks Quality becomes one picker over two stored values. The server holds a resolution cap and a bandwidth cap independently, which is what the player has always sent on the wire — useTranscodeQuality.ts has decomposed 1080p-high into {resolution, bitrate} for as long as it has existed. Presets live in the client rather than the contract so retuning what "High" means is a one-line edit here instead of a contract change four codebases have to agree on, and an older server keeps working because it only ever sees the two axes it already understands. A combination no preset covers still gets a truthful label rather than a picker showing the wrong entry: reachable by setting the axes separately through the API, or from a legacy value whose bitrate is off this ladder. Choosing an uncapped preset clears the bitrate rather than storing a sentinel, so "no cap" stays the absence of a value at every layer. Adds hooks over the canonical API alongside the legacy ones rather than replacing them wholesale — a key that is not in the manifest cannot be expressed, because SettingKey is generated from it, and the default for an unset value comes from the generated table rather than a literal at the call site. That last part is what stops the flip-off bug the Apple client carries a hand-written guard for. A test asserts every preset composes values the contract actually accepts, so a preset naming a resolution outside the enum or a bitrate outside the declared bounds fails here rather than 400ing when a user picks it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): resolve catalog playback preferences through the contract catalog/detail.go held the two hardest ladders in the codebase: subtitles resolved across four levels by hand, audio across three, each partially overriding the last through Has* flags. Both now call the canonical resolver, so the precedence lives in the manifest and this file cannot disagree with the contract about which override wins. Adding a scope is a manifest change rather than another branch here. The subtitle track signature stays on its specialized table — it identifies a concrete track rather than expressing a preference, so it is not a setting. Resolution keeps the memoization the old lookups had: the audio resolver still reads once per profile and once per library rather than once per file, which is what kept a many-track audiobook detail page fast. The test that guards it now counts resolver reads instead of GetProfile calls, since the guarantee is about scaling with file count rather than about which method does the reading. Four tests seeded the profile column directly. That column is a migration source now, not a read path, so they seed the canonical value instead — they were passing against storage nothing reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): close the unknown-key extension bag keyUsesUserScope returned true for any key the registry did not know, so a client could invent a production setting unilaterally and the server stored it as an unvalidated string. That is how six ui.* settings and five orphan keys reached production untyped, and it is the root enabler the design names. An unknown key is no longer a user setting, so the legacy write path rejects it and the canonical API — which validates every value against its own definition — is the only way to store something new. jellycompat's DisplayPreferences blobs ride the same table under synthetic keys and keep working: they are that subsystem's storage rather than user settings, and they move to dedicated storage in the follow-up rather than being dropped here. Also repoints the DisplayPreferences seed at the canonical resolver. Resolved at profile scope with no device on purpose — Jellyfin clients do not carry Silo's device identity, so a device override leaking into the seed would hand one device's settings to every Jellyfin client on the account. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(settings): enforce viewer quality caps through resolver constraints constraintsFor was the unwired half of the preferences-versus-restrictions seam: it returned nil, so a profile capped at 1080p by policy still resolved its stored 2160p preference at face value through the effective endpoint. The settings routes are mounted inside RequireViewerAccess, so the resolved access scope is already on the request context. Scope.MaxPlaybackQuality holds a literal member of the contract's quality enum ("1080p"/"2160p"), which is exactly what the manifest binds playback.preferred_quality's ceiling to under policy_input "max_playback_quality" — so the wiring is a direct map with no translation table. An empty value means the policy sets no cap, expressed by returning nil so the resolver leaves the preference alone. catalog.metadata_language deliberately stays unconstrained: the manifest notes record that the allowlist draft was circular (the policy input it would bind to is populated from the very preference it would narrow). The handler test covers both halves of the seam: a 2160p preference under a 1080p cap resolves to the cap with constrained:true/ceiling and the authored value reported in stored_value, the stored row itself is not rewritten, and an uncapped viewer gets the preference unchanged with no constraint noise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): publish user_settings change events Add a user_settings realtime channel so clients learn when a setting changed on another device without polling. The channel is modeled on user_state: non-admin subscribable, per-user addressed envelopes, null snapshot. SettingValuesHandler gains an EventsHub and publishes user_settings.changed after every successful PUT and DELETE on /settings/values/{key}. The payload carries only key, scope and profile_id — never the value. Admins receive every user's user-scoped events, so a value in the payload would leak private settings to admins; interested clients re-fetch over the scoped REST API instead. The payload is always non-empty because an empty Data falls back to a null snapshot in the hub. A nil hub (tests) skips publishing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): sweep expired mutation receipts daily Setting-mutation idempotency receipts were written with an expires_at that nothing enforced, so the table grew forever. Add a hidden daily system task (05:00) that walks every login account, opens its user store, and calls DeleteExpiredSettingMutations. A user whose store fails to open or sweep is logged and skipped so one broken store cannot stall retention for everyone else; the delete is idempotent, so the next run repairs anything missed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(settings): resolve metadata language canonically in access and policy Repoint the last legacy column readers onto canonical contract resolution (settings cutover task A4a): - access.Resolver and policy.ViewerResolver now resolve catalog.metadata_language through settingsresolve (profile scope -> contract default) via a shared access.PreferredMetadataLanguage helper, instead of reading user_profiles.preferred_metadata_language. Resolution is deliberately unconstrained: the policy input this preference feeds is the one a constraint would have to reference, which is circular — see the key's manifest notes. - playback start now resolves playback.audio_language canonically for the profile default instead of reading user_profiles.language, matching the catalog detail path. Series and library override handling is unchanged. - items.go needed no change: it already consumes the resolver-produced scope.PreferredMetadataLanguage. The legacy columns keep their values but are no longer read on these paths; a profile with only a column value now resolves to the contract default, and a stored canonical value wins. Tests pin both directions in access, policy (including scope parity, where the column is now a decoy), and the playback handler. Read cost is one batched store read per resolution, same as the profile-row read it replaces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(jellycompat): give DisplayPreferences its own table The Jellyfin DisplayPreferences blobs rode the legacy user_settings key/value table under synthetic jellycompat:* keys, which forced the legacy settings API to carry a prefix carve-out in its otherwise-closed unknown-key gate. They are the compat subsystem's storage, not user settings: the contract neither validates nor resolves them. Move them to a dedicated jellycompat_displayprefs table in both backends, keyed by (prefs id, client) per user, with the blob stored as opaque text served back byte-for-byte (deliberately not jsonb, which would re-serialize it). The data-copy migrations — per-user SQLite V16 and a paired SQL + Go goose migration for Postgres — are transactional and harmless to re-run, and both drive their key parsing and row classification from the new internal/jellycompat/displayprefs package so the backends cannot diverge, following the internal/settingsmigrate precedent. A jellycompat:* row that does not parse as a DisplayPrefs key (only ever writable through the removed carve-out) is recorded in user_setting_migration_rejects rather than silently deleted. With the last non-settings tenant gone, the jellycompatSettingPrefix carve-out is deleted: the legacy settings endpoints now refuse jellycompat:* keys like any other unknown key and never surface them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settings): serve admin user-settings through the canonical API Replace the ten string-registry /admin/users/{id}/settings* and device-settings* routes with the canonical contract surface: one list of every explicit value the target user has stored across all scopes, and set/delete at an explicit scope named in the query string. The admin handlers live on SettingValuesHandler and share the session routes' implementation rather than duplicating it — the same key/scope parsing, identity validation, contract scope allowance, value normalization and mutation-receipt idempotency, factored into keyedScopeFromRequest/completeIdentity and setValueAt/deleteValueAt. The only admin-specific parts are the target user coming from the path, profile and device ids coming from the query (an admin holds no session claim to the user being inspected, so its named profile is checked to exist), and change events attributed to the target user so their clients refresh. The list is a new UserStore read, ListAllSettingValues, implemented in both backends and pinned by the shared storetest conformance suite: the admin surface wants the stored truth (which overrides exist, for a per-row reset affordance), which no resolution-shaped read answers. The ten removed routes are recorded in the pre-lock removals table in docs/architecture/v1-scope.md per the v1 API rules; the web admin device-overrides page moves onto the new surface in the Phase B rewrite inside this same unmerged PR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(settings): add the cross-platform conformance fixture and its Go and web runners contracts/settings/v1/conformance.json is the spec's named drift gate: 21 hand-authored cases of {keys, stored rows, context, constraints, expected effective value + source}, every one executable against the shipped manifest. They pin the semantics most likely to drift across four resolver implementations: the full resolution ladder (series > library > device > profile > default), an absent identity dropping its scopes, foreign-identity rows never resolving, ceiling caps that report the authored value with constrained:true, the ordered-enum sentinels (auto below every cap, original above), null-on-a-nullable-numeric meaning unbounded and being brought down by a ceiling but ignored by a floor, allowlist falling back to the first allowed member rather than the (possibly forbidden) default, and playback.subtitle_appearance resolving device > profile only with the sparse device object replacing, not merging. Cases may inject a constraint binding onto a copy of a real definition so constraint kinds no shipped definition carries stay testable. The Go runner (internal/settingsresolve/conformance_test.go) resolves each case through the real resolver against the embedded manifest. The web runner (web/src/lib/settingsConformance.test.ts) runs the same cases through a new client-side resolver, web/src/lib/settingsResolve.ts, which mirrors the server's semantics; the TypeScript bindings now carry each definition's ordered flag and constrained_by binding so that resolver derives constraint behavior from the contract instead of hardcoding it. Both runners reject unknown fixture fields — schema drift in the fixture itself is drift — and both refuse a fixture authored against a different manifest revision. The fixture travels with the bindings: make settings-bindings vendors the copy the web runner reads, and make verify-settings-bindings fails CI when that copy goes stale. The Kotlin and Swift copies land together with their runners in the client repos, which will pick their own test-resource paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settings): review pass over the phase A stack Fixes the eight adversarially-confirmed defects the review of the unpushed phase A stack (40e0f77a..1f2c7fe4) found, each with a test that fails without its fix. Writers left behind by the language cutover (high). |
||
|
|
63e18cf37f |
fix(playback): prevent transcode resolution upscaling (#503)
* docs(playback): design transcode resolution clamp * docs(playback): plan transcode resolution clamp * fix(playback): prevent transcode resolution upscaling * refactor(playback): share transcode resolution tiers --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
271a2e1741 |
feat: emailed invitations, claim + household setup, and server-driven onboarding tour (#501)
* feat(invitations): add emailed pre-provisioned invitations
Admins can invite a specific person by email: the invitation pre-binds
role, access group, and library access, and the invitee only chooses a
password. Their email address becomes their username, so login gains an
email fallback (username lookup first, email column only on miss for
inputs that parse as a bare address).
- invitations table: single-use token (SHA-256 at rest) bound to one
address; a partial unique index makes resend-supersedes atomic; no
users row exists until accept, so a typo'd address can't squat a
username. Status is derived from timestamps, not stored.
- internal/invitations: repository, service, and branded email through
the shared internal/mail sender. When SMTP is off the claim URL is
returned for manual delivery instead of failing.
- Admin endpoints /admin/invitations (list/create/resend/revoke) beside
the existing invite-codes routes; public claim endpoints
/invitations/{token} (+/accept) rate-limited with the other auth
endpoints. Unknown/expired/revoked/used tokens are indistinguishable.
- Accept returns the same login response shape as signup, so clients
reuse their session plumbing.
Spec: docs/superpowers/specs/2026-07-27-invitations-and-onboarding-design.md
Plan: docs/superpowers/plans/2026-07-27-invitations-and-onboarding.md
Part of #215
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(web): add invitation admin tab, claim page, and household setup
- Admin → Users gains an Invitations tab: compose (email, role, access
group, libraries, note, first-profile and tour toggles), list with
derived status, resend, revoke. When the server has no SMTP the create
response's claim URL is surfaced for copy-paste instead of a fake
success.
- /invite/:token claim page: everything but the password was decided at
send time, so it asks for exactly one thing and lands the user signed
in. Expired/used links get an explanatory card, not a 404.
- /household-setup ("Who's watching?"): profile tiles plus the existing
ProfileEditorDialog, all through the existing /profiles endpoint —
no new backend. "Just me for now" is a first-class exit.
Part of #215
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(onboarding): add server-driven onboarding manifest and state
GET /onboarding/flow returns the ordered first-run tour for this server
and profile: steps for disabled features (requests, watch together,
recommendations, notifications) are filtered out server-side, surface=tv
drops steps needing text entry, and child profiles never see stops they
can't act on. Copy lives in Go, so a wording fix is a deploy — clients
render step kinds they know and skip unknown ones by contract.
setting_choice steps name an explicit write target (profile_field /
setting / device_setting) because playback quality is a profile column,
not a settings key — the tour writes through the same APIs the settings
screens use.
Per-profile completion state lives in the user store (SQLite schema v14
+ a Postgres twin table), keyed by (profile_id, tour_id) with monotonic
completed/skipped timestamps: finishing on one device silences every
other; a later progress write can never un-complete.
Part of #215
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(web): add the first-run feature tour
TourHost renders the server manifest as a modal overlay on Home: unknown
step kinds are skipped silently (the forward-compat contract), progress
posts per step, and setting_choice steps write real values through the
existing profile/settings mutations — by the last step the account is
genuinely configured. Skip is always one click and recorded server-side,
so no other device re-prompts. The tour ends by handing off to the
existing taste-seed picker, which now waits for the tour to finish
before its own redirect. Settings → Personalize gains a replay entry.
An invitation sent with show_tour=false plants a local hint that the
gate converts into a server-side skip for the first profile.
Part of #215
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(web): satisfy noUncheckedIndexedAccess in the tour's advance step
The Docker web build runs `tsc -b`, which applies the project's
noUncheckedIndexedAccess; the bounds check didn't narrow steps[next].
Look the step up once and branch on its presence instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(web): blur the whole app behind the tour, sidebar included
The tour overlay rendered inside the app layout, where an ancestor
creates a fixed-position containing block — inset-0 pinned to the
content pane, leaving the sidebar completely un-scrimmed. Portal the
dialog to <body> so the scrim truly covers the viewport, and raise the
backdrop blur from sm (4px) to xl (24px) so card titles and nav labels
aren't legible through it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(onboarding): name features by their UI labels in the tour copy
"Same movie, different couches" never said what the feature is called.
Every feature card now leads with the name the sidebar actually uses —
Watch Party, Requests, Watchlist, Calendar, Notifications — and says
where to find it, so the tour teaches vocabulary, not just concepts.
Server-side copy, so all three clients pick this up with no release.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(onboarding): add apps and Jellyfin-compat steps to the tour
Two new web-only feature cards near the end of the tour:
- "Take Silo with you" — native apps for iPhone/iPad/Apple TV and
Android/Android TV, with outbound TestFlight and Play Store links.
Steps gain an additive links field (label + url) that older clients
ignore; the web TourHost renders them as external-link buttons.
- "Already use a Jellyfin app? It works here" — Infuse/VidHub/Findroid/
Swiftfin connect via the Jellyfin API. Gated on
jellyfin_compat.enabled (default-on, so unset counts as enabled;
only an explicit "false" hides it).
Both steps are web-only: the apps card is pointless inside the apps it
advertises, and TV can't open store links. surface=phone/tv manifests
skip them, covered by tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(web): keep the tour card responsive on phone widths
Verified every step at 1600px, 390px, and 320px with an automated
overflow check. Fixes it found:
- Link buttons (apps step) now wrap and truncate instead of extending
past the card edge.
- The footer wraps at very narrow widths, so the handoff step's wide
primary button drops to its own line rather than overflowing.
- Progress pips hide on phones — decorative, and they crowded the
Back/Next buttons.
- The card scrolls within 85dvh so a tall step never pins its buttons
off-screen on landscape phones.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(web): render store links as branded badges in the tour
The apps step's plain outline buttons now render as store badges: the
Apple or Google Play mark with a store eyebrow (TestFlight beta /
Google Play) over the platform label — the familiar app-store badge
idiom. The brand is inferred from the link's host on the client, so
the server contract stays icon-free and non-store links keep the plain
external-link button. Labels drop the parenthesized store name the
eyebrow now carries.
Verified at 1600px and 390px with the overflow sweep: none.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
QuickandClaude Opus 5
|
9e7fe79590 |
docs: record live TV, IPTV, and .strm as permanent non-goals
Live TV, OTA/DVB tuners, IPTV, EPG/XMLTV guide sync, DVR, and .strm remote-URL shortcuts are permanently out of scope for Silo. The deciding factor is app store distribution: the first-party iOS, tvOS, macOS, and Android clients ship through Apple and Google, and a server that plays arbitrary remote stream URLs puts the entire client suite at risk of rejection or takedown, not just the feature. Secondarily, live TV is a separate product surface whose reliability burden competes with the core playback path. This was an undocumented boundary until now, and contributors spent real effort against it (#419, #420, #474). Write it down in docs/non-goals.md and summarize it in AGENTS.md so both humans and agents see it before proposing or implementing in this area. Refs #474, #419, #295 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
ece3578e91 |
docs: rightsize repo context for Claude 5 context engineering (#467)
Applies the practices from Anthropic's "new rules of context engineering for Claude 5 generation models" to this repo's always-on context. AGENTS.md (symlinked as CLAUDE.md) drops content derivable from the filesystem — the module-by-module structure tour, the Makefile target list, and most of the style section, all of which Claude reads directly from internal/, the Makefile, .golangci and web/.prettierrc. What stays is the part that isn't derivable: the Goose migration rules, the v1 additive-only API contract, multi-repo boundaries, and the workspace gotchas. Also fixes a dead pointer to .claude/skills/deployment-debugging, which does not exist; the runbook is the dev-environment-debugging skill. The external-contributor AI disclosure block moves to docs/ai-contributions.md, reached by a one-line pointer, so it costs nothing in the common case where no external PR is being prepared. issue-to-pr sheds the generic agent hygiene now covered by the harness system prompt and keeps the Silo-specific gates. Its commit trailer no longer pins a stale model name. scripts/jellycompat-diff.sh replaces the hand-typed curl/python one-liners the jellycompat-diagnosis skill used to carry as prose. It unions item keys across all returned items rather than reading item[0], which was hiding fields present on only some items. Verified: make verify-local-paths, bash -n, shellcheck, and a mock two-server run of the diff script. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
ee31a1f0e2 |
feat(playback): formalize resumable direct streams and stall observability (#464)
* feat(playback): formalize resumable direct streams and stall observability Implements #443: strong stat-based ETag + If-Range on original-file direct play (via http.ServeContent), stream-end outcome classification in RollingDeadlineWriter (stalled_reap vs client_gone vs completed) with a structured log event and Prometheus counters, the direct_stream_resume_v1 protocol-v3 capability, and a contract doc. Progressive remux is explicitly excluded from the resume contract. Code written by OpenAI Codex CLI (gpt-5.6-sol) from a Claude-authored spec; reviewed and verified by Claude. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden direct stream resume contract * test(playback): cover resume platform contracts --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104
|
3c56ed606c | fix(admin): enforce settings contracts end to end | ||
|
rxwatcherandClaude Fable 5
|
a679b5e1a1 |
docs(ebooks): backfill automation + rate-limit cooldown design
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
rxwatcherandClaude Fable 5
|
e18bc3c568 |
docs(ebooks): add enrichment architecture plan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick104andClaude Fable 5
|
1f9bd99990 |
fix(diagnostics): address round-4 review findings on PR #445
- schema: add crash/report.type conditionals (allOf if/then) so a crash/anr/native_crash/hang/abnormal_exit manifest requires `crash` and a `manual` manifest forbids it, matching ValidateManifest. - service: reject uploads where X-Profile-Id and manifest.report.profile_id are both present but differ (new ErrProfileMismatch, mapped to 400 profile_mismatch) instead of silently preferring the header; single-source and matching cases unchanged. Adds service tests for mismatch, match, and header-only attribution. - schema: require manifest.json as the first archive.entries element via prefixItems (contains retained for validators without prefixItems support). - schema: document that maxLength is a character-count bound while the server enforces UTF-8 byte length, via a top-level note and per-field notes on the free-text device_summary and crash fields. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh |
||
|
Quick104andClaude Fable 5
|
a6348b3dc5 |
fix(diagnostics): address PR #445 review findings
- bundle: reject tar entry names that differ from their trimmed form instead of normalizing padded names into the allowlist - repo: reserve expected bytes on receiving rows and count receiving+ready in the per-user byte quota so concurrent/multi-node uploads can't overshoot - contract: require the crash object for event report types and keep it absent for manual; add contract tests - settings/service: seed diagnostics.server_instance_id atomically via insert-if-absent and adopt the winning value across nodes - bundle/service: capture the embedded manifest.json during ValidateBundle and reject reports whose embedded manifest disagrees with the part-1 manifest (minus archive); add tests - admin: delete the DB row before the blob on DeleteReport; log bucket/key when the blob delete fails instead of leaving a visible report with a missing bundle - bundle: reject PAX/GNU tar formats and extension records that smuggle bytes past validation; add a PAX-archive rejection test - migration: add CHECK constraints for state, report_type, and platform - docs: add text/jsonc language identifiers to the two unfenced code blocks - cleanup: log-and-continue per report and aggregate errors so one poisoned report no longer blocks the whole run; update tests - tasks: give diagnostics its own cleanup interval key instead of reusing the opslog key, and bound the startup settings lookup Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh |
||
|
Quick104andClaude Fable 5
|
f965a489bd |
fix(diagnostics): align bundle contract with real tar writers
Two validator behaviors made the contract unimplementable for clients using standard tar libraries: - Any byte after the tar end-of-archive marker was rejected, but GNU tar, Python tarfile, and Apache Commons Compress all pad the archive with zero blocks to a record boundary. Accept up to 64 KiB of zero padding; any non-zero trailing data is still rejected. - uncompressed_bytes was computed as the sum of entry payloads, which no tar-producing client observes. Define it as the total decompressed tar stream (headers, end-of-archive marker, and padding included) — the byte count between a client's tar writer and gzip writer, and what gzip -l reports. Documented in the design doc and contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e3QjbPo96ed9Mn2qRiUkh |
||
|
Quick104andClaude Fable 5
|
4fa84a661a |
feat(diagnostics): client diagnostics server foundation
Implements slice 1 of docs/design/2026-07-19-client-diagnostics.md: the versioned contract (schemas, fixtures, Go validator), storage-validated diagnostics.uploads_enabled gate, account-scoped status endpoint, hardened streaming multipart ingest with quota reservation and a receiving/ready/ failed report state machine, S3 streaming puts, acting-admin report API (list/detail/download/delete with audit events), and the retention + orphan-reconciliation cleanup task. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XppCCycoaskCsW7ja1fZct |
||
|
Quick104andClaude Fable 5
|
7369d0afd8 |
docs(design): client diagnostics spec (crash reports + debug log upload)
Cross-repo spec and rollout plan for opt-in client crash reporting and debug-log upload to the user's own Silo server: silo-server ingest, storage, admin API, and retention; silo-android and silo-apple capture, consent, and upload. Disabled by default on both server and client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XppCCycoaskCsW7ja1fZct |
||
|
|
91e1164090 |
feat(metadata): local NFO metadata and sidecar artwork (builtin chain provider) (#390)
* feat(metadata): register builtin NFO provider and broaden parsing Phases A and B of the #216 local-NFO work, implemented test-first. Registration & hint-first identity (Phase A): - Migration seeds a reserved kind='builtin' silo.builtin installation and an 'nfo' metadata capability (default_enabled=false, priority 1 for movie/series) with a partial unique index and documented Down. - In-process builtin provider registry (internal/metadata/builtin.go); buildProviders returns the registered provider for builtin rows. - Guard rails keep the reserved row out of every plugin surface (user plugin-settings, installations list, image resolvers, preload, auto-update, store Delete, mutation handlers -> 409); silo.builtin is a reserved manifest id. - Startup sync materializes legacy content_level='' chains per level, then appends builtin capabilities disabled via AppendProviderToAllChains (idempotent); resolveEnabledProvidersBy priority now respects default_enabled=false. - NFO uniqueids seed the trusted-hint machinery via IdentityHintProvider with per-mode conflict policy (stored IDs win on scheduled refresh, NFO wins on manual refresh, Identify skips NFO); ID-less candidates are excluded from provider-priority tie-breaks and nfo never counts as corroboration. - Web chain-editor empty-state gate is now server-derived so builtin providers are reachable on plugin-less servers. Parser breadth & sidecar hardening (Phase B): - Parser covers the practical Kodi/Jellyfin field set for <movie> and <tvshow>: original title, tagline, runtime, dates, content rating, genres/studios/countries/tags, multi-source ratings with scale normalization, cast with roles/order, director/credits. Empty collections stay nil so merge early-returns apply. - findNFO parses candidates and falls through on read/parse failure or root-type mismatch, so a stray movie.nfo cannot shadow tvshow.nfo; GetMetadata gains the same ContentType guard Search has. - New FieldReleaseDates lock gates Year/ReleaseDate/First+LastAirDate in merge (Go) and the edit-metadata dialog (web), closing the gap where a manual refresh re-applied NFO dates over admin corrections. - Merge-contract tests pin NFO fill semantics, genres whole-list first-provider-wins, and NFO edits propagating on manual refresh only. - Docs: new admin wiki page (supported fields, merge semantics, naming-supplies-structure contract), index bullet, sidecar wording revision, v1-scope feature-detection note. Zero behavior change while the provider is disabled (default); pinned by CI-mode and DB-gated test suites. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * feat(metadata): ingest local sidecar artwork and read series-depth NFO Phases C and D of the #216 local-NFO work, implemented test-first, plus the mixed-library use-case pins. Together these deliver the headline case: a series absent from every remote database (e.g. a fitness library) scans into a fully presented show -> named seasons -> titled episodes tree from NFO files and sidecar art alone. Local sidecar artwork through the S3 image cache (Phase C): - The NFO provider implements ImageProvider: poster/backdrop/logo sidecar discovery with a fixed precedence map, symlink/non-regular rejection, an 8 MiB cap, and file:// source URLs at rating 0. Generic filenames apply only via the sidecar search paths, so a shared folder.jpg in a flat multi-movie directory applies to none. - file:// becomes a live local source scheme: routed into *_source_path (never *_path), accepted by every image enqueue gate, attributed as provider "local", excluded from cached-path detection. - The image-cache processor caches local files with lexical-on-logical confinement to the library roots, open-handle reads with re-checks, the same variant widths as remote art, and stable (7-day) failure classification. Keys land under local/{contentType}/{contentID}/{hash8}/{imageType}; superseded prefixes are cleaned on re-cache and item deletion. - applyIfBetter gains a local exemption so rating-0 local art can fill matched items without being stickily displaced; ImageRequest carries additive sidecar path context. Series depth (Phase D): - SeasonsRequest/EpisodesRequest carry additive local path context (series roots, per-season directories, per-episode file paths), derived from naming at match time and reconstructed on refresh. - season.nfo supplies season name/plot; NFO season numbers are advisory (directory-derived number wins with a Warn - naming owns structure). <episodedetails> gains aired/runtime/ratings; <basename>.nfo titles episodes and <basename>-thumb.ext supplies thumbs; filename SxxEyy wins over NFO numbers. - Episode NFOs work without a season.nfo (provider seasons unioned with on-disk seasons); SynthesizeFallbackEpisodes always runs after persist so NFO-less episodes keep synthesized rows. Season/episode file:// art rides the Phase C pipeline unchanged. - Migration adds season:1/episode:1 to the builtin NFO capability's default_priority (still default_enabled=false). Mixed sports-library use case (tests only, no product change): - Pins the classification contract for one library holding movie-shaped and show-shaped content (WWE PPV events as movies next to a "WWE SmackDown" show, NASCAR/F1/FIFA with partial TVDB/TMDB data): naming decides movie-vs-series per file before any provider runs; the NFO supplies metadata/identity but never flips type (ContentType guard); the per-root Type override is the correction path. - NFO-driven type classification at scan time is recorded as an explicit deferred open question. Part of #216 AI-use disclosure: implemented with Claude Code (Fable 5) via spec-driven TDD and agent-assisted implementation. * docs(metadata): document local NFO metadata architecture Add a single as-built architecture page (docs/architecture/local-nfo-metadata.md) for the #216 local-NFO feature: the builtin registration model, hint-first identity semantics, the file:// -> S3 artwork pipeline and its deployment constraint, series depth, the mixed-library classification contract, and known limitations. This replaces the working implementation plan, the per-phase specs, and the narrow sidecar-artwork note, which were planning drafts and are left untracked; admin-facing behavior remains in the wiki. Part of #216 AI-use disclosure: planned, drafted, and consolidated with Claude Code (Fable 5) using multi-agent exploration and adversarial review. * fix(metadata): address PR review findings on NFO builtin provider Fold in the valid, low-risk fixes surfaced by automated review on #390: - imagecache: extract validateCacheRequest so CacheBytes (the local sidecar season/episode path) enforces the same episode-requires-season guard as Cache, preventing distinct episodes' art from colliding under one S3 key. - image_cache_processor: close the sidecar symlink-swap window by rejecting the opened handle unless os.SameFile matches the Lstat'd file, so a leaf swapped to a symlink can't pull an out-of-root target into the public cache. - plugins: guard the reserved builtin installation row in the store's Update, matching Delete, so its version/enabled/capabilities can never be rewritten even if a mutation slips past the HTTP layer. - cmd/silo: bound SyncBuiltinProviderChains with a 30s timeout so a stuck DB round-trip fails fast at startup instead of hanging. - metadata: panic instead of silently no-op'ing on an invalid RegisterBuiltinProvider call (init-time programmer error). - docs: correct the media-folder-and-naming NFO paragraph to state season/episode NFOs and sidecar artwork are actively read. --------- Co-authored-by: Quick104 <31828688+Quick104@users.noreply.github.com> |
||
|
|
854d07cf8f |
feat(playback): add protocol v3 planning and recovery (#398)
* docs(playback): plan protocol v3 server implementation * docs(playback): incorporate protocol v3 review * feat(playback): implement protocol v3 server * fix(playback): persist empty route diagnostics * feat(playback): harden protocol v3 HDR routing * feat(playback): complete protocol v3 client contract * fix(playback): harden protocol v3 recovery * fix(playback): restore dovi_rpu strip filter for DV remuxes The v3 work renamed the Dolby Vision strip recipe to a dovi_split=mode=bl bitstream filter that does not exist in stock FFmpeg or jellyfin-ffmpeg; the probe failed closed on every deployment, disabling the new validated DV7-to-HDR10 route and regressing the previously working dovi_rpu=strip=1 remux path from main. Restore dovi_rpu across the probe, remux and HLS copy arguments, and the recipe-card constant. Also from review: validate the remux DV mode for every profile (garbage modes on non-P7 sources silently no-opped), reject preserve mode for P7 outright (a base-layer-only remux cannot preserve dual-layer DV), tag dvhe sample entries only for the explicit v3 preserve recipe so legacy web/jellycompat remuxes keep their pre-v3 hev1 labeling, and honor the token-frozen DV mode in the proxy remux path instead of legacy-auto. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): correct v3 planner policy and contract validation Review fixes to the v3 planner and wire contracts: - Bar Profile 7 sources from the non-strip progressive remux route: a base-layer-only remux can never deliver native dual-layer DV, so the planner no longer emits plans claiming validated Dolby Vision while the executed remux drops the enhancement layer. - Accept the device-quirks feature flag from either capability location, matching every other dual-location feature check. - Treat legacy hdr_unknown rows as HDR10 for HDR10-capable clients with a degradation warning instead of leaving them unplayable under v3. - Honor bandwidth_cap_kbps as a hard ceiling in every quality mode and wire the previously dead Metered signal into conservative auto rungs. - Degrade to the validated source-quality route instead of a terminal when only an implicit quality reduction demanded an unsupported transcode; explicit user-selected rungs keep terminal behavior. - Bound inner capability lists and strings; compare attempt keys exactly instead of case-folded; make ParseTrackIDV3 strict about canonical numerics; accept dvdsub/pgssub/dvbsub aliases and stop promising burn-in for unknown subtitle codecs; probe every h264 encoder rather than requiring libx264; normalize the file-level bitrate fallback. - Evaluate subtitle renderability against the engine each candidate route executes on, not always media3_direct. - Pin the with-quirks attempt-key preimage arity in the cross-language fixture so the Kotlin client stays in lockstep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 control-plane reliability Review fixes to the v3 session, store, and handler layer: - Bound concurrent replans with a slot semaphore: each replan pins a pooled connection for its advisory lock while issuing further store queries from the same pool, so an unbounded recovery storm could turn every connection into a lock holder and deadlock the server. - Make CompleteReplan a real compare-and-swap (base-revision predicate, ErrReplanSupersededV3) and map BeginReplan insert races to a replay instead of a raw unique violation. - Fingerprint start requests (request_digest column): an attempt ID reused with different input is now a 409-style conflict rather than a silent replay, and both replay paths check session liveness so dead sessions surface as retryable terminals. - Pre-delete expired attempt rows on SaveAttempt so a retry during the cleanup window cannot wedge on an unreachable conflict. - Align the in-memory store's semantics with Postgres and add DB-backed planstore tests (SILO_TEST_DATABASE_URL), including a regression test inserting every route-event name against the real CHECK constraint. - Session manager: v3 route-set updates own RemuxDVMode outright so a replan onto an SDR source clears a stale strip mode; replacement reservations survive unrelated legacy stream updates; replacement admission excludes the replaced session explicitly instead of decrementing totals it may no longer be part of; the admission CAS loop is bounded and decider errors are logged. - Map transient store failures to 500s instead of terminal 404/403s; authorize route events via identity-only projections after the rate limiter; keep sanitized diagnostics deterministic. - Merge the server-computed durable plan key into replan exclusions so unreproducible client history cannot re-select the failed route. - Remap tracks only when the effective edition changes (a same-file replan no longer switches audio to a lookalike track) and remap ID-only subtitle selections on edition fallback. - Cache the v3/shadow feature flags for five seconds instead of one settings SELECT per playback request; stop remote transports best-effort when the start call times out; carry dvm/tid claims and the transport-scoped job identity through the legacy audio-change re-mint; index playback_route_events(received_at) for the retention delete; run store maintenance for DB-less deployments too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): reap idle node jobs and gate WebVTT conversion - Add an idle reaper to the transcode node: a job untouched by manifest or segment requests for ten minutes is closed and unregistered. After a v3 replan retires a transport ID, a stale in-flight stream token could resurrect the old job via reconstruct and encode to end-of-file for nobody; jobs waiting on readiness count registration as access and are never reaped mid-wait, and reaping keeps the recipe so a still-valid token reconstructs on the next hit. - Reject bitmap subtitle tracks (PGS) on the .vtt conversion path with 415 before headers are written instead of spawning an ffmpeg command that always fails mid-response, and make the extract-format override fall back to source-driven mapping for bitmap codecs. - Drain error bodies on non-202 node responses so the HTTP transport can reuse connections. - Pin the transcode-dir cleanup separator-boundary semantics with a regression test (a session ID sharing another's prefix must not retain foreign directories). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): close v3 planner policy gaps from review - Clamp the final transcode bitrate to bandwidth_cap_kbps: the ladder has no rung below 480p/1500kbps, so lower caps were silently exceeded even though the cap is documented as a hard delivery ceiling. - Treat video-only media as audio-compatible instead of forcing an AAC conversion (or an audio_conversion_unsupported terminal) onto a file with no audio stream. Tracks whose codec failed to probe keep the gate. - Only promise a bitmap subtitle sidecar for embedded PGS with an engine that renders embedded bitmap: external/downloaded bitmap and embedded DVD/DVB published artifact URLs that always failed at fetch. They now fall through to burn-in or its terminal. - Accept client_video_transformations_v1 from either client_features or the nested context when validating client-executor transformations, matching the planner's dual-source reads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): probe and execute DV remuxes with one ffmpeg binary The v3 transformation registry probed the configured playback.ffmpeg_path while progressive remux execution resolved the process-global discovery path, so a deployment where only one binary carries dovi_rpu could plan a server_dv7_to_hdr10 route and then fail it at stream time. Resolution now goes through a shared ResolveFFmpegPath (configured path first, discovery fallback — the same rule the transcode pipeline already used), the dovi_rpu probe is cached per binary path, and the stream handler and proxy worker pass their configured path into ServeRemuxWithDVMode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): harden v3 replan identity and control-plane limits - Seed failure-replan track selections from the durable current plan before overlaying the request: after an alternate-version fallback the normalized request still carries requested-edition track IDs, so a replan omitting unchanged tracks was rejected as a track/file mismatch. - Remap ID-only audio selections across edition changes (parse the ID to an index like the subtitle remap already does) instead of leaving a stale file-bound ID to fail validation. - Release the node planner reservation when a prepared remote transport rolls back after the node accepted the job; repeated failed starts could otherwise pin max-job/bandwidth budgets for the full reservation age. - Size the replan semaphore below the PostgreSQL pool via a store capacity advisor: with max_connections at or below the fixed bound, advisory-lock holders could starve the inner store queries they need to finish. - Contain shadow-planner panics with a recover boundary; it runs on a bare goroutine where an escaped panic kills the process for what is telemetry-only work. Document why the memory store's session lock is deliberately a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(transcode): serialize node job teardown against reconstructs - Look up and touch manifest/segment sessions in one critical section so the idle reaper cannot unregister a job between the lookup and its liveness refresh. - Re-validate each reap candidate under the per-session lifecycle lock before closing it: Close removes the output directory, and without the lock it could race a token reconstruct and wipe the segments the fresh ffmpeg is writing. - Take the lifecycle lock in handleStop so a stop racing a RequireReady start's readiness wait blocks until registration and tears the job down, instead of 404ing and orphaning the ffmpeg until the reaper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
b7292a9473 |
fix(streaming): stop killing healthy streams at the server WriteTimeout (#361)
The main API server's WriteTimeout (120s) is an absolute deadline from request start, so every streaming response still being written at T+120s was cut mid-body with a clean close. Clients saw multi-GB direct streams truncate every two minutes; the Apple client's cursor-resume reconnect absorbed most kills silently, but one landing during backpressure or a demuxer resync exhausted its retry budget and forced a full player teardown (visible stop + historical audio desync seeding). Fix: internal/httpstream.RollingDeadlineWriter pushes the connection's write deadline forward with progress via http.ResponseController — a response that keeps moving lives indefinitely, a stalled one is still reaped within the window (180s default, SILO_STREAM_WRITE_STALL_TIMEOUT to override). ReadFrom delegates in bounded slices so http.ServeContent keeps its sendfile fast path. Wired into direct play, remux, downloads, the transcode-node proxy, and ebook serving; the server-level 120s guard stays for every other route. The metrics and request-logger response writers now implement Unwrap — without it http.ResponseController cannot traverse to the connection and SetWriteDeadline fails, silently disabling the fix (exactly what the first dev deploy showed). A middleware-chain integration test locks the whole path down against future wrappers missing Unwrap. Validated on dev: 200s/512MB direct and 300s/768MB via CDN sustained range-GETs (previously dying at 120s), zero duration_ms=120000 stream entries since deploy. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandGitHub
|
10e15798e0 | feat(plugins): add approved community catalog hub (#355) | ||
|
|
d68e70bb47 |
feat(autoscan): Sonarr/Radarr webhook intake without arr API keys (#353)
* docs(autoscan): add arr webhook intake spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add webhook intake schema migration Adds delivery_mode to autoscan_sources, the autoscan_webhook_endpoints table, and delivery_mode/provider_event_type on autoscan_events. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add built-in arr-webhook source identity Host-discovered scan-source entry so webhook-mode sources need no plugin installation; composite lister appends it to plugin discovery. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): persist delivery mode, webhook endpoints, event metadata Sources carry delivery_mode; autoscan_webhook_endpoints CRUD with SHA-256 token lookup and AAD-bound encrypted redisplay; events record delivery_mode/provider_event_type; CreateEvent gains SkipRunningCheck so webhook deliveries are never dropped by the poll exclusion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): share the consume path and add webhook IngestChanges Extracts consumeSourceChanges from PollOnce (marker semantics preserved, existing poll tests unchanged); PollOnce skips webhook sources; IngestChanges feeds deliveries through the shared pipeline without markers and without the running-event exclusion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add Sonarr/Radarr webhook payload parser Host-side arrwebhook package: provider inference, import/rename/delete path extraction with vanished-path-friendly previous paths, subtree fallback, exact-path dedupe, and no-op unknown events. Fixture-backed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(autoscan): add public webhook delivery route and admin endpoint management Public POST /api/v1/autoscan/webhooks/{token} with per-IP rate limiting, 256KiB body cap, 202-for-noop semantics, and token/body kept out of logs; admin create/rotate/delete endpoint routes; source responses carry delivery mode + webhook status/URL; create/update validate delivery mode against source identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add webhook delivery mode to Autoscan admin UI Webhook sources get a generate/copy/rotate webhook URL section, provider selector, delivery status, and a connection-free Add-source flow; activity rows badge webhook deliveries with the arr event type. Path rewrites stay editable in both modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): redact secret path params from request and activity logs The request logger and activity-log middleware recorded raw URLs, so bearer credentials in secret path segments (autoscan webhook {token}, webhook-sync {secret}) were persisted to app logs and activity_log. Redact the secret segment via the chi route params in both sinks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(autoscan): make webhook delivery reliable --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
04c4344f52 |
feat(metadata): reconcile artwork cache after public S3 provider changes (#349)
* feat(metadata): reconcile artwork cache after public S3 provider changes Changing the public S3 provider previously broke every cached image permanently: the DB keeps bucket-relative keys, the image cache pipeline treats a cached path as its durable dedup marker and never re-enqueues, and clients eat the 404s straight from S3 so the server never notices. Add a storage identity fingerprint (s3.public_storage_identity, seeded via SetIfAbsent at boot) and a reconcile_artwork_cache task whose startup trigger only fires when the identity changed; manual runs always sweep, doubling as bucket-data-loss recovery. The task probes a random sample of cached objects, then either bulk-resets (near-total miss) or per-row verifies. Missing provider-sourced artwork is reset to its *_source_path so the existing enqueue loop re-caches it; surfaces without a re-downloadable source (chapter thumbnails, collection artwork, library posters, branding refs, embedded book covers) are cleared so their owning pipelines refill them. Small upload-holding tables are always per-row verified so bulk mode cannot blind-clear an upload that survived migration, and transport errors never reset rows. Users never see broken images during the transition: reset rows serve the provider's original URL via the existing absolute-URL pass-through and thumbhashes are preserved. The storage settings page now warns that uploads cannot be re-downloaded when the identity fields are edited. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): harden artwork reconcile per code review Address the confirmed findings from the PR review: - Fingerprint the key prefix case-sensitively and slash-trimmed exactly as s3client applies it (new exported NormalizeKeyPrefix): a case-only prefix edit is a real storage move and must reconcile; a slash-only edit is not and must not. - Certify the storage fingerprint immediately after the artwork sweep succeeds and make the 4-object branding check non-fatal (reported in the task message), so a transient branding error cannot discard a completed catalog sweep and force it to repeat every boot. - Fail closed on conditional-task preflight errors in the task manager (previously fail-open ran the task), and retry transient settings reads in ShouldRun since the startup trigger fires once per process. - Track probe HEAD errors against a separate baseline so a flaky probe cannot consume the sweep's error budget. - Probe before counting: bulk mode skips the per-surface count(*) full scans entirely, and probe sampling drops ORDER BY random() (plain LIMIT answers "is the cache in this bucket" just as well). - Verify chapter thumbnails across a whole 500-file batch in one HEAD fan-out instead of per file, keeping the worker pool saturated. - Replace the 10 inline non-provider-scheme ARRAY literals in the enqueue query with the shared nonProviderImageSchemesSQL constant. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(metadata): guard bulk reset against degraded probes, certify only clean sweeps Address bot review feedback on the reconcile hardening: - A probe where more than half the HEAD requests error aborts the run: errored requests are excluded from the sample, so a partial outage could otherwise present a handful of surviving 404s as a ~100% miss rate and bulk-reset the catalog. Bulk mode additionally requires a minimum number of successful samples; thinned probes and tiny catalogs take the safe per-row verify path. - Track sweep errors separately from probe/branding errors (stats.sweep_errors) and certify the storage fingerprint only when the sweep completed with zero of them — skipped rows were never verified, so the next startup retries. Applied resets stay durable. - Give each ObjectExists attempt its own timeout so a stalled HEAD fails that attempt instead of pinning the retry loop to the run context. - Report branding assets checked (not just cleared) in stats.Checked. - Drop the dead settingsRepo/brandingSvc nil guards in cmd/silo and sync spec numbers with the implementation constants. Part of #348 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
CoffeeKnyteandGitHub
|
a2ef26bece |
perf: root-cause fixes for endpoints still slow after #292 (NextUp, series badges, resume tail, subtitle fonts) (#350)
* docs(plans): root-cause analysis for endpoints still slow after PR #292 Five endpoint groups stayed slow after the home/Continue Watching/Latest latency work shipped: Resume (110s p95), NextUp (17s p95), Latest (17s), /Items, and the home sections routes. The caps and caches from PR #292 are live in the deployed binary; they bounded how many rows the loops touch but not what each underlying query costs. Documents the four confirmed root causes (4.3M stale completed-with-position progress rows + missing resume index, unbounded next-up anchor scan, per-episode series rollup fanout, two index-starved history/scanner paths) with live EXPLAIN ANALYZE measurements and the fix plan implemented by the follow-up commits. AI-use disclosure: analysis and doc produced with AI (Claude) assistance. * perf(catalog): bound the global next-up anchor scan to recent completions The completed_episodes CTE in buildListNextUpQuery derived per-series anchors from the profile's ENTIRE completed history — DISTINCT ON over 233k rows joined to episodes for the worst bulk-import profile, then a per-series LATERAL that scans every episode of a fully-watched series before yielding nothing. 648 slow executions in a 19h window, 44.7s worst; this drove /Shows/NextUp (17.1s p95) and the next-up injection on the native home sections aggregate. Global queries now derive anchors from the profile's nextUpAnchorMaxRows (500) most recent completed rows — an ordered index walk on idx_uwp_profile_completed, with the hidden-items exclusion and date cutoff applied inside the bounded scan so hidden/old rows never consume the anchor budget. A next-up rail surfaces ~24 series; the 500 most recent completions cover every series that can realistically rank on it. Series-scoped calls (the show-detail tile) keep the unbounded shape: they must anchor on the series' last completed episode no matter how long ago it was watched, and are naturally bounded by one series. Measured on the live worst-case profile with the exact generated SQL: 44.7s worst / ~2.6s avg before; 10ms after (together with the one-time stale-resume-point data repair applied directly to the deployment DB — see docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md). AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat,userstore): aggregate series watch-state rollup in SQL The series Played/UnplayedItemCount badge on list rails (per-library Latest, library browse, search results) and series detail pages was computed by materializing EVERY episode of every series on the page (episodeRepo.ListBySeriesIDs) and then batching per-episode progress+history lookups in 500-id chunks. A 50-series page of an episode-heavy library (Sports) expanded to 32,467 episode rows and ~65 sequential queries — measured 17-18s per /Items/Latest request, and PR #292's cached Latest fast path pays it on every response for series libraries. The same fanout made /Items?searchTerm=... slow whenever the result set was mostly series (Meilisearch itself answers in milliseconds). New optional store capability userstore.SeriesEpisodeRollupStore, implemented by PostgresUserStore as one GROUP BY e.series_id aggregate with semantics identical to the chunked path (episode availability via episode_libraries, hidden-items visibility on progress rows, completed-history fold, in-progress = not watched with position > 0 — verified value-for-value against the old semantics on a real 1,586-episode series). enrichSeriesListUserData and enrichDetailUserData use it when present; SQLite-backed stores and rollup query failures keep the existing chunked path as fallback. catalog.SeasonUserDataFromCounts pins the counts-to-DTO mapping to EpisodeRollupUserData. Measured on the live worst-case profile against the real 50-series Sports Latest page: ~17s of chunked round-trips before, 119ms in one query after. Part of docs/superpowers/plans/2026-07-06-slow-endpoint-root-causes.md. AI-use disclosure: implemented with AI (Claude) assistance. * perf(catalog): bound superseded-episode completed walk to recent history The Resume / Continue Watching superseded-episode filter loaded a profile's *entire* completed history into memory on every request that contained an in-progress episode: CompletedProgressSnapshots paged user_watch_progress WHERE completed=TRUE with no upper bound. The 2026-07-06 slow-query comparison showed this surviving as a 60-116s Resume tail even after the in-progress index landed live, because the 4.3M zeroed Plex-import rows are still completed=TRUE and were re-walked every load. A completed episode can only supersede an in-progress one it was finished more recently than (the query gates on done_progress.updated_at > ip_progress.updated_at), so only completed rows newer than the oldest in-progress entry can matter. Compute that cutoff in SupersededEpisodeProgressIDs and pass it to CompletedProgressSnapshots, which — since the completed listing is ordered updated_at DESC — stops paging as soon as it crosses the cutoff. Import-heavy profiles whose back-catalogue predates their current in-progress items now stop on the first page instead of paging hundreds of thousands of irrelevant rows. Correctness is unchanged: no relevant superseding row is excluded. * perf(catalog): hard-cap superseded-episode completed walk at 5 pages The updated_at cutoff added in the previous commit bounds the completed walk on the relevance axis, but a very old in-progress entry sitting behind a large volume of newer completions could still page deep. Add a 5-page (2,500-row) hard backstop on top of the cutoff: normal profiles still stop on page one via the cutoff, and only the adversarial tail hits the cap. When it engages the tail of the completed set goes unscanned, so a superseded episode could momentarily survive on Continue Watching — we log a warning when that happens (with profile_id + rows scanned) rather than mis-filter silently, and it self-corrects once the stale in-progress entry ages out of the scanned window. * perf(playback): extract subtitle fonts in a single ffmpeg pass Embedded ASS/SSA font extraction spawned one ffmpeg process per font attachment, each re-opening the (usually CephFS-backed) media file. Anime releases carry 15-47 fonts, so the per-spawn file-open cost dominated and pushed GET /api/v1/stream/{sid}/subtitles/{track}/fonts to a 17-60 s plateau (p95 ~33 s in the live logs). Collapse the N spawns into one ffmpeg invocation that dumps every attachment to a temp dir (-dump_attachment:idx path ... -i file -map 0:t? -c copy), then read the files back. The file is opened once instead of N times, taking p95 from ~30 s to ~1-2 s with no change to output. Safety is preserved. The 32-attachment / 32 MiB caps still apply: attachment size is stat'd before read so an over-limit font never enters memory, and a watchdog polls the dump dir and kills ffmpeg if its on-disk output crosses the cap -- restoring the hard bound the old pipe-per-attachment reader enforced by killing at maxBytes+1, so a container with oversized "font" attachments can't fill the disk. Part of the slow-endpoint follow-up; see slow-query-analysis/subtitle-fonts-extraction-findings.md. * fix(review): report enforced font-byte cap; correct doc subtitle scope Address PR #350 review: - dumpFontAttachments reported the maxSubtitleFontBytes package constant in both over-limit errors instead of the maxBytes argument the caller passed, so the message misstated the enforced bound whenever a different cap was in effect (as the tests use). Interpolate maxBytes in both messages. - The root-cause plan claimed subtitle extraction was 'out of scope' while the branch actually optimizes /subtitles/{track}/fonts. Scope the out-of-scope note to subtitle *track* conversion and record the fonts single-pass work as deliverable 5. |
||
|
|
203a18ae83 |
feat(observability): OpenTelemetry logs+traces with secret redaction and slog standardization (#290)
* feat(observability): OpenTelemetry logs+traces with secret redaction Part of #265. Adds opt-in OpenTelemetry (logs + traces) alongside the existing stderr + opslog pipeline, plus secret redaction on all sinks. Default-off: with no OTEL_* / SILO_OTEL_ENABLED config, behavior is unchanged. Bootstrap (internal/telemetry): - Setup() builds one shared resource, a TracerProvider (parent-based trace-id ratio sampler), a LoggerProvider, and the W3C TraceContext+Baggage propagator from env. It installs NO MeterProvider — metrics stay on Prometheus, and the built-in no-op global MeterProvider keeps the trace instrumentation libs from double-emitting. Shutdown is deferred with a flush timeout. - Logs are bridged via otelslog fan-out (slog.MultiHandler), level-gated by the shared LevelVar and best-effort so a failing collector can't break the console or DB branches. stderr + opslog stay untouched. Secret redaction (internal/logredact): - A slog.Handler masks secret-keyed attributes (password, token, api_key, authorization, cookie, ...) — including .With-bound attrs, nested groups, secret-keyed group subtrees, and values behind a LogValuer — on the console and OTLP sinks, with a no-op fast path when a record has no secret keys. opslog.shouldRedact delegates to logredact.SecretKey so all sinks share one marker list. Rotation is infra-managed (no custom file sink): container runtime for stderr, collector/backend for OTLP, opslog partition-pruning for the DB. Documented in docs/architecture/observability.md. Verification: go build ./..., go vet, gofmt -l — clean; go test ./internal/telemetry/ ./internal/logredact/ -race pass. AI-use disclosure: implemented with AI assistance (Claude Code), including adversarial reviews that hardened the bootstrap and fixed two redaction leak paths; reviewed by the author. * refactor(observability): slog context+component sweep, sloglint gate (phase 3) Part of #265. Builds on the OTel bootstrap + redaction commit. Standardizes every log call site onto the context-carrying slog variants so records correlate with the active OpenTelemetry trace, and locks the standard in with a machine gate so future code (human- or AI-authored) can't drift back. - Call-site sweep: converted the remaining slog.<Level>(...) calls to the slog.<Level>Context(ctx, ...) form wherever a context.Context is in scope (background/init calls with no ctx are left as-is), across 183 files. Applied via a type-aware AST codemod. Log levels and message strings are preserved verbatim; a component attr (canonical per-package name) is added to direct package-level slog calls. Bound-logger calls keep their existing .With bindings. The main.go and telemetry package conversions rode with their file in the previous commit to keep each file within a single commit. - Enforcement (.golangci.yml): enable sloglint with context=scope, static-msg, key-naming-case=snake, no-mixed-args. After the sweep all four report zero violations repo-wide (tests included), so make lint / CI now blocks any regression to the non-context form. The gate ships with the sweep because it cannot be green until the legacy sites are converted. Metrics remain on Prometheus; no behavior change to /metrics or Grafana. Verification: go build ./..., go vet ./..., gofmt -l — clean; sloglint (all 4 rules) 0 violations repo-wide; log levels verified unchanged. AI-use disclosure: implemented with AI assistance (Claude Code), including the codemod; reviewed by the author. * fix(observability): honor per-signal OTLP protocol and secret WithGroup names Two Codex review findings on PR #290: - telemetry: OTEL_EXPORTER_OTLP_{TRACES,LOGS}_PROTOCOL now override the generic OTEL_EXPORTER_OTLP_PROTOCOL per signal, so mixed collector setups (e.g. HTTP logs + gRPC traces) build the right exporter. - logredact: entering a group whose name is secret-bearing (e.g. WithGroup("authorization")) now masks every leaf in that subtree, matching how slog.Group("authorization", ...) is masked as a whole. * fix(observability): address review feedback on telemetry bootstrap - Telemetry setup failure no longer kills boot: Setup returns usable no-op providers alongside the error and main logs and continues with telemetry disabled, honoring the best-effort contract. - Honor OTEL_TRACES_SAMPLER (always_on/off, traceidratio, parentbased_* variants); unsupported values fall back to parentbased_traceidratio. - Attach node identity as semconv service.instance.id instead of the non-semconv node.name. - Rename opslog retention-scope log attrs to target_component/target_level so they no longer collide with the canonical component routing key, and tag those lines with component=opslog. - Fix stale levelGated comment casing; use WarnContext in the telemetry shutdown defer; document the LogValuer double-resolve on the redaction slow path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
43d9056b01 |
fix(collections): repair broken builtin collection templates (#331)
* fix(collections): repair broken builtin collection templates A live audit of the builtin template catalog (all 40 MDBList URLs and all 10 TMDB franchise IDs fetched) found two dead sources, a silent bundle-apply collision, and several templates whose defaults contradict their descriptions: - Repoint mdblist_misc_a24 and mdblist_misc_criterion_collection to live lists; the original irvingbeano/shtluck lists were deleted on MDBList (404), so every sync of those collections failed. - Retitle mdblist_charts_popular_movies to "IMDb MovieMeter Top 100". It shared the "popular-movies" title slug with tmdb_popular_movies, and bundle apply dedupes by slug per library, so applying all_defaults silently skipped it. Poster regenerated from the raw plate with the new title; new handler test asserts builtin title slugs stay unique. - Raise the shared default limit 50 -> 100, give the IMDb Top 250 templates an explicit 250 (limit*4 fetch trim previously never scanned entries 201-250), and drop the limit on catalog lists (Criterion, A24) so they hold every owned title. - Correct IFC Films to MediaMovie (live list is 100% movies; as MediaMixed it was offered to TV libraries where it always synced empty) and fix the Trakt Popular descriptions (ratings-based, not "most-watched"). - Update stale limit docs in collection-templates.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(collections): raise import limit caps above IMDb Top 250 default The IMDb Top 250 templates now default to 250 items, but the template config forms rendered their Max Items input with max=200 and the user import API rejected limits above 200, so applying those templates from the direct galleries failed native validation or got a 400. Raise the cap to 500 on both sides, wired to shared constants: sync's fetch trim (collectionSourceFetchMax) never scans more than 500 source entries, so a larger explicit limit could never be satisfied anyway. collectionutil.MaxExplicitItemLimit backs validateOptionalLimit, and COLLECTION_MAX_ITEMS in lib/collectionTemplates backs all seven Max Items inputs (gallery forms + admin import/editor dialogs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e140bd9424 |
feat(metadata,scanner): trailers and extras for movies and series (#322)
* feat(metadata,scanner): trailers and extras for movies and series Remote provider videos (TMDB trailers/teasers/featurettes/...) are fetched through the unified match/refresh pipeline into the new item_videos table, filtered per-library via media_folders.trailer_kinds, merged across providers with site/provider dedup, and lockable via FieldVideos. The movie scanner stops discarding supplemental directories (Trailers/, Featurettes/, Behind The Scenes/, ...) and classifies them — plus Jellyfin-style filename suffixes (-trailer, -behindthescenes, ...) and series-root supplemental dirs — into the new media_extras entity backed by ordinary media_files rows (extra_id ownership, content_id/episode_id NULL so existing version/matching queries stay structurally blind to extras). Series Extras/SxxExx season-0 mapping is unchanged. Extras are playable watch targets via a GetWatchDetail fallback tier (episodes precedent), with contentid.ForLocal minting stable ids. API: ItemDetail gains additive videos/extras arrays (single + batch parity); library settings expose trailer_kinds. jellycompat now populates RemoteTrailers, LocalTrailerCount/SpecialFeatureCount, and serves real /LocalTrailers + /SpecialFeatures items playable through PlaybackInfo. Requires silo-plugin-sdk v0.9.0 (VideoRecord) before go.mod can bump; builds locally via go.work against the SDK feat/metadata-videos branch. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): trailers and extras sections, library trailer-kinds setting TrailersSection (YouTube thumbnails + youtube-nocookie modal) and ExtrasSection (plays extras through the standard watch controller) on movie and series detail pages; admin library form gains a trailer-kinds allow-list synced with the server default (all provider kinds), now also honored on library create. Part of trailers/extras capability work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): scan extra_id in scanMediaFiles; review cleanups scanMediaFiles (the plural row scanner behind GetByContentID/GetByFolder/ GetByExtraID and 20+ other queries) was missing the scan destination for the new extra_id column, which would have failed every media-file read at runtime with a column/destination count mismatch. Also: extend the batch equivalence test to seed item_videos/media_extras so the new videos/extras prefetch wiring is actually proven; drop the one-off pgxRows interface for the repo-wide pgx.Rows convention; reuse formatClock instead of a third duration formatter in ExtrasSection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): bump silo-plugin-sdk to v0.9.0 for VideoRecord Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(matching): exclude extras files from match queues and bulk content linking Dev verification caught extras media_files rows (content_id NULL by design) being swept into the movie/series match queues and the root-claim bulk relink: a '-featurette' suffix extra was matched onto its parent as a version, and a Trailers/ file minted a spurious local skeleton item that shadowed the extra's watch target. Add 'extra_id IS NULL' to the queue eligibility conditions, root/group claim relinks, observed-root content assignment, and the admin unmatched-files listing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): authorize local extras files through their parent item Dev verification: playback/start (and the shared MediaFileAuthorizer used by markers/subtitles/ebook reader) resolved file ownership only via episode_id/content_id, so extras files (extra_id only) 404ed. Add an ExtraLookup tier that resolves media_extras and gates on the parent item's access, mirroring the episode->series pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): resolve local extras through GetItemDetail for compat playback jellycompat PlaybackInfo (and any per-item consumer resolving arbitrary content ids) goes through GetItemDetail, which lacked the extras tier that GetWatchDetail has — so Jellyfin clients got zero MediaSources for extras. Add buildExtraItemDetail (minimal detail + ordinary playback surface, parent-gated access) as the fourth resolution tier, and map the extra type to Jellyfin's Video kind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): allow youtube-nocookie embeds in CSP; trailer modal a11y The frontend CSP's frame-src blocked the trailer modal's youtube-nocookie.com iframe (found on dev verification). Also add the missing sr-only DialogDescription and drop the redundant allowFullScreen attribute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address PR review findings for trailers/extras - Extras watch/item detail no longer stamp SeriesID/SeriesTitle for movie-owned extras (players key episodic post-roll flows off series_id); series-owned extras keep them (Codex). - processExtraFiles resolves the parent and upserts media_extras before the unchanged fast-path, and the fast-path now also compares mtime, so rematched parents / reclassified kinds / same-size replacements converge (Codex + CodeRabbit). - media_files upsert clears content/episode linkage atomically when extra_id is set (ownership mutual exclusion in one statement); the now-redundant MarkFileAsExtra helper is removed (CodeRabbit). - ScanFile's extras branch runs syncPresentLibraryState + reconcileLibraryMemberships so converting a primary file to an extra cleans stale library membership immediately (CodeRabbit). - media_extras migration adds the media_files FK as NOT VALID + VALIDATE to avoid a full-scan exclusive lock on large tables (CodeRabbit). - trailer_kinds input is trimmed/lowercased/deduped and unknown values are dropped instead of silently widening the allow-list to 'other' (CodeRabbit). - Extras authorization branches match the episode branch's posture: unconfigured lookup is a config error, nil extra is a 404 (CodeRabbit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
c29212b2c5 |
docs: handoff for account.capabilities_changed events
Design doc for a user-scoped capability-invalidation event on the existing events WebSocket, so clients refresh cached capability payloads (e.g. /downloads/capability) when admin permission or server settings change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0fb5afe479 |
feat(matching): split wrongly merged versions with watch-state reattribution; anchor group keys on provider tags (#319)
* feat(matching): split wrongly merged versions, reattribute watch state, anchor group keys on provider tags
Wrong merges (two titles normalizing to the same title+year key) stacked
different films as fake "versions" of one item with no in-app repair, and
explicit {tmdb-…}/[imdb-…] folder tags could not prevent it because the
content-group key ignored provider IDs entirely. Merges also silently
orphaned all per-user watch state.
- Anchor group keys on structured provider tags: same tag always groups,
different tags can never merge; untagged files keep title+year keys.
- media_identity_overrides: path-scoped (root/file) forced identities applied
during group inference, so admin splits survive rescans.
- internal/catalog/reattribute: shared user-state mover — exact moves for
file-linked rows, evidence-based user_watch_history classification via the
playback session log, newest-wins progress conflicts; wired into
rebindItemToExistingItem to stop merge orphaning (with S/E episode mapping).
- POST /admin/items/{id}/split (dry-run = full transaction + rollback, so
previews are exact), POST /admin/items/{id}/merge, GET /admin/items/{id}/files.
- Web admin: Split Versions dialog (files by folder → candidate search →
preview → split), Resolve link from ambiguous-roots diagnostics.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(reattribute): classify history before moving session log; cover managed downloads and series-scoped preferences
Review findings on #319, all reproduced against a migrated scratch database:
- moveFileSubset re-pointed playback_history_admin before the history
evidence query ran, erasing exactly the evidence proving a profile's plays
were all on moved files — their history stayed behind as ambiguous.
History classification now runs first; the pre-fix code demonstrably fails
TestRun_HistoryEvidenceClassification.
- Managed offline downloads (downloads.content_id/episode_id) were not
remapped on split or merge, stranding rows on the old id. Now moved per
file on splits and swept per id pair on merges/episode re-anchoring.
- Series merges left user_audio_preferences, user_subtitle_preferences,
user_series_playback_preferences (series_id-keyed) and the denormalized
user_home_item_dismissals.series_id behind. All four now move, mirroring
the provider-merge remap.
All five reattribute DB tests now verified green against PostgreSQL, with
new coverage for managed downloads, subtitle preferences, and dismissal
series ids.
Part of #318
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
42602b7896 |
feat(policy): access groups + embedded OPA policy engine with decision audit log (#282)
* docs(policy): add OPA policy engine design spec and implementation plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(deps): add OPA v1.18.2 SDK for the policy engine Pulls github.com/open-policy-agent/opa v1.18.2 (policy engine core for the upcoming internal/policy subsystem) and the transitive upgrades go mod tidy applied (otel 1.44, grpc 1.81.1, prometheus/common 0.67.5). Full build verified. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add OPA engine core, vendor scope policy, and parity suite New internal/policy package (dead code — nothing wires into request paths yet): prepared-query Engine with 25ms eval timeout and fail-closed decode, typed PDP.ResolveViewerScope, go:embed vendor bundle, capabilities lockdown for future admin-authored Rego, and vendor scope.rego reproducing access.Resolver.Resolve (library intersection, disabled-library handling, quality/rating ceilings) with a narrowing-only silo_custom.scope.override extension hook. Parity proven by 1368 dual-execution subtests against the real access.Resolver, including the nil-vs-empty AllowedLibraryIDs battery and quality/rating variation; rank tables are test-pinned to internal/access. Rego unit tests run via opa/v1/tester inside go test. Bench: ~106µs/op per scope decision incl. input marshaling. Also restores the OPA requirement to go.mod (the earlier deps commit ran go mod tidy before any import existed, so tidy dropped it). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed, corrected (quality.allowed raw-file-rank divergence), and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy document store, foundation schema, and compile-check policy_foundation migration: policy_documents (one enabled doc per domain via partial unique index — two enabled docs would define override twice and conflict at eval), immutable policy_document_versions, single-row policy_generation counter, and the partitioned policy_decisions log table (daily range partitions, no FK, denial partial index). PolicyStore: transactional version numbering (FOR UPDATE), activation that verifies compiled_ok and bumps the generation in the same tx, enable/disable with typed ErrDomainAlreadyEnabled, and a delete guard for documents with an active version. CompileCheck sandboxes admin Rego: locked capabilities (no http.send/net.*/opa.runtime), enforced silo_custom.<domain> package path, vendor+stub layering, 2s budget, structured row/col errors. Engine gains NewEngineWithCustom / NewEngineFromStore with WARN-and-skip for invalid custom rows. DB-backed tests verified against a migrated Postgres (concurrent version numbering, atomic generation bumps, activation guards). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (domain constants extracted). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add policy System lifecycle with hot reload and cross-node invalidation policy.System owns one long-lived Engine and reloads it in place when policy documents change: EventPolicyChanged on the existing ChannelAdmin bus (new cache event constant) plus a 60s generation-poll fallback for Redis-less deployments, with a generation-consistent snapshot read. Vendor compile failure is startup-fatal; store/custom failures degrade to vendor-only and the poll loop heals them; runtime reload failures keep the last known-good engine. NotifyChanged gives the future admin handlers synchronous local reload + cross-node publish. Wiring: constructed in integrated/api modes only, PolicySystem field on api.Dependencies (unused by routes yet), policy.eval_timeout_ms setting (hot-reloaded via configWatcher.OnChange; default 25ms). Verified by a full server boot smoke and DB-backed convergence tests (event + poll paths, degraded boot, last-known-good). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): add async decision logging with sampling, retention, and query repo DecisionLogger batch-inserts each node's policy decisions straight to the partitioned policy_decisions table via a non-blocking buffered channel (drop-and-count on overflow — logging never adds latency to or fails a decision). Scope decisions sample 1-in-N (default 50, setting policy.decision_log_scope_sample_rate); denials and eval errors always log; input/result JSON samples only at policy.decision_log_verbosity= verbose. Cursor-paginated DecisionRepository backs the upcoming admin log viewer. Retention via partman (daily partitions) and a PolicyDecisionLogCleanupTask honoring policy.decision_log_retention_days (default 14). PDP emits entries per evaluation; the System owns the logger lifecycle and settings hot-reload. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (removed an unused, unsynchronized PDP setter). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): add admin policy management API and capability endpoint /api/v1/policy/capability (authenticated feature detection) plus the acting-admin /api/v1/admin/policy surface: vendor Rego viewer, document CRUD with the one-enabled-per-domain conflict mapped to 409, immutable version creation (compile-checked; failed versions persist as audit history with structured row/col errors and can never activate), activate/rollback with synchronous reload + cross-node invalidation via System.NotifyChanged, stateless validate, throwaway-bundle simulate (never touches the live engine, never logs decisions), and cursor-paginated decision-log queries. Routes mount only when the policy system is wired, keeping proxy/transcode modes untouched. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here (seeded the FK'd test user; replaced an unchecked fmt.Sscanf with strconv). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add /admin/policy workspace with Rego editor, simulate, and decision log New Policy admin page (System nav group): documents list with one-enabled-per-domain conflict handling, CodeMirror 6 Rego editor (hand-rolled StreamLanguage mode) with server compile issues rendered as inline lint diagnostics, explicit Save-version vs Activate flow with confirm, read-only vendor module viewer, simulate panel with seeded example inputs, version history with rollback, and a cursor-paginated decision-log browser. Capability-gated via /policy/capability. Adds the three decision-log settings to Log Retention. First code-editor dependency in web/ (@uiw/react-codemirror + @codemirror/*), decided in the design spec. Implementation drafted by Codex (GPT-5.5) via codex exec; verified here (lint, format:check, tsc --noEmit, vitest policy suites). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for viewer scope resolution policy.ViewerResolver implements the ViewerResolver interface backed by PDP.ResolveViewerScope and replaces access.Resolver at all five construction sites: router viewer middleware, notifications scopes, the reconciler, jellycompat's scope filter, and the ABS resolver (which now accepts a pre-built resolver, preserving its PIN-at-login semantics). PIN/profile-token verification and disabled-library loading are extracted into shared exported helpers used by both implementations, so the legacy resolver stays compiled as the parity reference with identical behavior. The adapter lives in internal/policy (which already depends on internal/access transitively) — direct typed PDP calls, no new import cycle. Sites without a policy system (proxy modes, bare test routers) keep the legacy resolver until the cleanup phase. Verified: full test suite green (jellycompat TestBeginWebOperation* and one playback GPU test are pre-existing failures, confirmed identical on main), 1368-case parity suite, dedicated ViewerResolver parity/PIN/ nil-vs-empty/fail-closed tests, and a full server boot smoke. Implementation drafted by Codex (GPT-5.5) via codex exec; a first-pass reflection-based adapter was rejected and reworked into the typed in-policy adapter; reviewed line-by-line and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for acting-admin and permission gates vendor/permission.rego reproduces the acting-admin rule (admin role + primary-profile-or-none), HasEffectivePermission semantics for marker_edit, and the metadata-curation rule including the subtle admin-past-refused-bypass case that requires the explicitly ASSIGNED permission. Policy-backed middleware in policy_gates.go keeps all Go-side lookups (declared-profile primary check, item->library resolution, the 404-on-unknown-item path) and preserves the legacy status/body taxonomy exactly — proven by dual-execution middleware tests that run every scenario through both implementations and assert byte-equal responses. Permission decisions always log (allowed flag populated); simulate and the capability endpoint gain the permission domain automatically via the domain registry. Router swaps behind single constructor choice points with the legacy gates retained for policy-less wiring. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(policy): make OPA authoritative for download and playback admission decisions vendor/action.rego decides download eligibility (downloads enabled + user allowed), download-transcode eligibility (transcode enabled + user allowed + artifacts available), and playback admission (stream/transcode counts vs limits, zero = unlimited), with a tightening-only silo_custom.action override that can also clamp a quality ceiling (never widen — merged via quality.min). Go keeps everything stateful: config loading, preset-ladder enumeration, and live session counting. Downloads consult an optional ActionDecider (nil = legacy logic) mapped back to the existing sentinel errors and capability response. Playback gains a minimal AdmissionDecider hook at the exact point of the legacy limit comparison: counts snapshot under the session mutex, PDP evaluated OUTSIDE the lock, then revalidated under lock before insert (retry on count drift) — no admission ever decided on stale counts and no eval under the mutex. Deny reasons map to the legacy ErrTooManyStreams / ErrTooManyTranscodes sentinels, pinned by tests. Parity: combination tables driven against the real PresetsFor / ensureTranscodeAllowed / SessionLimits math; full suite green (known pre-existing jellycompat flakes only). Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (locking design verified line-by-line) and verified here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): satisfy tsc -b strict return typing in the Rego stream tokenizer The production build (tsc -b) rejects assigning CodeMirror's string | void next() result to string | undefined; tsc --noEmit did not catch it. Restructured the string-literal loop. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): clearer error when a decision is undefined for partial input Vendor policies index required input fields directly, so a hand-written simulate payload missing fields yields an undefined decision. Surface that as 'decision X is undefined for this input (missing required input fields?)' instead of 'empty result' — found while exercising the simulate API against a live server. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(web): set changeOrigin automatically when the API proxy target is remote Remote dev backends sit behind vhost-routing proxies that reject a localhost Host header; local targets keep the existing pass-through behavior. Enables pointing the Vite dev server at a hosted backend via VITE_API_PROXY_TARGET in web/.env.local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): redesign the policy workspace around the decision pipeline The first-pass UI was structurally generic: a five-column document table squeezed beside the editor, three equal-weight action buttons with hidden preconditions, raw version IDs, and jargon copy — nothing taught the model. The page now teaches it: - A pipeline strip states the mental model up front: Silo decides the baseline -> your overrides narrow it -> every decision is logged. Tabs renamed to Overrides / Baseline / Decision Log (ids stay stable for bookmarked URLs). - The document table becomes one card per domain (Library visibility / Admin & permissions / Downloads & playback) with plain-language descriptions, example rules, status pills (Live vN / Draft / Disabled), inline creation, and the enable kill-switch in place. - Selecting an override drills into a full-width editor with a visible lifecycle rail (Draft -> Validated -> Saved -> Live) and one contextual primary action per step; the unedited live source shows no actions until edited. Version comments appear only at the save step. - Simulate is reframed as 'Test before going live' with a human verdict chip (Allowed / Denied — reason / ceiling summary) above the raw JSON; internal generation counters no longer surface. - History uses 'Make live' with plain go-live copy; authors read 'User N'; the baseline tab explains that upgrades never touch overrides. Hand-written redesign (no Codex); verified via vitest, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(web): present the policy baseline as readable rules, not raw Rego The Baseline tab dumped five Rego modules into read-only editors. It now leads with what the rules actually do: one card per domain with plain-language statements of the shipped behavior and a note on what an override may change, plus content-rating and playback-quality tier ladders parsed live from the lib module sources (so the tiers shown are the ones the server enforces, not a hardcoded copy). The Rego source stays one click away behind a per-module accordion and remains the stated source of truth; unrecognized modules fall back to source-only. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(policy): add access-groups design addendum Groups with permission toggles become the everyday admin surface; the Rego editor is demoted behind policy.editor_enabled (default off). Restriction-only composition: group grants are an upper bound, per-user settings tighten further — same rule as the existing account/profile merge, one layer up. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): add access groups — group defaults with restriction-only composition New access_groups table + users.access_group_id (one group per user, NULL = today's behavior). Group grants are an upper bound composed with the user's own settings by strictest-wins rules — library intersection, MinQuality, AND'd booleans, strictest positive stream/transcode limits, permission-mask intersection, and a requests toggle gating CreateRequest. The merge happens in Go (access.ApplyGroupPolicy / EffectivePolicyForUser) before policy inputs are built, so vendor Rego, the parity suites, and the decision log are untouched; every enforcement surface (viewer scope in both resolvers, permission gates, downloads, playback admission, requests) consumes the effective policy and fails closed on provider errors. Changing a group's quality ceiling bumps its members' access_policy_revision, mirroring the per-user rule. Additive admin API: /admin/access-groups CRUD with member counts; PUT /admin/users/{id} + user DTOs gain access_group_id. Also demotes the Rego editor: policy.editor_enabled (default off, hot-reloaded) drives the capability endpoint's editor_available and 403-gates editor endpoints while the engine and decision logging keep running. Design: docs/superpowers/specs/2026-07-02-access-groups-design.md. Implementation drafted by Codex (GPT-5.5) via codex exec; reviewed (composition core + fail-closed call-site audit) and verified here. DB-backed group-store tests pending local Postgres recovery. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): add Access Groups admin page and gate the policy editor New /admin/access-groups: a card grid summarizing each group (member count + key restrictions), drilling into an editor that reuses the same LibraryAccessSelector and quality presets as the user editor, with toggles for downloads/transcoded-downloads/requests, concurrent-stream and transcode limits, and a permissions mask (all-assignable by default, narrowable to specific permissions). Delete warns how many members fall back to the built-in defaults. Copy states the composition rule up front: a group grants the most a member can do; their own restrictions still apply on top. The user editor gains a Group picker and read-only row; the Policy nav entry is now hidden unless the capability reports the editor enabled. Plumbing (types, hooks, user-editor picker, nav gating) drafted by Codex (GPT-5.5); the Groups page hand-built. Verified: 25 tests across the touched suites, tsc, eslint, prettier, and a production build. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): seed a Default Group and auto-assign newly created users Adds access_groups.is_default with a partial unique index (one default at most — the profiles is_primary pattern) and seeds a permissive 'Default Group' whose ceiling is a no-op, so assignment never changes anyone's effective access until an admin edits it. The seed is guarded against pre-existing defaults and name collisions; the Down migration only removes the row if it is still untouched. Assignment happens at the single INSERT INTO users choke point (UserRepository.Create): when no explicit group is given, access_group_id is filled by a scalar subquery on the default flag — NULL when no default exists. Every creation path (setup, signup, invites, OAuth, admin create) is covered by construction. Setting a new default via the API atomically clears the previous one in the same transaction. Deleting or unsetting the default is legal: new users then start with no group, which is pre-feature behavior. Implementation drafted by Codex (GPT-5.5); migration guards and the choke-point subquery reviewed line-by-line here. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): surface the default access group Cards show a Default badge; the group editor gains a 'Default for new users' toggle (with copy noting existing users are never moved); the delete dialog warns when removing the default that new accounts will start with no group. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): ship the Default Group with house-rule ceilings Seed values per product decision: 5 concurrent streams, 5 transcodes, transcoded downloads off, and a permission mask of marker_edit only (metadata curation excluded). Plain downloads and requests stay on. The Down guard matches the new values so it still only removes an untouched seed row. Only newly created users are affected; existing users are never assigned. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): retire per-user defaults — the Default Group is the sole default policy Removes both legacy 'user defaults' mechanisms now that the seeded Default Group owns new-user policy: - users.max_streams / max_transcodes column defaults drop from 6/2 to 0 (= unrestricted at the user layer), so group ceilings apply to new signups/invites/OAuth users instead of fighting stale per-user numbers. Existing rows keep their stored values — nobody is silently uncapped on upgrade. - The dead defaults.max_playback_quality / defaults.max_profiles settings validation goes away with its only writer (the User Defaults dialog, removed on the web side). Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): replace the User Defaults dialog with group-governed creation The Users page's 'User Defaults' dialog (defaults.* server settings) duplicated what access groups now do properly, and its values were only ever form prefill — no backend path applied them. The button now links to Access Groups, and the create-user form seeds unrestricted user-layer values (0 streams/transcodes, any quality, downloads allowed) so the member's group governs; per-user fields remain for tightening individual users. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(access): migrate existing non-admin users into the Default Group Existing users join the seeded Default Group on upgrade so one policy source governs the whole instance. Their per-user limits still holding the retired 6/2 column defaults are normalized to 0 in the same statement so the group's ceilings actually apply; deliberately customized values are preserved. Admin accounts stay ungrouped — scope/action decisions are role-blind, so grouping an admin would cap the server owner on upgrade. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): keep admins out of the Default Group and treat group moves as policy changes New-user creation now mirrors the migration's admin exclusion: the default access group is only auto-assigned to non-admin roles, so a fresh server owner no longer inherits the starter group's transcode denial and stream caps. Changing a user's access group now bumps access_policy_revision (the group carries permissions, quality, and limits, exactly like the per-user fields that already bump it) and triggers admin session revocation when the group actually changes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): enforce marker_edit through the PDP on marker write routes The Rego permission policy owned marker_edit but no Go caller ever consulted it: PUT/DELETE /markers went through a handler-local check that short-circuited admins and read only the user's own permissions, so group permission masks and custom policy overrides were ignored. Marker writes are now gated by router middleware like the other permission surfaces: a PDP-backed RequireMarkerEdit that evaluates the group-merged effective permissions (plus the legacy variant for proxy/test wiring without a policy system). The handler-local check and its user loader are gone. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert device/quality policy facts and honor the quality ceiling The download_transcode action check hard-coded an empty device ID and never asserted the requested quality, and no caller consumed ActionDecision.QualityCeiling — custom download policies keyed on those inputs were silently ineffective. Resolve now threads the request's device ID and requested quality into the action input, and a returned quality ceiling downscales the prepared transcode target (the ceiling applies to what is served, matching the serve-time rule in serveDownloadBytes). FileQuality and the content-rating pair stay intentionally empty for downloads — documented on downloadActionInput: those ceilings are enforced against the served artifact by the scope-derived access filter, and asserting the source's quality would wrongly deny capped transcodes. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(access): align the default-group seed assertions with the migration The DB test still asserted the earlier no-op seed (transcode allowed, unlimited streams/transcodes, null permissions); the shipped migration seeds transcode denied, 5/5 limits, and marker_edit-only permissions, so the test failed on any database with the migration applied. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): lock the Rego sandbox by builtin purity and bound compile work Exclude every nondeterministic builtin from the admin sandbox instead of denylisting names, so OPA upgrades cannot silently expose impure builtins while pure helpers like net.cidr_contains stay usable. Apply the same capabilities to the runtime engine, cap concurrent compile checks, and reject oversized sources before they reach the uncancelable compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): require literal booleans in vendor override and input checks Bare object.get truthiness treated any non-false value as satisfied, so a malformed override 'allowed' value could fail to tighten a base grant and hand-crafted simulate input could flip flag predicates. Compare against literal true so anything else denies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): surface decision log cleanup failures to the task manager CleanupDecisionLogsOnce now returns the first error alongside the deleted count so a broken partition manager or DB outage marks the scheduled task failed instead of reporting 100% success while policy_decisions grows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(playback): log admission decider errors before failing closed A policy-evaluation failure was silently mapped to the too-many-streams denial, making an engine outage indistinguishable from a real limit hit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(access): nil-guard the downloads user and restore the ABS legacy resolver effectiveDownloadUser dereferenced policy state before its nil-user check, and the ABS handler lost viewer-scoped filtering entirely when the policy system was unavailable because no legacy access.NewResolver fallback was wired like the other resolver paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): address admin policy review feedback - invalidate the version query by version_number, the key usePolicyVersion actually caches under - keep the goPrevious cursor-stack updater pure (Strict Mode double-invoke) - make version history rows keyboard-selectable like the document list - clamp download_transcode_allowed when downloads are disabled so groups cannot save a contradictory record Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): cap policy endpoint request bodies at 1 MiB The policy write endpoints (create document/version, set enabled, validate, simulate) decoded JSON bodies without a size limit, so an oversized payload buffered fully in memory before CompileCheck's 256 KiB source cap could reject it. Route all five through a shared decodePolicyRequest helper that wraps the body in http.MaxBytesReader and returns 413 with the repo's standard too_large error shape. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(access): forbid deleting or demoting the default access group Deleting the default group (or unsetting its is_default flag) left the server with no default: new non-admin users were then created ungrouped with max_streams/max_transcodes of 0 — unlimited — because the legacy per-user column defaults were retired in favor of the group's ceilings. The store now rejects both operations with ErrDefaultGroupRequired (mapped to 409); promoting another group remains the supported way to move the default, and atomically clears the previous one. The admin UI disables the delete button and the default toggle on the default group and explains the promote-another-group flow. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(web): keep unsaved policy drafts when a newer version activates elsewhere The editor state was keyed on the active version's id/sha, so a background refetch after another admin (or another tab) activated a version remounted the editor and silently discarded the dirty draft. PolicyEditorPanel now pins the seed it is editing against and only adopts an incoming seed when nothing can be lost: the editor is clean, the draft already equals the incoming source (the same-admin activate flow), or the selection moved to a different document. Otherwise the pinned editor stays mounted and an inline notice offers an explicit "Load live version" action. Part of #272 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UtnZ2Uewzo959hpneLrtRN * fix(policy): fail reloads on invalid custom sources and surface degraded/apply state A stored custom source that stops compiling used to be silently skipped on reload: the bundle widened to vendor-only for that domain while the generation reported fully applied. Reload is now strict — a bad enabled source fails the reload and the last known-good engine keeps serving. Boot keeps its vendor fallback for availability, but skips are recorded on the engine and exposed (with store-outage reasons) through System.DegradedState and additive degraded fields on GET /policy/capability. Activate/SetEnabled re-run CompileCheck instead of trusting the stored compiled_ok flag. Mutation endpoints also no longer conflate persistence with live apply: activation/enable responses carry additive applied/failed_step/ loaded_generation fields and return 202 when the store change persisted but the local reload failed. Addresses review findings C1, C2, and the degraded-signal gap (6.1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): type deny reasons across the contract and enforce profile_verified Deny handling used to branch on exact free-text reason strings in three Go consumers, and playback reported ANY unrecognized reason — including custom override free text and engine failures — as a stream-limit error. Decisions now carry a stable reason_code (custom overrides always get custom_denial); downloads, the metadata-curation gate, and playback admission switch on codes, with a new ErrPlaybackNotAllowed -> 403 playback_not_allowed mapping for non-limit denials. Rego tests pin every vendor code. The scope contract's tighten-only profile_verified output was also emitted but never consumed; a policy revocation now surfaces as ErrProfileUnverified (403 profile_unverified) instead of silently proceeding. Addresses review findings 6.2 and C4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): close the dual-library disabled-scope bypass in direct item authorization EnsureAccessible, EnsureAccessibleIDs, and FilterAccessibleContentIDs gated library access with allow/deny predicates over a single joined media_item_libraries row, so an item linked to BOTH a passing library and a disabled one satisfied the disabled check via the passing row — a direct-ID bypass of disabled-library scope on the detail, media-file, playback, and download paths. All library access predicates now share one helper (libraryAccessConditions) emitting independent EXISTS / NOT EXISTS subqueries, the semantics GetByIDsWithAccess already used, including the orphan-item membership guard for disabled-only scopes. SQL-shape tests pin every builder and a DB-gated regression test covers the dual-library item end to end. Addresses review finding C3 (plus the same shape in buildFilterAccessibleContentIDsSQL, which the review did not flag). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): serialize quota check and row creation under a per-user advisory lock The concurrent-download quota was check-then-insert with nothing serializing the pair: parallel creates could all observe free quota before any row existed, bypassing the cap and stacking artifact encode jobs. All four check->insert spans (ephemeral original, artifact-backed, series batch, managed batch) now run inside Repository.WithUserQuotaLock — a pg_advisory_xact_lock keyed by user, so the serialization holds across nodes. The artifact path keeps the limiter-before-Ensure ordering (a rejected request must not leave an encode job behind) by holding the lock across Ensure. Managed-entry replacement stays quota-exempt and lock-free. A DB-gated barrier test races 8 creates against a cap of 1. Addresses review finding C5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): assert served quality at create time for original and remux downloads Direct-original and remux downloads serve the source resolution unchanged, but create-time policy checks left file_quality empty — an over-ceiling source registered a row serveDownloadBytes could never satisfy. Resolve now runs a final download action check with FileQuality populated on those two paths (capped transcodes keep the ceiling-on-artifact behavior), a custom override ceiling below the served resolution denies, and quality_ceiling_exceeded maps to ErrQualityUnavailable. The ActionInput contract now documents exactly when file_quality and the rating facts are supplied so custom policy authors are not misled. Addresses review finding C6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(policy): guard activation against slow overrides and make eval timeouts observable A custom scope override that exceeds the 25ms eval budget compiled fine, activated fine, and then converted to 500s on every authenticated request — server-wide lockout authored in the admin editor. Activation and enable now run GuardEvalCost: the candidate source is evaluated on a throwaway engine against a canned representative input under the live budget, and a source that cannot complete is rejected 422 with ErrPolicySlowEval before it goes live. Runtime timeouts keep failing closed but now carry a distinct ErrPolicyEvalTimeout sentinel, an Error log, and a per-engine counter exposed as eval_timeouts on GET /policy/capability so intermittent near-budget policies are attributable. Addresses review finding C7. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt remediation files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
430224a1b9 |
perf: cut home-screen, Continue Watching, and Latest latency; cache shared home rails (#292)
* perf(jellycompat,sections): bound resume scan, batch leaf detail progress, widen section concurrency Three low-risk fixes from the section-fetch performance investigation (docs/superpowers/plans/2026-07-03-section-fetch-performance.md): - jellycompat: bound loadProgressPage at resumeScanMaxRows=300 so a single request never pages through more than that many in-progress rows. The cap is unconditional: it also covers the sparse-visible-set case (a heavy watcher whose recent rows are mostly dismissed/superseded, or a Series/Season-only request that matches no leaf in-progress row), where the page never fills and the loop would otherwise scan the entire history — previously an O(history) scan reaching tens of seconds. In the common case the loop exits far earlier, so the cap only bounds the pathological worst case; 300 leaves ample headroom to fill a ~20-item Continue Watching page. Beyond the cap the reported total is a clamped lower bound. Covered by TestLoadProgressPage_BoundsScanForSparseVisibleSet. - jellycompat: batch the leaf-item (movie/episode) progress lookup in GetItemDetailsByIDs via ListProgressWithCompletedHistory instead of a per-item GetProgressWithCompletedHistory (~100 sequential queries for a 50-item detail page). Series keep the per-item episode-rollup path (they own no progress row). Output is unchanged; a batch-lookup failure is now logged rather than silently dropping played state for the whole page. - sections: raise fetchAllMaxConcurrency 4 -> 6 to cut FetchAll wave count for large home layouts, staying within the default 20-conn pool. Part of the home/continue-watching latency work. AI-use disclosure: implemented with AI (Claude) assistance. * perf(jellycompat): keep Latest browse on the cross-library fast path under isPlayed /Items/Latest with isPlayed=false is the highest-frequency compat browse (~10.8k calls/day). The played overlay can't be pushed into SQL, so browse over-fetches and filters locally. The cross-library recently_added fast path (BrowseRecentlyAddedAcrossLibraries: one ~1ms index walk per library) was gated on Offset==0, so a heavy watcher who had already seen the newest items needed a 2nd chunk and fell through to BrowsePage — a whole-catalog MIN(first_seen_at) + GROUP BY HashAggregate over ~147k movies measured at ~755ms per call (0.8-1.6s observed end-to-end). Fetch the entire over-fetch budget (maxScannedRows) in a single merged fast-path walk instead of paging into BrowsePage, so the loop fills from one call. The clamp caveat (MaxLimit=1000 leaves a fall-through only for requestedLimit>200, off the Latest hot path) is documented inline. Part of the home/browse latency work. AI-use disclosure: implemented with AI (Claude) assistance. * fix(jellycompat): scope resume scan cap to resume path and bound the fast-path loop Addresses PR #292 review feedback: - Codex (P2): the resumeScanMaxRows cap was applied unconditionally in the general loop, which also paginates the completed (watched-items) list. Gate it on resumeFiltered so the completed path keeps exact TotalRecordCount and deep StartIndex pagination. Covered by TestLoadProgressPage_CompletedScanNotCapped. - CodeRabbit (Critical): the earlier raw-offset fast-path loop — the default Continue Watching shape and the sections-fallback route — had the same unbounded-scan bug and was not covered by the cap (the existing test forces EnableTotalRecordCount=true, routing around it). Bound it with the same resumeScanMaxRows guard. Covered by TestLoadProgressPage_BoundsFastPathScanForSparseVisibleSet. - CodeRabbit (Minor): tag the doc's fenced example blocks as text to satisfy markdownlint MD040. AI-use disclosure: implemented with AI (Claude) assistance. * perf(sections): cache shared user-agnostic home rails per access scope Home-screen rails that are identical for everyone who can see the same libraries (recently added, recently released, genre, trending on server, most watched, new to library, critically acclaimed, award winners, format showcase, seasonal, mood, trending discover, admin-curated lists, and library collections) were rebuilt from Postgres once per request, per user. Only the overlay on top of each row (watched flags, play position, presigned poster URLs) is actually per-user. Insert a process-global resolved-list cache at the FetchOne choke point in internal/sections. Each cacheable row is built once per access scope, held with a 15m TTL, and refreshed in the background 3m before expiry; singleflight collapses cold-miss stampedes into a single build. The per-user overlay still runs fresh in buildSectionsResponse, so no profile state is ever shared. Random and per-user rows (continue watching, next up, recommendations, hidden gems, forgotten favorites, activity feed, user collections) bypass the cache. The access-scope key captures every access boundary the fetch path enforces -- section identity (type + id + config hash) + item limit + accessible and disabled libraries + max content rating + excluded media types + name prefix + allowed-content-id allowlist -- and nothing per-user, so entries are safely shared. Empty membership is never cached (avoids freezing a transiently empty rail); background refreshes are bounded by a timeout. Scale (analytical, derived from the cache behavior -- not a measured latency): for the user-agnostic rows, Postgres section-query volume collapses from O(rows x concurrent requests) to O(rows x distinct access scopes) per 15m refresh window, because most users share a handful of access scopes. Illustrative -- 40 cacheable rows on a home screen, 1000 concurrent users falling into ~5 distinct access scopes: - before: ~40 x 1000 = ~40,000 section queries per wave of home loads - after: ~40 x 5 = ~200 builds per 15m window (plus one background refresh per row per scope), i.e. a warm home load runs zero section queries for these rows. That is a ~99% reduction in shared section-query load at that concurrency; the win grows with concurrency and shrinks as access-scope diversity rises. Design/plan doc added under docs/superpowers/plans/. * perf(jellycompat): serve per-library Latest via the cached recently-added section A jellyfin-compat per-library /Items/Latest rail is the same user-agnostic list as the native "recently added" library rail -- both order by mil.first_seen_at DESC. It was rebuilt on every request through directContentService.BrowseItems, missing the resolved-list cache entirely. Route per-library Latest for movies and series libraries through the native section fetch instead, so it reuses the shared cache. HandleLatest resolves the library's type once, and for a movies/series library builds a synthetic SectionRecentlyAdded with the same type + config + limit + access scope the native rail uses and calls FetchOne; the per-user overlay (favorites, progress, episode targets, presign) is extracted into buildLatestItemDTOs and shared by both the native and BrowseItems paths, so no overlay logic is duplicated. Cached *models.MediaItem values are read-only -- LocalizeItemModels deep-copies before any presign mutation. To let the two surfaces share one entry, resolvedListCacheKey no longer includes the arbitrary section ID: every cacheable section type derives its membership from type + config + limit + scope, never from its own ID (audited all 14 cacheable types plus the library-collection path; the sole s.ID read lives in the non-cacheable user-collection branch). A native recently-added rail and the compat Latest for the same library + scope now collapse to ONE cache entry, built once and reused. Access-scope isolation is unchanged -- the removed ID never carried access information, and every access boundary (libraries, rating cap, excluded types, content allow-list, name prefix) still keys the entry. Guardrails: the native path is restricted to movies and series libraries; every other library type (ebook, music, manga, mixed) is ignored and keeps its exact BrowseItems behavior -- important because an unfiltered recently-added fetch would otherwise surface non-video items to Jellyfin clients that only expect video. Deeper pages, played-filter and backdrop-required requests, a client asking for a type other than the library's own, and any FetchOne error also fall back to BrowseItems. Chosen over an alternative that gave the synthetic section a deterministic ID (which kept two separate cache entries): both returned identical data with similar complexity, so the shared-entry design won. * fix(sections,jellycompat): post-review fixes for the shared-list cache and Latest path Consolidates fixes from the branch's adversarial review and PR #292 review comments into one commit: - Latest fast path: fall back to BrowseItems when a request carries a genre, name-prefix, or person filter (the synthetic recently-added section cannot express these, so serving it unfiltered would return a wrong, broader set). Eligibility is decided by latestFastPathEligible and covered by a test. - Clamp the /Items/Latest page size to compatBrowseMaxLimit before building the section, matching the BrowseItems fallback, so a large client Limit can't drive an oversized recently-added fetch or explode the shared cache key with unbounded ItemLimit values. - Evict expired entries from the process-global resolvedListCache: resolvedListSet sweeps expired keys at most once per minute, bounding the map to scopes seen within one TTL window. Covered by TestResolvedListCacheEvictsExpiredEntries. - Log a short digest of the cache key (resolvedListLogKey) instead of the raw key in the background-refresh panic/error paths, since the key embeds user-controlled access-scope fields such as NamePrefix. Skipped review comments (verified already fixed or stale against current code): the resume fast-path scan bound and watched-items cap (04d2e795) and the docs fence-language tags (already addressed). Build, vet, and go test -race pass for internal/sections and internal/jellycompat. * perf(plugins): cache plugin installations in-memory, invalidated on lifecycle change ## Problem Every poster/image on a warm home rail re-read plugin_installations from Postgres to answer "is this plugin enabled?" and to acquire the plugin client (Source A: metadata chain buildProviders enabled-check; Source B: ensureClient -> loadInstallation). Plugin-resolved image URLs are never URL-cached, so the plugin source and the DB read behind it fired again on every identical warm request; 100% of images in the target library are plugin-backed. ## Solution - Guarded in-memory installation cache (map[int]*Installation + RWMutex) in plugins.Service. loadInstallation reads through it; the requireEnabled gate stays after the cache read so ErrInstallationDisabled semantics are unchanged. invalidateInstallationCache clears it and is self-registered as a lifecycle hook, so Service.OnLifecycleChange wipes it on install/enable/disable/update/ uninstall. - A generation counter closes an invalidate-vs-repopulate race: captured before installations.GetByID and re-checked under the write lock, so a row fetched before a lifecycle mutation is never written into a freshly cleared cache (would otherwise resurrect a just-disabled plugin). - Route the metadata chain enabled-check through the same cache via a structural InstallationEnabledChecker interface (nil-safe: falls back to the pool query when no checker is injected), wired in cmd/silo/main.go. ## Post-review fix (auto-update reliability blocker) AutoUpdateService mutated installations (new InstallPath, old dir deleted) on the default auto update policy without firing OnLifecycleChange, leaving the cache stale and breaking plugins with "stored plugin manifest mismatch" until restart. It now takes an onChange callback wired to Service.OnLifecycleChange and fires it once per Check run that mutated a row. ## Verification go build/vet, go test ./internal/plugins/... ./internal/metadata/... (-race). Tests: cache hit/invalidation, racing-invalidation guard, IsInstallationEnabled, auto-update fires onChange. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(jellycompat): batch per-item presign, and enrich series on the cached Latest path ## Problem List rails presigned each item's poster/backdrop/logo/still image individually (~160 singular resolver calls for a 40-item page where 4 batched calls suffice), and ItemsHandler carried a near-verbatim duplicate of the batch presigner. ## Solution (batching) Promote the batch presigner to a shared package-level presignCompatListItems (presign_list.go) with a generic collectImagePaths[T]; convert the per-item loops (cached home/Latest rail, favorites, batch loaders, userdata favorites) to one batched PresignImageURLsWithExpiry per image type per page; batch the season/episode collections; delete the three duplicate presign helpers. URL output is unchanged (verified byte-for-byte). ## Post-review fix (series Latest data-parity regression) The native cached Latest fast path built items via compatListItemsFromModels + buildLatestItemDTOs and never ran the series watch-state rollup, so a series library's Latest lost Played / UnplayedItemCount and page 1 disagreed with the BrowseItems fallback. enrichSeriesUserData is promoted to the ContentService interface and called on the native path (reused, not duplicated). ## Verification go build/vet, go test ./internal/jellycompat/... ./internal/catalog/... Tests: bounded presign invocation counts + per-item URL mapping; series rollup populated on the native Latest path. ## AI-use disclosure Implemented with AI assistance (Claude). * perf(sections): gate personalized rails out of the shared cache; widen refresh lead ## Problem 1. The shared home-rail cache whitelisted custom_filter/genre sections by TYPE alone, but those route through fetchFiltered -> ParseQueryDefinition and can carry personalized (per-profile) rules/sorts (watched, favorited, in_watchlist, in_progress, last_watched; sorts progress/date_viewed/plays). Their membership is per-profile yet the cache key excludes userID/profileID, so a personalized rail built for one profile was served to others in the same access scope for up to 15m -- a cross-profile watchlist/watch-state leak. 2. The background-refresh lead was tuned so steady traffic is served a warm entry from a longer soft window. ## Solution - Add QueryDefinition.IsPersonalized() (reusing the existing QueryFieldRequiresProfile/QuerySortRequiresProfile helpers). isCacheableSectionType now parses the section QueryDefinition and refuses to cache custom_filter/genre when personalized; non-personalized definitions stay cacheable. Seasonal/mood/trending build their definitions server-side and stay unconditionally cacheable. - resolvedListRefreshLead 3m -> 10m (soft threshold builtAt+5min instead of builtAt+12min). ## Verification go build/vet, go test ./internal/sections/... ./internal/catalog/... (-race). Test: personalized custom_filter/genre not cacheable; non-personalized are. ## AI-use disclosure Implemented with AI assistance (Claude). * fix(sections,metadata): post-review fixes for shared cache and plugin chain staleness Addresses three review findings on PR #292: - sections: canonicalize section config JSON before hashing so configs differing only in whitespace/field order share a cache entry (native + jellycompat rail sharing). Added TestHashSectionConfigCanonicalizes. - metadata: invalidate the resolved-chain cache on plugin lifecycle changes; the installation-enabled check already reads the invalidated plugin cache, but resolveChainCached could serve a stale provider chain for up to chainCacheTTL after a provider's availability changed. - jellycompat: move ctx to the first parameter of presignCompatListItems for consistency with the other presign helpers. Skipped the episode-image presign batching nitpick: the resolver already dedupes+singleflights, so it is a Minor perf-only item not worth the two-pass refactor risk in this pass. * fix(sections,jellycompat): harden shared rail cache and Latest fast path per review Addresses the eight findings from the deep review of this PR: - Detach the blocking cold-miss rebuild from the singleflight leader's request context (context.WithoutCancel + the shared 30s build timeout) so one client disconnect no longer fails every collapsed waiter and leaves the entry uncached. - Stop client-controlled values minting unbounded cache entries: the compat Latest fast path now always fetches a fixed 100-row budget and slices to the requested limit (one entry per scope+library instead of one per Limit value), and an unrecognized MaxOfficialRating string disqualifies the fast path instead of entering the global cache key. - Add release_date to the sections item projection/scan so movies served via the Latest fast path keep PremiereDate (Jellyfin default-set field) in parity with the BrowseItems fallback. - Fall back to per-item progress lookups when the batched leaf progress query fails, restoring one-item-at-a-time degradation instead of blanking played state for the whole page. - Derive cache eligibility from a single source of truth: fetchSection and isCacheableSectionType now share the userAgnosticSectionFetcher table, whose no-userID/profileID signature makes a fetcher drop out of the cacheable set at compile time if it ever gains per-profile inputs. - Decide Latest fast-path eligibility off the actual browse params the fallback would receive, so any filter later added to buildBrowseParams automatically disqualifies the cached path; share one compatDefaultBrowseLimit constant between both paths. - Extract AccessFilter.WriteAccessScopeCacheKey as the shared, security- critical serializer for all access-scoped caches (resolved-list, editorial candidates, audiobook groups); the editorial key now captures ExcludedMediaTypes, which its loaders already applied in SQL. - Strip leaked agent-transcript markup from the section-fetch plan doc. go build ./..., go vet, gofmt clean; go test -race on internal/sections, internal/catalog, internal/jellycompat passes (TestBeginWebOperation* failures are the known pre-existing flakes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
604bbf1a0f |
feat(playback): unified restart-resilient playback (native + jellycompat) (#174)
* feat(playback): unified restart-resilient playback via shared TranscodeManager Make direct, remux, and native HLS transcode sessions survive a server restart through one shared flow instead of per-method paths. A missing in-memory session becomes a reconstruct trigger, not a 404: the server rebuilds the session from a tiny durable recipe card plus the position the client re-supplies on its next request. - internal/playback/transcode_manager.go: shared TranscodeManager owning the transcodes map, recipe-card lifecycle, reconstruct single-flight + concurrency cap, LoadOrReconstructSession front door, ReconstructSession / ReconstructTranscode, and orphan cleanup. ~90% is logic moved out of the native handler (no behavior change), not new surface. - internal/playback/recipecard.go + recipecard_postgres.go: RecipeCard with a PlayMethod discriminator (direct/remux/transcode; empty decodes as transcode for back-compat) behind a swappable, nil-safe RecipeStore interface backed by transcode_recipes. - internal/playback/session.go: RegisterReconstructed inserts a rebuilt Session under its existing id (no UUID mint, no limit double-count, race-yielding). - internal/playback/transcode.go: CloseProcess keeps the output dir so a reconstruct winner keeps serving; Close removes it. - internal/api/handlers: drain the transcode lifecycle into the manager; wire reconstruct into the stream/segment serve paths; re-bind ownership to the live caller (refuse userID==0/mismatch); card-aware orphan cleanup. - migrations: add transcode_recipes (expires_at TTL, filter-on-read, indexed). Ownership stays two-factor: an authenticated caller AND a session.UserID that matches; the card stores no secrets and identity is re-resolved per request. Tests: recipe-card round-trip/legacy-decode/disabled-noop, RegisterReconstructed insert/race/concurrency, close-vs-close-process dir semantics, the LoadOrReconstructSession status matrix, and the reconstruct concurrency cap. AI-use: implemented with AI assistance (design, implementation, adversarial review). * feat(jellycompat): reconstruct transcodes across restart via shared manager Bring Jellyfin (jellycompat) HLS playback onto the same restart-resilient flow as the native path. Previously jellycompat owned a separate PlaybackHandler with a private transcodes map and a duplicated transcode lifecycle that never grew the reconstruct half, so an in-flight Jellyfin transcode died on restart and the next segment request 404'd. - Embed the shared playback.TranscodeManager and delete the duplicate lifecycle, so jellycompat gets reconstruct, the concurrency cap, the node-affinity rule, and the card lifecycle for free. - internal/jellycompat/playback_sessions_postgres.go: DurableCompatPlaybackStore, a write-through cache over jellycompat_playback_sessions behind the new CompatPlaybackStore interface (nil pool degrades to cache-only). This persists the load-bearing PlaySessionId -> UpstreamSessionID mapping (plus media sources, route item id, seek) so it survives a restart instead of vanishing with the map. - Write a recipe card on compat transcode start keyed by the upstream session id, using the native StreamAppUserID so the ownership re-bind matches; reconstruct the upstream session and the transcode seeked to the requested seg_NNNNN. - migrations: add jellycompat_playback_sessions (expires_at TTL + compat_token index, full PlaybackSession in data JSONB). Auth is mapped to the native user id before reconstruct so the same two-factor ownership check and userID==0/mismatch refusal apply unchanged. Tests: DB-gated (SILO_TEST_DATABASE_URL) durable-store round-trip proving a session written by one instance reloads in a fresh one (the restart case), plus a nil-pool cache-only path; existing handler tests updated to the manager. AI-use: implemented with AI assistance (design, implementation, adversarial review). * docs(playback): consolidate unified playback reconstruction design Replace the three overlapping playback docs (the native Postgres restart-resilience spec, the jellycompat plan, and the unification spec) with a single self-contained design at docs/superpowers/specs/unified-playback-reconstruct.md. The doc leads with the unified design — the one-idea reconstruct model, a strong visual flow of a restart mid-playback, the shared TranscodeManager + recipe card, the two swappable durable stores, security, the concurrency cap and node-affinity constraint, preconditions, and verification. The design history and rationale (reconstruct-not-rehydrate, phased delivery, Redis-vs-Postgres, token-as- descriptor, failure analysis) move to an appendix. It references no other md file. AI-use: written with AI assistance. * fix(playback): address review on restart-resilient playback Four fixes from PR review of the unified reconstruction work: - Rewrite the recipe card on audio-track change. HandleChangeAudioTrack only updated the in-memory session/transcode, so after a restart reconstruct resumed with the stale AudioTrackIndex/TranscodeAudio (and stale play method) from the start-time card. Re-save the card (direct/remux/transcode) with the switched state, mirroring the start-card pattern. - Guard nil TranscodeManager in LoadOrReconstructSession and ReconstructSession. StreamHandler.TM is documented optional (tests/minimal setups); a missing session previously panicked in recipeEnabled instead of returning SessionMissing. ReconstructTranscode already guarded nil; make the two siblings consistent. - Reject direct/remux cards in doReconstructTranscode before spawning ffmpeg, so a non-transcode card id can never enter the HLS reconstruction path. - Log a non-success status from the remote transcode-node DELETE in CloseTranscodeSession; a 401/404/500 was previously silent. AI-use: implemented with AI assistance. * fix(playback): harden restart-resilient compat sessions * feat(playback): token-carried reconstruction across restarts Build on the shared TranscodeManager (introduced earlier in this branch) so a playback session survives an API-server or transcode-node restart without the client re-negotiating, and retire the Postgres transcode_recipes store in favor of a recipe carried inside the signed stream token. - RecipeCard encodes the byte-affecting encode parameters and rides inside the stream token; LoadOrReconstructSession rebuilds the in-memory Session (and, for integrated transcodes, the ffmpeg process) on a cold miss, single-flighted per session and paced by a spawn semaphore. Removes recipecard_postgres.go and the 20260617233705_add_transcode_recipes migration. - transcodenode reconstructs a lost ffmpeg node-side from the forwarded token. - TR-lease: proxy/streamauth enforce a revocation deny-marker on every served segment, with a 500ms Redis timeout, a bounded per-session "allowed" cache (3s TTL, expiry-first graceful eviction), and a degraded-fail-open counter. Review hardening folded in: - Manifest/segment handlers do the in-memory session lookup first and only verify the stream token on a reconstruct miss (token HMAC was per-segment). - Copy-mode reconstruct never applies the encoded-only seg*dur seek, at spawn time or via the recovery path: RestartSeekTarget reports "unresolved" for a copy session whose manifest cannot yet map the segment, so the client retries instead of seeking to a fabricated source time. - Crash teardown is a compare-and-delete (CloseTranscodeSessionIf returns whether it matched); the crash closure tears down the playback session only when it matched, so a session reconstructed under the same id is not killed. - Reconstruct enforces the same per-user stream/transcode caps as a fresh start (RegisterReconstructedWithLimits), closing a token-replay slot bypass. AI-use disclosure: implemented with AI assistance (Claude Code), including a two-round multi-agent adversarial review whose findings drove the hardening. * feat(jellycompat): node-side transcode reconstruct via shared recipe store Make Jellyfin-compat playback sessions survive a server or transcode-node restart by reusing the shared TranscodeManager reconstruct path and a durable recipe store, on top of the durable compat session store added earlier in this branch. - Node-side transcode reconstruct goes through the shared recipe store; the recipe is persisted to the control-plane store (Redis) when a dedicated transcode node is used so the node can rebuild ffmpeg after its own restart. - Adopt the shared manager's API (3-arg OnFFmpegCrash carrying the dead session, guarded CloseTranscodeSessionIf, RegisterReconstructedWithLimits). Review hardening folded in: - Recipe lifecycle: noderecipe.Store gains Delete, called on deliberate teardown (stop, method-switch discard, node stop/force-reload) so a stopped session cannot be resurrected by a buffered request after a node restart; crash paths intentionally keep the recipe so a resume can reconstruct. - Crash closure tears down the upstream session only when the guarded transcode close matched, so a reconstructed successor is never left orphaned. - Copy-mode segment recovery surfaces a retryable not-found instead of a wrong-position restart, matching the native and node paths. - Durable Update is now a SELECT ... FOR UPDATE transaction, removing the lost-update clobber that could silently drop a transcode recipe. - Empty-token route resolution no longer falls back to an unbounded full-table scan; DB expiry filters bind the injected clock; the redundant re-Get is gone. AI-use disclosure: implemented with AI assistance (Claude Code), including a two-round multi-agent adversarial review whose findings drove the hardening. * docs(playback): consolidate restart-resilient playback design Replace the superpowers spec with a single architecture record describing the token-carried recipe card, the shared TranscodeManager reconstruct path for direct/remux/transcode, the jellycompat durable session + node recipe store, and the revocation-lease model with its fail-open tradeoff. AI-use disclosure: written with AI assistance (Claude Code). * docs(playback): correct jellycompat node-recipe rationale in comments The noderecipe / transcode-node / jellycompat comments justified the Redis recipe store with "a Jellyfin client cannot round-trip a token". The real reason: the node-hop token is server-minted and could carry the recipe, but the recipe is mutated in place under a stable session id (a /Sessions/Playing/Progress audio switch restarts ffmpeg without re-minting the client's token) and a third-party Jellyfin client cannot be driven to refresh a stale token, so the node must reconstruct from a server-authoritative, node-reachable store. Aligns the comments with docs/architecture/restart-resilient-playback.md §10. Comment-only; no behavior change. * refactor(playback): remove deny-lease revocation, defer to future PR The deny-lease stream-revocation mechanism (the internal/streamauth package, its silo:streamauth:<sid> Redis markers, the proxy Allowed() enforcement, and the admin Stop/Terminate deny write) only ever enforced on the offload-proxy topology and was a silent no-op on the integrated single box and the dedicated transcode node. Rather than ship a partial revocation feature that looks complete but isn't, remove it wholesale and defer a uniform cross-topology revocation design to a dedicated follow-up. Removed: internal/streamauth (package + tests); the LeaseDenier field, StreamLeaseDenier interface, and denyStreamLease helper in playback.go; the admin deny write; the router/main wiring; and the proxy verifyToken Allowed() gate. The unified-reconstruct core (recipe-token, LoadOrReconstructSession) is orthogonal and untouched. Known limitation (now on every topology): admin Terminate and user Stop tear down the live in-memory session and ffmpeg producer, but a still-valid stream token can reconstruct the session until its 24h TTL expires. No node-side byte-withholding ships in this PR. docs/architecture/restart-resilient-playback.md is updated to mark the revocation/deny-lease sections as deferred and to drop the overstated "instant revocation on admin kill" claim. * fix(playback): allow zero-caller bearer on transcode reconstruct The authless HLS transcode delivery routes (master.m3u8 / segment) treat the session UUID as the bearer credential, so a real request carries requestUserID == 0. The live serve path already allows this, but ReconstructSession hard-rejected a zero caller, so a request that worked before a restart became SessionMissing -> 404 after the in-memory session was gone, breaking the restart resilience these routes advertise. Match the live-path contract in LoadOrReconstructSession: allow a zero caller (UUID-as-bearer) and refuse only a non-zero caller that mismatches the card owner. The reconstructed session is bound to card.UserID either way. Adds TestReconstructSession_Ownership covering both cases. * fix(jellycompat): re-persist recipe on local audio switch A Jellyfin client switching audio on an integrated/local compat transcode restarted live ffmpeg with the new track but did not re-persist PlaybackSession.Recipe. The remote branch already re-persists via startRemoteTranscode -> persistTranscodeRecipe. After a central restart, reconstruct rebuilt ffmpeg from the stale Recipe.AudioTrackIndex, so the integrated session resumed on the original audio track. Persist the updated recipe (best-effort) after a successful Restart in the local branch, mirroring the remote branch, so the durable Recipe.AudioTrackIndex tracks live ffmpeg. Adds a regression test. * fix(playback): strip stream token from proxied transcode-node URL proxyToTranscodeNode appended the client's raw query string to the internal transcode-node URL and logged that URL on transport failure. When a remote transcode runs without a separate proxy node, that query carries ?st=<signed JWT> — a 24h bearer reconstruction descriptor exposing the media path and recipe claims — placing the token into internal requests and error logs. Strip the "st" param before building targetURL, preserving any other query params. The token is neither forwarded to the node nor present in the logged URL. Header-forwarding of the token (so the node can reconstruct) is a separate follow-up (#6). * fix(playback): fail open on transient limit-provider error in reconstruct During the reconstruct wave right after a restart (Postgres under peak load), a transient limit-provider DB error was collapsed into a hard 404, permanently stopping playback for a user within their limits. limitsForUser wrapped any provider error, RegisterReconstructedWithLimits propagated it, and ReconstructSession mapped every error to SessionMissing -> 404 - indistinguishable from a genuine over-cap rejection. Distinguish the two: tag provider errors with a new ErrLimitProviderUnavailable sentinel and, during reconstruct, fail OPEN on a provider error (admit via RegisterReconstructed + log a degraded warning) rather than refuse - mirroring the reliability-first fail-open-on-dependency-error philosophy. A genuine ErrTooManyStreams / ErrTooManyTranscodes over-cap still refuses. Adds tests for both the fail-open and still-refused paths. * fix(playback): forward stream token to transcode node as header The dedicated transcode node's reconstruct path reads the stream token only from the X-Silo-Stream-Token header, but proxyToTranscodeNode forwarded only the node-API bearer token (and #5 now strips st from the URL). So when the central API proxied to the node and the node self-restarted, it could not reconstruct from the recipe-complete native token -> 404. Capture st before stripping it from the URL, verify it at the API boundary (streamtoken.Verify + SessionID match, mirroring the node's own check), and forward it as X-Silo-Stream-Token. Best-effort: a missing/invalid token never blocks the live proxy, and the token is still kept out of the forwarded URL and logs. * fix(playback): restart node ffmpeg on native remote audio switch A native audio-track switch on an offloaded/remote transcode was a no-op at the node yet returned 200 with a fresh URL: HandleChangeAudioTrack restarted ffmpeg only when the API owned a LOCAL TranscodeSession, so for an offloaded transcode the node kept serving the OLD audio (the node consults the token only on a session miss). The replacement URL was also minted from identity- only claims, so a later node restart 404'd. For the offloaded transcode case (detected via session.TranscodeNodeURL), POST a fresh /transcode/start to the node with the new AudioTrackIndex (handleStart tears down and restarts ffmpeg) and mint the replacement proxy URL from a full RecipeCard so reconstruct survives a node restart. The encode recipe is derived from the durable session target fields plus the file, mirroring HandleStartTranscode. A concrete SegmentDuration (playback.DefaultSegmentDuration) is embedded rather than 0: the node's token completeness gate treats SegmentDuration<=0 as incomplete and falls back to a recipe store the native path never populates, which would 404 on a node restart - the exact resilience this path provides. A failed node POST now surfaces 502 rather than a false 200. Remux and non-offloaded (local) transcode paths keep their prior identity-claim URLs unchanged. Known limitation: Session does not persist the original SegmentDuration or SubtitleTrackIndex/SubtitleBurnIn, so a remote audio switch resets subtitle selection to none and assumes the default segment length; a client that started with a non-default segment length will resegment on switch. Making that state durable on the session is a follow-up. * docs(playback): scrub stale deny-lease/revalidator comments The deny-lease revocation mechanism and its "central revalidator" were removed earlier in this branch, but four comments still described them as live (transcode_manager.go, noderecipe/store.go, streamtoken/token.go, proxy/server.go). Reword them to match the shipped behavior: ownership claims are re-resolved at reconstruct, the noderecipe store shares Redis only with the node-session tracker, and a sub-TTL hard cut depends on a node-side revocation mechanism that is deferred to a future PR. * fix(jellycompat): surface durable playback-session write failures DurableCompatPlaybackStore.Update applied the in-memory mutation and then swallowed every Postgres commit-failure path, returning nil. Callers that promise restart resilience (persistTranscodeRecipe's recipe write, the upstream-session binds in streams.go) were told the session was durably persisted when only the cache held it, so a transient DB hiccup could leave the next restart reloading a stale row (wrong audio track) or 404ing. updateDB now returns the genuine DB round-trip error (begin/query/unmarshal/ marshal/exec/commit); Update propagates it while still applying the in-memory mutation so live state stays correct. A nil pool and a genuinely absent/expired row remain best-effort (return nil) — only real infrastructure failures propagate, so existing rollback paths fire exactly when durability is lost. Part of #174 * fix(playback): re-inject stream token into proxied transcode manifests API-proxied remote transcode manifests dropped the reconstruct token from their segment URLs, so playback died after a node or API restart. When a remote transcode has no separate proxy node, the client loads its manifest via the API-local path; proxyToTranscodeNode strips the signed token ("st") from the forwarded URL (keeping it off node URLs and logs, forwarded only as the X-Silo-Stream-Token header), and the node builds relative segment URIs from that token-less query. The segment URLs the client received carried no token, and the proxy only re-attached the header when an incoming segment request already had "st" — which it never did — so a restart made those segments non-reconstructable and they 404'd. proxyToTranscodeNode now rewrites the manifest body at the boundary: every segment and #EXT-X-MAP init URI gets the client-facing, API-verified token re-appended (new playback.AppendManifestQueryParam helper), so the client's later segment fetches carry "st" again and reconstruct after a restart. The token still never reaches the node URL or its logs. Only 200 .m3u8 responses are rewritten (Content-Length corrected); segments stream through untouched. Part of #174 * fix(playback): preserve subtitle/cadence recipe across offloaded audio switch Switching audio on a remote (offloaded) transcode with burned-in subtitles silently dropped them, and reset a non-default segment cadence. The offloaded audio-switch restart rebuilt the node start request from Session state, but Session/SessionStreamState retained no subtitle or segment-duration state (only the live local ts.Opts() and the RecipeCard did), so the branch hard-coded SubtitleTrackIndex:-1, SubtitleBurnIn:false and SegmentDuration:Default — signing that altered recipe into the replacement stream token. An audio switch then changed bytes beyond audio selection, and any later reconstruct kept the wrong no-subtitle/wrong-cadence recipe. Persist the byte-affecting recipe on the session: SubtitleTrackIndex, SubtitleBurnIn and SegmentDuration are added to Session/SessionStreamState, populated at start (finalizeTranscodeStart) and on post-restart reconstruct (ReconstructSession from the card), carried forward on every audio-switch state update, and read back when rebuilding the offloaded node request and its recipe card. The restart now reproduces the exact live stream. Also resolves the M-4b non-default segment_duration reset. Part of #174 * fix(playback): serialize transcode spawn paths with a per-session lock Reconstruct was single-flighted only against other reconstructs, so a restart-driven segment reconstruct racing a quality/seek/audio fresh start could spawn two ffmpeg processes writing the same output directory at once — segment corruption, partial-write closes, orphaned processes, and skewed active-job accounting. The atomic register-after-spawn (GetOrRegister / the reconstruct compare-on-register) prevented a map leak but not the concurrent disk writers, because the losing path had already spawned. The dedicated transcode node had the same split between handleStart and spawnReconstruct. Add a refcounted per-session lifecycle lock to both TranscodeManager and the node Server, held across "check existing -> spawn -> register": - reconstruct (doReconstructTranscode / spawnReconstruct) re-checks under the lock and yields to any live session instead of spawning a duplicate; - the native and jellycompat fresh-start paths take the lock around their spawn+register (the native path also closes any session a reconstruct rebuilt in the meantime so its fresh ffmpeg is the sole writer); - the node handleStart holds it across teardown+spawn+register. The refcount drops the map entry once no path holds/waits, keeping it bounded. GetOrRegisterTranscodeSession is removed — the lock supersedes it and keeping a register-after-spawn primitive would invite reintroducing the race. Part of #174 * fix(playback): serialize restart re-spawn under the session lifecycle lock TranscodeSession.Restart() releases s.mu across cancel -> wait-for-done -> re-exec and spawns ffmpeg into opts.OutputDir without holding the per-session lifecycle lock. LockSessionLifecycle's contract (fresh start, restart, reconstruct) requires restart to hold it too, but all five callers invoked Restart unlocked: native audio-switch and segment-recovery, compat audio-switch and segment-recovery, and the transcode-node segment-recovery. A restart racing another restart (audio-switch vs segment-recovery) or a fresh-start/reconstruct could land two ffmpeg processes writing the same segment directory -- mixed timelines, init.mp4/segment mismatch, and an orphaned-but-still-writing ffmpeg -- the exact concurrent-writer corruption the lifecycle lock exists to prevent. Add RestartSessionLocked (TranscodeManager) and restartSessionLocked (node Server) that hold LockSessionLifecycle only across the cancel->respawn transition, re-check that the handle is still the live mapped session under the lock, and return ErrSessionSuperseded rather than re-spawning a stale handle. Route all five call sites through them. The lock is released before callers wait on segments so recovery latency is unchanged. Tests: gating (restart blocks until the lifecycle lock frees, then spawns), concurrent-restart serialization, and superseded re-check on both the manager (covers native + compat) and node lock owners. --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
9cae868a27 |
feat(downloads): offline sync for mobile — downloads v2 (#258)
* feat(downloads): offline sync for mobile (downloads v2) Replace internal/download with a unified internal/downloads package and add fully-offline download + watch-sync support for mobile clients, across five independently-shippable phases: - Phase 0: reshape the downloads table and the /downloads contract to be device- and format-aware; add GET /downloads/capability; extend DownloadConfig (default-off keys); update the web download hooks/components in lockstep. This is the one approved pre-lock exception to the additive-only /api/v1 rule (the web app is the only consumer and is updated together). - Phase 1: managed device-library entries (create/list/PATCH/delete/serve), keyed on the X-Silo-Device-Id header. - Phase 2: offline playback manifest plus artwork/subtitle proxy endpoints that strip every presigned URL (inline thumbhashes + authenticated proxies). - Phase 3: prepare-to-file (remux + transcode-to-single-file) as a durable, leased artifact queue with startup recovery, hosted on the task manager; playback.PrepareFile emits one +faststart MP4. Adds the admin transcode toggle and per-artifact LRU cleanup. - Phase 4: offline progress reconciliation -- a clamped event_at LWW key plus a server-assigned synced_seq cursor on watch_progress; an optional clamped updated_at on POST /sync/progress and an opaque ?since= cursor on GET /progress (additive; existing callers unaffected). Security & reliability invariants, each with an acceptance test: 1. Server-owned sync ordering: ?since= delta delivery is driven only by the server-assigned synced_seq; the client clock is bounded (event_at, clamped to now+skew) and used only for last-write-wins on the caller's own profile. 2. Full profile+device authorization on every managed endpoint, with a per-profile content/library access re-check before serving any bytes/assets. 3. Durable artifact recovery: a transactionally-claimed (FOR UPDATE SKIP LOCKED), lease-heartbeat, attempt-counted queue with a startup sweep, so no crash strands a download in preparing and concurrent workers never double-encode. Migrations are timestamped Goose files: reshape downloads (device/format); download_artifacts (durable queue); watch_progress event_at/synced_seq. DB-backed acceptance tests skip without SILO_TEST_DATABASE_URL and run in CI; the invariant-1 progress test also runs against the real SQLite backend locally. Client repos (silo-android, silo-apple) consume the reshaped /downloads/* contract and the updated_at/?since= progress fields and require coordinated follow-up. Implements the maintainer-approved v1 capability proposal for offline sync (downloads v2). AI-use disclosure: implemented by Claude (Claude Code) from the approved design doc under docs/superpowers/specs, with human review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(downloads): series & season downloads + client-pull monitoring Build season downloads and a "monitor a series" capability on top of the downloads v2 (offline sync for mobile) work. Season downloads: - POST /downloads accepts season_number (with series:true) to download one season. CreateSeries/CreateSeason share one body via a listEpisodes closure and register managed entries under a shared batch_id (original-only). Episode files are resolved in a single batched query. Series monitoring (auto-download), client-driven: - New device-scoped download_subscriptions table with a Sonarr-style mode (all | future | latest_season | specific_seasons), a client-enforced delete_watched flag, and a max_storage_bytes cap. The server never deletes on-device files; retention and the hard cap are the client's, the server only soft-gates registration. - The client calls POST /downloads/subscriptions/sync on open / background refresh; the server registers the in-scope, not-yet-downloaded episodes (idempotent via the managed-entry unique index) and the device pulls them on its own schedule. No background worker and no dependency on the notifications subsystem. latest_season follows new seasons (>= subscribe-time season); future excludes the back catalog via air date. - Subscription CRUD + sync are profile+device authorized (device id from the X-Silo-Device-Id header only) with a per-request content-access re-check. The capability endpoint advertises season_download / series_monitoring / monitoring_modes. Also lands the downloads-v2 work already present in the tree: durable artifact (remux/transcode) preparation and offline watch-progress reconciliation, plus the design-spec updates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WIP: epitaxy pre-switch from feat/downloads-v2-offline-sync * test(downloads): fix deterministic ID collision in reconcile test Artifact IDs are time-sortable, so two artifacts created in the same moment share their first 8 chars; combined with a captured timestamp the two preparing-download IDs collided on downloads_pkey. Use the full artifact ID, which is unique per row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): support sqlite userdb backend for managed downloads With the sqlite userdb backend, profiles live only in per-user SQLite stores and public.user_profiles stays empty, so user_devices' profile FK made every managed create/subscription/offline-sync request fail with an FK violation. Drop the FK (shared Postgres tables must not FK profile tables — same rule as notifications) and replace the lost cascade with an app-level purge on profile deletion, wired through ProfileHandler for both backends. DB-backed regression tests cover the no-Postgres-profile-row path and the purge cascade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): dispatch encode kick asynchronously triggerDrain invoked the kick inline, and the kick (taskmanager RunTask) executes the encode task on the caller's goroutine — so a POST /api/v1/downloads with a bitrate quality blocked the HTTP request on the entire queue drain, ffmpeg encodes included, delaying the 202 by minutes on an idle queue. Dispatch the kick on a goroutine; the task manager already serializes concurrent runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): enforce per-user quota on the encode pipeline Two gaps let a user bypass MaxConcurrentPerUser entirely for prepared downloads: artifact-backed rows are created in 'preparing' (never 'queued'/'downloading'), which CountActiveByUser didn't count, and createArtifactDownload enqueued the encode job before limiter.Check, so even a 429-rejected request left a job the worker would transcode. Count 'preparing' as active and check the limiter before Ensure; managed replacements stay quota-exempt since they don't add a row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): protect ephemeral artifact links from LRU eviction HasActiveLink only counted managed (device_id IS NOT NULL) rows, so under a byte budget Cleanup could delete an artifact still referenced by a ready-but-unfetched ephemeral web download — permanently 404ing a row the API kept listing as ready (the artifact row is gone, so recovery can't re-queue it). Any non-terminal link now protects the artifact; only artifacts whose links are all cancelled/failed/revoked are evictable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): batch manifests skip bad entries instead of failing whole batch One deleted or access-filtered episode made GET /downloads/batches/{id}/manifests 404 for the entire season, so a client could no longer fetch manifests for the still-valid entries. Report unbuildable entries in a skipped[] array (revoked | not_found | error) alongside the delivered manifests, mirroring the create path's skip idiom. Also cut the batch cost: the shared series detail is resolved once per batch instead of once per episode, and buildSubtitles reuses the already-loaded media file instead of re-querying it per manifest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(migrations): wrap DO block in StatementBegin/End markers Under NO TRANSACTION goose splits statements on semicolons, so the dollar-quoted DO block failed every fresh install with 'unterminated dollar-quoted string' (SQLSTATE 42601). Already-applied databases are unaffected. Same fix is being applied to main; identical content merges cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): allow season 0 (Specials) in season downloads season_number was a plain int dispatched with '> 0', so requesting the Specials season was indistinguishable from omitting the field and silently broadened to a full-series download. Dispatch on pointer presence, treat 0 as the Specials season, and reject negatives with 400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): capability quality_presets is never JSON null PresetsFor returned a nil slice when downloads are disabled or the user lacks the permission, and Capability's []string{} initialization was immediately overwritten by it — so GET /downloads/capability serialized "quality_presets": null where the contract documents an array. Normalize at the source so every caller inherits the guarantee. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): subscription sync correctness + batched registration Three subscription fixes: - A paused subscription no longer syncs: PATCHing scope (or pausing and changing scope in one request) registered episodes for a monitor the user had just stopped, inconsistently with SyncSubscriptions' guard. - SubModeFuture compares calendar days (UTC): air_date is date-only, so the strict instant comparison permanently excluded episodes airing the same day the user subscribed; episodes with no air date now fall back to their ingest time instead of never registering. - Registration is one batched fetch (GetManagedEntriesByKeys) plus one batched INSERT ... ON CONFLICT DO NOTHING RETURNING (CreateManagedEntriesBatch) instead of a SELECT+INSERT per episode — a 300-episode series cost ~600 sequential round trips per request and every no-op sync re-walked the full set. RETURNING yields exactly the new rows, so the sync response's 'registered' count now honestly reports 0 in the steady state instead of the full in-scope count on every app open. The now-unused InsertManagedEntryIfAbsent is removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(userstore): stamp triggers own the event_at LWW key MarkProgressBatch (jellycompat series mark-played) advanced updated_at but never event_at, and both stamp triggers only defaulted event_at when NULL — so a queued offline event with a client time between the row's old event_at and the mark could win SetProgressIfNewer and resurrect a stale resume position that then re-synced to every device. Make the triggers authoritative instead of adding a tenth hand-written SET clause: whenever an UPDATE changes updated_at without explicitly changing event_at, the trigger advances the LWW key; writes that do set event_at (offline sync's clamped client event time) keep their value. Postgres gets a CREATE OR REPLACE migration; SQLite gets a v12 userdb migration that drops and reinstalls the trigger bodies (CREATE TRIGGER IF NOT EXISTS never replaces). Conformance tests cover both batch paths, the preserved-client-time invariant, and the v11→v12 upgrade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): lifecycle hygiene — squash migrations, dead status, stale-row sweeps Migrations: fold the 20260621 corrective migration back into the base Downloads V2 migrations (its columns/constraints already exist there) and fix the reshape Down, which re-added the narrow status CHECK without collapsing managed-lifecycle rows first — rollback aborted on any DB with preparing/ready/revoked rows; validated against a live row. Branch databases that applied the corrective migration need its version row removed: DELETE FROM goose_db_version WHERE version_id = 20260621020459. Code: drop the dead 'registered' status (nothing ever wrote it; the lifecycle is preparing -> ready; 'revoked' stays reserved for the planned admin revoke flow) along with unused KindDirect and ErrInvalidFormat. Sweeps: Cleanup now runs an age-based hygiene pass independent of the byte budget — cold terminally-failed artifacts (with .part leftovers), orphaned ready artifacts no download row references, and ephemeral web rows older than their convenience-record lifetime (also unpinning their artifacts and bounding GET /downloads growth). The byte budget remains the disk quota per the limits & restrictions design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): sync API doc with v2 fixes; HEAD on file route; Android handoff Document the contract changes from the review fixes: batch-manifest skipped[] shape, honest subscription 'registered' semantics, season 0 = Specials, always-array quality_presets, bytes_sent actual behavior, ephemeral 7-day retention, header-pairing requirement, progress-delta deletion caveat, and the ready/failed push event schema (new §9.4). Add an Android client handoff section (§11) mirroring the Apple one, register HEAD on /downloads/{id}/file for download stacks that probe before ranged GETs, and add season_number to the web create-request type. Flag the /direct-download session-token-in-URL tradeoff; a short-lived download-scoped URL is a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: consolidate download/progress helpers, prune dead code, gate sweeps Behavior-preserving consolidation from the Downloads V2 review: - appendVideoFilterArgs: one home for the burn-in/hwaccel -vf selection, shared by the HLS builder and the single-file prepare builder (the drift pattern that already bit tone-mapping once). - userstore.ResolveProgressState: one home for the min-resume/watched threshold rule, replacing five identical copies across both store backends and the offline-sync ingest. - Download file selection ranks resolutions via access.CompareQuality (adds 4320p, agrees with playback) instead of a private switch. - writeSubtitle uses the shared subtitles.SubtitleContentType mapping. - config.DefaultTranscodeDir replaces three '/tmp/silo-transcode' literals. - Read-side quality/revision defaulting helpers removed: insertArgs plus the NOT NULL/CHECK schema already guarantee the invariant. - Dead code removed: Repository.ListByUser, SubscriptionRepository. ListActiveBySeries, and the stale auto-register-worker comments (the design is client-pull; no worker exists). - Redundant left-prefix indexes dropped from the base migrations (their unique indexes serve the same prefixes). - recover()'s disk-presence sweep and the stale-row hygiene sweep run on startup then hourly instead of every 30s tick (both are O(cache size)). - gofmt/prettier fixes for pre-existing drift in handlers/playback.go and pages/Profiles.tsx. Deferred (noted for follow-ups): quality-ladder preset table collides with the drafted download limits & restrictions design, which specifies its own ladder helper; Download-literal construction consolidation and the managed-identity value object remain open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(downloads): draft download limits & restrictions design Design input for the follow-up v1 capability proposal (quality ceiling, batch size cap, per-user quantity/bandwidth overrides). Committed with downloads v2 because the remediation work explicitly defers the quality ladder refactor and revocation wiring to this spec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(progress): reject malformed updated_at; clamp negative progress inputs Review findings on #258: - A malformed (non-RFC3339) updated_at in POST /sync/progress previously parsed to the zero time, which clampEventAt treated as "now" — letting a stale offline event win LWW as a fresh server-time write. The item is now rejected with a per-item error instead. - ResolveProgressState now clamps negative position/duration before classification so no backend can persist negative progress through UpdateProgress/SetProgress. - The online-write event_at invariant test is table-driven over both SetProgress and UpdateProgress, which share the same contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): close review gaps — permission gates, file-access recheck, artifact-true manifests Review findings on #258: - UpdateSubscription now applies the same feature/DownloadAllowed gate as CreateSubscription and SyncSubscriptions; a PATCH could previously re-activate or widen a monitor and register managed rows after an admin disabled downloads or revoked the user. - Serving download bytes (managed and ephemeral) and /direct-download now mirror playback's per-file authorization via catalog.FileAllowedByAccess: library scope and the profile's max playback quality are re-checked at serve time, with artifact-backed rows checked against the artifact's resolution (a 720p transcode of a 4K source stays servable under a 1080p ceiling). - Offline manifests for remux/transcode entries now describe the prepared artifact (container, codecs, resolution, single selected audio track) instead of the catalog source file the client never receives. - ArtifactRepository.Requeue reports ErrNotFound when the row was concurrently swept; ArtifactManager.Ensure recreates the job in that case instead of linking downloads to a dead artifact id. - "No downloadable episodes" is a sentinel (mapped to 404 no_downloadable_episodes) rather than a bare error that surfaced as 500. - Subscription season_numbers are bounds-checked (0–9999) before the int32 narrowing in the repo could silently wrap them. - HandlePatchDownload reuses requireManaged instead of hand-rolling the same managed-identity checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
QuickandClaude Opus 4.8
|
91b5105f3c |
docs(search): add hybrid semantic search hardening plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
87159b0a38 |
feat(collections): add profile-scoped display filters (#191)
* feat(collections): add profile-scoped display filters
* refactor(collections): dedup display-filter helpers per review
Address code-review feedback on the profile-scoped display filters
without changing behavior:
- Widen CompletedHistoryItemMap to accept ProgressCompletionStore and
drop the duplicate completedHistoryItemMapForProgress copy.
- Extract the duplicated MDBList candidate retry loop into a generic
collectionutil.FetchMDBListWithFallback helper, used by both the user
and library collection syncers, and cover it with unit tests.
- Reuse validateOptionalLibraryIDs in HandleUpdateCollection instead of
an inline positive-ID loop.
- Import the shared COLLECTION_{WATCH,MEDIA}_FILTER_OPTIONS in the
template config form rather than redefining them locally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(collections): sanitize query_definition library_ids fallback
readSourceConfigLibraryIDs validated source_config.library_ids (finite,
positive, truncated, deduplicated) but returned the query_definition
fallback raw, so legacy rows could surface zero/negative/duplicate IDs
that the backend now rejects on save. Extract a shared sanitizer and
apply it to both paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(docs): This makes the agents annoying to work with
* Improve playback session handling
* Support collection source order in catalog filters
* fix(collections): address display filter review feedback
* refactor(catalog): remove duplicate collection query params
* Hide episode media scope for collection overlays
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
QuickandGitHub
|
b3198276f7 |
[codex] fix(subtitles): stream live transcribe_translate cues (#177)
* fix(subtitles): stream live transcribe_translate cues * fix(subtitles): harden live AI transcription |
||
|
Quick
|
562ae635d7 | feat: improve audiobook groups and notification refresh | ||
|
|
e99079abf8 |
Server-side Kindle→EPUB conversion (mobi/azw/azw3) for in-app reading (#171)
* Kindle->EPUB conversion: design + proven wasm build pipeline
Server-side MOBI/AZW/AZW3 -> EPUB conversion so the Android in-app reader
can render Kindle-family ebooks. Conversion runs in-process via libmobi's
mobitool compiled to wasm32-wasi, executed by wazero (pure Go) -- no cgo,
no external binary, arch-independent, sandboxed untrusted input.
This commit lands the design + the validated build artifact (spike done):
- docs/.../2026-06-17-kindle-epub-conversion-design.md (Codex-reviewed;
9 review fixes folded in: failure contract, strong cache key + negative
cache, wazero command-module specifics, FS-sandbox tightening,
double-gated capability, serve headers, .wasm guardrails).
- tools/mobitool-wasm/{Dockerfile,README.md}: reproducible build of
mobitool.wasm (wasi-sdk 25, libmobi 9062742, zlib 1.3.1->wasm), with a
smoke-conversion gate. Build proven on native amd64.
- internal/ebookconvert/mobitool.wasm (+ .sha256): canonical artifact,
built on amd64. go:embed target for the converter package (next).
Spike proven on amd64: -e EPUB path works with --with-libxml2=no (internal
xmlwriter); converts MOBI6/KF8/HUFF-CDIC/unicode -> well-formed EPUB;
verified end-to-end under wazero (WASI preopen + argv + _start). Build
gotcha: link libmobi against real (wasm) zlib, not --with-zlib=no, to avoid
miniz duplicate-symbol clash with mobitool's zip miniz. DRM gotcha:
mobitool prints "Document is encrypted" to stdout but exits 0 -> detect via
stdout + output validation, not exit code.
Not yet implemented: internal/ebookconvert Go package (wazero harness +
cache + singleflight), read-handler wiring, admin flag, client capability.
v1-scope proposal required before PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ebookconvert: converter core + cache (Codex-reviewed)
internal/ebookconvert: in-process MOBI/AZW/AZW3 -> EPUB via the embedded
mobitool.wasm on wazero. Converter compiles the module once and instantiates
per conversion (isolated). Cache adds on-disk, singleflighted, size-bounded,
negative-cached conversion keyed by file identity + module fingerprint.
18 tests pass (DRM-free->valid EPUB, DRM->ErrDRMProtected + no output,
oversize/corrupt/missing/timeout/cancel/after-close, 6/8-way concurrent,
EPUB structural validation incl. stored-mimetype + container rootfile,
cache miss/hit/key-change/singleflight/eviction/negative-cache).
Codex review fixes folded in:
- timeout/cancel classified before generic nonzero exit (WithCloseOnContextDone
surfaces sys.ExitError special codes); no more bogus "exit <huge>".
- DRM detection scoped to known mobitool diagnostic LINES (Document is
encrypted / DRM key not found / Invalid DRM pid / DRM expired / DRM support
not included) -> no false-positive on book text; Print Replica -> clear fail.
- WithMemoryLimitPages cap; capped stdout/stderr writers; MaxOutputBytes.
- read-only fs.FS input mount + dedicated writable out dir; documented that
FS isolation ultimately relies on running as a non-root user (memory-safety
is the WASM boundary). validateEpub now requires STORED mimetype + verifies
the container.xml OPF rootfile exists. Atomic moveFile. Closed-guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ebookconvert: wire Kindle->EPUB into the read handler + capability endpoint
Server now transparently serves Kindle-family ebooks as EPUB when the admin
flag ebook.kindle_conversion_enabled is on and the WASM converter initialized.
- handlers.EbookConversion (converter + per-request flag predicate) on the read
handler; HandleReadFile -> h.serveEbook. Kindle + enabled -> cached EPUB with
X-Silo-Ebook-Conversion: converted, epub MIME, ETag = exact conversion cache
key, must-revalidate. Failure (DRM/corrupt/oversize/unservable) -> raw
original + X-Silo-Ebook-Conversion: failed + no-store, so the client opens
externally. Context cancel propagates (not a conversion verdict).
- GET /api/v1/ebooks/capability advertises {enabled, source_formats,
served_format, header contract}; enabled only when flag on AND converter
wired (double gate) so the Android client can decide whether to flip
mobi/azw/azw3 to in-app.
- router: buildEbookConversion compiles the module once at startup (feature off
if it fails), cache dir is a sibling of TranscodeDir, flag read per request.
Codex review fixes folded in: ETag derived from the exact SourceKey cache key
(id+size+mtime+oshash+module version), not a weaker hash; no-store on the raw
fallback; open/stat failure of a produced EPUB falls back to raw per the
contract instead of 500. 10 handler tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ebookconvert): harden conversion cache, HEAD path, and artifact verification
Addresses adversarial review + CodeRabbit findings on the Kindle->EPUB feature.
Correctness:
- Stop poisoning the negative cache on transient timeouts. Introduce
ErrConversionTimedOut (distinct, non-wrapping ErrConversionFailed); classify
the per-call timeout as transient and propagate a caller's cancel/deadline
verbatim instead of reclassifying it as a conversion failure. remember() now
only caches deterministic verdicts (DRM / failed), so a one-off timeout under
load no longer wedges a convertible book onto raw-fallback for 6h.
- Detach the singleflight conversion from any single caller's context (DoChan +
context.WithoutCancel), so one caller cancelling no longer aborts the shared
work for the others; the cache is still populated for the next reader.
- enforceBudget never evicts the entry it is about to return, and skips other
conversions' in-flight "converting-*" temp files.
- Cache hits refresh mtime so the mtime-ordered budget eviction is a real LRU,
not FIFO.
Read path:
- HEAD is now cache-only via Cache.Lookup: a hit serves real converted headers,
a negatively-cached source serves the failed contract, a miss advertises the
converted representation cheaply without triggering a (minute-long, ~1 GiB)
conversion. The GET still delivers the body + authoritative verdict.
- The admin flag is read through a short-TTL predicate so the read path and the
capability endpoint no longer hit the DB per request.
Artifact / build:
- Add an in-code provenance test (embedded mobitool.wasm matches its recorded
sha256) and a self-hosted CI job that runs the ebookconvert smoke conversions
+ provenance check, so the committed wasm can't silently rot.
- Pin + checksum-verify wasmtime in the build Dockerfile (drop curl|bash).
Docs: correct the design doc cache-key + setting-name descriptions, document the
HEAD/timeout/LRU semantics and resource limits, note DRM-marker brittleness, and
fix the README markdown table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci: remove ebookconvert workflow
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com>
|
||
|
|
e084cdd1d6 |
Add unified literary works for ebooks and audiobooks (#107)
* docs: add literary works design and plan * feat(literary): add work link schema * feat(literary): add work domain primitives * feat(literary): persist work links * feat(literary): score work matches * feat(catalog): include literary work summary on item detail * feat(literary): expose work detail API * feat(literary): assemble work detail * feat(literary): add admin work linking primitives * feat(catalog): group literary items by work * feat(literary): auto-link works during book scans * fix(literary): narrow work match candidates * fix(literary): address work merge blockers --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
13c5e0ba2f |
feat(catalog): deterministic cross-server content_id (#155)
* feat(catalog): deterministic cross-server content_id Replace per-server Sonyflake content_id with a structured natural key derived from provider IDs (movie:tmdb:…, series:tvdb:…, episode:…, local:… fallback), so two servers holding the same title share one anchor for artwork, watch history, progress, favorites and ratings. - internal/contentid: derivation core, SeriesIDFromContentID transform, frozen precedence, SchemeVersion=1, embedded-series-anchor invariant. - internal/metadata/service.go: deterministic id at every mint site. - internal/catalog/history_source.go: resolve show via string transform for anchored episode ids; skip the episodes_pkey probe. - migrations/sql/20260612130000: collision-safe value remap across the 65-column reference graph + COLLATE "C", FK/trigger handling, audit map, working down. Benchmarked against an exact-cardinality copy of cprod-postgres (1.93M episodes, 775k history rows): 2.57x faster history page, 1.7x throughput at 100 concurrent users, 2.7x cheaper per content_id probe. * feat(catalog): re-ID untagged items to deterministic content_id at first match Untagged libraries get a path-derived local: content_id at scan time and only learn their provider IDs later, when the match worker confirms a result. Previously that id was never folded back in, so untagged-then-matched items kept a per-server local: placeholder forever and never converged across servers (re-ID was deferred to a migration rerun). mergeAndPersist now promotes a local: skeleton to its deterministic provider-anchored id at the moment of first confirmed match, via a single new gate (canonicalizeLocalContentID): - target id already taken -> merge onto it (existing rebind machinery) - target id free -> rename in place The rename is a single SQL function (silo_rename_content_id); FK children follow via ON UPDATE CASCADE added to the content_id family, so a fresh skeleton moves a handful of rows rather than the full-table remap the bulk migration does. The guard is one IsLocal prefix check, so tagged content and all refreshes pay nothing, and the move is self-healing under retry. Verified: gofmt/vet/build clean; migrate-validate passes; migration applies on the real schema (up/down/up), FKs gain ON UPDATE CASCADE while keeping ON DELETE; functional test confirms series PK move + series_id cascade + provider-id sweep, and movie rename. Follow-ups (noted in docs): recomposeSeriesChildIDs for a series that accumulated episodes before matching; a lockstep test for the soft-ref list. * fix(catalog): harden content_id parsing and merge per review Address review feedback on the deterministic content_id work: - history_source.go: gate the anchored-episode display-id transform on the full five-part episode shape (split_part parts 2-5 non-empty), not just the 'episode:' prefix, so a malformed id can't transform to 'series:broken:' and vanish at the media_items join. Shared anchoredEpisodePredicate drives both the null-poisoned join key and the series-recovery expression. - contentid.go: unexport the provider-precedence slices so no package can mutate the frozen SchemeVersion ordering at runtime. - contentid.go: add parseAnchored to validate the exact per-kind arity and numeric season/episode suffixes; SeriesIDFromContentID and IsProviderAnchored now fail closed on truncated/malformed ids (e.g. "episode:tvdb:296762"). - canonicalize.go: distinguish catalog.ErrItemNotFound from transient lookup errors (a real error no longer masquerades as "target free"), and allow a matched local source to be consolidated onto the canonical row instead of orphaning a duplicate. * refactor(contentid): URL-safe "-" separator in content_id Use "-" instead of ":" to join content_id components (movie-tmdb-228064, episode-tvdb-296762-1-5, local-<hex>). "-" is an RFC 3986 unreserved character, so a content_id is URL-safe verbatim: encodeURIComponent is a no-op and the id is its own tidy path segment (/item/series-tvdb-296762) with no %3A escaping. The stored value equals the URL value, so there is no encode/decode boundary and an operator can grep the id straight out of a URL or log. Every component is [a-z0-9]+ (or "tt"+digits), so "-" is unambiguous. Pre-release format finalization: this branch is unmerged, so no deployed data carries ":" ids — the migration mints the "-" form fresh and no re-migration is needed. Still SchemeVersion 1. - contentid.go: single `sep` constant drives construction and parsing so the two can never drift; all constructors/parsers and doc examples updated. - history_source.go: split_part transform and the anchored-episode predicate use '-'; kept in lockstep with the package via a code comment. - 20260612130000_deterministic_content_id.sql: derivation and season/episode composition emit '-'; LIKE filters match 'series-%'. - docs/architecture/deterministic-content-id.md: format spec + rationale for the separator choice; this is the design doc the change is derived from. Client-side: the web frontend treats content_id as an opaque string (no splitting/regex), so no client changes are required; existing encodeURIComponent call sites simply stop emitting %3A. * docs(contentid): show why hash/bigint rejected in probe-cost table Add Cross-server deterministic / Zero-join show transform / Human-readable columns to the index-probe-cost comparison so the trade-off is legible at a glance: the 128-bit hash and bigint surrogate are faster but each give up a load-bearing property, and the structured key is the only all-checkmark row. * docs(contentid): order probe-cost table to end on the structured key * docs(contentid): label fenced blocks and drop stray EOF tags Per CodeRabbit review: add 'text' language to three fenced code blocks (MD040) and remove accidental </content></invoke> artifacts at EOF. * fix(catalog): remap array-valued content_id soft references in deterministic id migration The value-remap migration (20260612130000) enumerates the reference graph by FK plus a scalar name+type sweep (text/varchar/bpchar). That misses trending_discover_snapshots.content_ids: it is text[] (excluded by the type filter), named content_ids not content_id (excluded by the name list), and cannot carry an FK — so the bulk remap left those arrays holding stale Sonyflake ids that resolve to nothing until the snapshot regenerates. A counterexample to the migration's "self-protecting, cannot orphan" invariant. Remap the array element-wise in both directions (Up old->new, Down new->old), preserving order and leaving collision/unmatched elements untouched; a WHERE EXISTS guard skips empty/unaffected arrays so array_agg never collapses the NOT NULL column to NULL. Mirror the gap in silo_rename_content_id (20260614120000) with array_replace for the single-value runtime rename so the two stay in lockstep. Verified on PG18: mixed/collision/empty arrays remap correctly and round-trip clean; runtime array_replace preserves order. Surfaced reviewing #155. The jellycompat restart-decode regression and the atomicity-wording nit are posted as review comments, not addressed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(jellycompat): pack content_id into compat UUID reversibly so item ids survive restarts Addresses the restart-decode regression raised in review of #155. With content_id now a structured string instead of a numeric Sonyflake, EncodeStringID sent every item/season id down the one-way SHA1 path, making decode depend on an in-memory reverse map. That map is cold after a process restart (the codec is a process-lifetime singleton), so a client presenting a previously-issued item UUID — resume-from-home, deep link, detail page, image, userdata — got "unknown compat id" until the item was re-listed. Make the encoding reversible instead of stateful: - internal/contentid: add Pack/Unpack, a bit-packed, fixed-budget (<=15 byte) binary form of a structured or local content_id. digitCount preserves provider-id leading zeros (e.g. imdb tt0944947); structured forms are self-delimiting; the local form fills the budget exactly. Provider ids that overflow uint64 return ok=false. - Shrink ForLocal to a 112-bit (sha256(path)[:14]) hash so a local id packs losslessly into the 15-byte UUID payload. 112 bits is far beyond any single server's local-item count. No other code assumed the old width. - internal/jellycompat: EncodeStringID packs item/season content_ids into the UUID (byte 0 = kind, bytes 1..15 = packed, non-zero tag distinguishes it from the numeric encoding); DecodeStringID unpacks first and re-packs to confirm, so an opaque id whose bytes merely parse is rejected and falls through to the map. Numeric ids and arbitrary names (genres, studios) are unchanged. Net: item/season ids decode by pure computation — stable across restarts and across instances — with no lookup table. Only the rare unpackable content_id and non-content names still use the in-memory map. TDD: round-trip property tests in contentid (all kinds, leading zeros, reject cases) and a cross-instance decode test in jellycompat that fails on the old hash+map path. Full contentid + jellycompat suites green; production code golangci-clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(migrate): make the migration run timeout configurable (SILO_MIGRATE_TIMEOUT) The boot-path migration runner hardcoded a 5-minute context timeout. The deterministic-content-id value-remap (20260612130000) does a full-table COLLATE rewrite + 65-column remap that needs ~20 min on a real dataset (615k items / 2M episodes), so it was cancelled at 5 min. Worse, Postgres keeps the orphaned backend running (holding AccessExclusive locks) until it notices the dead client at a statement boundary, while the goose session advisory lock releases on disconnect — so each 5-min boot retry piled a new attempt behind the previous one's locks. The migration never applied; the server boot-looped. Make the timeout configurable via SILO_MIGRATE_TIMEOUT (a Go duration like "60m"); 0 or negative disables the deadline for a one-off heavy migration. Default stays 5m. All three entry points (migrate-status, --migrate-only, boot) honor it. Required for the deterministic-content-id migration to apply on any real-sized database, not just dev — the 5m cap made the PR undeployable at scale. Follow-up (not here): on cancellation the runner should actively terminate its backend so a future timeout cannot orphan a lock-holding statement. TDD: MigrationTimeout parsing (default/override/zero/invalid) + MigrationContext deadline behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(contentid): require exact length for local ids in Unpack Tighten the tagLocal branch of Unpack from `len(body) < localHashLen` to an equality check. The local form fills the compat-UUID payload exactly (no padding), so a body of any other length is non-canonical; matching it exactly keeps Unpack a strict fail-closed inverse of Pack for the fixed-length branch, which decodes client-supplied UUIDs. Not applied to the structured branch (a review suggestion proposed the same change there): structured ids are self-delimiting and the compat layer pads them with trailing zeros to fill the 15-byte UUID payload, so ignoring trailing bytes is intentional and documented. Rejecting them would make every structured id fail to decode — the jellycompat cross-instance test guards against that. Not a live bug today (the only caller passes u[1:] from a 16-byte UUID, so body is always exactly localHashLen, and idcodec re-packs to verify), but it is the correct contract and zero-risk. Adds a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
88aa769fe6 |
feat(collections): surface server collections on the user Collections tab (#156)
* feat(collections): surface server collections on the user Collections tab The user-facing Collections tab only showed personal collections, which are usually empty — leaving most users with a confusingly blank page. Server (admin-curated) collections were reachable only inside each individual library's tab. Add a new GET /collections/server endpoint that aggregates visible library collections across every accessible library (honoring access scope, capped per library with a total_count for a See all link), and restructure Collections.tsx into two titled sections: Your collections (personal) and Server collections (horizontal teaser rows per library, linking into each library's Collections tab). Extract the shared CollectionPosterCard so the per-library grid and the new rows share one implementation. * fix(collections): match server-collections loading skeleton to row layout The Server collections section renders as one horizontal teaser row per library, but the loading skeleton showed a poster grid — so data arriving visibly reflowed the page from a grid into rows. Mirror the final layout (section header + per-library rows of poster cards) in the skeleton, and drop the now-unused COLLECTION_POSTER_GRID_CLASSES import. Addresses CodeRabbit review comment on PR #156. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Align server collections with shared carousel behavior - Add opt-out edge padding to reusable media carousels - Render server collection rows with shared carousel controls and spacing --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
5afe56cfc0 |
feat(jellycompat): add runtime-managed Jellyfin Web compatibility (#77)
* feat(jellycompat): install web assets at runtime * fix(jellycompat): recover stale web operation locks * fix(jellycompat): harden web component management * feat(admin): refine compat settings and restart status * chore(dev): add hot-reload docker compose stack * fix(dev): include npm in hot-reload backend * feat(admin): refine Jellyfin compatibility settings * feat(settings): improve jellyfin proxy summary * feat(settings): improve jellyfin web controls * fix(settings): update jellyfin web removal status * fix(settings): enable jellyfin web after install * feat(jellycompat): auto-select web ui version * test(api): update rate limit handler setup * feat(jellycompat): refine web ui install onboarding * fix(jellycompat): address web ui install review issues * fix(onboarding): mirror jellyfin api runtime status * fix(admin): remove global restart banner * fix(settings): gate restart required tracking * fix(jellyfin): ignore live settings for restart status * fix(jellyfin): avoid restart for live compat settings * fix(subtitles): normalize AI language codes * fix(catalog): support partial title search tokens * feat(branding): add white-label customization * Add push relay engineering plan - Document relay API contracts, APNs/FCM behavior, auth, storage, and ops - Capture implementation plan, provider references, decisions, and README --------- Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> |
||
|
|
6f9427d613 |
docs(process): v1 scope-lock process — proposal template, CODEOWNERS gate, agent instructions (#145)
Implements the process layer of the v1 feature-lock planner: capability proposals arrive uniform via issue form; the lock artifact (docs/architecture/v1-scope.md) is CODEOWNERS-gated; the shared CLAUDE.md/AGENTS.md guidelines gain the scope gate, additive-only API rules, and pre-push checklist for agent-driven contributions. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
df95e3cb95 |
feat(notifications): email notification channel
Adds email as a notification channel built on the shared SMTP core (mail.Sender). Email mode is a per-account preference (off, daily digest, or per-episode) stored in notification_email_prefs; delivery is an account-watermark sweep over notification_deliveries that dedupes cross-profile duplicates, advancing the watermark only after a successful send. Admin controls cover the channel kill switch, the per-episode allowance (off coerces those accounts to the digest), digest hour, and an external URL for deep links inside emails. Availability is advertised through /notifications/capability and the user settings page gains an Email section for opt-in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
a24279e3e3 |
docs(notifications): import notification design docs, add web push + v1.5 specs
Imports the notification system design folder (architecture overview, release-events/inbox foundation, APNs/FCM relay specs, outbound webhooks) and adds the Web Push spec (05, implemented in this branch), the shared outbound-email architecture note, and the v1.5 roadmap (06) covering the remaining work after APNs/FCM were deferred to v2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
QuickandClaude Fable 5
|
adeda3c87c |
docs(notifications): add notification system design plans
Imported the Continuum-era notification specs (durable inbox + websocket foundation, APNs relay, FCM relay, outbound webhooks) and amended them for Silo. Amendments from the 2026-06-11 review: wire contracts normalized to Silo naming, back-catalog seeding and per-series burst suppression, durable dispatch outbox, cross-library episode dedupe, forward-sync wake API, ticket-based websocket handshake, relay threat-model additions (egress IP, keyed collapse IDs), and webhook 4xx/SSRF hardening. Includes a self-contained design-decisions.html visual overview. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
39ba284c9d |
feat(ai): shared AI core — metadata translation, Whisper ASR, per-profile language, on-view translation (#127)
* docs: design + plan for shared AI core, metadata translation, Whisper ASR Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai): shared LLM client, segment translator, and job runner packages internal/ai/llm: OpenAI-compatible chat client moved out of subtitles/ai, plus /v1/audio/transcriptions (verbose_json) for the ASR work; one shared retry/backoff loop for both. internal/ai/translate: the batched indexed-JSON translation protocol generalized to text segments. internal/ai/jobrunner: dispatch/heartbeat/reaper/cancel lifecycle extracted behind a minimal store interface, with a semaphore shareable across job services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(subtitles): consume shared AI core LLMTranslator becomes a thin cue<->segment adapter over aitranslate; the service delegates dispatch/heartbeat/reaper/cancel to jobrunner; the local OpenAI client is gone in favor of internal/ai/llm. Behavior (prompts, wire protocol, job rows, recovery semantics) is unchanged. NewService now takes the dispatch semaphore so all AI job services can share one bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(config): shared ai.* settings, metadata translation job table, localization provenance columns ai.* connection keys (chat + optional separate ASR endpoint) load with a fallback to the legacy subtitle_ai.* rows — those are never renamed in SQL because encrypted values are GCM-bound to their setting key. New toggles: subtitle_ai.transcribe_enabled, metadata_ai.enabled. Migration adds metadata_translation_jobs, per-field provenance (provider|ai|manual) on the localization tables, and media_folders.auto_translate_metadata. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): localization field provenance with provider/ai/manual precedence Provider upserts keep manual values and never blank a field with an empty incoming value; new UpsertAITranslation/UpsertAIOverview methods write AI fields only over empty or ai-sourced values (force adds provider, never manual) — all enforced in single-statement SQL. Serving now merges only non-empty localized fields onto the base item, since localization rows are legitimately partial (AI rows carry no titles/artwork). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(metadata): AI translation service, refresh auto-fallback, and admin API internal/metadata/translation: job service over the shared AI core that expands an item to its season/episode overviews, skips already-localized fields (zero model calls on repeat runs), batches paragraphs through the generic translator, and persists per batch with provenance-aware upserts. MetadataService gains an AutoTranslator seam invoked after each refresh for libraries with auto_translate_metadata. Admin endpoints under the metadata curation guard: enqueue, list (poll), cancel; plus a status probe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(subtitles): Whisper ASR transcribe and transcribe_translate jobs New WhisperTranscriber: one ffmpeg pass extracts the audio track to 10-min 16kHz mono WAV chunks (temp dir cleaned on every exit path), each chunk goes to the OpenAI-compatible /v1/audio/transcriptions endpoint (verbose_json, per-request timeout sized to 3x chunk duration), segment timestamps are offset and built into wrapped cues. Chunks process playhead-first and stream live to the requesting session. The transcript is stored as an ordinary downloaded subtitle (provider 'transcribed'); transcribe_translate chains the existing translator and stores the translated track as the job result. Enqueue accepts an optional kind; status reports transcribe_enabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): AI services settings, metadata translate action, library auto-translate, generate-from-audio New AI Services admin page hosts the shared endpoint config (reads fall back to legacy subtitle_ai.* values, writes target ai.*) and the three feature toggles; the AI card moves out of Subtitles settings. The metadata editor gains a Translate-with-AI panel with job polling and force/re-translate. The library form gains the auto-translate toggle (threaded through the libraries API). The player translate modal gains a From-audio mode that lists audio tracks and submits transcribe / transcribe_translate jobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: gofmt import grouping in router and translation tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(catalog): per-profile metadata language and viewer-triggered description translation user_profiles.preferred_metadata_language threads through the access scope into catalog serving: presentation language now resolves explicit param -> profile preference -> library metadata language (native API and jellycompat). ItemDetail gains pending_translation_language when the viewer's language is missing a localized overview. New metadata_ai.on_view setting (off|button| auto) gates POST /items/{id}/translate-description: any profile with item access may request its language, with in-flight dedup and a 15-minute failure cooldown so page views never hammer a broken endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): on-view description translation with per-profile metadata language Profile playback settings gain a Metadata language picker (library default inherit). Detail pages: when the server reports pending_translation_language and metadata_ai.on_view is 'auto', the description translates on view with a pulse animation until the refetched detail comes back localized (45s timeout); in 'button' mode a small Translate chip triggers the same flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): expose metadata_ai.on_view in AI Services settings The on-view translation mode had no UI control, so it could only ever be 'off' — viewers got neither the auto translation nor the fallback button. Adds the off/button/auto selector to the Features card, and the config loader now warns and falls back to 'off' on a bad row instead of refusing to start. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): clear configuration hint when the transcription endpoint is chat-only A blank Transcription base URL falls back to the chat endpoint; chat-only gateways reject the multipart upload with an opaque 400 that reads like a pipeline bug. 400/404/405 transcription failures now carry a hint to set a Whisper-compatible endpoint in AI Services. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): wrap ASR cue text by rune count, not bytes Arabic/Cyrillic/Greek text is 2+ bytes per character in UTF-8, so byte-based wrapping broke lines at roughly half the intended visual width. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(web): steer transcription base URL hint away from chat-only gateways Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): block chat-only gateways for transcription, add endpoint presets llm.IsChatOnlyGateway (OpenRouter et al — no timestamped transcription API) is enforced in three layers: the settings API rejects ai.asr_base_url values pointing at one, the router disables ASR with a warning when the blank-URL fallback would land on one, and llm.Transcribe refuses outright. The AI Services page gains one-click transcription presets (Groq turbo/accurate, OpenAI, self-hosted speaches) plus the mirrored client-side check, and the settings API now also validates metadata_ai.on_view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): tighten ASR subtitle sync Three systematic timing-error sources addressed: cue offsets now use the segment muxer's exact per-chunk start times (segment_list CSV) instead of assuming index*chunk_seconds; the audio stream's start delay relative to the container timeline (common in TS remuxes) is probed via ffprobe and added to every cue; and the chunk length is now operator-tunable via subtitle_ai.asr_chunk_seconds (60-600s, default 600) since shorter chunks bound Whisper's within-chunk timestamp drift. Playhead-first ordering now pivots on real chunk starts, and a beyond-end playhead starts at the final chunk instead of restarting from zero. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): tolerate base URLs that already include the /v1 segment Providers like DeepInfra expose their OpenAI-compatible API under a base that contains the version segment (api.deepinfra.com/v1/openai); always appending /v1/... mangled those. endpointURL now appends bare paths when the base already carries /v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): prefer self-hosted transcription in presets and hints Preset order becomes self-hosted (recommended) -> Groq turbo -> Groq large-v3 -> OpenAI, and the settings hint plus the job-error hint lead with the self-hosted option. The self-hosted preset now fills the turbo CT2 model to match the recommended speaches setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(subtitles): request VAD and word timestamps for ASR cue accuracy Without vad_filter, faster-whisper servers report wall-to-wall segment times: cues linger on screen through silence (verified up to 91s) and paragraph-length segments become single 400+ char cues. Request vad_filter=true (skipped for hosted providers that reject non-OpenAI fields and run VAD server-side) plus timestamp_granularities word+segment, and rebuild cues from word timings: split at speech pauses, sentence ends, text capacity, and a 7s max duration; cap word-less segments instead of trusting their reported end; stretch sub-second cues to a readable minimum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
54b20f57d7 |
feat(audiobooks): player QoL — shortcuts, smart rewind, per-book speed, volume, chapter nav (#125)
Brings the web audiobook player to parity with best-in-class audiobook apps (Audiobookshelf, Audible, BookPlayer) on the quality-of-life axis, per docs/superpowers/plans/audiobook-player-qol.md: - Keyboard shortcuts active on every route while the player is mounted (space/K, arrow skips, volume, M, N/P chapters, shift+./, speed, E expand, Esc collapse), inert while a video session exists. - Configurable skip intervals with asymmetric defaults (back 10s, forward 30s) via a new player settings popover. - Speed control expanded to 0.5x-3x with a 0.05 stepper, presets, and per-book memory (LRU-capped, device-local). - Chapter prev/next transport buttons (prev restarts the chapter when more than 3s in, the universal player convention). - Volume/mute via the shared VolumeControl with a new theme-token "surface" tone; persistence shared with the video player. - Smart rewind: backs up 3-30s on resume scaled by pause length, plus a flat 10s on cold resume from the detail page; explicit seeks are never second-guessed. Toggleable in player settings. - Now Listening time label cycles total / remaining / remaining at current speed. Chapter flattening is deduplicated into lib/audiobooks/chapters.ts, the SkipIcon is shared between both player views, and new pure modules (smartRewind, chapter nav, prefs) come with unit tests. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Quick
|
ef78a1c5da | Merge branch 'main' of https://github.com/Silo-Server/silo-server | ||
|
|
8b70357703 |
feat(ebooks): first-class ebook libraries, scanner, and reader (#124)
* docs: define ebook architecture matching audiobooks * docs: plan ebook audiobook-parity implementation * feat: add ebook scanner parser foundation * fix: harden ebook scanner foundation * fix: handle ebook isbn labels * fix: guard ebook subtree scans * feat: scan ebook libraries in core * fix: preserve ebook scan people credits * fix: refresh ebook scan metadata safely * feat: persist ebook series membership * test: cover ebook series persistence decisions * fix: address ebook scanner PR review * docs: clarify ebook foundation PR scope * feat: add ebook metadata enricher * fix: harden ebook poster cache * feat: wire ebook metadata sync task * feat: expose ebook library metadata setup * feat: add ebook catalog scope support * feat: add ebook detail view * feat: label ebook file versions by format * feat: use file-size copy for downloads * feat: use file language in download dialog * test: cover ebook detail authors and downloads * fix: drop narrator credits from ebook scanner merges * fix: align ebook collection filters with book media * fix: drop asin provider ids from ebook enrichment * fix: force ebook people refresh for stale narrators * chore: omit ebook planning docs from branch * feat: add ebook detail related content * feat: add ebook reader file entrypoint * feat: render ebooks with foliate reader * feat: persist ebook reader progress * feat: add ebook reader controls * feat: extract ebook pdf metadata * feat: favor scanner isbn during ebook enrichment * feat: extract fbz ebook metadata * feat: count cbz ebook pages * feat: show ebook file page counts * feat: show ebook download summaries * feat: switch ebook reader files * feat: prefer epub for ebook read action * feat: surface ebook reader progress * feat: sync ebook reader progress cache * feat: hide ebook read action for unsupported files * feat: filter ebook reader file selector * fix: serve fbz ebook archives with reader mime type * fix: detect fbz ebooks from compound filename * fix: authorize fbz ebooks from compound filename * fix: scope ebook catalog facets * fix: reject narrator queries for ebooks * fix: build ebook recommendation text from authors * fix: include ebooks in embedding eligibility * fix: include ebooks in recommendation media mix * fix: include ebooks in recently added recommendations * feat: include ebook progress in recommendation signals * feat: include ebooks in continue watching sections * feat: include ebooks in catalog progress metrics * fix: read ebook isbn from epub metadata * fix: filter ebook asin provider aliases * fix: fall back from unsupported ebook reader files * fix: sort ebook catalogs by reader progress * fix: filter ebook catalogs by reader progress * fix: include ebooks in last watched catalog filters * feat: reflect ebook reader progress in item user state * feat: share ebook progress state across item surfaces * feat: report ebook scan progress * fix: include ebook activity in recommendations * fix: expose ebook reader progress on item detail * fix: support ebook subtree scans * fix: honor profile header for ebook item progress * fix: add ebook library default sections * fix: route ebook continue cards to reader * fix: hide watched toggle for ebooks * fix: route ebook watch tonight cards to reader * fix: route ebook hero actions to reader * fix: detect archive ebook reader formats by filename * feat: cache embedded ebook covers during scan * fix: encode ebook hero reader links * fix: persist non-epub ebook reader progress * fix: scope narrator catalog badges to audiobooks * fix: merge ebook reader progress during item repair * fix: label ebook progress filters as read * fix: show ebook related rails as book covers * fix: remove txt ebook reader support * fix: reject txt ebook reader files * fix: label ebook advanced filters as read * fix: label ebook personalized sorts as read * fix: remove plain text reader loader path * test: cover ebook unread catalog rules * fix: preserve ebook reader library context * fix: link ebook genres with library scope * fix: encode related rail item links * fix: encode catalog card item links * fix: encode hero and continue item links * fix: encode watch tonight item links * fix: encode recommendation and search item links * test: cover ebook scan format set * fix: label ebook search results clearly * fix: make global search prompt media neutral * fix: encode catalog read API ids * fix: encode item API ids * fix: include ebook reader vendor in docker build * fix: make ebook reader build clean * fix: clean ebook embedded descriptions * docs: plan ebook reader shell parity * feat: add ebook reader shell controls * fix: widen ebook scrolled reader flow * fix: remove scrolled reader content width cap * docs: plan ebook reader full parity * feat: persist ebook reader config * feat: add ebook annotations and bookmarks * feat: add ebook reader tools and aids * feat: add ebook advanced reader settings * fix: keep ebook reader panel in viewport * fix: use foliate sizing units for ebook scroll flow * fix: keep ebook settings controls readable * fix: simplify ebook reader settings controls * feat(ebooks): extract local covers during scan (#98) * feat(ebooks): extract local covers during scan * fix(ebooks): read nullable poster paths during cover scan * fix(catalog): coalesce nullable media artwork fields * fix(ebooks): group sibling formats by book identity * fix(ebooks): tolerate legacy ebook metadata encodings * fix(ebooks): decode PDF hex metadata strings * fix(ebooks): harden local cover extraction and format grouping Address review findings on the local cover scan: - Restrict generic sidecar covers (cover.jpg, folder.png, ...) to single-book directories, always accept images named after the book file, and apply exactly one cover per reconcile with sidecar taking precedence over the embedded cover. - Replace the read-then-write poster update with an atomic conditional UPDATE (ItemRepository.SetLocalPoster) so provider/admin artwork is never clobbered by concurrent writers, and refresh locally owned posters when the extracted cover bytes change (thumbhash compare). - Preserve UTF-8 PDF Info strings (including a UTF-8 BOM) instead of forcing everything through Windows-1252; the cp1252 fallback now only applies to non-UTF-8 bytes. - Select EPUB covers by manifest media-type with properties="cover-image" outranking the EPUB2 meta name="cover" id, so XHTML cover pages no longer shadow the real image. - Order CBZ pages naturally (2.jpg before 10.jpg, ch2/ before ch10/) when picking the cover page, via a single O(n) min-scan. - Bump the ebook content group key scheme to version 2 and reprocess rows written under older versions so pre-existing libraries gain sibling-format grouping instead of accumulating duplicates. - Group different formats only (a same-format sibling with colliding sparse metadata stays a separate item) and stop a joining sibling's embedded metadata from overwriting a provider-matched item. - Decode any IANA-labelled OPF/FB2 XML charset (windows-1251, koi8-r, shift_jis, ...) via x/net/html/charset, and wire the charset reader into FB2 parsing which previously had none. - Strip the full .fb2.zip double extension from filename-derived titles and group keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * feat(ebooks): add reader profiles and ruler (#99) * feat(ebooks): extract local covers during scan * fix(ebooks): read nullable poster paths during cover scan * fix(catalog): coalesce nullable media artwork fields * fix(ebooks): group sibling formats by book identity * fix(ebooks): tolerate legacy ebook metadata encodings * fix(ebooks): decode PDF hex metadata strings * feat(ebooks): add reader profiles and ruler * fix(ebooks): address reader ruler and profile review findings - skip renderer setStyles/render when computed styles and attributes are unchanged, so ruler position updates no longer re-style the book view - drag the ruler via a local draft that commits on release, with the surface rect cached at pointer-down - migrate font values persisted before the generic stacks (Inter, Georgia, Merriweather, legacy serif) so the font select never renders blank, with a Custom fallback option for unknown values - make the ruler band click-through and move dragging to a dedicated keyboard-accessible slider handle so links and text selection keep working under the band - share font stacks between options and profiles via READER_FONT_STACKS - surface the active reading profile, move presets to the top of the settings panel, and drop the redundant profile button aria-labels Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ebooks): resolve prefer-const lint error in readest document lib `pnpm run lint` failed on the branch because `direction` is never reassigned in getDirection; split the destructure so only the reassigned `writingMode` stays mutable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Quick <31828688+Quick104@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * Merge branch 'main' into work/ebooks-reader-base Brings the ebook integration branch up to date with main (audiobook library redesign, continue-watching rework and card affordances, quic-go bump, jellycompat fixes). Conflict resolutions favor main's generalized mechanisms and register ebooks with them: - media scope validation goes through IsValidMediaScope (now including "ebook" alongside main's "video" group scope), in Go and in the web filter/search types - continue-watching uses main's typed rails; reading-type sections pull resume points from ebook_reader_progress and the ebook library default section is wired to ContinueTypeConfig(ContinueTypeReading) - item_repo keeps main's derived select-list machinery (itemColumnExpr) and both poster accessors (GetPoster/SetLocalPoster for ebook covers, GetPosterPath for audiobook covers) - web cards/hero/watch-tonight adopt main's buildMediaPlayHref helpers, which now route ebooks to /reader/ebook and encode content ids; ebook affordances (BookOpen icon, Read verb, percent-read subtitle) carry over onto main's reworked components - LibraryForm ebook support ported into main's refactored useLibraryForm/libraryTypes modules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docker): copy foliate-js vendor into Dockerfile.dev frontend stage foliate-js is a file:vendor/foliate-js dependency, so pnpm install needs the vendor directory before the lockfile install layer. The production Dockerfile already copies it; the dev image was missed, breaking make dev-deploy with ENOENT on /app/web/vendor/foliate-js. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ebooks): render Continue Reading sections as upright poster cards All-ebook continue sections previously fell through to the horizontal 16:9 wide card; include ebooks in the poster-variant check so book covers render in their natural 2:3 framing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): stop related-rail highlight ring clipping on detail pages Move the current-item ring onto the cover artwork with a themed ring-offset color (matching the sidebar profile highlight) and give the scroll container top headroom so the ring is not cut off by overflow-x-auto. Applies to both ebook and audiobook detail rails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scanner): harden ebook scanning against data loss and bad metadata - Reconcile missing ebook files like video/audio, with real per-root walk failure tracking (failed/unmounted roots are excluded from deletion), symlinked-root support via the shared logical walker, and the empty-root cleanup allowance before any destructive reconciliation. - Create ebook items as 'pending' so enrichment can promote them to 'matched' (backfill migration included), and protect matched items from re-scan clobbering: title/year skipped, people/series fill-empty only. - PDF metadata: scan head + tail windows (non-linearized PDFs keep the Info dict at the end), require proper key delimiters, head values win. - Cap plain .fb2 reads like .fbz entries; drop .md as an ebook format. - gofmt internal/scanner/audiobook.go (pre-existing drift). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ebooks): make enrichment failures non-terminal with dedicated backoff state - Provider errors now record a failure (capped retries) instead of stamping last_refreshed, which permanently excluded items after transient outages. - Unconfigured metadata chains and the scan-window membership race skip the item without stamping or burning a retry. - Failure tracking moves to a new ebook_enrichment_state table, decoupling it from media_items.refresh_failures (shared with metadata refresh debt). - Preserve non-author people credits when persisting enrichment results. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(catalog): gate ebook progress on hidden history and centralize threshold - Apply user_history_hidden_items gating (video semantics) to the ebook watched/in-progress filters, progress sort plan, and Continue Reading. - Continue Reading pages past dismissed items via the shared collector and dedupes items across pages (also fixes the video path's latent exposure). - Centralize the 0.9 finished threshold as models.EbookFinishedProgressThreshold with a single SQL-interpolated mirror in catalog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(recommendations): correct watcher counting and wire ebook taste signals - itemWatchersQuery dedupes to distinct (watcher, item) rows so one binge-watcher can no longer satisfy minWatchers; the eligibility floor now counts distinct accounts rather than profiles. - Hidden-history gating on GetEbookReaderProgressForUser (signal reader). - Ebook reading produces canonical implicit taste signals (weighted like the equivalent movie progress ratio); ebooks join taste-seed candidates. - Stale GetRecentlyAddedItems doc comment corrected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): harden ebook reader endpoints and serve a Content-Security-Policy - Serve a CSP on all SPA HTML responses: blob/srcdoc book iframes inherit it, so script-src 'self' 'wasm-unsafe-eval' blocks script execution from malicious book content (sandbox alone is defeated by the WebKit allow-scripts requirement). Threat model documented on the constant. - X-Content-Type-Options: nosniff on frontend, jellycompat, and ebook file responses; MIME resolution can no longer fall through to octet-stream for an admitted ebook file. - Annotation PATCH: presence-aware field semantics (absent keeps, present sets/clears), invariant re-validation on the merged row, and an atomic SELECT ... FOR UPDATE read-merge-write. - Request size caps (413) on progress/config/annotation writes; Content-Disposition via mime.FormatMediaType; hidden-history gating in the shared ebook progress lister; FK-cascade indexes for reader tables. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(api): native read-state endpoints for ebooks - POST/DELETE /watched/{id} accepts ebook content IDs: mark read upserts progress 1.0 preserving the reader's file/location (or picks the preferred reader file for never-opened books); mark unread mirrors video unwatch semantics and deletes the progress row. - /history/remove accepts ebooks: hides via user_history_hidden_items without touching the reading position (hidden != unread; next reading activity resurfaces the book, mirroring video re-watch). - Access-filter checks match the video branch; shared logic lives in ebook_read_state.go. Sort metrics/user-state thresholds use the shared constant; profile-header fallback deduplicated. Clients: response is {type: "ebook", affected_count: 1, played: bool}; the existing watched SSE event fires. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): harden the ebook reader UI - Open-flow race: cancellation checked after every await with full stale-run teardown (no wrong-file progress saves, no leaked views/blob URLs); book.destroy() on cleanup. - Progress: monotonic stale-response guard; visibilitychange flush uses the refresh-capable client, pagehide uses keepalive; per-book cross-format progress documented as deliberate. - Settings: side effects out of the setState updater; local edits no longer clobbered by late server config; pending saves flushed on unmount/pagehide. - TTS: generation token so Stop actually stops (Chromium/Firefox synthetic events); Media Session uninstalled on unmount. - External book links: http(s) only, opened with noopener,noreferrer. - apiBlob 512 MiB guard with a user-facing error; fraction bookmarks navigable; search-result key collisions fixed; dead e-ink code removed; getLibrarySortRelevanceScope deduplicated; md format dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): mark read/unread affordances for ebooks - Item detail gets a Mark Read/Unread button; card menus drop the ebook gate and share type-aware labels/toasts (also dedupes audiobook wording). - Watched-state invalidation includes the reader progress query key so the Continue button and percent refresh after toggling. - Continue Reading dismiss copy for ebooks; dismissal path now URL-encodes item IDs (ebook content IDs can contain reserved characters). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: record the PR #124 review and hardening pass Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: rxwatcher <rxwatcher@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0bd4f8cb3b |
fix(jellycompat): restore CanDownload with a real Download route for Infuse (#123)
dd81a7ef set CanDownload=false to stop Wholphin's screensaver from
404ing on the nonexistent /Items/{id}/Download route — but the flag is
load-bearing for Infuse, which refuses Direct Play (Static=true
streaming) of items it believes it cannot download. With omitempty the
field vanished from the JSON entirely and Infuse playback broke, while
PlaybackInfo-negotiating clients were unaffected.
Resolve the underlying inconsistency instead of trading one client for
the other: implement GET/HEAD /Items/{id}/Download serving the original
file (range support, Content-Disposition, optional mediaSourceId for
multi-version items) under stream-group auth, and restore
CanDownload=true now that the route exists. Fixes Infuse playback and
keeps Wholphin's download callers working.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|